Kano Questionnaires

synoppsys: The Kano questionnaire is useful (at least on a small scale) as a supplement to the more qualitative data revealed in interviews and focus groups about features that are important to users.

I recently read about the Kano Questionnaire in Mike Cohn’s book, Agile Estimating and Planning and in another article by Cohn. It seemed like a really great way to determine quantitatively what user stories are important to our customers/users. Using Kano’s matrix, you can cross-check an individual’s response to a functional question (the feature is in the system) and to a dysfunctional question (the feature is not in the system) into one of three types of features: 1) must haves; 2) linear (the more, the better); 3) exciters and delighters (cool stuff the user doesn’t know that he wants).

In 2007, we released a web application at the college level for academic advisors to track student contacts. Now it’s being considered for release at the university level. We entered a pilot phase and just finished up focus groups with the participants. From the focus groups and various other surveys, we were able to glean an initial list of user stories. We wanted to confirm our qualitative research with hard data (as we write a proposal for funding) about what features are most important to the users.

We sent a questionnaire to pilot participants with 3 Kano-type questions. We picked three stories that emerged from focus groups that we wanted to confirm quantitatively.

Here is an example of the functional and dysfunctional forms of one of our user stories:

When you go to create a new note, you select a reason for the contact–or topic discussed–from a short list of reasons/topics you use frequently. How do you feel about that?

• I expect it to be that way.
• I like it that way.
• I am neutral.
• I can live with it this way.
• I dislike it this way.

When you go to create a new note, there is no list of reasons/topics from which to choose. There is only the free-text box for the note. How do you feel about that?

• I expect it to be that way.
• I like it that way.
• I am neutral.
• I can live with it this way.
• I dislike it this way.

20 out of 30 people responded. It did take some time to go through each of the individual responses and categorize according to the Kano matrix but we got some interesting results.

Most notably, the pilot participants were much more indifferent about one story which we thought was a “must-have” from our focus groups. Also, another story which we thought was a “must-have” showed more of an “exciter” quality. The third story had been a “must-have” in focus groups and was an overwhelming “must have” in the questionnaire.

The lesson for us was that the Kano questionnaire is definitely useful (at least on a small scale) as a supplement to the more qualitative data revealed in interviews and focus groups. It gives us the “numbers” to support (or not support) our assumptions. The results will be included in our proposal for funding to the university.