Project Progress (week 10)

During the spring break, I did the first data collection of my project. Although there is just one participant so far, I already got some valuable experiences.

First, I got some advices about the questionnaire design. There are some misleading questions making the participant feel confused and I need to correct them. The misleading questions influenc the participant'sperceptions about the website. He checked the number iondicating high difficulty in the ranking sclae becasue he felt frustrated when searching for the answer. It might cause problems on validity of testing participants' "true" feelings about the website.

Second, there are several serious problems about the website found by the participant which were not found by me when designing the tasks. I am excited to see whether the next participant will find the same problems or some other different issues. I found myself actually "learning" from the subjects.

The first usbility testing took us two hours. It is logner than I expected. One reason is that the participant #1 is not only an experienced web user, but also has experiences of creating website and online database system. He is more sensitive to deficiencies of website design than general users and pointed out many details. The data collected from "thinking aload" is the part I feel the most valuble. The other reason is that he is my friend. So he is willing to spend more time getting familiar the website and trying to find as many problems as possible. This is the advantage to recruit people whom researchers know to be participants.

The participant #1 is not an Engligh native speaker. But I found he doesn't have difficulty to complete the test. So I decide keep using Engligh to do the test even though the participant uses the same mother tongue with me.

Here are just few things I learned from this testing. When I finish transcription, I beleive there will be more ideas coming out.

Reading Response (Week 9)

This week's G & C chapter 8 talks about interviewing. It is relevant to me because my project involves interviews. During reading this chapter, I wrote down any ideas coming up in my mind on my "qualitative research notebook."

I decide not to take notes during the interview. Because I am not a native English speaker, what I hear will need to be translated to my language in my mind for understanding. If I take notes during the interview, my concentration on the participants will be distracted. I think the better way for me is to take notes immediate after the interview session is over.

I plan to carry out my first pilot study next week, so I also read the Silver's chapter 4. It also talks about the interviews. The other concern mentioned in Silver's book for interviewing is that: we have to be careful about the "voice and experience" issues (Silver, p. 114-116). This issue causes problems if the researcher directly sees what the participants said as the facts or experiences they actually have. Sometimes it is hard to know participants' retrospective account is what it was really like.

This issue reminds me the importance of "triangulation" or "multiple methods" to enhance the credibility of the data. G & C p. 129 mentioned the other way to enhance credibility related to interview technique: to interview a number of different people in different organizational positions (if the site is in a particular organization), and the cridibility of data you gathered is enhanced if it can be confirmed from several sources.

In my project, I also take the same principle into consideration. I try to involve several data collecting methods: observation, questionnaire, interview, and screen mouse activities recording to enhance the credibility. But at fist, seeking for similarities among the data collected by different ways is not my goal. The purpose of the usability testing is trying to find problems. Even though I always got different opinions from participants, I am still successful on the purpose of "understanding the topic." Especially in the stage of pilot testing, I will keep open mind and expect for the diversity emerges.

Reading Response (Week 8)

Silver's Chapter 10:
This chapter talks about suggestions for writing the research paper. After reading it, I was encouraged somehow. The most valued tip for me is that "Do not put off writing; write as you go." This tip is also mentioned by Dr. Shulman in the class.

The case study illustrated in p.337 gives a good example demontrating the benefit of "writing as you go" which is to help to clarify the research ideas. Something wrong only can be realized when conducting the actual writing.

"If you delay your writing, you are asking for trouble (p.344)."

G & C's Chpater 9:
This chapter talks about two group discussion techniques, focus group and NGT.
Focus group is one of the most common-used techniques in usability testing. Because people might bring up different opinions when coversing with others, it is usually the complement to the individual interviews. Focus group is also seen as a good avenue to obtain suggestions and information for define the research focus or for the questionnaire/protocol design in the preliminary stage.

I decide to conduct the pilot study first and then decide whether to use focus in my project.