User Study Lessons

Back to Home

The following is a set of lessons I learned while putting together user studies.  The context of the lessons can be found in the Project Journal.

User Study Lesson 1:  A good payment estimate for participants of user studies is $50/hour. (from journal entry Week 1)

User Study Lesson 2: When dealing with organizations like CPHS, try to think about all the details that would affect your human subjects (e.g. How will data be collected and stored?  How will subjects' identities be protected? What is the payment schedule?) CPHS is concerned with protecting human subjects from disclosure of identity and to make sure they are informed of all terms of the project, including compensation.  (For instance, in our study, as common with many study, we assigned all users with a code number and labeled all data by code number in order to protect their identities). (from journal entry Week 2)

User Study Lesson 3: Set reasonable correspondence deadlines for users. I contacted one potential trial participant and didn't set a clear deadline one when I needed to hear from him regarding his interest. I didn't hear from him for a few days and was in limbo - I didn't know if I should contact someone else or wait. I also wanted to start the study while I was waiting. I should have initially stated that if I don't hear from the user by X date and Y time, I would contact a different candidate. I think a reasonable telephone/e-mail correspondence deadline when dealing with users is within 36 hours to 2 business days. (from journal entry Week 2)

User Study Lesson 4: Test your questionnaires and all data collecting forms on colleagues or friends before sending them out. The quesstionnaire was used by Jen in an initial test, but I should have passed it by other people in another round. (from journal entry Week 5)

User Study Lesson 5: Consider awarding survey participants for timely submissions, and reducing payment for late submissions. It's frustrating when data is late, and money incentives may motivate users to submit data on time. (from journal entry Week 6)

User Study Lesson 6: Monetary payment that user study participants receive is considered taxable income. In order to bypass the paperwork involved in this, in addition to avoiding the delaying process of obtaining a University check, co nsider offering gift certificates as compensation for survey participation. I'v e found many user studies offer gift certificates. (from journal entry Week 7)

User Study Lesson 7: Communicate clearly with survey participants and confirm participant information before initiating the survey. I found this especially important when communicating with senior participants. Due to different hearing abilities and other factors, there were several occassions when the participant asked me to sp eak louder and slower. (from journal entry Week 8)