Spring 2019

Interaction Design Research

Groups · Ideas · Project Abstract · Progress Meeting · Presentation · Paper

The future is already here — it's just not very evenly distributed.
— William Gibson

Project Overview

In this course, you will complete a quarter-long research project. This project will be completed in groups of two.

At a high level, successful projects raise a question about human-computer interaction, and plan and execute a project for answering that question. Most projects build and evaluate a prototype system, but hacking is not strictly necessary. All projects require a study — obviously a much more thorough study will be expected of projects that do not involve system building. The goal of the project abstract draft (described below) is to help you scope your work appropriately.

To get a sense of what a good scope for a project is, here are the final papers from 2017.

For information on how the project will be evaluated, see the grading page.

Forming Groups

This project will be completed in groups of two. E-mail Nida if you'd like a larger group. You will be subject to lengthy and brutal questioning. No teams with a single member. Project groups will be self-paired. Use Piazza to help find team members. When discussing a potential partnership with someone, you should discuss your background (e.g., programming proficiency or other skills you bring), availability (e.g., do you plan to primarily work evenings or mornings? weekdays or weekends?), and motivation level (ambition for a turing award? Or to just barely graduate?). It's important to be honest with your partner up front, and follow through on commitments you make. If you make commitments that you don't deliver on, your teammate may fire you after meeting with the teaching staff and providing email notification as to the reason to you and the teaching staff. Keep a clear, consistent line of communication open with your teammate.

At the end of class Thursday April 13th, use the online submission system to submit the name(s) of who you will be working with. (All group members should complete a submission.)

For assistance in finding a group, go to the Piazza forum to post your ideas and communicate with others. Also, take a look at the opportunities for collaboration with individuals outside of the class.

Project Ideas

An important part of your course work is to articulate a research question and make a plan to address it. To do this, you'll need to: 1) read a lot of papers in an area of interest, and 2) generate a lot of ideas. Leveraging an existing project on campus will get you to a novel idea quicker, increase your impact, and be more rewarding. Here are some possibilities:

It will take some thinking to find a project where you can ask and answer a research question inside a 10-week quarter. You'll likely iterate on your idea several times -- take care not to fixate on one idea early on. Collaboration with people not enrolled in the class is allowed as long as their contribution is clearly identified. A higher level of quality will be expected with additional collaborators.

April: Progress Meeting

Course staff will meet with each project group to provide feedback on your progress. These meetings will happen during class time. Please use this google doc to sign up for a time slot. Use the online submission system to submit any materials you'd like to discuss (e.g., prototypes, data, draft writing.) Come to the meeting prepared to show and tell with preliminary results and how you plan to course-correct based on your early experimentation. Pilot results are welcome. How will you revise your system/design/experiment/framing so that your project really pops? What will the title of your final paper be? In other words, how will you summarize your research contribution in just a few words? This exercise will helps you focus and sharpen your efforts on what will best address your research question. This focus will be especially important as time gets tight: some things will matter more than others.

May: Study Preflight Check

Your analysis plan should include the following:

What is your current hypothesis?
What 3-5 things might you measure?
How does each of those things connect to your hypothesis?
Draw the primary graph of your results.
What statistical tests do you plan to use for counting and comparing results? List steps/code for at least one measure.
Include your study plan. A study plan is a written list of steps/instructions for conducting your study.

This will done in lieu of your reading for class that day.

On the day that analysis plans are due, we will be doing in-class pilot studies. As a group, be prepared with the materials you would be required to run your study. First approximations are okay, we do not expect fully completed intervention.

June: Final Presentation

At the end of the quarter, you will present your research results to the class and outside guests. We have invited a couple HCI luminaries. Feel free to invite interested friends and colleagues!

Each pair will have 4 minutes to present. This time limit will be strictly enforced, and your time begins as soon as the previous group ends. Any technical difficulty will be counted within your time. Three pairs will form a 'session'. There will be 3 minutes of Q&A after each session.
We recommend using Google slides and sharing your presentation in this folder by 11:59a the day of the presentation.
The TA will have everyone's presentations open on their laptop. If you decide to use your own laptop for presentation, please check it works well by coming early to the presentation session.
To reiterate: If you use your own laptop, then test (and debug) your laptop video projection before presentations begin. Time spent fiddling with display settings will count against your presentation time.
Structure your presentation like a pyramid — begin with a one-sentence statement of your research result. This will get everyone on the same page. Then, offer a short (e.g., 1 slide, 4 sentences) description of what you did and why you did it. Then, explain things in detail.
This presentation is short enough that you can write out everything you want to say long-hand. Do this! This will allow you to convey information efficiently and effectively. Read through it enough times so that you have it basically memorized, but not so memorized that you get flustered if you skip a word or someone asks a question.
Know your audience! You can expect that everyone in the class knows everything you learned in class. So, you don't need to re-introduce the whole field of HCI. A sentence or two to situate your work in the field is good, but spend the rest of the time telling us what you did.
When presenting, stand near your slides. And look at the audience.
Make sure axis labels on all graphs are large enough for the audience to read. Often, graphing programs generate labels that are too small. And in general, start the y-axis at 0, and show the standard error as "whiskers".
Make sure to state who your participants were in your study.
When describing your study, show an example (e.g. screenshot or sample question) of what the participants did or saw.

June: Final Paper

In addition to the presentation, you will present your findings in a final paper.

Page limit: Final papers should be 3-5 pages long in the two-column CHI format. While this may sound short, it is much harder to write an effective, complete short paper than it is to ramble. A good approach to writing a great short paper is to write a long one first, and then trim it down to the most vital parts. Appendices are acceptable and optional (they don't count towards the page limit), but won't be graded. Add one for materials you want an interested reader to see (for example, when we post your project on the website for next year), but don't need to be graded. Page limit includes references.

Much of the advice from above for preparing your presentation applies to the paper as well. Here are a few more suggestions for preparing your paper:

Find a paper that you particularly like because of how it's written, and use it as a template. This paper needn't be on the same topic, but a close mapping in terms of type of contribution (e.g. a tool paper vs. a theory paper) will give you more guidance as to how to structure your paper.
The title and abstract are the most important parts of a paper, and should clearly convey what you did. Motivate your specific problem (not the field as a whole), and focus on what you did. After reading the abstract, the reader should know what your contribution is – don't speak in generalities. For example, instead of saying "We analyze different methods for preparing cookies with interesting ingredients by running a user study.", say "We present three new recipes for chocolate chip cookies each employing a unique ingredient: jellybeans, tofu, and corn nibblets. Cookies were compared using a blind, within-subjects taste test with 30 individuals. The cookie with tofu was found to have superior mouth feel when compared with the other two, but subjects preferred the taste of the corn cookie by a 2:1 margin."
Review the Project Abstract assignment. Make sure you clearly address each of the important bullets from the abstract in your final paper.
To descript your study, have separate sections labeled 'method', 'results', and 'discussion'. (This is standard APA heading structure.) Clearly tie your analysis to your hypotheses.
Use pictures to show your interface and graphs to present your data. Graphs should generally aggregate across participants, and show standard error bars. (Only show individual data points if the reader learns something more by doing so.) Figures should be captioned with what you believe the reader should infer from the figure (e.g. Participants rated tofu cookies to have 25% better mouth feel. Differences between jellybeans and corn nibblets were non-significant).
When presenting empirical results, graphs are preferable to purely numerical tables. You are welcome to include numbers with the graphs.
Please don't include us in the authors or acknowledgements section. Your work is your own.

Groups who do excellent projects will be encouraged to submit their research to the UIST poster track. These submissions are due in early summer.

Tips on Running Effective Studies with People

If you have instructions, present them to participants in written form. You'll have a lot on your mind. Likely too much to remember to say everything you want. Written instructions also help insure that everything is consistent across participants.
Remember to instrument your software so that it logs all relevant user actions. How often did people ___?
Whatever you try first is almost guaranteed to have bugs so iteratively prototype your experiments just like you iteratively prototype interfaces. Test early and often and leave time for iteration in your schedule.
If you're asking someone to do a task, it helps to provide a good motivating scenario. Appendix A of this paper provides a pretty good example of a scenario for a design tool.
Here is an template for a consent form that you can use.

Notes on IRB

In general, when students undertake a behavioral science class project (like most HCI), IRB approval is not required. That said, here are three things to know:
NO data collected as part of a class project may EVER be published unless you receive IRB approval BEFORE beginning to collect data. This is not something that can change later. We encourage PhD students taking the class to apply for IRB, especially given that the IRB process for studying interfaces is streamlined, i.e., usually receives Exempt status. (This is why we offer extra credit for completing the CITI training course, so that you're set up to apply for IRB.) With IRB approval, you can publish your work if something great emerges.
Even for class projects that are not subject to IRB jurisdiction, you still have a responsibility to protect the welfare of those participating in your class project.
If you would like to record sensitive personal information (like health information) you MUST show me written IRB approval/exemption prior to doing so, even if you don't intend to publish your work.
Instructions for obtaining IRB Exemption:
- To qualify for IRB Exemption, your study must fall into one of the categories outlined here, most likely category 1 or 2. If it does not, you require an application for full review and this will take much longer to get approved. Information on that application process can be found here.
- Information on applying for IRB Exemption can be found here. To streamline the process, we have created templates for the files you are required to submit. To apply for IRB Exemption, download these templates, obtain the required signature from Professor Klemmer, and submit them online as soon as possible (it can take several weeks to get approved).