A9 · HCI Design

Assignment

Launch A/B Test

We are going to be using Google Analytics to run rate comparison you planned last week.

Since you will be statistically analyzing the results, recruit at least 20 users. Put another way, that means that each person on your team should recruit at least 7 people each: a couple dorm-mates and a few family members and you're basically there. You can also use social media like Facebook and Twitter to recruit. You can even recruit other people in class! Make sure to launch your test several days before the deadline. How you schedule your analysis around your implementation plan is up to you. Do what's best for your app and your group. Here's some ideas from Optimizely.

Results: Comparing Rates Online

Collect your results from your A/B test which should be comparing the number of users who, for instance, either clicked through a navigational funnel or didn't, viewed a particular page or didn't, or pressed a button or didn't, in both your A and B conditions. Present your data and the results: show the inputs to your chi-squared test, your calculations, the resulting chi-squared value, and the p value. Tables of both your observed and expected values are the best way to visualize your data and calculations. This tool is a helpful resource for calculating the chi-square statistic (note that it corrects for small sample sizes). However, you must show your work for your calculations. Watch this video to see how it's done. Remember that the chi-squared test compares numbers of users, not times. If you find yourself using a different measure, e.g. time spent on a page, then find the appropriate statistical test to show your calculations and interpret its results. The TA's will not help with anything other than the chi-squared test, so do so at your own risk.

Next, discuss your findings: can you draw solid conclusions or are additional experiments needed? What changes would you make based on these results? What other things would you test in the future?

Fit and Finish

For all you designers out there, get ready. It's time to make it pretty. Concentrate on completing the changes based on the results of your in-person test. Attention to detail will serve you well. Make sure the app is optimized for the mobile interface. Same drill as the other weeks, keep updating your development plan.

Also this is the time work on branding your application. Create a finalized app name that encompasses and brands the function of your app clear, and submit it with your PDF.

Submit

The data that you collected. Organize this information so that we can get a sense of the raw data that you gathered. Explicitly state how many participants were in each condition. What feedback (quantitative or qualitative) did you receive? We recommend organizing this information into a table or spreadsheet. (Online Test Results)
Summary of your findings. You should present any statistical analysis that you performed, and any insights or discoveries that your test helped inform. (Online Test Results)
List of the changes you would make (as a result of the online tests). We understand that your test may have concluded only recently, and you would have had time to only make small changes. But we would like you to think about additional features or improvements that are motivated by your results. Add these to your development plan if you have time to complete them. (List of Potential Revisions)
Your final app name. This is the name that will be used in your final presentation. (Fit and Finish)
The URL of the prototype version that you want us to see. Do not forget to give us login information. Note: the URL must work at least until your assignment is graded. If it doesn't work, you'll receive no credit. (Fit and Finish)
PDFs of your development plan from last week and updated development plan. (We recommend making a Google spreadsheet and saving it as a pdf; this gives you a snapshot for comparison.) (Goals, Updated Development Plan)

Note: since we may grade your assignment up to a few days after submission, per the honor code, we expect that the prototype URL show the state of your prototype at the time of submission. You will very likely be updating your prototype after submission, but please do so on another version.

Submit your formatted pdf here

Evaluation criteria & Grading rubric