Assignment 8: Test your prototype


The goal of this assignment is to test your prototype with two people to further streamline your app and inform your A/B test question and design.


Step 1: Develop a protocol

Protocols, or "usability scripts," help keep tests consistent across testers and facilitators. Write a user testing protocol that covers:

  • Preparation and setting up
  • Gaining informed consent using this form
  • Executing the test, identify who does what
  • Written instructions that you will read to the testers, and any other materials (e.g. questionnaires, interview questions) that will be used during the session
  • How your observations will be recorded
  • Debriefing the tester and a team debrief

Submit your testing protocol and signed consent form for each participant. Immediately after each test, do a quick debrief with your team and write down any reactions or thoughts that came up.

Step 2: Watch people use your prototype

Observe at least two different people test your interactive prototype. Try to find representative testers who you would expect to use your app.

One person will facilitate the test and interact with the tester, and the rest of the team will be in charge of taking notes/photos/video/etc. This time your user will not be writing down the problems they find for you. It's your job to learn what the people testing your prototype are thinking; the feedback they provide you will be invaluable for your next iteration. Your goal is to find ways to improve your interface.

Submit a photo or sketch of each tester using your prototype. As with the needfinding assignment, these photos with captions should show breakdowns and design opportunities. Contextualize them by capturing the action, e.g. using over-the-shoulder shots, and the setting. Look for other breakdowns and pain points in your interface and try to understand what the problems are and how you might fix them. When possible, modify/update your prototype before running the next participant.

Step 3: Compile your results

After testing, take some time with your team to reflect on your findings. Go through all the notes and other records. Try to be objective; don't write problems off. Discuss as a team and define some general patterns in people's behavior. When you identify some interesting points, talk deeply about them - ask each other questions, recreate the different tests, analyze the decisions people made, other paths they could have taken, and so on.

Submit a detailed and understandable list of changes that you will implement as a result of your testing and discussion, with justifications. Fix the bugs that are either small and easy to fix, or too severe to ignore. Make sure that you do this before moving on to the next step of this assignment.

Step 4: Create a meaningful redesign for A/B testing

A/B testing is a powerful web design tool that leverages random assignment and the easy-to-use chi-squared statistical test. In general, it requires measuring count, or frequency, data of some kind, e.g. number of heads in a series of coin flips, or number of users out of all visitors to visit a certain screen in your app. From your user testing, you should have identified many design breakdowns or opportunities and their potential solutions. Each solution will manipulate, or have consequences, on the user’s interaction in some way. For instance, changing the size or location of a button may increase the likelihood that a user follows an optimal navigational path. In this case, you can measure the effect of this manipulation by counting the number of users to follow this path, and those who didn't, using the chi-squared test. Identify and redesign ONE component of your prototype that resolves a breakdown or leverages an opportunity with an outcome that users can be binarily classified into, e.g. clicked or didn't click. The redesigned component needs to be noticeably different from the original design. Design is often a slow and iterated process, so select something small and manageable in scope. See the lecture videos for more information.

Creating simple paper mockups of your redesign is highly recommended. Then, electronically create and submit two separate URLs for your redesign and your original application - don't write over your old code!

Submit a description of the online test you will run for the next assignment. How will you measure your manipulation? What are the possible outcomes and their interpretations?

In Studio

In studio, you will present your ideas informally to the other teams: What were some major findings? What changes did they translate to? What are you going to do moving forward? Then you will work with your studio leader to prepare for A/B testing!

Student Examples

Here are three student examples from last year.

  • Example 1 - This is an example of an A+ level assignment. This group obviously put a lot of thought into their in-person test, and was able to motivate their redesign from the conclusions they drew from the in-person test.
  • Example 2 - This is an example of a B level assignment. This group lost points for not including their consent form for the in-person test. We also wished the feedback was more substantive beyond obvious usability bugs (one of which had been mentioned by the TA in a previous assignment). For the online test description, we were not convinced that measuring click rates was the right metric to measure success.
  • Example 3 - This is an example of an A level assignment. We liked the clean and well captioned photos for each participant testing their app. They also tested more than the required two users.

What’s this for? A UX agency perspective

By Mike Davison, Community TA and UX Project Manager

Testing your high fidelity prototype with users closes the circle. It is vital to ensure your solution meets the needs identified during the first assignment, and that the agency has not simply spent months drifting further from the problem.

It also allows you third party reflection and suggestions for tweaks to the design. Everything we learn here and correct is a problem we don’t have to live with because it has already been coded and is too costly to’s a valuable phase of the process.

Remember this - feedback is not criticism, feedback is not personal. User centred design works best when pride is left aside, and the feedback of others is incorporated into your design thinking!

Common "I like" Feedback

The following statements are common feedback given on this assignment. We call these statements 'I like' feedback because they are a way to express positive aspects of the submission. You could think of these as elements to aim for.

  • Redesigned an interesting and useful part of their app
  • Was thoughtful about the design breakdowns they found
  • Ran well thought-out user tests
  • Tested tasks important to users of the app
  • Created a thorough set of testing materials (plan, script, etc.)
  • Took interesting photographs
  • Had prototypes that were well-designed and bug-free


  • Your testing protocol and signed consent forms, as well as any materials you gave to the user as part of your tests (either as text, PDF or a scanned image). (Testing Protocol & Documentation)
  • Captioned photos for each participant testing your prototype. (Photo Documentation)
  • A detailed list of changes you will implement in your next iteration, with justifications. (Planned Changes Based on Test)
  • The URL of the original prototype you tested and the URL of the implemented alternative redesign of one interface element. Including submissions of the paper mockups would be very helpful. (Alternative Design)
  • Description of online test. (Description of Planned Online Test)
  • Copy of last week’s and this week’s plan submitted via PDF in a readable and easy-to-compare format. Link to Google Sheet with history should also be submitted to help with grading. (Updated Development Plan)
Submit your formatted pdf here

Evaluation criteria & Grading rubric

The rubric below contains criteria that are worth one point each and will be graded independently and in a binary fashion.