TerpGenius: Online Tutoring System

by Team Giga
  • Patrick Campbell – Created and shared a Google Doc to take notes during user testing; served as an observer and recorder for two different sessions; contributed to report.
  • Joe Stout – Brought a digital camera and tripod to video record the user testing sessions; served as an observer and recorder for two different sessions; contributed to report.
  • Olivia Outlaw – Modified the IRB consent form to fit the project; scanned the signed informed consent forms; served as an observer and recorder for two different sessions; contributed to report.
  • Nathan Yang – Crafted Paper Prototype #1 and Paper Prototype #2; took pictures of all of the paper prototypes and uploaded to the wiki; served as an observer and recorder for two different sessions; set up the structure of the report; created post-study survey and posted survey results; contributed to report.

Section 1: Abstract

TerpGenius is an online tutoring system designed for Computer Science students at the University of Maryland, College Park. Its goal is to allow students to seek help from tutors and to allow students to market their tutoring services. During the user testing sessions, we asked our participants to perform the following tasks: (1) browse tutors, (2) register / sign-in, and (3) find an instant match. The first paper prototype, titled "One, Two, Three, Match!", has a simple layout that utilizes iconography. It features three large, square buttons on the home page, specifically "Browse Tutors," "Find Students," and "InstaMatch." The first prototype only uses the user's location to perform an instant match. The second paper prototype, titled "Search Engine and Questionnaire," features a Google-like search bar on the home page. Unlike the first prototype, it features an in-depth questionnaire so that the instant match process can perform a match using more information about the user. To evaluate both prototypes, we asked the participants to voice their thoughts while performing a specific task. At the end, we asked them to compare and contrast both prototype versions by filling out a survey. During each session, we recorded the participants' thoughts and also took notes on how they completed the tasks. We recorded any difficulties or confusions that users experienced while completing a task. In the first prototype, we found that people liked the simplicity of the registration page since it did not span multiple pages. They also remarked a disadvantage of the first prototype, specifically the fact that it does not ask for many details about the user. Consequently, the instant match process did not have many parameters to consider when matching a user with a tutor. In the second prototype, we discovered that people enjoyed filling out a series of questions that can help them find the best match with a tutor or tutee. They also felt that the second prototype guided the user in accomplishing tasks. However, users did point out that the second prototype was more cluttered since they had to go through an in-depth process to register and fill out a questionnaire in order to get instantly matched.

Section 2: Task Descriptions

We formulated the three primary tasks by brainstorming the main objectives of TerpGenius. We then used feedback from the peer review sessions to refine the main tasks. The tasks browsing tutors, user registration and sign in, and instant match are all important features of an online tutoring system. Being able to quickly and efficiently perform these three tasks will convince users that TerpGenius is a useful tutor and tutee matching system. Below is an outline of the three tasks that were performed on our paper prototypes by the four participants.

1. Browsing Tutors

This task requires the user to navigate to a list of tutors available to either review or contact when the student is in need of a tutor. We did not give specific tasks to the user other than to simply find where they can begin browsing for tutors from the homepage of both paper prototypes. Browsing tutors from the homepage quickly is key for users who do not want to register and are in a rush to find a tutor at the last minute. The peer critics indicated that some users would like see the tutors available before committing to a registration and then determine afterwards whether they want to register as a user. We agreed with the critiques and made two different processes, but kept it as a primary task for user testing.

2. User Registration and Signing In

User registration is how we can build the user base of TerpGenius and is pivotal to the customized experience we want to provide students and tutors. From the browse tutors page, we asked our participants to register an account on TerpGenius and sign in after receiving an email confirmation of their registration. This allows the user to begin customizing his or her student or tutor profile for other users to see and to introduce the instant matching feature we want to include in our final product. The peer review critics gave us the direction we needed for both designs and emphasized that we need to focus heavily on the design of the registration process. Consequently, both prototypes utilize different registration processes so we could get useful comparative feedback from the participants. By approaching the registration process differently but still keeping it as a main task, we are iterating to a much better design. After the users registered on both prototypes, they were asked to log in to the system.

During the sketching phase, we did not consider user registration as a primary task. However, Professor Froehlich mentioned that the registration process could make or break the success of TerpGenius. We realized the importance of gathering information from our users in a way that is not too demanding in order to find the best match and keep them as users of the system. Thus, the first paper prototype had a simple registration form that did not ask for too many details from the user, while the second paper prototype required users to go through several pages in order to register.

3. Instant Match

Instant match is a feature of TerpGenius that allows a quick matching of a tutor to a student based on information provided by the user such as their distance, price, expertise, gender, and rating preferences. Performing an effective instant match requires extra information from the user besides the basic email address, birthday, gender, username, and password requirements of most websites. We made the instant match process separate from user registration (e.g. an instant match does not occur immediately after user registration; the user has to manually request an instant match). This way the user can choose how they would like to find students or tutors, either by manually browsing and filtering tutors and students or by performing an instant match.

We asked the participants to perform an instant match after they had signed into the website with their newly registered account. Once completed, a tutor would be matched based on the information they provided. During the peer critiques, there were many comments about making the instant match process quick and simple so the user would not view the task as burdensome. This led us to make the two prototypes handle the task very differently in order to determine the usability of each instant match version and decide on the best final design. One instant match version is more visual and interactive, while the other is similar to a one-page registration form. This way we still have the same task as before but different approaches to observe testing with.

Section 2: The Paper Prototypes

The goal of our website is to match students and tutors together in a way that is accessible and efficient. It was decided that the important features to have include methods for browsing multiple tutors at once, register with little hassle, and find an instant match at the click of a button. The two prototypes implemented these features using two separate interfaces: 1) a menu-based homepage that is broken down to 3 different buttons and (2) a Google-inspired search engine. Both designs offer a unique experience for the user and clearly showcase varying similarities and differences.

From the moment you visit either prototype, the main page will feel open and free of interference. The features are clearly presented from the very beginning and some widgets, such as the ever present logo that returns you to the main page, are always accessible. The task of browsing tutors was designed in both cases to be available from the start as this is likely to be the first action of a new visitor. There is nothing more frustrating than being denied access to an application that requires you to create an account or log in prior to exploring the system. In addition to this utility, it was deemed necessary that browsing tutors should have selective filters and options to sort the results in multiple ways. This allows the user to highlight what is important to them and get a better understanding of what the website has to offer.

Obvious differences exist between the two prototypes, and this is paramount for evaluating understandability, usability, and approachability. The main page designs reflect this immediately: one version consists of buttons and the other focuses on an all-purpose search bar. While browsing through tutors, users are offered two distinct page layouts with different information displayed and a variety of sorting and filtering options. The largest differences between both prototype versions lie in the registration and instant match feature. The button-based interface focused on quick and simplistic form and contrasts the other's thorough and guided matching system. Finally, a grayed out button on the first interface restrains the new user from using the InstaMatch feature, nudging them to create an account while prototype #2 guides them through a tunneled survey.

Paper Prototype Design #1: One, Two, Three, Match!

A visitor is greeted with three distinct buttons and a sign-in header which all work together to complete our three tasks: browsing tutors, registering, and instant matching. The "Browse Tutors" (Figure 01) button brings them to a catalog of other popular users, displaying information such as hourly rates and 5-star ratings. Browsing can be filtered by subject, location, or other tags and lets you quickly find your favorite starred tutors. At any time, the user can return to the main page by clicking our TerpGenius logo which leads to our second and third tasks.

The "InstaMatch" button will still be grayed out and displays a tooltip that asks the user to register or sign in (Figure 01) by clicking either the grayed out button or sign in header leading to our single registration form. After several text fields asking for login information, general identity, and what kind of user they wish to be (Student, Tutor, or both), they press the "Sign up" button and receive a confirmation message along with an email (Figure 05). The once grayed out "InstaMatch" button is now unlocked and clicking it will assign the user to a tutor or student (Figure 02). A portrait of the user is the focus of the page with their identity, rating, and hourly fee in plain site, and after the user is done learning about the person they can either accept or decline the matching (Figure 06). Selecting "No - I would like a different tutor" attempts a new match while choosing "Yes - I would like [name] as my tutor" displays a confirmation message leading them to arrange an appointment or contact the user (Figure 08).

The decisions that went into the design of this prototype were focused on making the system easy to explore and as hassle free as possible. Through very few steps the user can sign up and immediately make full use of the site while still allowing new visitors a very good look at our marketplace.

Paper Prototype Design #2: Search Engine and Questionnaire

The homepage for prototype #2 remixes Google's search engine design and immediately conveys to the visitor that they can type something in and get results back (Figure 17). In this case they will get a list of tutors relevant to their query, be it by class, name, location, subject or topic. Next to the search button is "Browse" and clicking this directs them to our second browsing interface, highlighted by a different layout and up-down voting system (Figure 18). Clicking the TerpGenius logo still returns you to the main page, and there is still a sign-in header that brings the user to our registration page.

The largest difference between the two prototypes is the registration process. Prototype #1 featured a very quick system but the InstaMatch feature has less parameters to match users with. For the second design, the registration process is more guided and involved, but as a result the matches are going to be more optimal. The web page asks a single question at a time and at any point the user can go back to a previous one (Figure 21). After specifying the necessary information a registration page and email are sent to the user like the last prototype and they can now contact and arrange meetings with fellow users (Figure 25). To use InstaMatch however, more information relative to the user's learning preferences is required and uses the same guided design. At any point the user can skip the process and finish at another time, but when it is complete they are directed to a page containing their ideal match (Figure 37). Multiple pictures of the user are displayed along with their experience, hobbies, and favorite subject. Just like prototype #1, they can "Skip" the match to get another, or "Accept" and get a confirmation display allowing them to schedule an appointment or send a message.

Section 3: Testing the Paper Prototypes

Pilot Testing

Pilot testing was a great learning experience for us, particularly on how we were to go about with the user testing. By going through the computer-user interaction ourselves we learned that the most effective way observe someone using the system is by given them the task and leaving the rest up to them. At first it was thought we should give them an overview of each new page as they explore, but the end product will not have our instructions, just the webpage. It also gave us a much better understanding of the system and how an end-user would perceive it.

Participant Recruitment and Demographics

We are designing a website that hopes to pair computer science students of the University of Maryland so we focused on recruiting people who fit this description. The students we recruited were all undergraduates working on individual projects located in the AV Williams building as well as the Computer Science building. The locations we chose are perfect for finding our demographic; there are many rooms in AV Williams dedicated to helping students with questions and the Computer Science building was filled with students reviewing with each other.We simply approached each candidate asking if they were a student at UMD and had a moment to help us test our two prototypes.

Study Method

The original room we had selected was the undergrad study room across from the Computer Science advisers (a.k.a. the “Fish Bowl”). During our first recording a group of students entered and made hearing the participant's comments difficult so we moved to the adjacent office hours room. Each participant was read the informed consent paper, and after they read it themselves and signed it we instructed them that we would give them three individual tasks. They were also asked to think out loud and give feedback on what they liked, recognized, or were distracted by. Most of the recording was done with a DSLR on a tripod focused over the participant's perspective and lasted about 20 minutes each. After they completed all of the tasks for both prototypes we asked them to complete a Google survey on one of our laptops.

Analysis

The most crucial data we collected was from the user's vocal thoughts as they we navigating our prototypes. By letting us know when they thought something was odd or missing and even if they simply liked the look of the webpage we are learning different ways to make the entire system more approachable.The post-study survey gave the users a medium to formulate or reiterate their thoughts after going through both prototypes, something they could not do while experiencing them.

Section 4: Results from User Testing

As a result of our user testing, we overall found that our participants were generally able to complete our specified tasks without much help. However, there are still things that we found needed improvement. For our first prototype we found that our users did not like the lack of information given to them on the interface and they were not big fans of the rating system. For instamatching they preferred to have more information on the tutor they matched with (e.g. price, history, etc.). For our second prototype we found that most of the problems came in the registering and instamatch process. These problems include things like leaving out the next button on some of the steps in the process which has a lack of consistency, restricting the participants to selecting only one price range for tutors (almost all our participants tried selecting more than one), and the amount of time and effort our participants spent on these questionnaires which we should try to minimize more. Generally our users seemed to prefer the first prototype when it came to simplicity, but when it came to context they seemed to prefer the second prototype because even though they may be putting a lot of time and effort into giving feedback they are more satisfied and confident in the feedback they get when it comes to finding the right match for them.

Section 5: Learnings from this Project Assignment

The user testing was definitely useful information and was necessary in our design process, but parts of the assignment were much more difficult than expected. The construction of the paper prototypes definitely were time consuming, but Nathan did an extremely good job making the user interfaces based on our chosen designs. This could have gone very differently and been much harder to complete the task without great paper prototypes.

The most difficult part for us was finding willing participants. Upon deciding computer science students would be our audience for this assignment, we chose to go to the fishbowl in A.V. Williams to find participants. It was difficult to pry students away from their current assignments to participate in our study for about twenty minutes, and many students said they were too busy. Also, while the video part of our assignment is only for our final assignment, sound was difficult to capture without an external microphone since we had to conduct our trials in crowded rooms. Most of our first hand experience and note taking took care of this problem so it was not too difficult to complete this analysis.

If we had this assignment again we would make more of a schedule for with our participants rather than ask random students. This did work out in the end for our group but it would have made our group time more efficient. Other than that small part our group functioned well during this assignment and had a successful paper prototype user testing.

Section 6: Appendix

Paper Prototypes

Paper Prototype Design #1: One, Two, Three, Match!

IMG_20131104_194317.jpg
Figure 01: The default home page of TerpGenius. The visitor is not logged into the TerpGenius system. As a result, the "InstaMatch" button is grayed out and displays a tooltip stating, "Register first or sign in to use InstaMatch." The "Browse Tutors" and "Find Students" buttons are enabled and clickable.

IMG_20131104_194454.jpg
Figure 02: After logging into TerpGenius, the user is brought back to the home page. The email and password text fields get replaced with a user drop-down menu that displays the user's profile picture, their name, and provides links for editing their account systems, viewing their profile, and signing out. The blue "InstaMatch" button appears only if the visitor is logged into the TerpGenius system. It is hidden by default for anonymous users. Notice how the "InstaMatch" button is no longer grayed out and is clickable.

IMG_20131104_194613.jpg
Figure 03: The screenshot above depicts the "Browse Tutors" page. By default, the list of tutors is sorted by distance. Users can click on the blue "Distance" link to sort by other criteria such as experience, price, and rating. A search box appears at the top of the "Browse Tutors" page, allowing users to filter the list of tutors by keywords. The "Tutors" menu link on the left hand side is grayed out, showing that the user is currently browsing only tutors. They can, for example, click on the blue "All" link to view both tutors and students. The "Starred" link displays a list of bookmarked tutors or students. Users can enter a subject inside the "Subject" text field and a zip code inside the "Postal Code" text field to filter the results.

IMG_20131104_194716.jpg
Figure 04: The registration page appears when the user clicks on the yellow "Sign up" button in the upper right hand corner. The "Sign up" button is hidden if the user is logged into the TerpGenius system.

IMG_20131104_194741.jpg
Figure 05: After the user successfully fills out the registration form, they are brought to a confirmation page that asks them to check for an email from TerpGenius. Inside the email, the user can click on a link to activate their account. If the user provided the wrong email address, they can click on the "change it here" blue link. If for some reason the user has not received the confirmation email, they can click on the blue link "Click here to resend," and the system will send out another confirmation email to their email address.

IMG_20131104_194821.jpg
Figure 06: When a logged in user clicks on the "InstaMatch" button in the upper right corner or the square button on the home page, he or she is immediately shown a potential match. The user can accept the match by clicking on the button with green text or reject the match by clicking on the button with red text.

IMG_20131104_194845.jpg
Figure 07: If the user rejects a match, another match will be displayed on their screen. In this case, a different tutor was displayed. The user can see how long the person has been a tutor and see their rating based on a 5-star rating scale. The name of the tutor, hourly rate, degree earned, and location are displayed above the tutor's profile picture. Users can click on the "Read reviews" link to drop-down and read reviews about that tutor. They can also click on "Biography" to read a short biography on the tutor.

IMG_20131104_194904.jpg
Figure 08: The user gets a "Congratulations!" message after accepting an instant match. Afterwards, they can schedule an appointment, send a message to the tutor, or they can go back to the Instant Match process by clicking on the "Change tutor" button.

IMG_20131104_194935.jpg
Figure 09: The user would see the above figure if they had accepted "Robert Lesher" as their tutor. The buttons "Schedule appointment," "Send a message," and "Change tutor" appear just like the previous figure.

scan.jpg
Figure 10: A close-up view of the "Browse Tutors" page.

scan0009.jpg
Figure 11: A close-up view of the "Registration" page.

scan0010.jpg
Figure 12: A close-up view of the Registration confirmation page.

scan0011.jpg
Figure 13: A close-up view of getting instant matched with "Robert Lesher."

scan0012.jpg
Figure 14: A close-up view of what happens after accepting "Robert Lesher" as a tutor.

scan0013.jpg
Figure 15: A close-up view of getting instant matched with "Joan Smith."

scan0014.jpg
Figure 16: A close-up view of what happens after accepting "Joan Smith" as a tutor.


Paper Prototype Design #2: Search Engine and Questionnaire


IMG_20131104_195040.jpg
Figure 17: The default TerpGenius home page, inspired by the Google home page. Users can type in keywords into the search box and click on the "Search" button or press ENTER on their keyboard. There is also a "Browse" button that allows users to browse for tutors and students. A "Sign In" and "Register" button appears in the top right corner.

IMG_20131104_195107.jpg
Figure 18: Performing a search or clicking on the "Browse" button on the home page brings the user to the "Browse" page. Users do not have to be logged into the system in order to perform a search or simply browse for tutors or students. They can filter the list by tutors, students, or view both tutors and students. The results can also be filtered by distance within a certain zip code and also sorted by distance, experience, price, and rating. The rating system is based on the up/down rating system featured on Reddit and StackOverflow. On the "Browse" page, each tutor result has a profile picture, name, degree, hourly rate, specializations, location, and an up/down rating.

IMG_20131104_195157.jpg
Figure 19: Clicking on the "Register" button will bring the user to the first registration page as shown above. They can click on "Sign in" if they are already a member. Otherwise, users can click on the green "Start!" button to initiate the registration process.

IMG_20131104_195220.jpg
Figure 20: The first registration question asks for the user's gender.

IMG_20131104_195234.jpg
Figure 21: The second registration question asks for the user's birthday. After inputting the month, day, and year, the system will automatically calculate the user's age in years so that the user can verify whether or not their entered birthday is accurate.

IMG_20131104_195249.jpg
Figure 22: The third registration question asks for the user's zip code or city. The user can click on the "Back" button to go back to the previous question. The green "Next" button brings the user to the next page, if they filled out a valid zip code or city.

IMG_20131104_195304.jpg
Figure 23: The fourth registration question asks whether the user is interested in finding a tutor, student, or both tutors and students. Clicking on one of the rectangles will direct the user to the next page. The "Next" button was omitted, so the user does not have to make an extra click.

IMG_20131104_195320.jpg
Figure 24: The final registration question asks the user to enter a valid email address and password.

IMG_20131104_195333.jpg
Figure 25: After entering an email address that does not already exist on the system and a valid password, the user is brought to a registration confirmation page. The user is asked to check their email for a link from TerpGenius. The email contains a link that when clicked, will activate their TerpGenius account.

IMG_20131104_195548.jpg
Figure 26: Clicking on the blue "Sign in" button in the upper right hand corner of most pages (assuming the user is not logged in), will bring the user to a sign in page. After entering credentials, the user should click on the "Sign in" button to login. Alternatively, if the do not have an account, they can click on the "Sign up" button to be brought to the initial registration page.

IMG_20131104_195633.jpg
Figure 27: After the user logs in, the "InstaMatch Questionnaire" process begins. Clicking on the red "Skip" button throughout the questionnaire process will send the user back to the home page, allowing them to complete the survey at a different time.

IMG_20131104_195705.jpg
Figure 28: The first InstaMatch question asks how much of an auditory learner the user is. The gray circles are essentially radio buttons displayed horizontally rather than vertically. Clicking on one of the gray circles will fill the selected circle with a dark color like dark red, denoting the user's selection.

IMG_20131104_195714.jpg
Figure 29: The second InstaMatch question asks how much of a visual learner the user is.

IMG_20131104_195731.jpg
Figure 30: The third InstaMatch question asks how much of a kinesthetic learner the user is.

IMG_20131104_195745.jpg
Figure 31: The fourth InstaMatch question asks what kinds of material help the user learn. The gray rectangles are like checkboxes. When one gray rectangle is selected, the background of the rectangle turns dark gray, and the font changes to white so that the label is still visible.

IMG_20131104_195758.jpg
Figure 32: Once the user gets through the first set of questions, a page with a 50% progress bar is displayed. This tells user how far they are in the questionnaire.

IMG_20131104_195814.jpg
Figure 33: The next set of questions asks about the user's preferences in tutors. The first question asks if the user prefers male tutors or female tutors. At any time during the questionnaire, the user can click on the "Back" button to return to the previous question or click on the "Skip" button to return to the home page and resume the questionnaire at another time.

IMG_20131104_195826.jpg
Figure 34: The second question asks whether the user prefers tutors that are 5 miles away, 10 miles away, 25 miles away, or 50 miles away. Users can select more than one option.

IMG_20131104_195836.jpg
Figure 35: The third question asks the the user to pick out the rate they are willing to pay for a tutor.

IMG_20131104_195848.jpg
Figure 36: After successfully completing the InstaMatch questionnaire, the user is asked whether or not they are ready for "InstaMatch."

IMG_20131104_195912.jpg
Figure 37: If the user in the previous figure says he or she is ready for InstaMatch, then the first instant match appears. On this page, the tutor's name, major, degree, specializations, up/down rating, brief biography, experience, hobbies, and favorites appear.

IMG_20131104_195944.jpg
Figure 38: Accepting an instant match will bring the user to a "Congratulations" page where they are given three buttons to either schedule an appointment with their chosen tutor, send a message to their tutor, or change tutors by getting a different match.

scan2.jpg
Figure 39: A close-up view of the "Browse" page.

scan20001.jpg
Figure 40: A close-up view of the initial registration page.

scan20002.jpg
Figure 41: A close-up view of the registration page asking for gender.

scan20003.jpg
Figure 42: A close-up view of the registration page asking for birthday.

scan20004.jpg
Figure 43: A close-up view of the registration page asking for the user's zip code or city.

scan20005.jpg
Figure 44: A close-up view of the registration page asking to see if the user is interested in finding tutors, students, or both.

scan20006.jpg
Figure 45: A close-up view of the registration page that asks users to fill out their email address and password.

scan20007.jpg
Figure 46: A close-up view of the confirmation page that gets displayed after registering for a new account.

scan20008.jpg
Figure 47: A close-up view of the initial InstaMatch Questionnaire page.

scan20009.jpg
Figure 48: A close-up view of the question that asks how much of an auditory learner the user is.

scan20010.jpg
Figure 49: A close-up view of the question that asks how much of a visual learner the user is.

scan20011.jpg
Figure 50: A close-up view of the question that asks how much of a kinesthetic learner the user is.

scan20012.jpg
Figure 51: A close-up view of the registration page that asks for the types of material that help the user learn.

scan20013.jpg
Figure 52: A close-up view of the 50% progress bar that displays when the user is half way through the registration process.

scan20014.jpg
Figure 53: A close-up view of the question that asks the user to pick a tutor gender preference.

scan20015.jpg
Figure 54: A close-up view of the question that asks users to set a distance preference when finding tutors.

scan20016.jpg
Figure 55: A close-up view of the question that asks the user to state how much he or she is willing to pay for a tutor.

scan20017.jpg
Figure 56: A close-up view of the confirmation page that shows up when the InstaMatch questionnaire is completely filled out.

scan20018.jpg
Figure 57: A close-up view of the instant match with Jane Hill.

scan20019.jpg
Figure 58: A close-up view of what happens after accepting "Jane Hill" as the tutor.

Signed Informed Consent Forms

*Please ignore the first four pages of Consent Forms.pdf.


Raw Notes from User Testing Sessions

Session #1

Monday, November 4, 2013 6:07pm EST

Prototype #1: One, Two, Three, Match!

Browse Tutors
Clicks browse tutors
Good search filters.
Says browse tutors isn’t confusing

Registration
Doesn’t see anything wrong with it when asked
Comments: nothing to add to the page

Sign-in
Signs in with email and password at top of page

Instant Match
Clicks browse tutors, explained again clicks instant match
Doesn’t understand how instant match works, what it is based on
Different tutor: clicks next.
New tutor: clicks accept.
Asks about other buttons like tutor history, met a tutor and has a history of tutors used. For using the same tutor on sign in

Prototype #2: Search Engine and Questionnaire

Browse Tutors
Type in tutor name and search
Click dropdown, filter by different areas- understood
Comments: Likes the second prototype browse tutors better
More tutors per page.
Would like more filters sort by gender or degree
Upvote, downvote what is that?
Topics were obvious
Likes the star system from the first prototype better, easier comparison
More time to see positive and negative
Everything else good

Registration
Clicks register top right
Click male
Completes birthday, and next
Add zipcode, next
Clicks tutor (out of tutor, student, both)_> should have a next button for consistency (radio)
Enters rest of information, gets to congrats page.

Sign-in
Sign in button clicked (email password, signin/ sign up buttons)
Mentions forget password, forget email, or remember me buttons

Instant Match
Clicks instant match>start
Doesn’t understand radio buttons or scale at first
Understands process and starts answering radio questions
Highlight buttons explained, continues
Searches 5-10 miles
Asks if the range of the money is more than 1 selectable, requests checkboxes
Instant matched.

Overall

Likes the second prototype better, felt like more questions, customized. More detailed instant match page, requested 5 star rating system again
Asks if skip goes back to the homepage. Likes this design.

Session #2

Monday, November 4, 2013 6:39pm EST

Prototype #2: Search Engine and Questionnaire

Browse Tutors
Had no trouble finding how to browse tutors
Likes the pictures and colors on the browse tutors page, she likes the simplicity too

Registration
Clicked on register button to register
Predicted birthdate had dropdown menus
Figured out that zip code was text field
She would not read the successful registration page that asks you to confirm email address

Sign-in
Typed in username and password and pressed Sign In button

Instant Match
Clicked on instant match button to perform an instant match
She likes the bubble selections - like radio buttons
She figured out that the learning styles were multiple selection
Recommends that each page should have a page that shows how far you are in the process - like a progress bar
She recommends having a no gender preference for tutors
Was a little confused that the price rate recommendation was not multiple choice
Was able to figure out how to accept an instant match

Prototype #1: One, Two, Three, Match!

Browse Tutors
Clicked on Browse Tutors button to browse tutors
She likes Prototype #1’s Browse Tutors layout a lot better than Prototype #2
She likes 5-stars over up/down voting

Registration

Clicked Sign Up button to register
She likes registration page of Prototype #1 since all the fields are on one page and you don’t have to display a progress bar
She likes that the Prototype #1 had less words on the confirmation page after successfully registering

Sign-in
Entered username and password and clicked sign in button
She clicked on the TerpGenius logo to go to the home page

Instant Match
Pressed instant match button to do an instant match
She prefers Instant Match in Prototype #2 since it had more information
She likes the color coding on Prototype #2

Overall

Overall, she likes the simplicity of Prototype #1. She’s used to doing registration all on one page. In Prototype #1, she wants more information about the tutor on the instant match page like Prototype #2.

Session #3

Monday, November 4, 2013 7:01pm EST

Prototype #1: One, Two, Three, Match!

Browse Tutors
Would not take high school person because he is in college. Wasn’t sure what the stars were for.

Registration
Thought the process was easy.

Sign-in
Typed in email and password into text fields and then clicked on the "Sign in" button

Instant Match
Would read the person’s bio before accepting. Thought task easy and simple (to him it is a good thing).

Prototype #2: Search Engine and Questionnaire

Browse Tutors
To browse for tutors. He would type the course he is looking for and click “Search”. Browse page a little more cluttered than other prototype but still good. Good that specialties are available. Knew how to find if someone has good/bad reviews and prices of people without anyone telling him. Picked the person with better reviews but higher price because the grade is most important to him (not money).

Registration
Straight forward process

Sign-in
Clicked on Sign in button and filled out email address and password to login,

Instant Match
Wants to know why if he already found a tutor, why do instant match? No price on match page.

Overall

Liked the first prototype better. Less cluttered, so it seemed more smooth. First prototype better for browsing tutors (for first time users). Second prototype provides more options for browsing but may take a first timer longer to pick up.

Session #4

Monday, November 4, 2013 7:48pm EST

Prototype #2: Search Engine and Questionnaire

Browse Tutors
Type in UMD tutors in search bar and then press search button. (Would not press browse)
Type in Java tutors and then press search.
The browse tutors page shows up and he picks Sally Mae.
He would contact Sally Mae to see where she lives.

Registration

Clicks on Register button to register for a new account.
Enters date of birth by using the drop-down menu.
He checks his email after getting the confirmation page.
He says the registration page is like a typical registration page.

Sign-in

Clicks on the Sign in button and enters credentials.

Instant Match
Clicks on the InstaMatch button.
The bubbles look like radio buttons. He would prefer the bubbles to be vertically oriented instead of horizontal.
He did not know what “kinesthetic” means - should add a description of what kinesthetic means -- does it mean he’s a runner or a sports person?
He wasn’t sure if the learning styles were like checkboxes. He suggested making the font smaller or use a drop-down menu.
Asked what the skip button was for. Should put the skip button on the same line was the Back button - he didn’t know that the skip button was on the previous pages.
The selection with distances was confusing. Does selecting 10 miles include 5 miles?
One picture on the instant match is enough. It’s not bad to have more than one picture - but more than one is not necessary. You don’t want to see that many pictures of the tutor.

Overall
Sees the overall sign up process just like other websites.

Prototype #1: One, Two, Three, Match!

Browse Tutors
Clicked on the Browse Tutors button.
Asked what the browse tutors page was for -- does it show a list of all tutors in the database?
His first priority is picking a tutor based on price and then gender.
He prefers the 5-star rating system over the up/down rating system.

Registration
Signs up by clicking on the Sign up button.
After the confirmation page, he will check his email for the confirmation link.
To get back to the home page, he will click outside the Registration page to go back to the home page -- but nothing happens, so he refreshes the page. If refresh does not work, he will re-open the website inside a new window.
He would click on the logo to go back to the home page. He initially thought that the logo was in the background of the registration confirmation page and you had to click outside of it to make the popup go away -- but we explained the registration page was not a popup.

Sign-in
Clicked Sign in button to sign in with credentials.

Instant Match
Clicked on the big InstaMatch button to get to Instant Match. He says the instant match page was standard. He says the picture should be small and not large -- like a thumbnail. He would send a message to the tutor after selecting one from Instant Match

Overall

He likes Paper Prototype #2 better since it asks more questions, so it does a better match. The designs are fine, but the radio buttons should be vertical rather than horizontal. The thumbnails on the browse tutors page is great. He would like a small picture in the instant match and not a big picture nor multiple pictures.

Post-study Paper Survey Responses

Blank Survey

post-study-survey.png
Figure 59: We asked participants to fill out the above survey on one of our laptops after they finished testing both paper prototypes.

Results

Which paper prototype did you *like* better?


post-study-survey-chart01.png
Figure 60: A pie chart depicting which paper prototype(s) the participants liked.

Prototype #1: One, Two, Three, Match!
2
50%
Session 2, Session 3
Prototype #2: Search Engine and Questionnaire
2
50%
Session 1, Session 4
I liked both equally.
0
0%


What features did you *like* in the paper prototypes?


I liked how the second prototype had more questions than the first prototype. Answering questions can help me find a better match. I also like the 5-star rating system. Paper Prototype #2 does a better job matching since it asks more questions about myself. (Session 1)

The entire registration process was easy to complete. I like how you can read a tutor's biography before accepting him or her. All of the tasks were easy to complete. (Session 3)


Prototype #1 had a simple design. I like how registration could be completed on one page. (Session 2)

I like the 5-star rating system over the up/down voting system.
I like the Instant Match feature in Prototype #2 since it displayed more information about the tutor.
I like how you have thumbnails on the browse tutors page. (Session 4)


What features did you *dislike* in the paper prototypes?


The browse tutors page in Prototype #2 is a bit more cluttered than the first prototype but still good. It has more options but may take first time users some time to get used to them. (Session 3)

I don't like having multiple photos of a tutor when I do an instant match. A small picture is fine -- but even one large picture is too much. Also, radio buttons should be displayed in a vertical fashion rather than horizontal. I don't like the up/down rating system. The 5-star rating system is a better indication of who is a good tutor and who is a bad tutor. (Session 4)

I don't like having to go through over ten pages just to fill out a registration form or questionnaire. The registration confirmation page on Prototype #2 had too many words. I wouldn't read all of that stuff. (Session 2)

The up/down rating system is not as good as the 5-star rating system. The 5-star system makes an easier comparison, so that the user doesn't have to parse the number of positive and negative votes. (Session 1)

How would you improve the design of TerpGenius?


You should stick a price tag next to the instant matched tutor in Prototype #2. Also, why would I use instant match if I already found a tutor by browsing? Overall, I loved the design of TerpGenius. (Session 3)

When asking what range a person would be willing to pay for a tutor, you should use checkboxes instead of radio buttons since some people may not mind paying tutors at different rates. It would be nice to be able to sort the list of tutors by gender and degree. (Session 1)

I didn't know what kinesthetic means. Can you provide some sort of description? I thought from the picture that it meant someone who plays sports or runs a lot. The learning style boxes didn't seem like checkboxes. Try making the font smaller or use a drop-down menu instead. The Skip button during the questionnaire should be on the same level as the Back button and not below it. (Session 4)

I didn't know it even existed until after a few pages through the questionnaire. If you're going to have a questionnaire get displayed on several pages, you should incorporate a progress bar so the user knows how much they have completed and how much is left to complete. (Session 2)

How likely would you refer TerpGenius to a friend?


post-study-survey-chart02.png
Figure 61: A column chart depicting whether or not the participants would refer their friends to TerpGenius. A score of "5" represents that the user would be very likely to refer their friends to TerpGenius. A score of "1," on the other hand, denotes that the user would very unlikely refer their friends to TerpGenius.
1 = Very unlikely
0
0%

2
0
0%

3
0
0%

4
1
25%
Session 2
5 = Very likely
3
75%
Session 1, Session 3, Session 4