Improving Pratt Institute's Online Application Process for Prospective Undergraduate Students.
Client: Pratt Institute
Overview
Pratt Institute's website is the first and the most important touchpoint for thousands of prospective students when considering applying for courses.
The client's primary aim was to understand how prospective students perceive the institute, learn about the programs, and move through the application process on the website.
Team Members: 3 UX Consultants
My Role
Tools: Figma, Tobi Pro Lab
-
Participant recruitment as part of a research ops team.
-
Study, test planning, moderating and facilitating eye tracking studies.
-
Analysing gaze plots and metrics from the study to provide actionable design recommendations.
-
Design solution prototyping.
Duration: 7 weeks
In August 2022, Pratt redesigned their website and wished to conduct research to analyse the impact and effectiveness of the redesign on their target audience: Prospective Students.
Exploring the website's strengths and limitations: Research Roadmap
Evaluating the needs & challenges of key users: Prospective Undergraduate Students
Participant Recruitment:
A research ops team was formed to combine recruitment efforts and a screening questionnaire was used to recruit students in the 20-23 year age group.
08 Participants
Age Group (20-23 years)
06 Undergraduate Students
4 Seniors, 1 Sophmore, 1 Junior
4 Seniors, 1 Sophmore, 1 Junior
02 Graduate Students
First Year Students
The Key Challenges
To ensure the study provided the most accurate results our aim was to recruit high school students as they were exactly the right target audience our study aimed to focus on - prospective undergraduate students. However, our research ops team ran into the following challenges:
Difficulty recruiting high school students:
While high school students were our key target participants, recruiting them wasn’t possible as it required legal as well as parental consent.
Expanding The Target Participant Group:
Additionally, recruiting undergraduates in the 18-23 age group proved difficult and time limitations forced us to expand to recruiting graduate students.
Measuring the desktop sites effectiveness and efficiency through Eye Tracking
Adopting a scenario based approach coupled with retrospective think aloud.
Considering we had to expand out participant group, our strategy involved providing them with a scenario to facilitate better task understanding.
​
​
Scenario: You are interested in pursuing a Bachelors Degree at the Pratt Institute. You want to know more about the college and the program.
01
TASKS
Find out more about the duration of the program and a few courses of interest.
02
Find out more about the application requirements and deadlines for the program.
03
An important part of college is finances. Find information on the total cost of the program and possible scholarship opportunities.
04
Now that you have found this information, start the process of applying to the program.
Understanding users mental models, thoughts navigating the website through Retrospective Think Aloud
The Retrospective Think Aloud method helped us gather valuable qualitative data. The process involved showing users a video replay of their gaze and prompting them to explain their thoughts and reasoning during the test.
Identifying limitations in the websites content and navigational patterns through detailed analysis of qualitative & quantitative data.
Eye Tracking Data Analysis
Analysed Tobii Pro data and metrics such as gaze plots, heat maps, areas and time of interest which was further strengthened with the System Usability Scale.
User Task Flow Analysis
Analysed trends in user flow and navigation patterns across participants and tasks to identify user pain points and challenges.​
​
Quantitative Data Analysis: System Usability Scale
Usability limitations have severely impacted the websites performance and user experience.
Pratt Institute's desktop site received a score of 61.3 on the System Usability Scale which was below the average benchmark score of 68.
​
The usability of Pratt's desktop site was worse than 79% of other websites raising some serious concerns over its impact on user experience.
61.3
Pratt Desktop Site's SUS Score
SUS is a global measure of system satisfaction that is reliable even with small sample sizes was thus used to analyse users' experience interacting with the website.
Key Findings: A Summary
Identifying scope for design intervention
We provided the clients with a problems list that acts as an overall summary of our findings and the their severity thus guiding them towards the problems that require the most amount of intervention.
Severity ratings here are categorised according to ​Nielsen Norman's Severity Rating for Usability Problems
Analysis: Key Insights
Pratt Institute’s website met the target task completion rate of 75% for only one of the four tasks. Which points to low website effectiveness.
However, the participants that did complete the task could do so efficiently. (Refer table 2)
.png)
Table 1: Effectiveness: Measured by task completion rate
.png)
Table 2: Efficiency: Measure by average task completion time
Analysis & Synthesis
KEY FINDINGS: HIGHLIGHTS REEL
The highlights reel reflects the challenges faced by the participants and is effective in conveying the limitations of the existing website.
Analysis & Synthesis
Identifying key areas of design intervention to maximise impact & optimise the websites user experience.
Based on the insights covered in the highlights reel and the severity and frequency of the problems identified, we decided to narrow down our recommendations to the following 4 key focus areas:
01.
Improving discoverability of important information regarding programs and applications.
02.
Provisions for users to keep track of their navigational routes (backtracking).
03.
Restructuring content hierarchy on the website to match users mental models and goals.
04.
Minimise users cognitive load by limiting similarities in the navigation categories.
Key Finding 1A
Key information regarding programs and applications is not easily discoverable.
“
The huge block of text gets me immediately disinterested
- Participant 6
”
.png)
Usability Concern:
Key information about program duration is hidden between a huge block of text.
All 8 of our participants failed to find details on program duration.
Inconsistencies in content across program pages affects discoverability of fundamental information.
Usability Concern:
While the overall structure of program pages is consistent, the inconsistency in content adds to the users cognitive load and reduces the efficiency of the website.
.png)
Recommendation 1A
Highlight Vital Content to Increase Discoverability
Improving visibility and consistency of key information significantly reduces users' cognitive load.
Design Solution:
Clearly stated program duration in a separate section on the landing screen to facilitate discoverability and consistency across pages.
Key Finding 1B
Lack of appropriate visual signifiers greatly affects discoverability
“
I had trouble with the task of finding application requirements and deadlines.
- Participant 2
”
4/8 of our participants struggled to find program application requirements.
Usability Concern:
The side navigation does not visually signify an accordion. Thus it's not intuitive for user's to click on it to find more information.
Recommendation 1B
Adding Visual Signifiers to Facilitate Seamless Access to Content
Design Solution:
Adding drop down arrows to signify an accordion will increase the discoverability of vital information.
Facilitating access to key content through the accordion will help reduce the users’ motor load.
Key Finding 2
Backtracking on the website proved to be difficult for participants.
“
The part that I was frustrated was that there is no back button
- Participant 7
”
Usability Concern:
User frequently fixated their eyes on the top left of the screen in search of back button.
4/8 of our participants found it difficult to go back to the pages from where they started browsing the website.
Recommendation 2
Use navigational aid to keep track of users’ location on the site.
Design Solution:
Using breadcrumbs as secondary navigation aid helps users' easily understand their location and navigation path on the site.
Key Finding 3
Content hierarchy on the website does not match user goals
“
I thought this [section] would be useful for the next task
- Participant 2
”
7/8 of our participants utilized the Programs of Study section on the Homepage to learn about Pratt’s Majors
Usability Concern:
Key program information is located at the very end of the homepage limiting accessibility and increasing motor load.
Recommendation 3
Restructure content hierarchy to prioritise key information
The priority of content should determine the order and hierarchy of content on the homepage.
Restructuring the hierarchy to include content thats most relevant will help reduce the motor load on the users.
Design Solution:
Programs of study is key information for prospective students and is the most accessed content on the page. It should thus be highlighted earlier in the page.
Key Finding 4
Similarity in navigation categories increases the likelihood of confusion
“
“Too many options given on the navigation bar. There was a lot of repetition”
- Participant 4
”
5 out of 8 of our participants struggled to interpret the category names from main navigation.
Participants find it challenging to differentiate between categories such as Courses and Academics and find them very similar. This causes them to take additional time and visit different pages while looking for important information.
.png)
Usability Concern:
Increased cognitive load on the users while looking for the right category to find specific information.
Recommendation 4
Further testing & analysis to improve category taxonomy
Redefine the content in the top navigation to help users efficiently navigate the website, thus improving their overall experience.
Design Solution: Card Sorting
Finding an organization structure that best matches users’ mental model through card sorting.
Design Solution: Tree Testing
Understanding navigation pathways to help shed light on the hierarchy and placement of information.
Design Delivery & Client Feedback
"This is just a universal moment of thanks for all the work. Because this is a body of work that I think could end up genuinely informing work that we do in the future."
The client was extremely happy and impressed by our recommendations. They believed that our findings echoed a lot of internal conversations and questions and aligned with a lot of technical hurdles they were noticing.
​
The client specifically mentioned that the findings surrounding "program duration" hadn't really come up in conversations and was a really interesting finding considering that for a lot of prospective students its a way for them to measure their commitment. They found it interesting because its not something they had necessarily thought about.