Duolingo
Helping language learners view their progress in a clearer and more meaningful way.
Project Overview
Project Brief
How can we help Duolingo users see their language learning progress?
My Role
UX researcher, UX designer
Duration
80 hours
Background Info
Duolingo is one of the most popular language learning apps currently on the market. I’ve used it for years and, while I generally enjoy it, I was curious to explore potential improvements.
Disproving Early Assumptions
As an avid Duolingo user, I had felt how demotivating it could be to lose a long streak. I wanted to know if other learners experienced this problem too, but my early user interviews uncovered a more important problem.
“I’m not really any closer to being fluent than I was six months ago”
Losing a long streak wasn’t nearly as motivation-crushing as not knowing if you were really improving. Duolingo currently relies heavily on gamification to keep learners engaged.
Streak: how many days in a row you’ve used the app
Experience: earn XP by completing lessons
Leaderboard: compete against other learners
Quests: new ways to earn gems each day
But these metrics only show you how much you use Duolingo. They don’t tell you anything about your language proficiency. For a language learning app, that’s a pretty serious and urgent problem to address!
Learners feel like they’re getting better at using Duolingo, not at speaking a language.
Problem: losing a long streak hurts motivation
Problem: learners don’t know if their language skills are improving
Learners enjoy using Duolingo - it’s fun! They get lots of cool stats, but none of these really show how much they’re learning.
Solution: provide a way for learners to measure and view their language learning progress
Learners need a clearer and more meaningful way to see their progress. They want to feel like their language skills are actually improving! By testing proficiency at regular intervals and then comparing these results, learners can compare their results to see how much they’re learning.
Understanding the Problem
Research Methods
Competitor analysis: I analyzed three direct competitors and two indirect competitors to figure out how other products go about measuring and showing learning progress
Subject matter expert interview: I interviewed a learning expert to learn more about why measuring language learning progress is so challenging
User research interviews: I conducted four interviews with current Duolingo users to understand what meaningful progress indicators look like to them
Does anyone measure learning progress effectively?
The short answer is, not really. The competitors I researched faced a similar stumbling block to Duolingo: they measure how much people are using the product rather than how much they’re learning.
Why is measuring learning progress so difficult?
According to the learning expert I interviewed, there are good reasons why so few apps actually show learning progress.
Learning is really complicated: it’s different in the short-term and long-term, and how much someone feels like they’re learning can be vastly different to how much they’re actually learning
Learning stats can be misleading: stats like words learned are impossible to measure 100% accurately because words can be forgotten or learners may struggle with difficult conjugations
Misjudging progress can hurt learners: bother under- or overestimating progress can leave learners feeling less motivated and less trusting of any feedback they receive
“There are a lot of ways people can tell they're learning, but it's hard to measure them and turn them into a score.” - Learning expert
Key Insights: What Does Meaningful Progress Look Like?
I interviewed four Duolingo users to better understand their frustrations and learn more about what indicators of progress are meaningful to them.
Can I use the language in real life?
Their greatest frustration came from not feeling like they were improving in real-world scenarios.
Learners feel like they’re improving when they:
Have greater confidence when speaking
Are able to understand more in conversations, books, TV shows, or other media
How do I know what I need to work on?
Getting individual feedback more closely mirrors a classroom learning setting. It can help learners know where they’re improving and what they still need to work on.
Am I getting closer to my goals?
Duolingo’s language trees are long, even more so after the Nov 2022 redesign. Learners can feel discouraged, like the course is neverending.
Reaching milestones or specific goals helps learners feel a sense of achievement. It shows that they have actually progressed.
Why is this so important to address?
“I lose my sense of motivation”
When learners are less motivated, they:
Feel less confident
Are less likely to keep learning a language
Lose trust in Duolingo because they don’t feel like the app is following through on its promises to them
50% of my research participants had stopped using Duolingo because they didn’t know if they were making progress.
Developing Personas
Based on my user research interviews, I identified 3 distinct personas, each with different behaviors, goals, and frustrations.
Self-motivated Selena
Cares most about language proficiency
Wants to gain fluency
Most likely to subscribe to Duolingo
Goal-oriented Gabrielle
Cares most about language proficiency
Wants to get good enough to use the language in specific contexts
Dabbler Danny
Cares most about gamification and app usage stats
Least likely to subscribe to Duolingo
Self-motivated Selena and Goal-oriented Gabrielle are the ones most heavily impacted by not being able to tell how much they’re learning. I worked with these as my primary personas.
Defining the Feature
How can we measure and display progress?
As there aren’t yet established patterns for how to do this in ed-tech, at least not in the language learning sphere, I had to cast a wide net when thinking of potential solutions.
Improvement = positive change over time
I decided to focus on progress quizzes as a way for learners to see their proficiency and progress.
By measuring how much you know at the start of a section and the end of it, you can see how much you’ve learned during that time.
Wireflow illustrates how the feature works
Designing Progress Indicators
Design process
Even though Duolingo has an extensive UI library, I still started by sketching ideas by hand. This helped me better explore different options, rapidly iterate, and better understand elements of the app’s existing design.
How much have I actually learned?
I designed a standard progress quiz to test how much a learner has improved from the beginning of a section to the end.
Uses questions that already exist from a group of units
Exists within the main language tree so learners must complete this before progressing
Is the same length as a regular lesson, to minimize disruption
In my prototypes, I spaced the progress quizzes five units apart. In an ideal world, I would collect data about how long it generally takes learners to complete a unit and use this to optimize the frequency. I would also want to test different placements to see how learners respond!
What if I want to see my progress in between quizzes?
Quizzes are intentionally spaced far apart, but being able to view progress is too important for it to only sometimes be accessible.
Enter: the Progress Hub.
Always accessible as it’s nestled under the existing ‘review’ tab
Contains full progress quiz history
If a learner accidentally skips the progress quiz results screen, is returning to the app after a break, or just doesn’t remember their last score, they can find it here.
Results screen is easy to accidentally skip
Progress hub allows learners to view their progress at any time.
Simulating Realistic Scenarios with Open-Prompt Quizzes
“I wouldn’t feel prepared to actually speak to someone in Spanish”
Current speaking exercises test pronunciation rather than active recall. An open-ended prompt would be much more similar to how we actually speak! It would have to be:
Level appropriate: have a similar topic and complexity level to the learner’s current place in the course
Optional: these questions are more disruptive and so should live outside of the main learning tree. I don’t want someone to lose the streak they’ve worked so hard for because they don’t have the brain space that day for a more involved lesson!
Creating reflection prompts
These reflection prompts show progress in two important ways:
When something feels easier than it used to be, it’s a sign of improvement
Greater confidence provides a sense of progress, which helps learners feel more motivated to keep learning!
“I appreciate this because I’m someone who can often forget how far I’ve come. It’s a good reminder that you are making progress.”
How much detail is helpful?
Duolingo typically just provides a few brief stats at the end of a lesson. Showing all the information here would be overwhelming and inconsistent with the rest of the app. However, more detailed feedback helps learners see where they have room for improvement.
End of quiz results screen:
XP earned
Scores of first and second quiz
Visual indicator of score improvement
Full comparison screen:
Full answers for both quizzes
Visual indicators of score improvement
Comparison of reflection prompt answers
“I don’t think I’ve ever gotten to hear myself speak. That would be really useful to me.”
Priyanka, usability tester
Grading Open-Prompt Questions
How do you make a score for a subjective answer?
There’s no single ‘right’ answer for an open-prompt question, so I had to figure out how to turn a totally unpredictable answer into a tangible score. I explored different options and evaluated them based on value to the learner and ease of implementation.
Self evaluation: learners evaluate and compare their own answers
Peer review: two users learning each other’s language are paired and they grade the other person’s response
Tutor feedback: personalized feedback from a 1:1 tutor
Basic score: number grade based on factors like word length, and sentence complexity
AI-generated score: basic score plus AI finds spelling and grammar mistakes
Balancing usefulness and feasibility
A fully AI-generated score would be, by far, the most valuable to learners. But it would be so resource and time intensive to create that I doubted whether it was really possible - imagine developing Grammarly for every language offered on Duolingo!
The basic score is much more feasible and still gives the learner a concrete score. This grading system could be implemented in a more simple way at first and updated over time to include more complex factors.
Matching Duolingo’s Existing Branding
Of course visual design is important for a cohesive experience. But so is embodying company values and understanding how learners expect the app to work.
Introducing lessons
Most lessons start with a speech bubble from Duo. I thought this would be a great way to set expectations about the difficulty of the upcoming quiz.
Learning should be fun
Instead of having a 1-5 scale for the reflection prompts, I injected some fun by using expressing illustrations of Duo.
Digestible info
Learners expect to get a number that shows their score at the end of the lesson. It should be quick and easy to read.
Testing & Iteration
Research methods
I conducted moderated usability testing via Zoom with six usability participants. Four of them aligned very strongly with the primary personas.
What I wanted to learn
How easily can learners understand the quizzes and their results?
What information is most valuable to learners?
How easily can learners find the progress hub and detailed score information?
How might this feature impact learners?
“If the app showed me how to find it once, I’d be fine”
While all the testers found the progress hub quickly and easily, they weren’t confident that they were looking in the right place. I added feature onboarding to guide learners through the different parts of the progress feature.
Helping learners interpret their results
My participants found the idea of levels as a grading metric confusing - it reminded them of the lesson levels. Some of them even thought a higher level might mean that the quiz itself was more difficult!
I changed the levels to a numerical score out of 50, so it wouldn’t look like a percentage. I combined this with providing more detail about how the score is calculated, both in the form of an information screen and by providing a detailed breakdown of the different factors that impact the score.
This had the added benefit of being able to use the same progress bar indicator as the standard progress quizzes. This increased consistency within the feature and made the change in score more visually obvious.
Thoughts & Reflections
Balancing usefulness and feasibility
Giving learners a better way to see how much they’re learning is simultaneously hugely important and extremely challenging. I had to get creative about how to design a feature that would provide as much value to learners as possible while still being feasible.
Consistency within an existing product
Consistency goes a lot further than just following UI guidelines. This project gave me a greater appreciation for how many factors impact the cohesiveness of the user experience: visuals, microcopy, layout norms, and amount of information displayed at once, to name a few.
Limitations of working alone
If I’d been working with a team on this project, there are a few aspects of the feature I would love to explore in more detail! I’d love to have been able to work with:
Curriculum designers: to explore how the feature might change at different stages of the course
Business stakeholders: to figure out how this feature would work within Duolingo’s freemium model
Engineers: to understand more about the AI and machine learning aspect of grading the open-prompt answers