Context

Duolingo, the world's most downloaded learning platform, offers over 100 courses teaching 40+ languages. This project was part of my graduate studies coursework in 'Usability,' where I collaborated with a team of four to assess user experience and usability of the iOS app.

Duration: 3 months (Sept to Nov 2022)

Please note: This study refers to the version of Duolingo available from Sept to Nov 2022. The app has since undergone significant changes. Also this test is for educational purposes only and is not affiliated with Duolingo.

My Contributions

  • Conducted a Heuristic Evaluation of the application's main features.

  • Conducted a Competitor Analysis to identify market trends.

  • Conducted Usability Studies with participants to uncover usability issues.

Heuristic Evaluation

Key Heuristic Violations:

  1. User Control & Freedom:

    • Lack of preparation material before quizzes can induce panic and demotivation.

    • Absence of an option to pause and resume lessons leads to frustration and rushed learning experiences.

  2. Aesthetic & Minimalist Design:

    • Excessive animations and notifications may overwhelm users, particularly experienced ones.

    • Unorganized setting pages hinder navigation and task completion, requiring more effort from users.

  3. Flexibility & Efficiency of Use:

    • Inefficient customization options for language learning components can lead to boredom and frustration.

    • Skipping learning components without revisiting them may result in imbalanced language proficiency.

  4. Consistency & Standards:

    • Inconsistent placement of icons across languages impairs user recognition and navigation.

    • Varied user controls and categorization methods across pages create usability barriers and hinder content retrieval.

Competitor Analysis

Key Trends across Competitors:

1. Customized Lessons: Users benefit from a simple pre-lesson test that assesses proficiency, ensuring tailored and efficient learning experiences.

2. Enhanced User Experience: Hands-free operations for stories, podcasts, or audio clips cater to users with busy schedules, enhancing accessibility.

3. Confidence-building Features: Lesson tips are available to support users in completing tasks with confidence, fostering a positive learning environment.

4. Motivational Tools: Progress trackers, such as daily streaks and leaderboards, actively encourage regular engagement and participation.

5. User-Controlled Learning: Customization options empower users to personalize their learning journey according to their preferences and needs.

6. Multitasking Flexibility: The ability to learn multiple languages simultaneously accommodates users with diverse interests and goals, enhancing flexibility.

7. Engaging Learning Environment: Gamification elements contribute to an enjoyable learning experience, fostering engagement and motivation.

8. Revision Support: Review and summary options are provided to reinforce learning and aid in retention, ensuring lasting knowledge acquisition.

Usability Test

Objectives:

  1. Evaluate the user experience for language learning to ensure alignment with user needs and expectations.

  2. Assess the ease of accessing and using the application's features.

  3. Utilize research findings to identify design opportunities and formulate actionable recommendations.

  4. The objective of this test is to assess the efficiency and overall usability of the app, identifying issues in:

    1. Information architecture

    2. Visual design

    3. User experience

Moderator Script and Test Packet:

Once the screener document was prepared, we proceeded to design the test environment and structure. It outlines the following sections:

  1. The Moderator Script

    1. Script

    2. Pre-Test Questions

    3. Post-Test Questions: System Usability Scale (SUS) questionnaire + Qualitative questions

  2. Collateral

    1. Consent Form

    2. Task Sheets

    3. Note Taking Sheets

  3. Evaluation Scales (for SUS Questionnaire & Error Rate Calculation)

Test Design

  1. We scheduled a 30-minute session with each participant for the test.

  2. Each test session had one moderator and one notetaker.

  3. Tasks were screen recorded for later review and comparison.

  4. Participants were asked to think out loud (task time was not recorded).

  5. All tests were conducted in-person.

Key Themes and Recommendations

  1. Information Architecture

    1. The app's current lack of categorization makes it challenging for users to locate desired options. For example the Settings page

    Recommendation: Improve information categorization to enhance user searchability and optimize app real estate. Introduce a search option in pages with diverse information.

  2. Consistency

    1. Inconsistent icon placement across different language courses leads to user confusion about available options.

    2. Some icons and their functions are not intuitive to users, and 'Help' pages do not align with the updated app version.

    Recommendation: Utilize more universal icons or label the icons clearly to enhance user understanding. Maintain consistent icon placement across languages to build user trust.

  3. Effectiveness of Lessons

    1. While the lesson experience is positive, users find the learnings may not be practical in real-world settings.

    2. Some users note an inconsistency in lesson difficulty.

    Recommendation: Enhance lessons with native touches such as audio or scenario-based structures to improve user engagement. Consider incorporating these options to remain competitive in the language learning space and align with user expectations.

Reflections

I thoroughly enjoyed working on this project since I had the opportunity to lead my team and work on the end-to-end user testing lifecycle. It was interesting to see how different users perceived and interacted with Duolingo in different wats.

One of the biggest challenges I faced during the project was recruiting the desired mix of participants for the study within the timeframe we had set. Another personal challenge for me was to ensure I remained bias-free throughout the study and refrained from intervening while participants performed the usability tasks.

Another personal takeaway is that not every feedback needs a solution. For example, some users complained that a less-used feature is buried inside pages and suggested mentioning it in the main menu. However, considering the app’s limited real estate, it may not make sense to clutter the app with non-essential features in the menu bar.

End of page