Practice Module
Spring 2023
My Role
Senior CX Product Designer — User Research, User Flows, Design Concepts, Visual Design, Hi-fi prototyping
Team
Lead Product Designer - Fabio Secci
Lead UX Motion Designer - Giga Khurtislavi
Senior UX Copywriter - Dhanesh Misir
Timeline & Status
1 month — Tested as A/B experiment around 4 weeks
Overview
As AI technology appeared on the market, we saw a huge potential to start generating quizzes with excellent quality based on our vast knowledge base.
I was responsible for creating end-to-end design direction & vision of the new practice experience, whenever students browse through our Question Page.
We ran the experiment for four weeks on the US market and saw good traction from our users with good engagement metrics and a high level of completing quizzes.
With the release of GPT-4 technology from OpenAI, we saw massive potential in using their API and our vast knowledge base to create new high-quality content that could make a unique value proposition for our new & existing users. After playing with GPT-4, we discovered we could generate high-quality quizzes based on our Question page content.
This finding gave us the idea that we could allow users to deepen their knowledge of questions they visit and practice a simple quiz consisting of 3 related questions with a multi-choice answer generated with AI's help. Based on this, I prepared a user flow to start broader technical discussions with engineers and stakeholders to kick-off discussions.
Quiz Flow
Quiz Generation
To make quizzes, we needed to generate a bigger sample based on hand-picked question pages to check the quality of AI-generated quizzes and ensure that the error level was negligible. Also, we were experimenting with prompts where we gave GPT-4 more context for better-generated content, like: “Imagine I’m a teacher, preparing for an exam,” or providing real answers to the questions.
Based on such exercises, we managed to create a great generic prompt, that allowed us to have high-quality generated quizzes with quality scores over 90%. Based on this prompt, our development team was unblocked and was able to start massive quiz generation on the most popular question pages.
Based on my experience, I started preparing the first drafts of the Practice Module on our question page. Our team also tested a fullscreen practice experience some time ago, and testing results were promising. This allowed me to reuse some experimental designs, which needed to be adjusted to fit the question page layout.
First Desktop Practice Module Drafts
Mobile Experience
The mobile version from the beginning felt more straightforward and contextual because of the more simplified UI on the mobile web version. Based on this mobile draft, we had initial approval to proceed with this approach. We only needed to spend a little more time polishing the copy and finding a better name for a feature.
First Mobile Practice Module Drafts
After getting approval for the drafts, I needed more time to work on the desktop version. I decided to remove the quiz element from the sidebar on the question page and make the quiz element look similar to the Intercom chatbot on the bottom right corner of the page.
Desktop Practice Module Release Prototype
I also worked more closely with our Senior UX Copywriter - Dhanesh, who helped us to shape feature communication and proposed naming “Level Up”. We also tried to remove all possible negative words, as we know they could increase the anxiety level of students.
We also create a state for each possible result of the Summary screen to encourage students to practice more after they finish the quiz.
Summary Screen States
Extension Web Store Marketing Materials
Results
We ran an experiment for around four weeks for all users on the US market. We saw massive user interest with a completion rate of over 70%. Interestingly, overall, the mobile web version had a better CVR to Start Session and completion rate, while the Desktop version had smaller CVR metrics but still higher than we expected. Currently, the project is on pause, but we plan to extend the number of questions and make quizzes more integrated into the Question Page experience.
Brainly Extension
June 2023
