Defining the goal of the study
Why and Where is Research needed?
WHY?
Number of users having access to the application is greater than Number of projects created.
WHERE?
The two major areas critical for creating a new project is the
New project user flow
Training the bot to create a successful project, which happens in the settings panel on the right side of the interface
I highlighted the need of research and presented a research proposal to the leadership team, which was accepted.
WHY?
I highlighted the need of research and presented a research proposal to the leadership team, which was accepted.
Learning : Vague goals like "finding user pain points" aren't enough. Defining what to test—whether it’s a specific user flow, a section of the interface, or UX writing—makes research more targeted and actionable.
WHY?
Methodology
Research methods
Step 1: Leveraging existing data
COLLABORATED WITH DEVELOPERS
I collaborated with developers to access backend data and dashboards highlighting user activity levels.
This helped identify users to interview across three groups: frequent users, occasional users, and those with access who haven’t used the application yet.
DATA FROM PAST RESEARCH + CUSTOMER QUERIES
I reviewed past research data, gathered frequent queries from customer care, and monitored Slack channels for insights
Gathering this data guided strategic participant recruitment for usability testing and interviews, informed tailored research materials, revealed patterns, and refined research goals
WHY?
My efforts in reached out to cross-functional departments was much appreciated by the manager.
Learning : Collaborating with other teams and exploring existing data can enrich research and save time. Research isn't always formal - observing users or noting offhand comments can uncover valuable insights.
Step 2: User Interviews - 9
INTERVIEW GOALS
Understanding overall user sentiment
Identifying persona
Existing onboarding process pain points
Use cases
Gains/Wins
RECRUITMENT AND RESEARCH MATERIAL
Participants were recruited based on their frequency of usage. The findings gathered through user interviews helped in preparing for the usability testing material.
Learning : Recruit participants with a purpose - may be based on personas or product familiarity.
PROCESS
We conducted 9 user interviews via Zoom, guided by clearly defined interview goals. Key observations and insights were documented in an Excel sheet during the sessions
Using these notes, we synthesized an empathy map to capture user thoughts, feelings, and behaviors, providing a foundation for deeper analysis
PARTICIPANT DATA
Users who have created 2+ projects - 3
Users who have created 1 or 2 projects - 3
Users who have 0 projects - 1
I was a note taker for 4 interviews and interviewer for other 5. I recruited participants and scheduled timings based on their availability.
Learning : Resources aren’t just about budgets or tools—they include the participants willing to share their time and insights. Maximizing the value of these interactions is crucial for meaningful outcomes.
Interview Findings
User sentiment
Users feel empowered and are excited to use MyAI Builder in their daily life.
“ I do see potential for My AI builder and the need"
Personas
Personas identified:
1. I am getting started
2. AI Builder
Advanced users
70% of user base is of AI Builder and I am getting started persona. (Non - technical)
Onboarding process
9/9 participants appreciated the help provided by Paul - a team member. This highlights the need of creating a robust help support in the application to improve efficiency.
Top use cases
1. Creating course assets
2. Literature review
Report creation
Cost transparency
Users want to know how the 'paid' model will look like in the future as currently they are skeptical of investing their time in a tool that they might not be able to afford later on.
Transfer from ChatGPT
9/9 users were part of a challenge where they got ChatGPT license and now are moving towards MyAI Builder as their license is ending.
Step 3: Usability testing - 8
TESTING GOALS
Pain points related to User Interface
User flows
What features do they use while training the bot
Identifying task specific pain points
ZOOM AND ONLINE USABILITY TESTING TOOLS
Two kinds of usability testing were conducted.
3 usability testing through Zoom
5 user interviews using online tools such as Userlytics and Lookback as I was also testing which online tool is better for future usability testing.
PROCESS
Predetermined tasks were formed catering to each testing goal
Usability testing was conducted, each session was recorded and notes were taken in an excel file.
Notes from the excel file was imported to a FigJam file, which was created to visualize and find patterns. I segregated the pain points/wins and observations based on the tasks
Task completion time and rate were noted
Frequency of each finding was also noted to prioritize recommendations and understand the impact (positive, negative, neutral) each finding is creating
PARTICIPANT DATA
The persona identified through Interviews and previous research were used to recruit participants.
2 - AI Builder persona
1 - Advanced user persona
5 - I am getting started persona
Usability Testing Findings
Task success and Completion rates
Task Success: 8/8 participants successfully completed the "Create a New Project" task.
Beginner Struggles: The "I am Getting Started" persona struggled with project configuration tasks.
Efficiency Issues: Advanced personas completed configuration tasks, but with longer-than-expected completion times.
Confused Terminologies
Technical terms: Participants found terms like "In GPT Interactions" and the distinction between "Project Description" and "System Prompt" unclear.
Ineffective Tooltips: Tooltips were unhelpful, particularly for 'AI Builder' and 'I am Getting Started' personas.
Cluttered space
Overwhelming Panels: Participants found the panels cluttered and visually overwhelming.
Missed Actions: 8/8 participants overlooked the 'Update Settings' button due to poor visibility.
Unnoticed Features: 40% of features were missed, with unnecessary UX copy contributing to visual noise.
Information architecture
Logical Grouping: 8/8 participants expected features like sharing settings to be grouped with related interface options.
Workflow Misalignment: Features were not organized around participants’ workflows, creating friction during onboarding and task completion.
Navigation Challenges: Participants struggled with unclear labels and an unintuitive dashboard structure.
Trouble shooting style
Conversational Troubleshooting: All 9/9 participants opted for a conversational approach to troubleshoot issues instead of using system settings like verbosity controls.
Natural Language Interaction: 'I am getting started' persona who were using this application for the first time asked the application itself to help them find a feature
Lack of system feedback
Feedback Deficiency: Participants expect progress bars during file uploads to estimate the time required.
Guided feedback: Participants noted a lack of help by the system, guiding them on their next steps. (For eg. the system didn't have a progress bar when uploading a file)
Recommendations
OVERALL RESEARCH STATEMENT
Users are excited about MyAI Builder's potential and eager to integrate it into their workflows, but they often feel overwhelmed by its extensive feature set, using only 40% of its capabilities. Simplifying the user experience and prioritizing the visibility of key features could drive higher adoption and better engagement.
COMMUNICATING FINDINGS
I created presentations of research findings related to business goals to the leadership team and detailed UX findings to the design and developer teams.
Post the presentation I created a final recommendation report, which was sent to the design team.
As a designer on the team, I started a Design + Developer meetings where we discussed design team's plans addressing the research findings to understand each team's priorities and feasibility.
Learning : If findings aren't used, revisit communication - present clear insights aligned with business goals, mindful of each department's efforts.
Core design improvements suggested
HIGH LEVEL GOALS
Simplify Navigation and Interface Design
ACTIONS BEING TAKEN
The design team is working on updating the information architecture, cleaning the interface, catering to recently identified personas.
I conducted a card sorting activity with the design team, helping align team and stakeholders on category structures.
Improve Terminologies and Use Natural language/Conversational help
I developed a tone-of-voice guide and implemented UX writing across the product
Improve Onboarding and Guidance
Slack channel is created and additional weekly meetings are scheduled where users can come and ask their queries or discuss difficult in creating projects.
I highlighted the need to create a community of AI enthusiasts that would encourage word of mouth marketing and inspire others to experiment.
Enhance Feedback Mechanisms
The dev team is working to show a progress bar to users while they are uploading their files. Design team is working on error messages.
Challenges
Technical terms needed to be understood before conducting research.
Collecting information across departments as there are individual contributors who spoke to customers directly.
Recruiting participants from the 'Advanced users' persona was difficult as there weren't many.
Budget constraints in using online testing tools so I was experimenting using the free tools available.
Conducting research in an agile environment where features change frequently and keeping research relevant and up-to-date with evolving products
Appreciation from Leadership
Recognized for driving impactful research insights that shaped design decisions. One screenshot captures a Zoom meeting where the design team presented ideas influenced by my research, and the other is a Slack message from my manager appreciating my presentation.
LET'S CONNECT
I am everything design! I am resilient to the core.
Get in touch to find out more about how I create digital experiences to effectively reach and engage audiences.
Email: shailees0406@gmail.com
Contact: +1 (602) 576-4588
Design by
Shailee