Synopsis
Background
Goal
Weeks
What did I do?
Challenges
Impact
Methodology
Total responses
213
Total questions
23
Tools used
Qualtrics
Excel
Miro board
Process
Step 1: Set goals and survey familiarization
I collaborated with the product manager to define the purpose of this analysis, focusing on two key objectives:
Understanding user challenges during the task
Gaining insights into the AI audience to inform the design of ASU's new AI products.
I exported the Qualtrics survey to excel and went through the questions and responses to familiarize myself .
08
Likert scale questions
04
Yes/No questions
11
Open ended questions
Step 2 : Coding
Tagged responses without bias or focusing on the goal to ensure an unbiased analysis
Used a Miro board to organize tags and recorded significant responses under each tag for reference
Included quotes to provide context and aid in presenting the exact meaning during later stages.
Step 3 : Classifying
Identified related tags and assigned parent codes to form broader categories.
Ranked categories based on their relevance to the survey analysis goals, considering tag frequency and significance
Streamlined categories by omitting irrelevant or edge-case tags, ensuring alignment with analysis objectives.
Engaged in multiple revisions to finalize categories, accommodating overlapping themes from different questions.
Step 4 : Clustering
Identified overarching themes and relationships within the classified data to derive actionable insights and guide the next steps in the analysis process.
Used color coding in Excel to quantify the frequency of tags, aiding in prioritizing findings, assessing problem severity, and shaping recommendations. Categorized responses based on persona and created spreadsheets for each of them to dive deeper into finding patterns.
I identified key patterns and insights in:
Positive, negative, and neutral comments were linked to participants' technical expertise and the time they had to explore the project.
Variations in technical knowledge led to the creation of distinct user personas, each with unique expectations and requirements
Identified specific resources required by different personas to enhance their experience
Determined feature preferences and necessities corresponding to each persona.
Major findings
67.6% users were General/basic users that needed more resources, help from teammates, and more time to explore tools
23.4% users were Advanced users that found amount of resources to be satisfactory and asked for a platform of advanced projects
9% users didn't complete the project because of insufficient time, lack of trust, and them considering the tool to be technical
43% users said that significant time was saved by using ChatGPT license
Efficiency with "low level work" to allow people/resources to focus on "harder work"
Writing(21%) and Brainstorming(20%) emerged as 2 major use cases
API Access(22.5%) and Image generation(17%) were the top 2 requirements
ChatGPT(22.5%) and Slack(20.4%) being 2 top resources used
Recommendations made vs accepted
Accepted
Accepted for ASU product
Not implemented
Recommendations
Action taken
Result
Additional support
Started support slack channels and contact modes
Decreased number of people reaching out to program managers directly
Onboarding for non-technical participants
Scheduled weekly sessions with OpenAI for participants to learn about the product
40% decrease in users not completing projects because of no technical knowledge
Creating collaboration opportunities amongst different teams
ASU product allows creating projects collaboratively
80% projects have collaborators
Creating GPTs to help user ask their querries
ASU product has a bot replying to user's queries
The bot is in the ideation stage
Workshop/presentations by challenge winners
Not Implemented because of lack of resources
N/A
Mentioning project license, timelines and access limitations
Not implemented because the partnership keeps updating regularly
N/A
