DESIGN
PROCESS

Design Process + User Feedback

DESIGN PROCESS

An Overview of the Design Process for the Prototype

Using a user-centered design process, I integrated user research seamlessly into my design process, ensuring user feedback drove the final design. The feedback also forced many iterations of the project, which made vast improvements to the solution.

DESIGN PROCESS + USER FEEDBACK

I followed a user-centered design process to create my virtual escape room game. As an experienced user experience designer, I was able to apply many techniques from my practice to this project. I cover some of my methods in the Introduction Section, but highlight my user interactions and design iterations below.

STEP ONE

The first step was to collect data on a target user base and focus area. I decided to focus my project to something that was meaningful and personal to me: the education of my oldest daughter, Sophie. My target user base became elementary school students between the ages of 7-12 years old and grades 1st through 5th. Next, I collected data on this segment and found that mathematics was a key indicator in many other subject areas so I concentrated my project on math. I gathered data through interviews and simple surveys of what students were learning, how they were learning it, and how it impacted other parts of their education, either directly or indirectly. In total, I had 6 interactions with users from this demographic. From this initial front-end research, I came to the conclusion that some students learned math concepts through memorization but lacked the experience to apply those concepts to real-world issues.

"During school, my teacher shows us how to do the problems and then asks us to copy her. It's hard to create the same problems when I have to read a problem like Sally buying 4 apples and giving 2 of them away. I think I need more practice doing this." - 2nd Grader

PROBLEM TO ADDRESS: Traditional teaching methods and tools do not adequately prepare students to apply concepts to real-world situations.

PROPOSED SOLUTION: Teach students, specifically 1st through 5th graders, how to apply math concepts to real-world problems using a gamified stealth assessment.

HYPOTHESIS: Running stealth assessments will provide rich information about students' learning profiles so that educators can focus on key gaps in the classroom without being biased by uncontrollable factors.

STEP TWO

I next reviewed the motivational concepts from class to see which methods would help me solve my problem. Some directly addressed the issue, while others influenced the outcome, but were tangential to the core problem, like the Protege Effect or Learning by Making. Then, I thought about how to incorporate something I was passionate about into my final project and settled on a blend of virtual escape rooms mixed with puzzles and mysteries. From here, I sketched out a few ideas and ran them against some of my initial users from the first step to get some feedback. The overwhelming response was to move forward with some sort of mystery game, which ended up being this virtual escape room idea.

"That sounds so cool! I love puzzles and if the story was about something I like, then I would love playing it." - 1st Grader

STEP THREE

After sketching some initial ideas, I started to work out the design architecture, including the user flows, user experience, and how I would create the project for some initial testing later on. After brainstorming ideas, I decided that a web-based approach would be best for what I was trying to accomplish for this project. I referenced the theoretical and empirical frameworks from class to ensure that I incorporated them appropriately. This helped me design a few early features, such as the game settings and puzzle hints.

I conducted quick workflows in Miro and walked through them with my daughter to see if there were any missing elements or details that would help communicate the instructions better. This led to the idea of adding an instructions page that detailed the rules of the game for first-time users. I also walked through the workflow with another parent of a 2nd grade student.

"I don't understand how to play this game. I think you need instructions so that people can understand how to play before they start. A video will probably be better." - 1st Grader

"Sounds like a neat idea. I think it might be challenging for some kids to apply the concepts they learn in school to problems like this, but it's worth a shot to try something different. Applying math like this makes total sense since that's how we use math in the real-world." - Parent of 2nd Grader

Diagram of my escape room architecture and framework created in Miro.

STEP FOUR

I worked on an initial low-fi prototype using scratch paper just to get the idea down. I then moved these low-fidelity mockups to web pages and started to build in some of the key features like how to submit answers and create hints that users can display easily without exiting the game. This was the most time-consuming part of the project since there were many things to consider with the design and a lot of logic to develop into the prototype. I ran the initial puzzles with 4 more users to solicit their feedback.

"I like the storyline a lot, but there is probably too much to read. I just want to jump down to the question and work backward from there." - 3rd Grader

"Some of the questions were too hard for me. I need help to understand what they are talking about." - 1st Grader

This initial feedback made me rethink the difficulty of the questions. I used math concepts from the appropriate grade level (1st grade), but realized they hadn't learned how to apply those concepts to things outside of simple math problems. It took a few iterations to level-set the difficulty of the questions, but it helped a lot getting this done in this step before I had spent too much time on it.

Initial wireframes of the game UI and educator room builder.

Initial wireframes of the game navigation. Note that in this wireframe I allow players to jump across puzzles. This means they do not have to solve puzzles in a sequential order.

STEP FIVE

I finished an early version of the prototype and wanted to run a few research studies to check the questions' difficulty and the overall timing. My goal was to get the game down to under an hour, so fine-tuning was needed. I also wanted to make sure the narrative was consistent with what students at this age expect and fine-tuned the communication by replacing words or phrases in the clues. I also wanted to make sure the experience was appropriate - Did students understand how to use the hints? Did the students know how to navigate the website? What happened if they inputted an incorrect response? What happens if they accidentally refreshed the page? I ran the prototype with 4 users (2 from the previous study and 2 new) using a mix of in-person and virtual (via Zoom) qualitative and quantitative research.

INSIGHTS FROM RESEARCH STUDIES

4 / 4 users completed the game

4 / 4 users used hints

2 / 4 users struggled with navigation, especially exiting a hint prompt box

1 / 4 users became frustrated with the length of the content (reading)

4 / 4 users inputted the incorrect response (made errors)

3 / 4 users asked for help from someone

The average completion time was around 1 hour 10 minutes
2 users were in 1st Grade, 1 in 2nd Grade, 1 in 3rd Grade
Average age: 7.75 years old
By Clue #3, I noticed that students scrolled down to the questions instead of read the entire clue first. This was insightful since it saved them time.
Clue #6 was difficult for 1st graders since they didn't know the term "dozen"
Clue #7 was difficult for grades 1 and 2 since it uses multiplication
Clue #8 was easier to solve than I expected
Clue #10 was easy once they discovered what a cipher is since they only had to solve for the first letter (Finn)
Some of the words I chose, the students didn't understand
Some of the names were difficult to pronounce

"I really like this game! There was a lot of words for every question and I just wanted to answer the question so I scrolled to the bottom." - 1st Grader

"I didn't get some of the questions and what they were asking. The hints didn't help me in some cases." - 2nd Grader

"I didn't understand why I needed to read all the ingredients and steps if the question didn't ask me about them." - 3rd Grader

In-person user testing with participant (1st Grader).

ITERATIONS

I made several iterations of the design throughout the design process in order to get to the right level of questions and content. The feedback was extremely helpful in getting this information. It should be noted that there were biases in some of the testing due to the circumstances. Hence, the information should be treated as informative research in helping to shape the design and not be meant for validation. Here are a few I noted:

Test participants who tested multiple times had prior knowledge in future tests.
Some test participants were relatives
In-person testing enabled me to help participants with user experience or navigation issues
The prototype, specifically the platform I used, has limitations that do not reflect the final design of the game. It was primarily meant to collect user feedback and note design improvements for when it is fully developed.
The tested prototype was based on 1st Grade Mathematics. Students who feel below or above this grade level may have found the questions too easy or too difficult. However, I included a few questions that aligned with older grades to see if the feedback would change.
Testing structure and setup have their own challenges to consider such as network connectivity and lag or the PC the user was using in remote testing.
The number of participants is extremely low and not diverse. This will obscure any quantitative data, thus, the research insights should be used mainly as informative research.

Escape Room Themes. Survey results suggested these were some of the highest interest areas for students between 1st and 3rd grade.

Example of a Question from the Game. A few participants noted they didn't learn what "dozen" meant yet, so I tweaked the question in the final version.

Example of a Question from the Game. Several participants struggled with this question so I had to tweak it for the final version.

Example of a Question from the Game. Several participants struggled with this question so I had to tweak it for the final version.

NEXT STEPS AND FUTURE IMPROVEMENTS

There are several opportunities for future improvements to the game, including more questions in a question bank that can be dynamically inserted into the games based on previous successful answers within a time frame.

Read more about these improvements in the section: Overall Assessment.