OVERALL
ASSESSMENT

Strengths and Limitations

Suggestions for Improvement

Summary

CONCLUSION

This section covers my overall assessment of the experience including suggestions for future improvements.

STRENGTHS AND LIMITATIONS

There are many strengths and limitations of this learning experience over other learning tools, especially ones tied to traditional classroom settings. These tradeoffs are often highly debated, as noted by all the supporting research in the previous section: Game Design. My analysis argues that the strengths of this learning experience clearly outweigh the weaknesses. Educational escape rooms are an additional learning and teaching tool for educators and students to use that supplement traditional teaching tools. It should be a replacement for traditional methods, but serve as a complementary medium where traditional tools may fail.

I believe the the biggest strengths to Educational Escape Rooms are:

A tool that can be used for as a stealth assessment that directly addresses the challenges associated with today's educational assessments.
An interactive medium that can serve as a platform to teach intangibles such as team collaboration, communication, body language and posture, and social-emotional learning.
A tool to improve student engagement and motivation as evidenced by the research.
An immersive medium to teach more visually-weighted materials.
A tool to improve individual learning needs through dynamic questions and analysis of student data captured by digital tools like eye-tracking, biometrics, and facial recognition.
A tool to improve the accessibility of teaching by utilizing digital aids like screen-readers, auditory feedback, and haptics - modalities that are more difficult to supply during in-person lectures.
Research shows that Escape Rooms are evenly appealing to both genders (only two gender categories were reported in the research studies).
Helps eduators upskill by learning new methods and pedagogies to teaching complex information.
Used to teach remote students who cannot attend class in-person including global-based educational programs.
Provides a personalized and interest-driven learning experience in the form of storylines and narratives.
In addition to teaching students about core subjects, students gain skills in digital literacy, which is an important 21st century skill.
Educational escape rooms apply concepts to real-world situations in a situated learning environment.
Educational escape rooms promote active learning, which is known to improve engagement and motivation.

A few of the limitations to Educational Escape Rooms include:

The cost of implementation can be expensive, although there are very inexperience alternatives like Google Slides. Digital escape rooms need to be hosted and maintained.
Good educational escape rooms take up many resources including the educator's time. Not many educators have time to create escape rooms, especially if they cover narrow topics or lessons. Designing a good escape room takes a lot of planning.
Troubleshooting escape rooms can be an issue. Many problems could arise during the game that may affect the students' experience. This effect could be negative and may bias learning results.
Digital educational escape rooms could be reliant on good network connections. Students playing remotely, may have an adverse user experience due to technicalities beyond the educator's control.
The themes could bias the learning results. Students may be motivated and engaged with content for the wrong reasons.
Some themes may not be appropriate for educational materials such as references to pop culture and movies. Governance will be important, especially in school districts with influential governing bodies.
Puzzles are difficult to create. Educators have to strike a blend between making them too easy and too difficult. This puts a lot of pressure on educators to get this balance right.
Escape rooms are typically only completed once by a player. If a player completes the game, then the liklihood of that player doing the same room again is low. This means escape rooms have a short-life and cannot easily be updated.
Educational escape rooms typically cover a narrow topic. This means that educators must create multiple rooms to teach broad concepts.
Some students do not perform well under these learning conditions. For example, students who need extra accommodations (such as students with IEPs, English Language Learners, etc.) may be anxious about the timer and could introduce errors that they normally would not perform in other learning enviornments.
Digital escape rooms lack physical connection between peers and the educator. In a phsyical classoom, the student has access to the educator and can ask for help unprompted. In the digital environment this is much more difficult to do, especially if there are many students participating.
There is not enough research published about how educational escape rooms improve learning. Most studies I found were short. Until there are longitudinal studies conducted, we don't know the long-term effects from educational escape rooms.

SUGGESTIONS FOR IMPROVEMENTS

After extensively reviewing the academic research and data from the user studies I conducted, I found many areas of improvement I could focus on for future iterations.

It's important to highlight a key disclaimer of this project - after I had chosen to use a website with password protected pages as my main platform for the prototype, it was difficult to move away from this solution even when I found better platforms and technologies I could have used. This is reflective of the high switching costs involved with moving to different platform in the middle of the project. I should have done more thorough research in escape room solutions before settling on my proposed tool. I chose this tool because I was familiar with it and I had tested and tried 5 digital escape rooms that used a similar solution (website + password protected pages). As I was finishing up my high-fidelity prototype, I discovered "escape room kits" where educators could use prefabricated templates to create their rooms and puzzles. This would have expediated the design process and would have created a better user experience. However, I am proud of how my room turned out given the limitations of the platform and was delighted to receive a lot of positive feedback from my test participants.

Identified Key Areas for Improvement:

Besides using a different tool altogether, I think my biggest constraint was time. There was not enough time to think through the logistics of the room and puzzles to address all the research and goals I had initially proposed for this project. Creating an escape room from scratch requires a lot of planning, experience, and iterations. If I had to redo the project again, I would have spent more time creating a "puzzle bank" of questions that I could dynamically switch out in the prototype until my user study results achieved high scores.
There were many tools and plugins I could have developed or purchased to make the prototype better. Since I wanted to make this a cost-effective project, I did not purchase these plugins (which were fairly expensive). The plugins would have significantly improved the experience. One example is the count-down timer. There is an upgraded plugin that starts the timer once the player clicks on the "Start" button. However, this plugin alone costs $20/month. I did not think that price would dramatically change my results so I avoided the upgrade (at least for now). This was only one plugin of many I could have used throughout the game so the costs would start to rack up quickly on top of the website hosting fee.
Choosing a website required building the website from scratch which adding a lot of time to the project. Building the website architecture, including widgets and assets like buttons and images, took a couple of weeks and that didn't include any time working on the actual game. A lower fidelity version could have been sufficient for this project such as using a platform like Google Forms or Slides. I participated in 4 Google Slide virtual escape rooms and they worked just fine. Yes, the experience wasn't great and the low-fidelity nature of the UI affected my motivation and engagement to keep playing, but I could see a viable option of using one of those solutions to test early mockups like content or flow. If I had to start the project again, I might have used Google Forms or Slides for my low-fidelity prototype and then decide whether to change platforms for the high-fidelity prototype.
With more time, I would have created more variables for players to select and actually get them to work! For example, I could have created different levels of the same game and when a user selects their "Grade Level" from the settings screen, it would have taken the player to the appropriate set of website pages. This would have been nice feature, but I didn't think it would significantly alter the user research results. When the participant age changed, I manually switched out the questions, which was acceptable for what I was trying to achieve.
I would have conducted more competitive analysis and used that information to improve my game. I stumbled across a few really great educational escape rooms late into the project and could not figure out how to mimic of the features I noticed. For example, here is one I played recently called Training Room Escape that utilizes plug and play assets from an escape room kit that can be purchased. These additional features such as placing puzzles onto objects, zooming in and out of the room, moving objects around, clicking on different parts of the room or objects in picture, and variety of answer forms would have substantially enhanced the game.
I ran several user studies, but I could have done more and with a diverse group of users. Given the time, I had only conducted 8 user studies, split across my low-fidelity prototype and my high fidelity prototype. I also interviewed an additional 3 users including parents. This number is relatively low which means the results must be critically analyzed. As stated in the previous section, due to the low number of participants, most of the research is categorized as formative research with an emphasis on the qualitative component of the interactions. If I had more time and resources, I would have ran more user studies with a larger demographic pool covering a variety of users (age, gender, location, assessment scores, grade levels, grades, ethnicity, income level, etc.). This wider study would have provided more detailed information about what worked and what could be improved to allow more users to play the game.
I struggled a lot with content and asking the appropriate quesitons for puzzles. I felt like I was constrained by my own narrative at some points and realized it was much more difficult than I had expected. I think the questions should have utilized the information in the clues better. However, this would have shifted the focus away from math and more so on reading comprehension if there was hidden clues and evidence baked into the paragraphs. I noticed that players who were strong at math, but not as strong in reading or reading comprehension, struggled with understanding what they just read. This, in turn, affected their ability to solve the math question because they weren't able to figure out what the question was asking them. In a few cases, after I explained the question differently, they understood the math concept and got the solution. A huge insight from this work was the deep interelationship between math and reading in this type of assessment. Educators should note any biases involved when evaluating scores.
In one article, it proposed using an Learning Management System to display scores and progress while the students are conducting the escape room. The proposal was to understand the difference between mastery and performance. If the educator wanted to focus on mastery, they would display this "Progress Chart" at the front of the class so students can go back to questions they didn't score well in to get a better score. If the educator wanted to focus on performance, then this chart would be hidden from view. I think incorporating something like this into the prototype would be beneficial and valuable to educators. I would have probably mocked up a screen and tested it with educators if given more time.
Digital escape rooms should reflect physical escape rooms. However, there are many limitations of digital tools that make this simulation challenging. For example, a person in a physical room can walk around the room looking for puzzles, while a person behind a screen must navigate the space while staying in a fixed physical position. Objects or assets that require the use of three dimensional space can be challenging to manipulate on a screen. For instance, objects that have depth that are needed to solve a puzzle like turning a physical key in a lock. Yes, this can be simulated in 2D, but may adversely affect the user experience. To address these challenges, one option is using new and emerging technologies like Augmented Reality or Virtual Reality. These technologies give users depth in space and could even allow users to virtually walk around the space. This immersive experience could close the gap between a physical escape room and a virtual escape room and still retain all the benefits listed above for a virtual learning experience. If I had more time, I would explore this technology as a platform for these virtual escape rooms.

SUMMARY

Overall, I loved this project because it started from theory and ended in practice. This course gave us the tools to understand how to create engaging and motivational experiences for learning and then challenged us to apply those theories in a practical way. This process not only helped me understand the concepts better, but allowed me to test and iterate my prototypes until I felt they reflected the theories appropriately. Without such a firm foundation, I think I would have been lost trying to recreate something I found online and then, I would have not been able to explain why I had designed it that way.

I now have a strong fundamental understanding of how to create engaging and motivating learning experiences that I can apply to my own work as well as a strong portfolio piece that I can share with others. Given how passionate and excited I am about my work, my plan is to continue refining it, building it, and eventually launching or sharing it with others so they too can use it as a teaching tool. It's an exciting space in education, especially during the Covid-19 pandemic where distance learning is gaining traction. Digital learning experiences could spark new pedagogies and uncover insights we never knew existed. Most importantly, it gave me a deeper understanding of my users and how they experience learning. This eye-opening, empathy-driven experience gave me the fuel to create more meaningful and accesible learning tools that everyone can and should enjoy.

ACKNOWLEDGMENTS

To Professor Dede, Toks, and the T545 Teaching Team,

Thank you for an incredible semester and learning experience! This was an amazing class to cap my learning journey at Harvard. I appreciate all the mentorship and advice that you shared with me throughout my HGSE experience (including both classes last semester). Best of luck on all your future endeavors!

Cheers!