Tuesday, May 5, 2015

Play Testing and Feedback

We tested the game numerous times with a variety of people. We tested among groups of people, and individuals. It was all very valuable feedback and I look forward to continuing with the process.

While all the test groups were valuable and offered great feedback, I'd have to say, testing it with my kids brought the greatest satisfaction. Pretty quickly I decided that if I could make a game they could enjoy, I'd be happy. 

Among the groups we tested were:

EAE Cohorts.
Both during concept and development of the game these students proved to be extremely valuable resources for testing. It was helpful that they all spoke the same gaming language, and were hard at work making their own games as well. The feedback I received from this group was usually very precise and detailed information. They were all quite astute at quickly recognizing the key problems with the game, and what options we might explore to make it better. Being gamers themselves, they had a wide range of gaming experience, and were also very honest. They were able to tell me whether or not they liked the game, whether they would play it and even pay money for it, and where it compared to other games in the market. They pointed me to games that I could look at to learn things that would be valuable in our development, and that was extremely helpful. Even when the game wasn't the type of game they would normally play, they could still let me know what was working and what wasn't. It was very valuable having this feedback, and I hope stay in touch with a lot of these guys after graduation.

Family and Friends.
This next group was a fun group to test with. Most of them were very complimentary toward the game, simply because it was me making it (that might be an assumption on my part). It felt like I might have been able to make any game and they would have spoken well of it. That's not to say there wasn't some real honesty in the feedback. I have the kind of relationships with these people to ask for and receive honest feedback, and that was usually never a problem. My kids were by far the most fun to test this with, and I loved seeing each of their personalities come through as I tested the game with them. At first they were just thrilled to have their dad ask them to play a game. Then they moved to being very complimentary toward the game, and told me how cool it was, and what a good job I did. (I should add that they're all under the age of 13, so no teenagers yet.) They struggled with much of the frustration with the game that other testers did, and I could tell they did, but I definitely had to pry out of them the things they thought weren't working well. I had to let them know that they wouldn't hurt my feelings by telling me the frustrating parts, and that the more they told me the more it would help me make the game better. When they realized this they were not only helpful in telling me where it was frustrating, but also suggested ideas that might solve a lot of the problems. It was truly great to hear their ideas, some of them were spot on with how I would solve it, some of the solutions sparked new ideas and insights that were really good. 

If I continue to make my own games in the future, I will definitely focus on developing games that they would enjoy. If possible it'd also be fun to develop games with them if that's what they wanted, they've got some great ideas. Who knows.

The CS Capstone Students.
These students were shown the game on multiple occasions throughout the year. They were shown all the prototypes that were created and gave feedback. This was helpful, because it represented a group of people that approached development from the engineering perspective.

Here's some of their feedback:

  • The transformation mechanism is not intuitive enough (e.g. it was confusing to people that a green bullet caused the triangle to turn into a square).
  • People thought red should be lowest then white and then green (in order to be consistent with how the bullets transform you in that green leads to green and red leads to red).
  • Some people mentioned that it would be better if the transformations looped (e.g. the white would become a triangle if hit with a green bullet).
  • People liked the mini-games.
  • Several people were entertained watching us play the game but were intimidated to actually play the game.
  • People seemed intrigued by the concept of change and how flux explores that and most everybody said the game was cool.
  • Some people said they really liked how elegant and smooth the game looked.
  • Some people liked the animations.
  • Several people felt the first ten levels were too redundant -- different elements need to be introduced sooner.
  • The win/lose screens need to be transparent, so they can see what they actually did to win or lose.
  • The additional elements make the game more interesting and fun like the pass through walls and reflecting walls.

Professional Colleagues.
I took the opportunity to test with some of my colleagues that I've worked with. I made sure not to do this at work, as it was a conflict of interest, so I tested it with them outside of work. This was very helpful for a lot of the reasons that the EAE cohorts were helpful, but unique in that these colleagues had a lot more professional experience than the EAE cohorts. I'm not saying that that's a good or bad thing, just different. It offered a unique perspective that was hard to get elsewhere. They were able to see design and development problems clearly, and many times had very efficient and clear solutions. They offered advice on how they might do something differently, and how they would implement things themselves. It also proved that there are a lot of really bright and talented people who could make some great games and apps themselves if given the right opportunity and environment. 

The Foundry
Around the middle of the semester I was accepted into the 10th cohort of the Foundry at the University of Utah. The Foundry is a startup incubator composed of entrepreneurs, mentors, and students looking to get their ideas off the ground. A group of student entrepreneurs go through a 12-week course where they test and refine their idea for a startup, with the help of the mentors and other members of the cohort. This was a really cool experience with some great mentors, Russ McBride and Josh Maag. I tested the game within the cohort and received helpful feedback and ideas for the game itself, but equally valuable was the advice outside of the game design that was offered.

The Foundry turned me on to a process and philosophy that I had heard about before, but was only marginally familiar with, which is Lean development. Eric Reis wrote a book called The Lean Startup outlining some of its core ideas and implementations. It's tough to boil down everything that it is into just a few sentences, but as I understand it's a process of making assumptions or hypotheses, then testing these hypotheses in small, simple, experiments that help you learn what you need to further your hypotheses.  The Foundry is run on this philosophy, and much of what Russ and Josh advised toward was this process, what's the next step, what are the experiments that you can run to test your idea and how do you do it in a quick efficient process. 

It wasn't only valuable to get feedback on my own project, but on my fellow students' projects as well. The Lean mentality and process was applied to building and launching products and companies, but it could also be applied to game development. And in fact, as I read and became familiar with this process, it resonated as a process that I've heard used among successful game companies like Nintendo. This makes sense, as much of the Lean philosophy is derived from the Japanese company Toyota, and the way they manufacture their cars. 

It's a process that I think is critical in a creative place like a game or animation company. What I took out of it is that often we just don't know where we're going to end up in making a product or game. If we're willing to admit that, test our hypotheses in simple, inexpensive ways, and learn as much as we can from these experiments, we'll have as good a chance as any at getting to a product or game that might resonate with people. 

The time I spent at the Foundry was very helpful, I only wish I had more of it.


No comments:

Post a Comment