We are in Washington, D.C. this week to share our progress and meet our peers at the Fourth Annual IES Research Conference (Sunday, June 7 through Tuesday, June 9, 2009) at the Marriott Wardman Park Hotel.
The annual conference brings together grant and contract recipients who are conducting education research funded by IES and the U.S. Department of Education. More than 850 researchers from around the country attended last year’s conference. Approximately 1,000 researchers and policy makers are expected to attend the 2009 Research Conference.
Katie Culp and Wendy Martin will preside over our booth at the poster session. If you are attending the conference, please stop by and say hello. (Here’s a copy of our IES 2009 Handout [PDF] for those unable to attend.)

The IES conference highlights a big week for our project team, which is marked by the official launch of this project website and our participation in two other conferences (Games + Learning + Society and Games for Health.)

We have four puzzle games in development in Spring 2009, brief descriptions of each follow.

1. The Energy Puzzle (see above) presents players with a pattern of moving shapes representing different molecules. The player’s stylus exerts a certain amount of energy. When it’s applied to a part of the pattern, the energy produces a change in the movement. This surge could causes a chain reaction, some of which dissipate quickly, while others produce a more permanent change (create glucose or oxygen).
   Underlying Idea: Energy causes transformation, but does not itself transform. 
    
2. The Data Puzzle presents players with a flawed representation (illustration and text) of such topics as air quality and greenhouse gasses. Players are then provided an updated (corrected) illustration and asked them to identify a difference between the two images. Once the difference has been found, players must correct the caption that accompanies the second illustration to match the change in the representation.
   Underlying Idea: Visual representations are interpreted and can be misunderstood.  
    
3. The Sound Puzzle focuses on the transformation of gas into liquid and solid states using sound as an analogy. The player is greeted with a cacophony of sounds made by invisible objects (gaseous state). Through a variety of interactions, possibly blowing into the mic, the player filters sounds down to a rhythmic pattern based on the player’s input (liquid state). This pattern can be further turned into a specific, solid, visible sound source, a box or a bell, through strategic tapping on the touch screen. 
   Underlying Idea: Moving from invisible (gas) to visible (solid) states is hard to grasp. 
    
4. In the Logic Puzzle, players are coaching a debate team and have to identify the truth value and logic of causal propositions to be used in the debate. These propositions are parsed for them (literacy support) and turned into logical chunks consisting of at least one “because” premise and at least one “therefore” conclusion. Players have to check the truth value of each premise statement by searching efficiently through a small database of information declare the premise either true, or false or that there is either insufficient or no evidence to support its claim. The debate centers on using information about plant and animal life as well as lore to classify vampires. 
Underlying Idea: In a democratic society, students need practice in checking both truth and logic in causal arguments to participate in the ongoing science debates, such as climate change.

We see portable games as useful tools for enhancing 7th grade science curricula, not a substitute. It’s an important distinction. Portable games can bring literacy supports (sound, image, video), a motivating context (narrative) and discrete explorations (educational mini-games) into the classroom, providing educators with another way of bridging science content and concepts. We are developing games built on the Nintendo DSi platform that are designed to accompany 7th grade science texts typically available in classrooms throughout the country. These games will be independent of each other, allowing teachers to decide which ones they consider useful supports for their curriculum. Professional support to help teachers manage the games and identify how they might extend their use in the classroom with other activities and discussions, will also be provided on the web along with links to other resources.

Three major design goals anchor our game development efforts:

1. The games focus on the aspect of a topic, in this case photosynthesis, students find most difficult to grasp and which often lead to misconceptions.  We are not trying to illustrate central aspects of a topic for which good illustrations exist in other media.
2. The games support a specific type of thinking rather than providing information about the topic. Even though they may aid class discussion on the topic, the games will focus on the critical thinking skills that can be applied elsewhere.
3. The games provide the literacy support struggling readers need to participate in thinking critically about the topic and will focus on meaning-making and interpretation rather than decoding. With the multiple media they support (audio, video, image and text), portable games can assist teachers in reaching children of varying reading levels.

We are currently working on the first set of games.

Determining the success of a commercial game is all about dollars and cents. Have you met your development costs? Are you turning a profit? Is your audience satisfied? Our endeavor is very different.

We are trying to encourage a certain kind of thinking among young people (7th graders) in the sciences. We are also trying to model an approach for incorporating gaming into existing curricula in a way that leverages the strengths of game-based activities to support classroom instruction and deepen student learning.

So how do we measure our success? We will gathering feedback on our game modules throughout the development process from students and teachers in afterschool and school settings. Beyond that, we will conduct a randomized clinical trial  to determine whether the use of our game-based activities and support materials improve students’ student learning.

The main research questions for this summative evaluation are: 

• Does the use of Super Sleuths as an enhancement to the science curriculum have a positive impact on student achievement in science?

• Does the use of Super Sleuths as an enhancement to the science curriculum have a positive impact on students’ literacy skills?

Participating teachers will be randomly selected for treatment  and control groups. Treatment teachers will use the game with their students for one life science, while a group of control teachers will continue to teach as usual. The study will be conducted during the 2011-2012 academic school year with results available in 2013.

As part of the formative phase of the Possible Worlds project, we are conducting exploratory research into aspects of children’s abilities to engage in problem solving and scientific reasoning as they play four commercial video games: Auditorium, World of Goo, Portal, and Crayon Physics. Using a think aloud protocol and recording game play sessions with Flip cameras, we have been working with approximately twenty 4th–7th grade girls and boys in New York City for twelve weeks as they verbalize their thinking about in-game problems and solutions. The games are “small” and manageable in terms of children’s time and efforts and their slow pace facilitates frequent pausing, enabling the players to think aloud as they play and describe their activities. Finally, the problems children encounter in these games are relatively well defined and based in physics principles.

The goal of this work is to achieve greater insight into how children draw on their problem solving and scientific reasoning capacities in order to solve problems they encounter in video games. Specifically, we are interested in children’s understanding of game feedback and how it influences their choices about strategies to complete levels in the games. We are also investigating whether players’ explicit framing of game problems helps them develop more successful strategies.

We will describe how children’s play in these games intersects with their use of specific problem-solving strategies and how their use of those strategies varies developmentally.  This effort is part of an ongoing focus in our work to understand how developmental and cognitive differences shape children’s engagement with games, with an eye toward developing  educational games that support children’s development as problem solvers.

How can portable gaming devices support science teaching and learning? In this article, we briefly summarize the classroom affordance of Nintendo DSi.

  

1. Multiple Orientations

The Nintendo DSi, in particular, has two screens, supporting multiple orientations that can be used to create visually complex and engaging student experiences.
[ Dual Screens ]

1. Activity can flow across both screens.
2. Activity and point of view can be separated, such as one screen displaying a cross section of the pond and the other providing an inventory of scientific instruments to study the pond environment.
3. Finally screens can be viewed in portrait (left column) or landscape (right column) mode.

4. Record Observations

Students can record research notes and observations as well as capture sounds in the field, then upload their scientific data to the class wiki.
[ Microphone ]

5. Capture Video, Photos

Students can record images as well as video of themselves and their point of view via two cameras (inside and outside) the DSi.
[ Cameras ]

6. Language Support

Struggling language learners can hear text spoken, providing an important literacy scaffold.
[ Speakers ]

7. Collaboration

With wireless capabilities built into each device, students and the teacher can communicate with each other without requiring a school-wide wireless network.
[ Built -In Wi-Fi ]

8. Online Research

The Nintendo DSi includes a web browser built especially for the platform, enabling students to search the web and teachers to assign web-based research to students.
[ Web Browser ]

Rather than focus on science facts and how we can use portable game systems to make learning a set of facts fun, we are trying to create game mechanics that  promote questioning. probing, and re-conceptualizing of scientific phenomena and processes. The goal is to encourage and prepare young people to look critically at their science misconceptions and develop the habits of mind necessary to make sense of other phenomena they encounter. Crafting these “thinking” game activities requires a significant investment in instructional design as well as substantial trial and error, prototyping and revision in game development to get right. Thankfully, our game development partners, 1st Playable Productions (Troy, NY), understand the need for  rapid prototyping and an iterative design process. We are really excited to work with them.

1st Playable is led by Tobi Saulnier, a veteran of both handheld and console games. Her team has created a number of titles for the Nintendo DS platform, including Club Penguin, Ben 10, and GoPets, among many others. The company prides itself on collaboration and uses a game authoring tool called Game Maker to generate frequent game iterations. These prototypes can be quickly evaluated by the instructional designers and tested with students before coding begins on the proprietary Nintendo development environment. Game Maker prototypes provide an emulation of the physical look as well as the interactions and mechanics of the DS on the screen of a PC, using a mouse to represent a stylus. Our development of the prototypes for our first module of games continues to progress. Early builds are addressing the transformation of energy during photosynthesis, balancing systems, and a logic game based on evidence and argument. As these mini-games solidify, we will share screens and sample videos here in the Development section of this site.

Our game modules and support materials seek to tackle three key themes: structure gameplay in a way that supports the use of evidence and argumentation; focus game design on developmental targets; and craft experiences that meet real world needs and constraints of students, classrooms and teachers. To inform and refine this development, we are engaged in three inter-related, but distinct strands of research. 

1. Formative testing to inform the iterative development of game components.  For example, during the 2009-2010 school year researchers will be running after-school programs in several middle schools, in collaboration with science teachers.  We will use these sites as testbeds to try out early versions of games with students and teachers.  Researchers attend to the appeal and usability of the game components, but most importantly they focus on whether or not the games are engaging students with the kinds of thinking and exploration the games are intended to support.  Rapid reporting of these findings back to developers ensures that game components can be adapted and adjusted to meet students’ needs.  
2. Research on games and learning that reaches beyond the specific needs of our development team.  We will be conducting a range of studies over the life of the project that will help us to:
   • Develop new methodologies for understanding whether and how children are learning from a variety of electronic games.
   • Test and refine some of the theoretical premises driving the design decisions behind our game modules.
   • Explore how current thinking about cognition, learning, and science teaching are supported or challenged by evidence drawn from children’s experiences with educational electronic games.
3. Evaluation of efficacy of one of the game modules, through an experimental trial to be conducted during the 2011-2012 school year.  This study will allow us to test whether full implementation of one game module has a demonstrable effect on students’ understanding of the concept addressed by the module. 

Taken together, this work will provide an empirical context for understanding the development of our game modules, contribute to educators’ and educational researchers’ understanding of the roles electronic games can play in supporting students’ science learning, and introduce new models for research to the educational game development community.

How do children draw on their capacity for scientific reasoning to solve problems they encounter in video games? It’s a key question that we need to consider when working through ideas for game modules that encourage scientific thinking. To help us understand this question, we set up a series of afterschool gaming sessions over a 13 week period beginning in January 2009 and ending in May 2009, so that we could observe how young people think through game challenges. The research team worked with 20 4th-7th grade boys and girls in two afterschool programs in NYC. We used Deanna Kuhn’s conceptual model for “strategic competence in inquiry” as our theoretical framework, developing “think aloud” protocols to use during our observations. These protocols were designed to help children verbalize their thinking about the problems they perceive in games, the cues they identify in order to overcome obstacles, and the decisions they make in order to solve those problems.

We took these protocols into the field using four commercial video games: The World of Goo, Auditorium, Crayon Physics, and Portal. We chose these games because they are manageable in terms of children’s time and efforts, the problems children encounter in these games are well defined and based on physics principles, and they have a slow pace that facilitates frequent pausing, enabling participants to think aloud as they play and describe their actions. For further analysis and coding, each game play session was recorded with a Video FlipCam. As students played, researchers asked frequent questions to probe their decision making and reasoning, such as, “What do you think you have to do here? How do you know? Tell me more about what just happened.” 

We are now examining the students’ varying theories and strategies in each game to see how the feedback that the games provide affect a revision of strategy. By looking at students’ thought processes as they are happening in the game, we hope to understand how learning is refined, re-conceptualized and incorporated into existing knowledge.  

Watching kids play the game Portal provides a really clear opportunity to demonstrate, in a simple way, the notion I’ve been getting at when I’ve been talking about “possible worlds,” and in the material I wrote about Winnicott & Bruner that some of you read recently.  When you watch kids playing Portal, one frame to put around it is to view their actions as a process of asking the question “what are the rules here?” (always a tacit question in a new game, maybe?) and more specifically “what about this place works the way my world works, and what about it is different?”

Once you “get it” in Portal, the thing that is different is pretty clear, and also pretty difficult to make sense of or draw any larger principles from.  But while that’s a limitation, it also makes it very easy to see trace the kids’ experience of stumbling upon the “what’s different,” testing it out, and drawing it into their conceptual understanding of the play space and the rules under which it operates.

So this is a simple example of a broader principle - a) that virtual gamespaces are expected to operate according to some set of internally coherent rules and b) that those rules may or may not be consistent with the rules of the world in which we normally operate.  Kids’ openness to this phenomena is an opportunity that we’re going to leverage - a way to get a toehold that will, we hope, help us to dis-equilibrate their existing scientific misconceptions.