Computer Scientist and Game Designer Andy Nealen Does A Little Bit of Everything
To sit in Andy Nealen’s office is to take a dizzying tour of academic disciplines. In a single visit, the conversation might jump from the historical separation of architecture and civil engineering, the design of the Sydney Opera House, the psychology of bounded rationality, language acquisition, to three-dimensional computer modeling. Then there are the bookshelves, crammed with topics ranging from game design to the history of Dungeons and Dragons to calculus.
“I make no distinction between disciplines,” says Nealen, an assistant professor of computer science and engineering at the NYU Polytechnic School of Engineering. “I think people who are good at their designs know a little bit about everything. Or a lot of everything.”
Nealen’s own academic background reflects this philosophy. Although he is now a professor of computer science at the NYU School of Engineering, he started his career as an architect and civil engineer in Germany, where he lived from the ages of nine to thirty-eight. After ten years working in those fields he grew tired of large-scale projects that had so many variables outside of his control. He decided to start over as a computer scientist with the ultimate goal of designing games. “The idea of being able to do all my experimentation and thinking on a laptop was glorious and really liberating,” he says.
To start his career in computer science, Nealen had to begin nearly from scratch as an undergraduate. Then, his plan to go directly from there to the game industry was temporarily sidetracked by an offer to get a MS at the Technische Universitat Darmstadt, then a Ph.D. at the Technical Institute of Berlin. During his doctorate, he focused on three-dimensional modeling, although he was still able to dabble in game design. As soon as he turned in his thesis in 2007, he began collaborating with Hemisphere Games on a project, and in 2009 they launched the puzzle game Osmos, which collected awards including Apple’s 2010 iPad Game of the Year and the 2011 Apple Design Award. The game’s popularity continues even today: a few weeks ago, it was parodied on The Simpsons.
After finishing his doctorate, Nealen moved to the New York City area to be a computer science professor at Rutgers University, where he stayed for four years. Although he loved his department, his predilection for breaking down disciplinary borders brought him to the School of Engineering, in part drawn by the promise of the new Media and Games Network (MAGNET), which opened last September.
MAGNET is a multi-school colocation that brings art, computer science, and engineering into an airy shared space within the MetroTech Center commons. It includes faculty and students from the School of Engineering; the Steinhardt School of Culture, Education, and Human Development; the Tisch School of the Arts; and the Courant Institute of Mathematical Sciences. “In knowledge acquisition, I don’t think tribal- ism is a good thing,” Nealen says. “And that is why I left the comfort of a large computer science department to dive into this inter- disciplinary mess. It’s a beautiful mess. It’s the kind of mess that’s exciting.”
The ambient world of Osmos features dreamlike visuals and a minimalist, electronic soundtrack.
Nealen’s current research at the School of Engineering focuses on three major projects. The first project involves virtual cinematography, which will automatically fill in a three-dimensional camera path based on two-dimensional camera views. Movie and video game directors could use the technology to swoop through a scene and get a sense of the layout without having to do any manual work to design the three-dimensional camera path. The project may have applications for other fields, too. For example, it may allow architects or land- scape planners to automatically render blueprints into three-dimensional spaces, to allow clients a virtual walk-through.
The second is three-dimensional modeling and animation, a continuation of his doctoral work in Germany. Currently, in order to change the shape of a figure in a computer-generated scene, an animator or director must give the shape back to a modeler and wait for it to go through several iterations of design. Nealen’s goal is to allow the animator or director to alter a shape directly in a scene, without going through these time-consuming additional steps. The work is tentatively scheduled to publish later this year.
Nealen’s third project goes back to his love of game design—it aims to model human perception of a game’s difficulty with artificial intelligence. Using the popular iPhone game Flappy Birds, the AI plays different variations on the game and reports back as to how hard it was to play. The technology could eventually help game designers identify which versions of their games are the most difficult, allowing them to tweak the games’ parameters to find the sweet spot of both difficulty and enjoyment.
“I MAKE NO DISTINCTION BETWEEN DISCIPLINES. I THINK PEOPLE WHO ARE GOOD AT THEIR DESIGNS KNOW A LITTLE BIT ABOUT EVERYTHING. OR A LOT OF EVERYTHING.”
In his work at the school so far, Nealen has collaborated with the Game Center at Tisch. But his artificial intelligence project has implications for risk management and decision theory because it involves modeling how well a human can perform un- der pressure. This could eventually lead to joint efforts with NYU’s neuroscience and economics departments.
“I’m interested in the way that an environment and audiovisual feedback impacts performance,” Nealen says of these potential collaborative projects. “That is the kind of thing I want to move into.”
Work and Play
Despite juggling his research and teaching responsibilities, which include graduate courses in game design and computer graphics, Nealen seeks a healthy dose of extracurricular stimulation. Back in his office, alongside his eclectic book collection, are dozens of video games, empty boxes from game consoles, and around twenty board games—overflow from the 300 tabletop games he keeps at home. His desk space is littered with more: a chessboard, a complex version of tic-tac-toe made from wood and marbles, and various card sets including, a student prototype.
These aren’t just for fun. Nealen is a juror for a range of festivals that require playing thirty to forty video games a year. He is also a new board game critic at Paste magazine and is prototyping three board games of his own. The playtime feeds back into Nealen’s programming and video game design, too. Unlike a computer game, which can hide computation in the background, a board game lays everything out in the open, requiring a refined understanding of each mechanism and how it contributes to interesting play.
So does he like his job? “Oh god,” Nealen laughs. “Yes, it’s the best.”