Summary

​

• Designed a humanlike social agent capable of real-time face-to-face interaction

• Built the system to emulate the social behavior of members of a specific subculture based on extensive ethnographic participant-observation (task analysis, field studies)

• Evaluations of the system (usability testing, contextual interviewing) by nearly 100 human subjects indicate that the system effectively “passes for” a member of the target subculture

​

My Roles

​

Interaction designer, programmer, UX researcher

​

Objective

​

The aim was to design a virtual performer of “free improvisation,” a form of music-making wherein musicians are more or less at complete liberty to play as they wish.

Challenges

​

The potential “users” of this system — improvisers — largely decline to state what behaviors they expect of another player, whether human or machine. For example, as a cellist named "Carl" in Chicago put it to me when I tried to understand what constitutes an ideal improviser, he refused the very idea:

​

"there’s no such thing as a ‘good’ improviser."

​

Improvisers go to great lengths to avoid any indication of what they want others to do or that they find particular ways of playing good or bad. That means that the only basis of designing a system to play like them is observation of their actual behavior.

To make matters more complicated, however, the way they play tends to sound almost as if it were completely structureless:

​

Design

​

Given these constraints, this project demanded participation in and observation of music-making itself (i.e., task analysis, field studies). The first phase of this focused on locating members of this subculture themselves. After building contacts, I spent considerable time playing with other improvisers (as a woodwind player), observing interactions in live performance, and analyzing recordings. 

While improvisers frequently refuse the idea of a “typical” profile of free improvisation, my ethnographic work on this practice made it clear that a handful of parameters define this form of music:

• avoidance of 

  • pulse-based rhythm (i.e., groove, beat, etc.)

  • tonal harmony (chords, melodies, etc.)

• exploration of harsh or dissonant sounds (i.e., rubbing a styrofoam ball on a snare drum)
   

Evaluation

​

I tested the system with experienced improvisers in Chicago by asking them to play with it like it was another musician and talk about how it compared to a human player, a mix of contextual inquiry, usability testing, and open-ended ethnography. After initial tests in Chicago, I have also had nearly 100 improvisers in various locations (mostly Berlin and San Francisco) interact with the system and compare it to a human player. This portion of the project was funded by the Berkeley-Mellon Fellowship, the Fulbright Commission of Germany, and the Berlin Program for Advanced German and European Studies.

​

Most test subjects have found that the system is much like a human improviser. For example, “Wouter,” a Dutch woodwind player in Berlin, found that playing with it:

"didn’t feel radically different than playing with an unknown musician,

meeting them for the first time and playing, not knowing what to expect,

not knowing what the other person will do,

so, in that sense, 'comfortable.'"

Only a handful of performers have taken issue with the way the system avoids pulse or harmony or its focus harshness or dissonance. Thus the initial hypothesis behind the system’s design has proven valid.

But while few criticized its basic design principles, many have a lot to say about how their feelings about this system remind them of things they have had to deal with when working with other human players (see next case study).

​

Related Publications

​

2016. “Balancing Defiance and Cooperation: The Design and Human Critique of a Virtual Free Improviser.” Proceedings of the International Computer Music Conference. Utrecht.

​

​