I’ve been playing a lot of Bastion lately, the indie game by Supergiant that has popped up on a whole bunch of Best of 2011 year-end lists. One of the music tracks has a lick in it that sounded oddly familiar, but I couldn’t place it at first. Then suddenly one day it hit me: Taiwanese pop superstar Jay Chou 周杰伦.
Regular visitors to my website probably know of my marginally unseemly fixation on Jay Chou; I even wrote an article for Time Out Shanghai in 2010 entitled “Why I Love Jay Chou.” He’s a trans-media pop star (as all the biggest ones seem to be these days), recording albums, starring in movies, hawking toothpaste and motorcycles (at $6 million, a record endorsement for an Asian artist). Western audiences who missed him in foreign fare such as Curse of the Golden Flower 满城尽带黄金甲 might know him best for his Hollywood debut as Kato in Michel Gondry’s Green Hornet last year. Studying the lyrics to his songs was my primary method for learning Mandarin, and I still harbor dreams of releasing a tribute CD one of these days. He was even the subject of my first ever post on this blog.
Check out this video for the second track on his November’s Chopin 十一月的萧邦 album from 2005, “蓝色风暴” (Blue Storm). (Note that Chopin is more commonly rendered 肖邦 in mainland China, but 萧邦 seems to work too, as discussed here.) Pay particular attention to the closing moments, from about 4:36.
Now check out this track from the Bastion soundtrack, by Darren Korb, starting around 0:26.
I don’t think the odds are so slim that I might be the first person to notice this, given the slender overlap between Jay Chou and Bastion’s respective fan bases. Clearly, both artists are using the same loop from some sample library. I have no idea which library, but after conferring with some of my Berklee colleagues, the consensus is that the instrument in question is most likely a bouzouki, a fretted Greek lute.
I’m straining to remember, but I don’t think I’ve ever used a canned loop in one of my compositions. (I may have used some stock phrases on King’s Quest back in 1998, but slowed way, way down beyond recognition to create an ominous background texture.) I’m totally down with the idea of creating a meta composition out of several streams of patterns or recorded material, the way that Charles Ives or Luciano Berio or David Shea might weave a larger fabric out of existing sounds; in fact, I think this is pretty much the job description for a video game audio lead. But using a stock loop out of a sample library just takes all the fun out of it. At the very least, if you want to keep it fresh, roll your own loops. Moreover, when creating a composition out of layered loops, it’s too easy to ignore the contrapuntal interactions between the different layers; you can miss the chance to think through all the alternate configurations of notes that might make your musical point more purposefully.
Most critically, there’s a regularity and periodicity that really feels anathema to the subtle irregularities of human performance, and it’s too common to come up with something artificial and rigid, chopped up evenly along the bar lines: every eight beats (or whatever) another layer comes in or out. And when a layer stops, it stops abruptly, with none of the resonance or decay of a natural sound, since it must be truncated precisely on the bar line, in order to seamlessly connect back to the beginning of the phrase. When I was fielding composer demos back at Ubisoft, this characteristic was grounds for immediate rejection.
The same objection applies on a macro scale, too, in game music implementations that simply loop a piece of music indefinitely (Bastion’s primary mode of musical organization). At best, this kind of repetition can lead gamers to tune out the music, reducing its impact, and at worst, it leads to active irritation. In any event, the power of music to support the emerging drama of a narrative is lost. In fact, the desire to eliminate loops and fades (two of the most common signifiers that you’re listening to a game soundtrack) was a guiding impetus behind the design of the EndWar music system.
At least for Jay, the bouzouki sample is only a minor flourish, in a fairly ridiculous duet with DTMF touch tones, almost a punch line at the end of an eclectic song that started with Gregorian chant.
But anyway, let’s get back to my Jay fetish. Here’s a clip of me sitting in with the house band at Harry’s Bar in Suzhou on a few very loose renditions of Jay tunes towards the end of 2010. (Be patient; you’ve got to breach the Great Firewall for this clip.)
Rather, what I’m here to tell you about today are the prints from my 29 Giraffes series that are also included in the show. I usually do a pretty good job of documenting my pieces when they go up, but I never got around to blogging about this series when it was first exhibited at [the studio] in Shanghai, back in August 2009 (check out the press release, flyer, and this fine review from That’s Shanghai magazine). So allow me to take a few moments to fill you in.
First exhibition of 29 Giraffes at the studio in 2009
29 Giraffes represents my first foray into visual art. The earliest source file I could find dates from July 29, 2007. You can see some of the images on Flickr and additional images (including some early tests) on Facebook.
These images were a natural extension of my work in sound, and the original goal was simply to create an album cover for my CD Radiospace 040823 (as featured last night on Gregory Taylor’s radio program Remember Those Quiet Evenings!). That piece performs algorithmically modulated granular synthesis on a live radio signal, sort of sandblasting the sound into new patterns and textures. I got the idea in my head that the album art ought to be generated the same way, and this stubborn notion delayed the release of the album (recorded in 2004) by about five years. It took a while to conceive of how to translate the process into the visual domain, and then another long while before I realized I could use software I already knew (Max/MSP, specifically the lcd object, before I learned Jitter) to pull it off, and then an extended period of experimentation and testing before I felt the results were worth sharing with anyone.
After all that effort, I kind of hate the image I rather hastily chose for an album cover, but I was in a hurry to get the CD pressed in time for the exhibition opening. If I had waited a little longer, this is the image I would have used (and still hope to use for a reissue one of these days).
In the same way that Radiospace samples bits of radio, these images sample fragments of digital photographs. The final images manipulate images of neon lights from Shanghai’s Nanjing Dong Lu pedestrian corridor (the stretch of street where the Sofitel in which I recorded Radiospace 040823 is located), not too far from the Bund (taken on a photo shoot with Jutta for my birthday in 2007, four years and one day ago). The software excises little chunks from these images and statistically arranges them into new patterns, according to various parameters that I can set (min/max size of the image fragment, location in the original image, density, opacity, etc.). The final compositions are comprised of one or more layers (sometimes quite a few) of these statistical passes (horizontal or vertical), which I think of as analogous to brushstrokes, over the black digital canvas.
The boundaries of these digital brushstrokes into which fragments of photographs are statistically pasted are derived from curves I’ve drawn by hand into tables with a mouse. My earliest studies involved Gaussian patterns and other types of statistical distributions, but I eventually decided I wanted to incorporate a more tactile, hand-drawn element. I felt at the time the need to emphasize that these works weren’t simply the cold, rational, impersonal result of a some obscure mathematical formula. Rather, I was involved in an intuitive and iterative process with my software, guiding the generation of new material, and then responding to it to see if I liked it or not, shaping its evolution much as I imagine an artist in a more traditional medium would.
When I moved to Shanghai in 2004, I read that Shanghai was the second largest city in the world, behind Mexico City. These images convey something of the density of urban life I experienced in one of the world’s most bustling metropolises, the exhilarating disorientation and striking juxtapositions. I think of this work in terms similar to those Robert Hughes used to describe the Merz collages of Kurt Schwitters:
Their common theme was the city as compressor, intensifier of experience. So many people, and so many messages: so many traces of intimate journeys, news, meetings, possession, rejection, with the city renewing its fabric of transaction every moment of the day and night…
Kurt Schwitters, Merz 410: "Irgendsowas" (1922)
The final images have nothing to do with giraffes. When I started developing software to manipulate digital images, my earliest test subject was a photo of a giraffe I took while visiting my parents in Kenya in 2005. I started using the term “giraffe” as shorthand for the whole project, since it was quicker to explain to my Ubisoft coworkers that I was staying in over the lunch hour to work on my “giraffes,” rather than my “algorithmic image manipulation software” or whatever. There aren’t 29 of them either; the number was chosen as arbitrarily as the name, and I kept both to emphasize the idea that arbitrariness (or artistic intuition) is a key part of the piece.
The original giraffe photo I used as the basis for my first visual studies
An early Giraffe study, lines between random points in Gaussian distributions, with colors drawn from the original giraffe image
In addition to the first show at [the studio] and the current exhibition, several of these images were included in my solo show “The Point of Departure” at the True Color Museum in Suzhou about a year ago (read the little blurb I wrote about the series at that time here). I also sell these images as limited edition prints on archival paper, and several are already in private collections. If you’re interested, by all means, drop me a line! I guess these are the images CNNGo was referring to when they described my work as “very pretty.”
Going over my notes from the time of the first show, I’m reminded of several follow-up avenues still unexplored. The unanimous feedback I received from that first exhibition was that people would like to see these images larger; at the time I was constrained by what I could fit on one screen at once, but now that I’ve gotten into Jitter, I should look into that. Also, right before I left Shanghai last year, I collected a bunch of additional potential source images from backlit fashion advertisements, and I had the idea of doing digital collages based on awareness of different body parts, but I haven’t jumped on that yet either. As Morton Feldman said, “Now that things are so simple, there’s so much to do!”
I just got back about three weeks ago from a wonderful, inspiring, and very successful visit to San Diego. I was in town at the invitation of the San Diego Museum of Art to premiere my new piece Mobile 4 at the museum’s Summer Salon Series. While I was in the neighborhood, I also had the opportunity to sample some of the energy and diversity of the city’s eclectic arts community. And I learned what a California Burrito is.
Mobile 4 is a cross between a sound installation and a chamber music piece, scored for ten channels of real-time electronic sound plus a Laotian mouth organ called a khaen (performed by Christopher Adler), guitar (Colin McAllister), and accordion (me). It was designed to be an ambient experience in a gallery, with musicians and speakers scattered throughout the room, rather than as a concert with a beginning and end. There was no central stage to serve as a focal point; instead listeners were free to come and go. The electronic sound was continuous throughout the evening, and for about 45 minutes, we three musicians joined in, adding a living layer to the installation. Then as we finished in gradual succession, we all just got up and wandered away, perusing the paintings.
Quince, Cabbage, Melon, and Cucumber
The walls of SDMA’s Gallery 16 are lined with Renaissance and Baroque Spanish art, providing an ideal setting: El Greco, Bermejo, Hieronymus Bosch’s Christ Taken Captive, Francisco de Zurbarán’s Agnus Dei (~1640), and my favorite, Juan Sánchez Cotán’s Quince, Cabbage, Melon, and Cucumber (~1602), with which I feel Mobile 4 demonstrates a particular aesthetic affinity.
Specifically, I was in town at the invitation of Alexander Jarman, curator of the museum’s Summer Salon Series. (Alexander is also an accomplished artist in his own right; while I was in town I got to see some of his collages-in-progress, beautiful stuff!) I met Alexander while he and his colleague Paul were in Suzhou last year for the opening of an exhibition of works from the San Diego Museum of Art’s collection at the Suzhou Museum (as reported here). I was just finishing up my residency at the True Color Museum in Suzhou, and the museum director invited them over for an evening of traditional Chinese music, tea, and conversation, over the course of which they also had a chance to check out my Point of Departure show. After I moved to Boston last November, Alexander and I kept in touch, and as this year’s Summer Salon Series was coming together, he invited me to present a new work.
We were a pick-up ensemble for this performance; Christopher is a friend of a friend of my good friend the writer Lisa Movius from Shanghai, so I dropped him a line after I read on his website about the interesting work he’s doing. Christopher has spent a good deal of time in Southeast Asia mastering the khaen, a rich and resonant instrument in traditional music of the region; check out his webpage for more information and some videos and recordings of his performing. The khaen struck me as a good match for the reedy sound of my accordion, and Christopher recommended his frequent collaborator Colin to round out our ensemble.
I was super pleased with how everything went, and I’ll try to get some documentation up online soon. I had my back to the gallery entrance, but I was informed that there were a lot of people pressing to get in while we were performing. The piece seemed particularly well-suited to a gallery setting; I almost wished we hadn’t distributed so many chairs, so that people would have been more encouraged to walk around during the piece and look at paintings.
* * * * *
Mobile 4 was a milestone for me on three fronts: new investigations into mobile structure, a real-time score display, and ten channels of algorithmic sound.
On the structural side, there are two primary behaviors in the piece: one is a sustained tone/drone that is kind of brushed in and out, Morton Feldman style; the other is a set of algorithmic melodies that is constantly being updated. The drones are simply chosen randomly from the notes of the current scale. Melodies are chosen from a table of available melodies algorithmically generated from the notes of the current scale. There are five melodies available at any given time. Periodically the program will replace one of the old melodies with a new one (randomly choosing from available pitches and durations, with the maximum melody length and the time between melodies varying according to a random walk).
Each sound source (instrument or speaker) behaves independently, but all are aligned to the same rhythmic pulse, and the global statistical balance between the melody and drone behaviors varies according to another random walk. If we happen to venture too far over on the drone side, we enter a transition phase, in which all sound sources gradually converge onto one of the tones in the current scale, which then becomes a pivot tone, allowing for a common tone modulation to a new, algorithmically-generated scale.
The khaen is a diatonic instrument, so Christopher brought three different instruments (G minor, A minor, and Bb minor), allowing access to the full chromatic scale over the course of the performance, though only one diatonic scale at a time. This limitation was built into the structure of the piece; when it’s time to transition to a new key area, first we decide if we want to transition to a new khaen, and then we pick a pentatonic subset of the available diatonic scale. (These aren’t standard pentatonic scales, by the way, but any 5-note subset of the diatonic scale is fair game, creating a lot of interesting variety, sometimes with a major feel, sometimes minor, sometimes with a prominent tritone, etc.). New scales tend to happen every 3-6 minutes (if we go more than 5 minutes without a transition, I start to nudge one to be more likely to occur). This has the effect of kind of “cleaning out the ear,” similar to what we did in main menu music of EndWar.
The cool thing is that the three live performers plus all of the electronic sound are coordinated, even though the melodic and scalar material is being generated on the fly. We’re all keeping a common pulse, playing from the same scales, from the same pool of melodies, with the same density of musical material, all converging to the same common tones and modulating together. The result is that musical material is passed all around the room, allowing for a nice, mid-level coherence that keeps the piece from sounding too random or arbitrary. Accomplishing this kind of coordination is difficult in a traditional open form piece, where the musical material is written out in advance, and even harder to accomplish when there are real-time processes generating new material all the time.
So the centerpiece of this new work was a system for disseminating algorithmically generated melodic content to acoustic performers. This was accomplished by means of a Jitter patch I wrote that displays the notes to play in a scrolling musical notation, similar to Guitar Hero, but using traditional notation, scrolling right to left. Using a computer display to guide acoustic performers is something I’ve been mulling over for years; it’s always seemed a clear opportunity to apply design concepts from video games to issues of open form classical composition. My first practical investigation was when I performed Christian Marclay’s video piece Screen Play the Shanghai eArts Festival in 2008 (you can read about that experience in greater detail here). I fleshed out this idea further in my Zhujiajiao Drinking Game (2009), which provides indications to performers when they should blow on beer bottles (to produce sound), and when they should drink from them (to change the pitch). Traditional music notation was a logical next step, and I have plans to expand and apply this system to some piano studies as well as larger ensemble pieces.
Ten channels of real-time, coordinated sound is a new milestone for me. Previously, six channels was my maximum, in my installations Breaking New Ground (2008) and Kaleidoscope Music (2009, and coming soon to Axiom Center in Boston). Well, technically, under the hood, my Self-Portrait, Dusk, at the Point of Departure (2010) incorporates 36 channels of audio, but it all gets mixed down to 4 channels before being sent out the speakers. The exciting thing about having ten channels of sound is that you can really start to articulate a spatial texture, where a listener is not parsing individual signals coming from specific points in space, but there’s a spatial density that emerges, a real sense of depth, kind of like what I was exploring in my 18 channel video piece.
This idea of coordinated multiplicity is really important to my work. In this piece the electronic sound was generated by very basic synthesis, triangle waves for the most part. I chose them because the timbres blended well with the reedy tones of the accordion and khaen; the overtones would sometimes fuse into a larger aggregate sound, but then fracture off into different points in space. If you’re wondering, we used the Anchor AN-1000X speakers, which worked well for this piece in a gallery setting.
As the title suggest, Mobile 4 is the fourth in a series of pieces exploring mobile form. The title refers to the mobile sculptures of Alexander Calder, an important touchstone for my video game audio work. A system like the music engine I designed for EndWar is analogous to a mobile sculpture, in that the individual elements of the composition are fixed (the short snippets of recorded phrases), but the relationship between them is in constant flux. The first of my Mobile pieces was composed in 2004 for flute, cello, piano, and soprano, premiered by the Ensemble Sorelle at the Seattle Art Museum. The second piece is very open ended, for any number of instruments, based on the cries of street vendors who used to pass through my first Shanghai neighborhood. Mobile 3 was premiered at the 2Pi Festival in Hangzhou, a laptop composition based on recordings of traditional Chinese percussion instruments plus electric guitar. In fact, these days most of my pieces contain some element of mobile structure, and I’m not particularly strict about which pieces earn the “Mobile” moniker; I thought long and hard about whether my Zhujiajiao Drinking Game should be titled Mobile 4, but in the end I decided that the social game aspects of that piece were more predominant.
* * * * *
In the process of preparing my performance/installation, I had ample opportunity to explore SDMA, located in the heart of San Diego’s gorgeous Balboa Park (which I’m told is the largest urban park in the US, including several museums in addition to the SDMA, the San Diego Zoo, the world’s largest outdoor pipe organ, and a lovely cactus garden), and I’ve become quite a fan! In addition to the pieces mentioned above, I found Samuel van Hoogstraten’s Feigned Letter Rack with Writing Implements particularly revelatory, a seventeenth century Dutch work with a flatness and painterly self-awareness that to me seemed to presage cubism and Magritte (respectively). It also brought to mind this amazing sequence from Andrei Tarkovsky’s Stalker.
Another work that caught my eye was Jean Hélion’s Composition in Color (1934), evoking as it does the kinds of mobile structures I mentioned above. And speaking of Calder, there’s a great one near the entrance of the SDMA, which I took as an opportunity for a photo op.
I also really dug the Rubén Ortiz-Torres show, the big From El Greco to Dalí exhibition (where I made the happy acquaintance of Spanish Impressionist Joaquín Sorolla), and the work of Gustav Stickley, whose furniture exhibition provoked the question addressed by this year’s Summer Salon Series: “What does a city need?”
In between rehearsals, Alexander was my tireless tour guide to San Diego’s busy arts scene. We checked out Double Break, a new art gallery and shop, not far from Balboa Park. I also got to meet super friendly and passionate Jfre from Disclosed Unlocation (we enjoyed a long, whiskey-fueled rap, together with my friend Ellen, closing down the charming dive bar Nunu’s following my performance), David from Agit Prop, and the busy folks at SD Space 4 Art (a live/work space where we got to check out a dance rehearsal in progress). There’s a lot of great energy in San Diego’s arts community, and it felt fantastic to be part of it, if only for one fleeting week. And everyone with whom I spoke was full of praise for the enthusiasm and imagination that Alexander brings to the scene, through the Summer Salon Series as well as his other diverse efforts. I heartily join my voice to the throng!
I also popped in (on Alexander’s recommendation) to the San Diego Museum of Contemporary Art’s aptly titled Phenomenal show, featuring works exploring light and perception by James Turrell, Robert Irwin, Craig Kauffman, Mary Corse, and Larry Bell. And I was totally floored by Jennifer Steinkamp’s absolutely jaw-droppingly wondrous Madame Curie, a huge, seven-channel algorithmic (pre-rendered) video installation, commissioned by MCASD specifically for their space, a tour de force, impeccably executed. I dig it for its formal beauty, the layering of the digital branches and their gentle algorithmic swaying, the fixed perspective that allows for the evocation of a larger virtual space, the sheer scale of the thing, all in addition to the sly and ominous allusion to Marie Curie’s research into radiation. Digital video done right!
To round out my visit, I made my first visit to the San Diego Zoo since I was quite young. Having been on safari numerous times on visits to my parents in Kenya, I couldn’t muster much enthusiasm for large animals in contained spaces, but I really enjoyed the aviaries, and kept thinking of Messiaen notating birdsong. I also really enjoyed the lemurs.
While I was in town, it was great to meet up with my high school pal Ellen, who I met at show choir camp near Estes Park, CO, in the summer of 1991. For the ten or so people in the world who have my Titled Untitled cassette from 1993, the song “Missing Ellen” was written for her (read more about such juvenilia here). We seem to meet up every five years or so like clockwork; the previous visit had been in San Francisco during the Game Developers Conference in 2003 or so, until we met for dinner in Boston last month, and suddenly three weeks later our paths crossed again in San Diego! I’m grateful to her for showing me around Seaport Village and Old Town and generously driving me to/from the airport.
The fantastic last day of my visit ended with a visit to the house of a friend of Alexander’s on Mission Beach, frisbee on the sand, a swim in the chilly Pacific Ocean at dusk, a roller coaster ride, a burrito, and the fireworks from Sea World over Mission Bay, followed up by a bonus second dinner of Vietnamese food. Can’t wait to go back!
* * * * *
A big thank you to Alexander, Brittany, Greg, and everyone else at the museum for their help and support with this project. Also a big shout out to Ferino’s Music for repairing my accordion, severely damaged in transit, in time for the gig. If US Airways ever assures you that their Gate Valet service is safe for musical instruments, don’t you believe them!
As previously posted, I had the pleasure of doing some shows with my friend Yan Jun 颜峻 from Beijing recently. As I was digging up some links to introduce him to Boston area friends, I found this clip of a performance we did, together with Beijing-based Bruce Gremo, at the 2008 Shanghai eArts Festival (shot by Amsterdam-based artist Emile Zile, who I met after the concert; read his account here). We performed at the gracious invitation of Defne Ayas and Davide Quadrio of Arthub Asia, who curated the Final Cut section of the eArts Festival, which transpired in and around Xujiahui Park in Shanghai and also featured performances by B6, Aaajiao, Feng Mengbo 冯梦波, Dead J, Alizia Borsari, and Elliott Sharp, among others (Elliott Sharp is featured in the second half of this clip).
Here we’re performing a video score by Christian Marclay entitled Screen Play, which consists of a bunch of black and white footage from old movies, mostly evocative of sound in some way (telephones, slamming doors, ocean waves, dancers, phonograph records, etc.), overlaid with simple, abstract shapes in bright colors. The piece is about half an hour long. There are no clear indications how the score should be interpreted; rather, it serves as an inspiration, a framework for improvisation.
As I watch this clip now, my first reaction is, “Wow, it worked!” It’s become something of an established practice to do these kinds of live, improvised accompaniments to new video or old films, but in my observation, there’s one problem inherent in the format: synchronization. No matter how skilled the performer, it takes a certain amount of time to register changes in the video and interpret them as sound. So in preparing for this performance, I specifically set myself the task of finding a solution, and reviewing our work two and a half years later, I’m pretty pleased with the results.
Synchronization requires anticipation. This was one of my primary lessons when I studied conducting back at St. Olaf. In 4/4 time, if you want the orchestra to come in loud on the one, you need to make a big gesture on four of the previous measure; you need to stay a beat ahead. In traditional music notation, sound is represented on a grid in which the x axis is time and the y axis is pitch, so it’s easy to peek ahead on the timeline. Or in waveform representations, x is time and y is amplitude. But a video, unlike a graphic representation of sound on a page, is a time-based medium, and x and y can’t help you; time is time! There’s no way to look ahead and prepare for what’s coming next.
Christian, Bruce, and Yan Jun
To address this issue, I took a tip from some of my favorite videogames, Frequency, Amplitude, Guitar Hero, and Rock Band, developed by Harmonix Music Systems (just up the road here in Cambridge, MA, in fact; I just gave a talk there last month). In these games, as I imagine anyone reading this is already well aware, notes are represented by little colored dots coming towards you on the screen, and when they get to a certain point towards the bottom of the screen, you hit a button on your controller to sound the note. Notes are coming at you on a timeline, so it’s easy to look ahead and prepare for new notes to come, just like in traditional sheet music. This is a true video score.
To approximate this kind of prescience in Christian Marclay’s piece, I wrote a Jitter patch (the first time I used Jitter for a project, in fact) that plays back the movie in 4 separate windows, each window offset by one second. So I was able to see every event in the film coming from three seconds away and count down to activation: 3-2-1-play!
My Screen Play Jitter Patch
My Screen Play Jitter Patch
The window in my Jitter patch that displays the largest image (the actual current time) also doubled as my control surface for generating sound. To play along with the video, I was literally drawing on it with my mouse. The timbres I was playing employed very simple synthesis techniques, lots of bright cross modulation, and a bit of granular synthesis. The idea was that my buzzy tones would correspond to the bright, abstract graphics in the score, whereas Bruce (performing on his amazing homemade digital flute controller, the Cilia) would evoke the representational black and white clips, and Yan Jun (working with lo-fi electronics and voice) was more of a moody glue, bridging the two worlds.
I’m a big fan of Christian Marclay. His solo show at the Seattle Art Museum in 2004 is one of the best exhibitions I’ve ever seen, a fecund amalgamation of wit, inventiveness, historical awareness. He represents the full range of what a sound artist can be. He makes sound, of course, in performances, recordings, and installations. But he also makes silent pieces about sound, or about the ephemera surrounding contemporary sound production, and he also makes video pieces that suggest the contours of sound in another medium.
This playfulness is evident in Screen Play in the choice of images, their clever editing, and their relationship to the abstract graphics. He’s clearly toying with different ideas of sonic representation in the way these graphics are deployed, at times stretching five lines across the screen to evoke a music staff, at times drawing a waveform as on an oscilloscope, at times merging into the underlying scene (as when a bright yellow ball becomes attached to a man’s spoon as he’s slurping soup).
Jitter Patch in Action
I realize that for Christian Marclay, this synchronization issue is probably not a problem at all. Screen Play was conceived for the kind of freely improvising musician exemplified by downtown New Yorkers like Elliott Sharp. For a certain type of resourceful performer, the video is a way to nudge the music along, to create an overall contour and form that may not have otherwise emerged, and which provides the potential for greater large scale contrast and recurrence than an unmediated free improv jam. It’s kind of like a time-based version of a graphic score, such as Earle Brown’s December 1952, Cornelius Cardew’s Treatise, or Christian Wolff’s Edges.
However, there are a few sudden transitions in Screen Play, in particular a sequence that alternates a slamming door with a full screen of saturated color, that strike me as contrary to this ethos. That bit, and a few others like it, seem to call out for big contrasts and tight synchronization, and I think at these moments one could legitimately criticize the score for setting up an expectation that the performer cannot fulfill. But I’m happy to report that, by applying a simple technique from videogame design, we nonetheless nailed it.
Using my Jitter patch to perform this score actually felt a lot like playing a videogame. It gets at what I consider to be the heart of gaming: to develop a skill and apply it to new challenges. This aspect of gaming is very much like performing music; from a certain point of view, any musical performance can be considered a game. I’d estimate that this modified approach to performing Screen Play lies somewhere near the midpoint between downtown New York free improvisation and Guitar Hero, and I think there’s a lot more interesting work to be done along this continuum.
On the One!
Thanks to Defne Ayas and Mathieu Borysevicz, I think, for the pics. And thanks to Arthub Asia for the invitation to participate!
At PechaKucha Boston earlier this week I presented the US premiere of my Statement of Purpose. I primarily think of it as a composition, but you might also call it a performance piece. I suppose “multimedia lecture” might be most accurate. It was written in September 2008 for presentation at PechaKucha Shanghai and thus adheres to the PechaKucha format: 20 slides of 20 seconds each. In Boston as in Shanghai, I think it seemed to go over pretty well.
Statement of Purpose is consciously indebted to John Cage’s Lecture on Nothing, one of the pieces collected in his book Silence, which I first read many years ago. The thing that struck me about these early Cage lectures and essays is that, in many cases, the message is in the form of the work, not the content. Rather than just describing his ideas about rhythmic structure, he demonstrates them; you experience them directly.
So in my piece, which I have described as an update of Lecture on Nothing for the digital era, I’ve adopted a mobile structure, as opposed to a linear lecture format. I take great inspiration from Alexander Calder, because in his mobiles, the individual elements are fixed, but the relationships between them are in constant flux. So here my lecture is arranged topically, around nodes of ideas. The main idea-nodes are
Aspects of Music and Audition
Stasis in Sound
Dynamism and Interactivity
The Current State of Videogames
The Nature of Multimedia
On each of these topics, I wrote a bunch (around six to ten, I think) of one sentence statements, single ideas that could be presented in any order. Then I wrote a program that generates a script by randomly picking one of these idea-nodes, picking some of the ideas associated with it, picking another idea-node, etc. Pauses are added between each statement to vary the density of the lecture over time (using a random walk, aka a “drunk” function or brown noise), in the same way that a tide or a rainstorm has a changing contour over time. Indications about when to clear my throat, gesture to the screen, take a swig of beer, etc. are also algorithmically scattered throughout the script, as a kind of textural element, subverting the ephemera of a typical lecture scenario.
I also interspersed a purely musical element, consisting of a set of low drones plus a set of brief melodies in a higher register (outlining an A mixolydian scale) all sung on a textless “ooh.” It’s pretty arbitrary; I thought the piece could use it, and I like the texture that results. But it also serves to focus attention on the abstract structure of the piece, rather than the content, and to suggest that the piece as a whole may be considered in musical terms.
There’s another type of behavior, too, statistically less likely to occur. While 16 of the 20 slides use the above formulations, the remaining 4 are shuffled riffs on standard salutations and closing statements: “Hello,” “Good evening,” “My name is Ben Houge,” “Thank you for your attention,” “Good night,” etc. The idea is that through repetition and dislocation, these phrases become formal (rather than syntactical) elements; it’s very similar to what I’m doing with radio broadcasts in Radiospace. Having another type of behavior helps vary and articulate the overall form. I also just think it’s funny, and I sensed that the audience was similarly amused. Humor is like music, in that it plays with audience expectations, as when I end my piece with a cordial, “Hello, everyone.”
The slides were generated using very similar techniques to those I employed in my 29 Giraffes series, but substituting text for little chunks of photographs. The colors, in fact, are algorithmically extracted from the same Nanjing Dong Lu source material I used in my Giraffes. Here again, the emphasis is more on the texture that emerges from all this superimposed text, rather than on the text itself; as with the algorithmically generated script, the slides communicate through form, rather than content.
The whole piece has an audio accompaniment, too, one 20 second audio clip per slide. To create this backdrop, I processed a recording of myself reading the text of the piece using a bunch of custom software I had lying around at the time, programs I had developed for other pieces. You can identify bits of Psalmus, Study for Eventual World Domination (my contribution to The Bike Bin Project), Radiospace, and a granular synthesis demo I did as a videogame audio engine prototype. Looking back, the evocations of these pieces that crop up (as of the Giraffes) provide a nice snapshot of my digital workspace in September 2008, which was part of the idea.
To assemble all of these elements, I selected the 20 slides I wanted to use of the many I had generated, then I wrote a program to shuffle them. Same for the 20-second audio segments I generated. In the end, it’s a combination of arbitrary decisions and procedurally generated bits, which is really how just about any artwork comes together, digital or otherwise.
The result is that ideas come and go, freely floating. I’ve referred to a lot of my pieces as “meditations,” and the term is certainly apt here. Ideas recur, sometimes in different media (text from the slides may pop up again in the spoken presentation or recorded backdrop). They “interpenetrate,” to use one of John Cage’s favorite terms. They reinforce each other, and they add up to a way of thinking, which is very much my way of thinking, a network, a web of ideas, all connected.
It’s a good time for me to revisit this piece. Especially in the past year, I’ve been thinking a lot about the meaning that can be conveyed through pure structure. I think this has come to the fore as I’ve been increasingly active in visual media. In music, we take this for granted; you could say that music traditionally conveys meaning through structure alone. Music is the most abstract of the arts; representation or mimesis in the pre-recording era was by far the exception (think of the timpani evoking thunder in the “Scène aux champs” of Berlioz’s Symphonie Fantastique). In some cases you can say what a piece is “about,” because you know something of the circumstances of its composition, or because of a prefatory note by the composer. But principally, music’s meaning is all in the relationships of different frequencies, rhythms, velocities, timbres, etc., and, more importantly, the calculus of how this all changes over time. You would never say that even as abstract a composition as The Art of the Fugue is meaningless.
So coming, as I do, from a background in music, it’s only natural for me to approach my visual art in the same way, applying the same types of structures that I use in my sound work to visual information, and it’s been surprising to see how the conversation unfolds differently. A prominent arts person (don’t worry, no one you know) came to see my show in Suzhou last fall, and I was kind of amazed when she asked me what my piece was trying to convey. A musician would never think to ask such a question. As Elvis Costello said, if I could have written the song with any other words than the words I used in the song, I would have written a different song, wouldn’t I?
Of course there was a bit of a conscious impulse to poke a hole in the sometimes punctilious proceedings of a standard PechaKucha event (I have my Seattle School cohorts to thank for any vestigial confrontational aesthetic). As when I sneakily built an ambient electronic piece from mildly acrimonious pre-show chatter at Opensound a few weeks ago, I like the idea of snapping people into a different state of awareness with some new or unexpected realization. I also like the pacing of it; PechaKucha is usually about people cramming as much as they can into their 6:40, but my script actually includes indications to pause for as long as 10 seconds. But both times I’ve presented this work, the audience seemed to get it and dig it; it’s not just some avant-garde stunt. The message was conveyed.
Statement of Purpose was my first project after leaving Ubisoft at the end of August 2008. The deadline was tight, less than a month, as I recall, and I liked the idea of doing a new piece completely from scratch to emphasize my new trajectory as a full time, independent artist. I remember staying up all night to get it done, with an urgency that had been missing from my corporate gig for quite a while. I consciously wanted to make a statement about the main issues I was setting out to address in my work, my mission, as I considered it (and still do). Check out some documentation from that performance, and a video excerpt below.
I originally wanted to generate my slides and script in real-time using custom software, which I feel is technically still in keeping with the PechaKucha format, but in Boston as well as in Shanghai, the organizers very understandably wanted to stick to a standard set-up for all speakers. This is still something I want to explore, though, particularly the idea of giving cues to a performer on the fly, exploring the idea of real-time score generation (which is exactly what happens in a music videogame like Guitar Hero, and which I’ve already started to explore in pieces like my Zhujiajiao Drinking Game, more commonly referred to as Beer Hero). I’ve been contemplating a revision of this piece for a long time, to include this real-time score idea, write some new modules, add some Chinese text, incorporate multiple screens of real-time generated imagery, and blow past the 6:40 PechaKucha time limit to create a full, hour-long presentation. If anyone would like to sponsor and/or host such an event, please let me know!
Two super exciting bits of news about my Shanghai Traces video!
I’m pleased to announce that Shanghai Traces has made the shortlist for YouTube Play, the Guggenheim’s new Biennial for Creative Video. Here’s the full press release. The entire YouTube Play shortlist is on display at http://www.youtube.com/play (keep an eye out for AleaBoy!), as well as at kiosks in the Guggenheim Museums in New York, Berlin, Bilbao, and Venice, through October 21.
I also just realized that I am already at liberty to announce that Shanghai Traces has been selected to be screened at Seattle’s e4c Gallery early next year! Check out their announcement. I’m going to adapt the piece to run across four monitors at 4Culture’s innovative downtown storefront gallery for digital art, and once it’s up, it will be in rotation for a full year! I’m also planning some Seattle performances around that time; when it’s all nailed down, you’ll be the first to know.
Here’s the video in question:
(Read more about the genesis of Shanghai Traceshere.)
The Guggenheim says they received over 23,000 entries from 91 countries for YouTube Play, which they eventually narrowed down to 125 for the shortlist (and, yes, they promise they watched them all). For more info, be sure to check out YouTube Play’s companion blog The Take.
The next step is adjudication by a celebrity panel comprised of Laurie Anderson (a longtime hero of mine), Animal Collective, Darren Aronofsky (I hope he digs up my glowing twitter review of The Fountain from a year or two ago), Douglas Gordon, Ryan McGinley (whose work I just saw at UCCA in Beijing a few months ago), Marilyn Minter, Takashi Murakami, Shirin Neshat, Stefan Sagmeister, Apichatpong Weerasethakul (who film Uncle Boonmee Who Can Recall His Past Lives just won the Palme d’Or at Cannes; can’t wait to see it!), and Nancy Spector from the Guggenheim. The jury will select up to 20 of their faves to be presented at a special event at the Guggenheim New York on Oct. 21.
I’m particularly stoked about the prospect of Laurie Anderson spending a few minutes getting to know my work, after all the time I’ve spent getting to know hers. I attribute her, in a roundabout way, to connecting in my brain the world of classical music that I was studying in college with the world of pop music to which I’d been listening (and which I’d been writing) growing up. She was also the reason for my only visit to New York so far, to catch Songs and Stories from Moby Dick at BAM in 1999. Though I wonder how she’s going to find time for all this adjudication with her new performance piece in full swing.
Anyway, wish me luck!
And of course any day now Shanghai Traces should be up at Glamour Bar on Shanghai’s historic riverfront. Since they wanted to show it on a big 42” screen, I obligingly created a high resolution version of the piece, which you can view below (click the four arrows icon in the lower right corner to go full screen).
This piece was first exhibited as part of the “Re-Visioning History” show that opened on May 22, 2010, at OV Gallery in Shanghai. Less than a week later, representatives of the Cultural Bureau came in and shut down the show, walking off with a print from my video.
It’s not clear why my work was singled out for confiscation. The show was centered around the work of Zhang Dali 张大力 and Ren Hong 任虹, with several other artists (myself included) invited to present new work based on a collection of historical propaganda posters on loan from Madame Mao’s Dowry. Most analysts agree, and I wouldn’t argue, that my work was probably the least politically oriented in the show. It may simply have come down to my work being most portable and closest to the door. Oddly, they only took one of the two prints I made from the video, ignoring the video itself.
The principle objection seems to have been to the work of Zhang Dali. His work was drawn from a 7-year project called “A Second History,” in which he used his connections to gain access to the national archives in Beijing. He managed to locate the original, historical photographs that were used as the basis for various propaganda posters, which he then incorporated into work that presented both versions of history side by side, in digital prints, silk screens, and paintings. He had already exhibited this work several times without major incident, most recently at the Guangdong Museum of Art, though the rumor is that after the Guangdong show he was pegged as someone to keep an eye on. The most plausible explanation I’ve heard for the OV Gallery show’s closure is that the powers-that-be didn’t want to come across as enemies of culture by officially censoring the show (especially with heightened foreign attention during the Shanghai Expo), so instead they got the gallery on technicalities like selling catalogues and exhibiting foreign artists without a license. So it in all likelihood had nothing to do with my art (which does not mean I’m not mentioning it in every grant application I write from this point forward).
And then suddenly on June 22, one month after the original opening, the gallery was permitted to reopen. They even returned my piece, which is actually a bit disappointing; I kind of liked the idea of it hanging over some Cultural Bureau functionary’s desk somewhere. There was a little reopening party on June 26, and the show’s run has been extended through August 5, to make up for the period that the gallery was closed.
All of the hoopla surrounding my piece’s confiscation and the gallery closure has somewhat deflected attention from the work itself, with which, in fact, I am quite pleased.
My video employs custom computer software (developed in Jitter) to algorithmically emulate this process. The program excises sections of the original propaganda poster and pastes them onto a new digital canvas in constantly varying configurations. The composition unfolds in six overlapping “phrases” of about one to two minutes, each of which define an area to be statistically filled with snippets of the original image in varying sizes and densities. Sometimes the program focuses on one part of the source image, resulting in a consistent shape or repeated gesture. Sometimes the differences are greater: a small detail may be enlarged, or an image may be reduced to a texture or color. As new images are overlaid, the foreground is constantly receding into the background; the present forms a canvas for the future.
As with Shanghai Traces, I feel this is a particularly good pairing of subject and medium. The theme of the show, and the subject of this video, is the process of the present becoming the past, forming history. The medium of generative video provides an apt opportunity to evoke this process by means of another process. The video explores the unique properties of the digital medium; you couldn’t obtain the same results (the repetition of images in varying sizes, scales, and degrees of fadedness, not to mention the systematic evolution over time) using traditional paper collage or any other medium.
The video explores modes of propaganda. Slogans on posters, plastered in profusion, represent perhaps the most common form of propaganda: persuasion through sheer repetition, with no attempt at a reasoned argument. You see that happening in this piece, as the program tends to pick from roughly the same area of the source image for stretches at a time. But while working on this piece, I was also reviewing the work of the Russian Constructivist and Suprematist artists of the early 20th century, who were using pure, abstract forms to convey ideas about relationships and society: structural propaganda, still very much intended to alter society. The blocks of images that comprise my video at times resemble some sketches of Kasimir Malevich. The fragmentation of images in my work can at times feel violent as figures are chopped in half and hand-holding friends are sundered. On the other hand, one figure from the source poster can be cloned indefinitely to form an anonymous crowd or disintegrate into an abstract texture.
I don’t deny it: this is a long excerpt to post on Vimeo. But I also wanted to showcase the large scale ebb and flow of the piece. The pacing of this work is slower than other works of mine, such as Shanghai Traces, and things can take longer to come into focus. The viewer must constantly reevaluate the evolving, emergent structure of the composition. There’s a constant fluctuation between foreground and background as the images slowly fade out, and the rate of fade itself is constantly varying. Independent elements may suddenly coalesce into a balanced structure, only to be ruptured by some new element, which may seem out of place until it becomes a key component of some new structure, or it may simply get covered up and forgotten. The eyes and brain are constantly popping between phases of meaning and order, continuously addressing what to me is one of the most important and fundamental structural questions (something I grapple with in audio as well as visual pieces): “What makes things the same, and what makes them different?”
This is not to say you have to watch the video for a long time to appreciate it; part of the point of the piece is that its generative nature allows you to make your own beginning and ending as you enter or leave the installation, so feel free to start the video somewhere in the middle and watch for as long as you want. The piece also works in an excerpt as brief as a single frame, as you can see in this Flickr gallery.
The video was originally exhibited without sound. For this excerpt, I’ve added some ambient light traffic, recorded from my rooftop studio at True Color Museum, Suzhou, China, where I’m doing an artist residency through the end of July 2010.
BTW, if you read Chinese, you’ll spot some odd characters popping in from time to time. These are from the second batch of simplified Chinese characters (known as “二间”) that were announced in 1977 and then rescinded in 1986, which indicates a time frame in which this poster was originally published. 皃 has since been reverted to its original form 貌, 乙+心 is now 意, 尸+一 is now 展, etc. 片 is also now written slightly differently. For an exhaustive accounting of which characters were changed when, check out this site!
Hey, wow, new video! This piece is a bit quirky and personal, so I should probably fill in a bit of context. But first, imagine that you are viewing this piece on a huge bank of 24 TV screens, the sole light source in a huge, black warehouse, which is how I would ideally like to present it. [Note that you can turn HD on/off in the video below; it will load faster with HD off, but if you’re up for it, turn HD on, click the icon to the right of the play bar to make it full screen, and turn scaling off.]
I find myself constantly refuting the notion that art made with computers is somehow cold, impersonal, rational, unfeeling, etc., etc. In general, I refute the idea of absolutes in art, that a work must be, for example, either rational or emotional. In my work, both elements are present, and this one swings perhaps farther than most to the emotional side.
All art (including digital art) has some kind of inspiration, and in this case I was inspired during my trip to St. Paul, MN, last winter by the intricate patterns formed by barren tree branches, and how those patterns would shift with just the slightest change in perspective or movement of the branches. I think the first time I consciously started paying attention to tree branch patterns, I was looking out the window of Famous Dave’s on 7th in St. Paul, where I was having lunch with my parents, my brother, my sister-in-law, and my two little nieces. Later I noticed that the same kinds of patterns were occurring right outside my brother’s living room window. I spent a lot of time, last winter in St. Paul, sitting in the stuffed chair of my brother’s living room, working on my computer, opposite this window (to the point that the chair came to be referred to as “Uncle Ben’s office”), and as I gazed at the branches outside, I kind of started to identify a bit with this tangled mess of branches and what they might represent.
I was working on a couple of video projects during my two months in St. Paul, notably Shanghai Traces, and also collecting source material for my foolhardedly ambitious plan to produce backdrop videos for my live pop show. I really wanted to capture some of the unique topographical features of winter in St. Paul (i.e., snow), but I could never seem to find just the right combination of meteorological conditions and presence of mind to go out and actually tape them. So in the end I spent the last 10 minutes of my St. Paul visit standing in my brother’s snow-covered front lawn, videotaping those branches as the sun was setting, just before I hugged everyone goodbye and my brother drove me to the airport.
So my new video installation takes those ten minutes and makes them last forever. A lot of still art can be said to freeze a moment in time, but that’s not the same thing as prolonging a moment indefinitely. In a photograph, for example, whatever was happening at the moment when the photograph was taken is not happening anymore; it’s been stopped. But here, the moment is still happening, and it will never stop happening. It’s not the same thing as looping a video segment, either. In a loop, it would happen repeatedly, which is not the same thing as happening continuously. As in Shanghai Traces, I think this is a really good pairing of subject and medium.
How is this miraculous feat accomplished? By using the same techniques I’ve developed to make sound continue indefinitely in videogames over the past thirteen years or so: shuffling, staggering, offsetting, layering. These techniques are some of the most fundamental in my toolbox, but they’re endlessly applicable to a wide range of real-time organizational challenges. In this piece, each of the 24 screens is independently picking a section of the video to play for a certain amount of time, then picking a new section to play, and so on. The duration and position in the original video are not completely random, but constrained by previous behavior, so that the overall distribution of images across all 24 channels is constantly shifting. It’s very similar to the granular synthesis techniques I’ve used in my audio works, mixing together little chunks of a larger sound to kind of homogenize it into a steady texture (see the sustained textures in Radiospace for a good example).
As is quite obvious, the original video was shot without a tripod, which gives the piece a performative element (not that the world needs another flimsy performance video document). The unsteadiness in my hand as I’m holding the camera is the other subject of the piece, creating motion and the subtle changes of perspective that (in addition to passing breezes) animate the primary subject matter. It focuses attention back on the person holding the camera and the minor endurance test of holding the camera still for 10 minutes in below freezing weather. This idea of endurance echoes comments by Richard Karpen and Mike Min (that the drama of a performance arises from the struggle of a person pushing against his or her limitations). In other words, the motion of the camera in the video is a visualization of my own failure to hold it still, despite my best efforts, which you are free to view as a metaphor for the attempt to hold back time itself.
The end result displays all kinds of interesting formal and textural qualities, byproducts of the same behavior being multiplied across 24 screens. The original video was shot at dusk, so there’s a gradual transition from yellow to blue hues; as my piece runs, the various screens are constantly changing their position along the spectrum, forming new groups and contrasts. The motion of the different screens prompts a different organizational tendency, a kind of counterpoint, sometimes seeming to move together, at other times in contrary motion. When screens pop to a new image, a rhythmic texture emerges as well. The eye is drawn to the sudden popping of a screen to a new point in the source video, but because the new image shares the same perspective as the previous one, it can create a kind of paradox; you know something’s changed, but you’re not sure what. The eye and brain are constantly engaged (although on this small video rendering it may be hard to tell; again, think of a big bank of TV monitors), as the viewer is constantly challenged to re-evaluate what’s the same and what’s different as groups form and dissolve.
The audio for the piece is basically just the audio from each of the 24 individual screens mixed together. It happens to include the sounds of several different transportation mechanisms, which nicely underscores the idea of imminent departure. Occasionally you’ll notice the audio cutting out or in at the same time as one of the screens popping to a new image, reinforcing the structure of the piece. I wasn’t completely happy with the sound I captured on the camera’s little built in microphones, so I wanted to filter it a bit, and once I got into filtering, I really liked the mood I got by notching certain harmonic sets of frequencies. But I also really liked the neutrality of the unfiltered sound, and I couldn’t decide if this was too much meddling or not, so in the end I have it both ways, with the notch filters algorithmically fading in and out. The filters’ base frequency changes at longer intervals, which gives the piece a higher level structure and periodically refreshes the ears by establishing a new tonal center. For a public installation, I would revisit the filtering behavior; ideally, if I could present this piece in the big warehouse I’m dreaming of, I’d tune the piece to the room’s resonances.
I have no idea when I’ll actually have a chance to mount this as a public installation. Ideally, it should be displayed on a big bank of 24 TV screens mounted in an 8 x 3 array in a huge, dark, empty space. (If it strikes you that such a bank of TV’s would resemble the banks of monitors displaying airline departure times at an airport, you might be interested to know that in fact I did the first draft of this software while waiting overnight at Dulles International Airport in Washington, DC, for my connecting flight to Nairobi last January 29, seated across from just such a bank of monitors.) I like the idea of encountering it first from a distance, the images gradually coming into focus as you approach, with the bank of screens generating the only light in a room so big and dark you can’t see the walls. There should be a bench in front of the screens, or pillows, so people can hang out for a while, or maybe some stuffed armchairs, like at my brother’s place!
This is kind of the worst possible combination for a digital installation: expensive, but subtle. Typically, if someone invests in a big, 24-screen video wall, I guess they want something big and flashy, not quiet and contemplative like this. But if anyone would like to be the first to present it, that honor is yours for the taking!
I’d like to dedicate this piece to the Minnesota Houges, with love and gratitude.