Hey, look at this thing I made:
This is what I spent the last 4 days doing, instead of everything else I should have been doing instead. I’m calling it “Study for Insomnia.”
It began as a demo for artist Cindy Ng 吴少英, whose video work is presented in the same room as my 路口 sound installation at Art+Shanghai (you have until November 1 to see the show, so hurry!). She was asking how my piece was constructed (audio “phrases” strung together and overlapped in real-time by a computer, providing constant variation), and I suggested that it would be possible to use the same technique to deploy her video work. I made a quick demo to show her last weekend before we collaborated on one of her “Ink Walk” performances, and she was intrigued to the point that we’re now planning a full-on collaboration. I’ve continued to flesh out my little demo over the last few days into something I’m pretty happy with, and it ought to serve as a pretty good template for whatever we end up doing together.
Like most of my electronic pieces, there’s no pure synthesis going on; everything is manipulated from source recordings, in this case, video and audio recordings of rumpled bed sheets (yes, the inspiration came while lying awake a few nights ago). The unpredictable patterns that result are similar to other kinds of natural phenomena from which I draw inspiration (e.g., rain, fireworks, traffic), chaotic on a small scale, but consistent on a large scale. Like my other installation work, the piece can continue indefinitely, with no beginning or end; here, for convenience, I’ve recorded only a brief excerpt.
I guess you could call this my first video piece. I’ve been using Jitter, the video-processing add-on for Max/MSP for over a year, although this is the first time I’m presenting the results publicly. The first time I used it was in the software I developed for my performance (together with Yan Jun 颜峻 and Bruce Gremo) of Christian Marclay’s Screen Play at the Shanghai eArts Festival in October 2008, but it was used only to prepare cues for myself, not projected for the audience to see. (In fact, in the very first version of this little demo, I used a snippet from Screen Play as video source, a shot of rolling waves.) I’ve also used Jitter in some consulting I did for a videogame company last winter, analyzing audio signals to generate game levels.
I’m not sure yet if this will ultimately serve as the background texture of a bigger piece later on, or if it’s fine just as it is, as a super ambient standalone piece. I could see it working as a backdrop for a live performance. Or it could work in a gallery setting (ideally with 4 channels of sound). For what it’s worth, in terms of scale and mood, the 4-day process of putting this program together felt very much like writing a pop song.
What’s happening in the program is pretty simple. I’m picking little bits of my original bed sheet video and slowing them down, rotating them, and fading in and out from black. There are four asynchronous layers of this activity happening at once. The speeds are different, and there are slight hue offsets for each phrase. I adjust the contrast, hue, and saturation, and I add some feedback. For each video phrase, I pick an audio phrase from my original wave file (recorded separately from the video), and fade in and out, in time with the video. I’m also analyzing the average brightness of each video stream and using that to control the pitch of a bank of filters that the audio runs through before hitting the speakers. For each phrase I pick a different pitch multiplier (over roughly 4 octaves of a just intonation minor scale, not that it matters). Hue, feedback, filter Q, cutoff frequency, and probably a few other things are slowly modulated by random offsets to keep things interesting. I found there are lots of settings that produce different outcomes, all acceptable, so I implemented the old John Cage dictum: “Say we have a problem and 100 solutions; instead of chosing just one of them, we use them all.”
I ran into a tricky little trigonometry problem while working on this piece. I had to figure out the zoom ratio while rotating the images so that I wouldn’t go off the image and add weird black edges to the composite. I pondered long and hard, and I think I was at least looking in the right direction, but I eventually had to enlist the help of my pal Micah Sittig, who teaches physics over at SMIC Private School, and to whose class I gave a little presentation on music acoustics last March. He solved the problem in about a minute. It’s good to have smart friends. Thanks, Micah!
Crazy coincidence: yesterday while working on this piece, I thought I’d take a break and listen to something new, and a CD of music by Esa-Pekka Salonen came to mind. I picked up this disk following one of the premiere performances of his fellow Finn Kaija Saariaho’s second opera Adriana Mater in Paris in 2006, but I had never listened to it. I feel like a CD, if I have any reason to believe it will be good, really ought to have a focused and concentrated first listening, and consequently, I have a huge stack of shame of CD’s I’ve purchased, but haven’t yet had the time to devote to a proper first listen. Especially with all the hoo-ha about the new music director Gustavo Dudamel taking Salonen’s place at the Los Angeles Philharmonic, I’ve been thinking lately I really ought to listen to it. So today I picked it up and unwrapped it, and to my amazement, the last piece is also called Insomnia! I’d already been planning to call this new piece of mine something about insomnia for a few days, so I swear I didn’t steal the idea, unless it was buried somewhere deep in my subconscious. Anyway, I just listened to it, and it’s really a really nice piece, though somewhat more detailed than mine. Isn’t that nuts?