Sound That Surrounds, A Reflection on Immersive Audio in Three Movements
Sound often plays second-fiddle in discussions about emerging media forms. But what happens when makers build experiences around it? In this dialogue, three fellows at MIT’s Open Doc Lab share insights about their current projects, how they build relationships between audio and the world that listeners are passing through, and the role of participation in their work.
Each of them has also provided “field notes” with more details on their projects. Learn more about Pan-terrestrial People’s Anthem, a project that combines compositions and poetry by Andrew Demirjian; an audio journey through the Anthropocene titled From Here to Where by Halsey Burgund, and A Father’s Lullaby by Rashin Fahandej, which cross-examines poetic and spatial justice using site-responsive public installations.
MOVEMENT I: Creation
> Jump to Movement II: Location or Movement III: Participation
Demirjian: Let’s start by talking about “immersion.” What does this mean in your work?
Burgund: I think about From Here to Where as a sort of dual immersion because you’re listening in a car and the car provides this little audio bubble, where you and your companions are all hearing the same stuff. But then there’s this skin that separates you from the outside. (Listen here)
So, you are immersed in this audio bubble traveling through an expansive visual landscape that you are also immersed in and connected to. These two layers of immersion — one audio and proximal and the other visual and expansive — are connected to each other along the entire journey.
This feels different to me than the VR type of immersion, which takes you away from the real world. I tend to be a fan of the kind of immersion which connects you to what’s around you. You’re looking at stuff, you’re smelling things, you’re feeling the wind, you’re experiencing the natural world. Then that’s augmented by this extra layer of audio which is not “real” in the sense that it’s not being generated by physical objects that are in your proximity but has been carefully designed to bring you closer to your physical environment.
Using an augmentation to connect rather than to segregate is something that I think a lot about, although there are challenges because the technology always ends up mediating in some way that pulls you away from the analog world.
The necessity to use the technology is something that I get frustrated with sometimes. It’s a double-edged sword that enables you to do these things but then piles requirement upon requirement onto you and your public, your listeners, your experiencers.
I try to just make it disappear — press play and then put your phone away — but even that is far from ideal. I would much prefer it if you could just walk naked through the environment with no technology whatsoever and experience the piece! I feel like that’s the ultimate level of accessibility. I think a lot about accessibility both in terms of socioeconomic accessibility and physical accessibility. Universal design is a high ideal but one that I find useful to consider throughout the creative process.
Demirjian: Rashin, Halsey says that he likes to use augmentation to connect people to nature, to their surroundings. In my piece, I’m using music to disconnect people from their belief systems related to the nation-state. Do you feel you are using sound to connect or disconnect listeners?
Fahandej: I am interested in the use of technology to bring attention back to our bodies, and intentionality to our actions — a kind of immersion that does not start and end with technology like a VR headset, which takes you away from your surroundings.
The experience at the installation site for A Father’s Lullaby is an encounter with the unexpected: the sound of lullabies sung by men coming from treetops across the public plaza. The melodies move with you in the space and persist in a subtle way, while fading in and out as you move through multiple sources of sound.
I am interested in using technology to create a new social memory of space. And the public space allows for accessibility and diverse voices to co-exist. Similar to Halsey, I try to create an immersion to give you pause and make you aware of your surroundings, blurring the boundaries of real and imagined.
My projects deal with social issues, constructs and systems. In this particular piece, I ask people to participate by thinking about memories of their childhood, tender relationships of love, and then to sing and record a lullaby. The project is about racial disparities in the system of incarceration in the United States, the absence of many fathers as a result, and the impact of that on children, women, and lower-income communities. The entry point is recalling our own deep personal experiences in order to connect us with a broader socio-political issue.
Similar to your project, Andrew, it invites audience members to disconnect from an established social belief system.
Demirjian: You’re thinking about making new spaces, new memories that you want to see. This is like making a new anthem for a future you want to see. How can sound be used to make a future memory or create an aspirational place?
Burgund: Andrew, you were talking earlier about wanting to in some ways disentangle your listener’s sense of connection to or dependence on feelings of nationalism.
It was interesting to listen to your pieces because there is obviously a huge entanglement in the creation of the piece since you’re taking audio snippets from different anthems and bringing them together. In some places, you are weaving together regal sounds of horns and drums and it all sounds very anthemic. But then there’s a voice piece that is so spooky!
That one is so haunting, so beautiful. The breathiness and close-miking is very personal and intimate. You can feel yourself right there with the singer. I don’t know if you sped that up or slowed it down, but the effect was really amazing. It connected me to that individual singer, and it connected me to all the other people out there who feel connected to their country, their heritage.
What’s fascinating about your piece is the two sides, the connecting and disconnecting at the same time.
Demirjian: It is so great to hear you talking about Ami Yamazaki’s vocal performance. When I present this work in talks it’s not a piece I tend to play because it’s so intimate. There’s something about listening to this with headphones on that makes it such a moving experience.
How do you create an anthem that emphasizes interdependence and not exclusion?
Burgund: What country was that?
Demirjian: Ami is singing her interpretation of one of the snowball poems in a book made from a bunch of national anthems. I’m not doing any time stretching, there is only a little delay and reverb. She was riffing on the phonemes and language of the constituent parts of the words in the lyrics. She was deconstructing them and pulling them apart and compressing the fragments together. It is her performance of a poem.
Burgund: So you’re like double remixing. You’re mixing the lyrics and she is remixing the remix and you’re mixing them again.
I don’t know national anthems other than my own. To what extent do you think that matters to experience your piece? Does being able to pick out a bit of the Latvian anthem and juxtapose that with the Hungarian anthem enhance the experience by creating some sort of intellectual connection between those countries?
Demirjian: I think what matters is knowing that it’s a remix of all national anthems; it’s not necessary for people to have a wide range of knowledge. It could be more like a kind of game to see if they can identify the source. It could add a different level…
Burgund: Rashin, you were born in Iran and grew up and spent a lot of your life over there before coming to America. How do you feel when you listen to something like this?
Fahandej: National anthems are a genre that I have little regard for. I’ve seen very thin lines between patriotism and aggression against those whom we consider others. They have been used to rage wars, to oppress and to exploit in the name of borderline and patriotism. The reexamination of the anthems to break their rigid boundaries and to redefine them gives them a beautiful poetry in Andrew’s piece.
Demirjian: What I was trying to get at with this piece is that the old way of thinking about nations as separate and competing for resources has to be changed. How do you create an anthem that emphasizes interdependence and not exclusion?
MOVEMENT II: LOCATION
> Jump to Movement I: Creation or Movement III: Participation
Demirjian: I want to ask you both about your process for composing for unique spatial situations. What was your process?
Fahandej: Creating sound for a public site is a vastly different process and outcome than a gallery setting, where variables are predictable. At a public site, uncontrollable elements will swing elbow-to-elbow with the artwork, shifting and changing the experience with every nudge. This co-presence and the emergence of artsound in public sites is at the core of A Father’s Lullaby.
There are two layers of sounds demanding different kinds of action and interaction. The sound of lullabies and singing projected from tree tops from multiple hubs across the public plaza can be heard if you are a passerby. If you come closer, your body triggers motion sensors connected to speakers in the raised garden level. The stories are fragmented portraits, audio documentaries of fathers on federal probation. This is the part that is not about transcendence but about attentive, active witnessing.
Soft voices find their way in the crowded noises of street corners, as if someone is calling you, singing for you. You have to be close to the sound source to hear the voices, and your next movement in the space triggers the next part of the story and memories to unfold. The experience for the listener happens in the form of discovery.
I am very curious about how these moments change and reshape our relationship to a particular public site. Even individual experiences can create a collective social memory of a site, and therefore create a shared experience that we carry with ourselves. Fathers told me that when they sing for their kids at home they think of this piece. A woman told me she called her elderly father after spending time at the installation site.
Burgund: I think sound can so quickly generate nostalgic feelings; smells, too. For me, visuals don’t tend to have the same effect of bringing up memories. I listen to a lot of podcasts while wandering around in my daily life and will frequently come to a street corner and remember something I heard while at that location in the past.
I love that idea of creating a memory pinned in my mind to a physical location. It lets you bring yourself back in time to a different moment where you first learned something or experienced something meaningful to you. For whatever reason, your brain decided to write the memory deep into its folds and burn in those connections between location and sound.
I’m a hundred percent positive, Rashin, that there are people out there who experienced your Boston Center for the Arts (BCA) piece who will walk by that same location after it’s gone and remember some meaningful, beautiful aspect of it and be brought back.
Fahandej: Thanks, Halsey. I think that’s in a way the beauty of sonic experiences: they hover and float from one space to another in our imagination, the boundaries are fluid. This allows for a dream-state experience, drifting between what is perceived as real and imagined, as current and past. I feel like visuals sometimes gets locked into a certain particularity that makes it hard to break away from prejudged notions.
Burgund: What’s interesting about sound is that it doesn’t have edges, right? There’s no frame like in a visual situation. Even in 360, there’s still the frame of your own ability to see. Your own biology imposes a frame whereas your ears have edges defined by distance but not by direction.
Obviously, you can’t hear many things from far away but the directionality of it being all encompassing is really exciting to me and makes sound so markedly different than sight.
A piece of music is sort of invisible until you let it play its course. It has to have time in order to exist.
Control is a big part of this. With a piece in a concert hall, there are notes written down that are played over the prescribed period of time, and people listen. That’s time-based art, right? Music is inherently time-based, as opposed to painting or photography or anything that you can theoretically experience in a split-second via your eyes.
A piece of music is sort of invisible until you let it play its course. It has to have time in order to exist.
As a composer, you think about making things happen over a timeline. But if you change the paradigm and say that the music isn’t going to proceed until the listener moves through it, things start to get a bit wacky. It’s almost as though the music is waiting there for you. It begins at your instigation when you arrive and changes based on what you do thereafter.
So, it’s just a different kind of thinking about ways that different musical elements interact with each other and how this decreased level of control that the creator has can be managed or exploited.
It’s almost like the multiverse. You think, “Here we are in our own universe of my version of the piece, but maybe there are an infinite number of other universes and every single possible different decision any person or any object could have made in this universe would have spawned a new universe just for them.” Controlling for this is kind of scary, so you just have to give up and take advantage of the excitement of all the possibilities and revel in the fact that no decision is permanent.
You have to give up your control-freak attitude, which I have a lot of. But I’ve learned to embrace it and accept that some experiences are going to be more like what I’d want somebody to experience, and some are going to be less. It’s not that some are better and some are worse. It just means that there’s that there’s a wider variety.
In any case, what is so exciting and so fun for me when leaving more things open to chance is that you can encourage wonderful moments that I don’t think I’d be able to create out of my own brain. So, for From Here To Where, I thought of it as a physical journey representing a temporal journey from a time when the Earth is dominated by human beings to a time when there were little or no man-made effects.
There are obviously lots of ways to get between those two points with the audio. I treated it as a series of 30-minute-or-so sections based on the morphology of the landscape. You start in the city, which is quite flat, and then go through the outskirts and into the foothills. After that, you follow along with a river for a while, traveling upstream, and then eventually come up over a ridge and descend a bit into a vast high plateau. Each of these sections had a particular feel to me: the river section became a swamp with frogs and insects chirping, the open flat areas were dominated by sounds created by the wind, and the high plateau felt like an ocean, so I filled it with whale sounds, snapping shrimp, and edged it with waves. Over the whole journey, the audio is becoming more and more “natural” and less anthropogenic.
Demirjian: What about you, Rashin?
Fahandej: I think about sounds in visual forms, like sculptural objects, with their proximity to our bodies. Vibrations can surround and summon our bodies from near and far to surrender in a foreign space, to give in to intimate moments. I also rely on narratives in sound — not in linear form but fragmented stories, abstracted forms which are felt regardless.
At BCA, where the piece was installed, I thought of each lullaby as a story. The idea was to create sound compositions that allowed these chains of stories to create mega-narratives, which can be experienced in segments or as a whole.
There were themes that threaded through some lullabies — such as water, journey, and sailing. Looking back at the history of slavery and its relationship to our current incarceration system, I thought of passage as the origin, the transatlantic journey and the middle passage.
For the first composition of the series, I worked with longtime collaborator Krista Dragomer. On site, the passage and wave quality of sound is exaggerated through the use of space. You will encounter the audio at multiple locations along the path with a small delay, as if you are inside the piece and sound is traveling with you on the same pace. If you stand on one side, you hear the echo and rippling effect of sound coming to you from afar. We also used sounds of water and waves; but more than the ambient sound, it is the repetition in words and sounds that create a wave quality, dripping sounds.
Burgund: I love the notion of the ability to experience your piece at different time scales, from the very short to the much longer. It reminds me of fractals in the natural world, which is something that I think a lot about in my own work: Does the overall shape of this mountain ridge line mimic the shape of rocks found at its base? And is there a way that audio experiences can be built using musical elements that repeat over different time scales?
Andrew, can you tell us more about your methodology for composing for space?
Demirjian: This goes back to the idea of accessibility. If you make compositions for multi-channel artworks that take months to create, only the people who come to the exhibit can experience it.
What I was trying to think about with Pan-terrestrial People’s Anthem is how to make a spatial composition that is accessible from anywhere. And that’s how I got really interested in the ambisonic audio aspect of VR on YouTube. I was composing about half of the songs in a quadraphonic or octophonic speaker configuration, thinking about a person who might experience this with headphones on.
So, for instance, track 6 takes little fragments of these repeating themes and staggers them in a way so that the phrase is moving around the listener. Other times I would keep all of the sounds fixed. Placing sound parts up, down, left, and right three-dimensional through the listener experience involves composing for a circular space. As the listeners experiences the pieces, if they move the video, the panning in the mix changes.
So, I wanted to emphasize a lack of a fixed perspective to bring out the theme of porous borders by allowing the panning in the mix to be fluid and changing based on where you turned the video or moved your head. It was great to work with Contour Editions because they were up for releasing the work in video as well as audio formats.
MOVEMENT III: PARTICIPATION
> Jump to Movement I: Creation or Movement II: Location
Demirjian: What is the role of participation in your work, and how do you see your projects evolving in the future?
Burgund: For me, From Here To Where is a bit different from my typical work in that I usually have a real-time contributory component to pieces, much like Father’s Lullaby, where participants make their own recordings in certain locations and those are assimilated into the piece for others to hear.
Fahandej: Andrew, can you talk about the contributory aspect of the remixing in your project? How did you find your material?
Demirjian: Contributions to this project are mostly from people who have posted anthems online for each county. The majority of these are recordings of national orchestras. For some of the smaller countries, what was available online was limited: someone playing it on a keyboard or cheesy synth.
Burgund: Casiotone?
Demirjian: Full-on Casiotone. There may have been about a dozen that were these cheesy synth ones. One of the pieces has keyboards cut together. So, there was a power dynamic happening with how your anthem was recorded and what was available online. The wealthy countries had complete orchestras and tiny islands only had a one-person synth version.
My basic process was to put the songs into the same key and then search for little fragments that stood out by themselves or parts that could be blended nicely with others. I tried combining two parts together until there was a nice match, then built on top of that. It was very painstaking. When I was working with the lyrics it was easier to use computer programs to filter and analyze the 27, 523 words — but the music was all done one phrase at a time.
I find it helpful to develop constraint systems when I’m in a daunting situation like this, trying to represent 195 songs in 10 songs. So, I would come up with ideas like: Can I make a song that is built exclusively out of drum samples or just the last two notes of anthems? This is how track 3 and track 7 developed. They start out like audio science experiments and then I refine them and try to give them form.
This project is born from participation and will cease to exist without it. To participate is a conscious decision to become a part of a social change.
Fahandej: Mine is a multiyear and collaborative project. I am testing the limits of art in public space as a tool for social change, creating new social memories that give permission for new forms of interactions to emerge.
The project started locally but will expand to global on a virtual site. My goal is to create sound installations on multitudes of city blocks in every city in United States, possibly creating a heat map of lullabies collected and projected locally. Every site is the space for participation, collaboration and reflection. The voices are collected from the immediate community, so you will be able to hear yourself or someone you know at the site you pass by every day.
On another level, I am seeking a global perspective, where the localities come together to create a web of experiences and connections. I call this the “Poetic Cyber Movement for Social Justice.”
I have been working with Halsey and we have been using the Roundwareplatform. Now, at FathersLullaby.org, everyone can record their voice, singing a lullaby or sharing memories of childhood or reflections on the issue. Your voice will instantly be a part of a geolocated map online. You can also listen to other recordings by choosing locations where Roundware curates the sounds in that particular space, or listen to every recording individually.
This project is born from participation and will cease to exist without it. To participate is a conscious decision to become a part of a social change. The project in a way mimics our human society, an organic form; nothing could be taken for granted as-is, because everything is in constant shift and change.
In the United States, currently one in three adults, regardless of race and gender, has a criminal record. That is over 73 million people. This is extremely frightening. How did we get here?
I grew up in Iran. These exercises of power bring me chills. The rate at which we have been criminalizing and disenfranchising people in this country is an exercise of power only fit for authoritarian governance, not a democratic society. I think we have taken the word “democracy” for granted, as if it is something we have and will have forever. I believe in collective actions as our main hope to bring about positive social change.
This article was commissioned by and originally published on Immerse. It is cross-posted here with permission and much appreciation.
