Jacob Sundstrom
There are essentially two intertwined processes at work in the piece: audification of an image with subsequent audio effects applied, and electroencephalography (EEG) data driving the effects. EEG data — or my thoughts in raw form — were recorded via a clinical grade EEG device while viewing the images and interpreting them in my mind. This occurred for each of the 10 Rorschach images and thus, there are 10 segments of EEG data corresponding to each image. Only channels covering the parts of the brain related to visual processing were used. Only 8 of the animations have been published.
A program of my own design then ‘audified’ the images and applied effects to the audio version of the image, the parameters and the degree to which was determined by the EEG data accompanying the image. In this way, the algorithm and program — as well as myself, to a certain extent — function as the interpreting psychologist in a Rorschach test. In this instance, however, the result is a purely visual interpretation in the form of an animation that describes my thoughts in their raw form over time. Each frame in the animation describes a particular moment in the EEG data of the corresponding image and the changes in the animation describe the changes in thought patterns throughout a viewing.
I obtained the control data from EEG readings of my own brain while looking at the images using a montage referenced to the ears. I created a simple slideshow of black backgrounds and the images on a black background. The slides alternated black, then image, with 10 seconds of image and 5 seconds ‘rest’ during which my eyes were closed. The closing of the eyes functioned as delimiters in the EEG signal; eye blinks are very clear so even though I did not start the slideshow and EEG recording exactly simultaneously, it was relatively trivial to find where images occurred. The order of images was fixed, unfortunately, and I also had the misfortune of viewing the images beforehand, though I did make an effort to not ‘study’ them. Upon taking the recording, I recorded my conscious interpretation (not audibly, as this would destroy the EEG signal I was intended to record) and used this as a guideline when I began the manipulation.
After some research, I settled on using the data in 5 EEG channels: O1, O2, P3, Pz, and P4. These five electrodes cover the area in which visual information, both from external reality (light stimulus) and from internal reality (“the ‘mind’s eye”), is processed. The brain communicates with itself in different regions to achieve coherence; according to Schlegel et al., this occurs primarily in four core regions of the brain: the frontoparietal cortex including the dorsolateral prefrontal cortex, posterior parietal cortex, the posterior precuneus, and the occipital cortex. In their study, however, subjects were required to hold a mental image in their head that was manipulated; this required working memory—which is located in the frontal cortex—and as such functions as an executive (master) system that organizes and recruits subsidiary systems (slave) that are more particular in their function. Thus, I chose to monitor the slave systems of the posterior precuneus and the occipital cortex located at the previously mentioned electrodes. The particular frequency band chosen to manipulate parameters was somewhat arbitrary: I experimented with several different controls on pre-determined modification parameters and selected the one with what I considered the best results.
With this in mind, it should be made clear that this study is in no way intended to be a quasi-scientific comparison of my brains’ response to different images. Different frequency bands of different electrodes were often used on each image with different modification parameters. This is the case for two reasons: first, the same modification parameters (for instance, delay or noise reduction) often do not work the same way on two different images, however closely related they may be—what makes one image interesting creates a jumble of noise in another; second, it is far more interesting to work with the control data and ‘bend’ it to do one’s bidding. It is the chisel with which Michelangelo carved David, only the chisel changes shape and its reaction to the marble. Lastly, I’m not doing science. There are much easier ways to compare data and frankly, direct comparison between different sets of data does not necessarily make interesting art.
The filtering of the EEG data was done by converting an EDF file to a WAV file via Octave. A subset of the data was then subjected to an FFT from which relative amplitudes of three frequency bands (delta, theta, alpha, beta, gamma) were derived. The modification parameters of each image were derived empirically, since what works with one image may result in total incoherence in another. Thus, it was vital that I played with each image for several hours before applying the EEG data to determine the best effects to apply. Only after a satisfactory combination had been reached did I then begin using the values obtained from the EEG; this also required several hours of tweaking to find the best response from the various choices of electrodes and frequency bands. It should also be mentioned that I often use clear controls to modulate some of the data: that is, over the course of an interaction there is a value (line) such that it begins on 0 on the first iteration and linearly ascends to 1 in the last iteration. This provides some more modicum of control in the temporal manipulation and can allow one to fade from control using alpha in one electrode, to alpha in a different electrode. It also allows one to temper an effect temporally such that it may grow more intense in the middle and falls away at the tails where the loop ends and begins.
During manipulation, images are converted to a WAV file via a Bash script, loaded into a buffer in SuperCollider, manipulated, then saved as a RAW file which could then be viewed as an image. The animations involved iterations on the same image using the same parameters with the same mappings such that, for instance, the alpha changes in O1 are made to effect the delay time applied to the audification of said image over the course of the EEG data. As previously mentioned, my conscious interpretation was used as a guide for modifications; the manifestation of the conscious interpretation was dependent on the kind of effects that worked with the given image. Moreover, it was used only as a guide since I am aesthetically interested in discovering images that I did not conceive of. If I relied only on my internal conceptions, I would never come up with anything new or interesting; I would remain within my understanding. How boring.
______________________________________________________________________________________
Jacob Sundstrom
http://jacobsundstrom.com
University of California, San Diego
Although trained as a composer, Jacob Sundstrom’s (b. 1988) work has branched out into several mediums and
fields. Among these interests are installation, visual, glitch, data bending, and EEG-based art. His work
focuses on the friction between and among medium, performer(s), and process: using the process against
the performers, the process against the medium, the medium against the performers, and visa versa and so on.
Out of this friction arises fascinating, beautiful, and often surprising results.
Sundstrom’s musical work has been performed by ensembles including Inverted Space, the JACK Quartet, and
the Seattle Chamber Players. His visual work has been showcased by the Center on Contemporary Art in Seattle
and he was a featured artist at the Georgetown Art Attack Special Events at CoCA Un[contained] at Equinox
Studios. As a researcher, his work on EEG-driven musical interfaces has been published in Frontiers in Human
Neuroscience and presented at the Society for Neuroscience’s annual meeting. For what it’s worth, he studied
music and philosophy at the University of California, San Diego and music at the University of Washington. He
is currently a PhD student in computer music at the University of California, San Diego.