In fall 2020 of my senior year at Columbia, I decided to take an intensive fall class (just 6 weeks) called Computational Sound. I went into the class not knowing anything about digital music, or even much music in general. I’m really glad to say that I’ve learned about and worked on Audio DSP, Live Coding, and Automated Composition since.
Our final project was an open-ended assignment using Web Audio (our primary tool for the semester), as long as it reflected one of the three main topics we covered in class.
Now, I’m not knowledgeable enough to work on music composition or theory projects, but I’ve always enjoyed interactive, projects that generated customized results for the user. I decided to make a website that accepted file song uploads and manipulate the audio sound using both old and new techniques, falling into the category of Digital Signal Processing.
First, I did a lot of self-learning on how to upload and access files over the web and retaining the data with JavaScript and Web Audio’s Buffer Source API.
To play the song, I simply connect the Audio Buffer Source Node to the audio context destination so we can hear the sound. In my case, I connected my buffer node to a gain node to control the volume of the sound before connecting it to the destination.
I wanted to apply a concept from class, so I tried modulating the audio with a sine oscillator. To make the sound have more of an effect, I also created a delay node which the modulation gain connects to. I allow the user to select on the slider what modulation index and frequency effect they would like, and the effect is a vibration-esque sound.
Next, I decided to perform a bit of array manipulation, demonstrating understanding of the buffer source while using a variety of code techniques.
- I accessed the audioBuffer contents directly using the code getChannelData(n) parameter of the arrayBuffer, then called the javascript function reverse.call, which reverses the contents of an array allowing us to play the audio backward. To my surprise, the sound generated was very smooth and no tinkering was needed.
- I also thought it would be fun to slow down / speed up a song. I was able to do this by converting the channel data of the sound buffer to 32-bit arrays. I rearranged the array depending on the speed the user selects (the default is 2.0) and converted modified array back into buffer data.
When the speed is at 2.0, the effect that comes across is comparable to Alvin and the Chipmunks :)
Finally, I wanted to attempt fading out the vocals of the clip. I’ve always wondered how to obtain instrumental tracks of a song and if it was possible to generate the acoustic, stripped version from computers. It turns out the latter is much harder than the former.
The vocals, I learned, is usually recorded pan center in a stereo file in both channels, so I looked into the process of removing it. The logic is to invert the polarity of one side of the stereo signal, so I subtracted the left channel from the right channel, and then mix the left and right channels together, which cancels out anything identical and leaves a single mono track. I tried improving this technique by applying a low-pass filter to one channel before I inverted the channel to retain some of the bass and drum parts. Overall, the effect is achieved, though you can still hear small whispers of the vocals (from the reverb).
After watching the in-class demos, I was inspired to add two more target sounds.
I decided to do even more with array manipulation and attempted to shuffle the entire audio source! Again, I converted the channel data of the sound buffer to 32-bit arrays. Then, I created subarrays of 3-sec snippets of the sound, and joined the subarrays back into buffer data randomly. I had no idea how this would sound, but I liked the result.
I also learned about a new audio effects library (from my peers) called Tuna, so I decided to see what could be useful. I used the PingPongDelay effect to create an echo to my sound. I also added a high pass filter to lower the volume of the loudest parts of the signal, creating a smooth room-echo.
Last but not least, I decided to prettify my website! I used to be really interested in front-end development back in high school, and one of my favorite resources to use was Bootstrap since they had free templates. It took me a while to figure out how Bootstrap worked again (I had done 0 web development since going into college), but I was so pleased with the results. I also wanted to generate some graphics related to my audio player. I was inspired by spinning vinyl from codepen, and I also generated a frequency chart based on the entire track each time. I wanted to have a dynamic chart, but I felt like there was too much going on on the page, and a basic UI rule is to avoid that!
Overall, this was a really fun project and I’m very proud of it. I would recommend this class for any CS major vaguely interested in learning more about audio and sound.