GitHub GitHub Universe 2020: Play - Creative Coding, 12 Dec 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Introduction to web audio - GitHub Universe 2020

Description

Presented by Ruth John, Digital Artist, Independent

Shall we talk about web audio? Annoying background sound, auto playing videos... but wait! What about games, apps, XR, web assembly giving us the power to compile a whole new world in this environment? In this introductory session, Ruth will walk through the concepts behind web audio and show an example of how to start using it.

For more from GitHub Universe 2020, visit https://githubuniverse.com

As always, feel free to leave us a comment below and don't forget to subscribe: http://bit.ly/subgithub

Thanks!

Connect with us.
Facebook: http://fb.com/github
Twitter: http://twitter.com/github
LinkedIn: http://linkedin.com/company/github

About GitHub
GitHub is the best place to share code with friends, co-workers, classmates, and complete strangers. Millions of people use GitHub to build amazing things together. For more info, go to http://github.com

A

Hello- everybody, it's lovely to be here, my name's ruth and I'm a creative coder based in the uk.

A

I use web technology as my primary environment, which means most specifically, I usually work in javascript as my primary language, I'm here today to talk to you about programming audio with javascript, and for that I'm going to use the web audio api.

A

This is one of many web apis which come out of the box within a browser environment.

A

So if you build websites or web apps with html css and javascript, there's a lot of apis at your disposal, things like geo location, device, sensor, control, browsing resource management and media apis.

A

You can find out more on the mdn docs web api page so back to audio. Let's address one thing very quickly and that's audio on the web. People are not keen on that.

A

However, with the rise of games, xr experiences and art audio plays a huge role so having a good low level api to create and manage audio is very worthwhile.

A

There's a lot of functionality within the web audio api. I have 10 minutes of your time, so I'm going to give you a quick demonstration.

A

One thing to understand about the api: is it works around the concept of nodes? Each bit of functionality is a node. There are input, nodes, ways of loading or creating sounds. There are modification nodes these make up quite a bit of what you can do. You can adjust volume, apply filters, add echo or spatialization you can even analyze the sound.

A

Then there are output nodes by default. The sound is output to the speakers which by and large, is how you'll work, but you can also output the buffers and files.

A

Here I've got a web page with a button on it. Here is the html. I've got a button. I've added some styles to it with the css, but what I'm going to show you is how to make a sound when we press that button, so we're really interested in the javascript here.

A

I've grabbed a reference for the button and added listener, for if the button is clicked, the first thing that we need to do before anything else is create a context. This gives us all the functionality of the audio.

A

Api, the audio api only allows sound on user gesture I.e. The user must interact with the page to play noises.

A

This means you can't auto play them when you load a website or app, which is a big pain point with audio anyway. So in our case, this means we can start it once the button is clicked.

A

The way we do that is just starting is checking the state of the audio context and resuming it if it has been stopped.

A

I've already created a name for our function and we just need to call it within that click code.

A

The first thing we need to do is create an input node. Now there are a few different types of input nodes to the web audio api. You can load a file. You can get the audio from the html. If you've used an audio or video element, you can create a buffer and fill it with your own data. To play.

A

This method is good for things. Like white noise, you can fill a buffer with random numbers, I'm going to create an oscillator as my input node, which plays a clean, sound wave like a sine wave, a square wave, a triangle wave or a sawtooth wave. The difference between these types of waves is the shape of the actual wave and we tend to have these shapes by default with electronic oscillators, as they are easy to generate electronically.

A

So, let's create an oscillator using the aptly named, create oscillator function, I'm going to set the wave type to triangle. I do this with the type property.

A

We want to start and stop the oscillator for which the oscillator gives us methods, so we do oscillator start and oscillator stop we'll add a parameter for when we want to stop the oscillator it takes in the time it wants to stop. In our case, I'm going to be one second from when we start it to do this, I'm going to use the audio context.currenttime property, which returns the time in seconds since the audio context was instantiated.

A

In this case, I want the oscillator to stop one second after this code, which means, I just add one to this property.

A

One important step, which is easily forgotten, is connecting unknowns together. This is called connecting the audio graph. We need to connect our input node to our output node by default. The destination property is the speakers. So we need to do nothing else here now we should be able to play our oscillator by clicking on the button.

A

Great, let's add a couple more things to make that sound a little bit more interesting. Let's change the frequency of the oscillator. The frequency is the pitch so whether it sounds low or high by default. It's 440, but let's make it a little bit lower.

A

I also want to make the frequency higher, as it gets played so over the seconds that we're actually playing this oscillator. We can do this with a method called exponential ramp to value at time. All that really does is a bit of a mouthful. Is it moves the value which you're specifying over a specific amount of time, but as a nice curve, rather than just.

A

Linearly now that sounds a bit. How should I put this forced, let's also drop the volume of the sound too, so it tails off as we play it, I'm going to do this by creating a modification node for volume. It's the gain node with the web audio api. So let's create a gain. Node.

A

And not forget to connect it to a graph. We can chain our nodes with the connect method. So here I can say my oscillator is connected to my gain, which is then connected to the destination to the output.

A

On my gain, node, I'm going to use the same exponential ramp to value at time, it's available on most audio node parameters within the web, audio api, so we're shifting that value in a nice curve over time.

A

I want the value to be almost naught and to the value to be just under a second. So it's just tailing off nicely.

A

Let's see how that sounds, pretty good. It's worth noting here that there are quite a few different modification nodes that you can play with.

A

There's a wave shaper node, which changes the shape of the wave there's a convolver node, which gives you echo, effects, there's filters which can cap or allow specific frequencies through um there's spatialization there's plenty to play with, but this is just a very quick introduction, because I'm nearly out of time, you can check out more about web audio on mdndocs, where I've written some introductory and advanced tutorials to help you discover more.

A

Thank you very much for watching I've been ruth. You can find me on twitter and github as roomira. Thank you very much for having me.