GitHub GitHub Satellite 2020 - Play, 8 May 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Creative Coding in p5.js - GitHub Satellite 2020

Description

Presented by Cassie Tarakajian

Have you written code before, but never thought about how it could be used for art? What are the artistic possibilities for software? This session will introduce the foundational concepts of creative coding through p5.js, a JavaScript library with a focus on making coding accessible and inclusive for artists, designers, educators, beginners, and anyone else. This session will explore how software techniques, such as algorithms, randomness, data, and interactivity can be applied to creative work. Familiarity with JavaScript or code syntax is recommended.

Cassie Tarakajian (they/them) is a technologist and artist based in Brooklyn, New York. Their work focuses on creative tools, working as an engineer at Cycling ’74 and as the lead developer and maintainer on the open source p5.js Web Editor. They also teach creative coding workshops and classes, and serve as an adjunct professor at NYU ITP. Past artistic projects range from generating sonnets from Wikipedia contributions to teaching computers how to love as a member of the band Lullabies for AI.

GitHub Satellite: A community connected by code

On May 6th, we threw a free virtual event featuring developers working together on the world’s software, announcements from the GitHub team, and inspiring performances by artists who code.

More information: https://githubsatellite.com
Schedule: https://githubsatellite.com/schedule/

A

B

Hi, my name is Cassie, and today I'm gonna be talking about the basics of creative coding by doing a little live coding and p5.js. So if you've just seen some of the past talks and you're feeling really inspired or kind of like intimidated like well I, don't even know how I could possibly ever do this and I think this talk is for you, but first I want to introduce myself hi I'm Cassie I am a technologist and artist and educator. So that means that I do a few different things.

B

One thing I do is I work on the software for creative tools, so this is a picture of a graphical programming. Language called max/msp. It's a good musical programming language. It's managed by a company named cycling 74 that I've worked with for a few years. I also teach here is a picture of me teaching a max workshop this past summer, down to some high school students and I also teach at a grad program at NYU, called ATP or interactive telecommunications program and talked creative coding there for about two years.

B

I also have my own creative practice. This is a picture of me performing in a musical project called lullabies for AI, which was about writing songs to teach our AI children how to love and exist in the world. But really what I want to talk about today is p5.js. Something is near and dear to my heart.

B

P5 is an open-source JavaScript library for creative coding, the focus on making coding, accessible and inclusive, and it's managed by this foundation called processing foundation, is created by a person named Laura mccarthy about seven years ago, and so you would use p5 to make interactive audio visual art. So that means stuff. Like you know, you can draw shapes you can do. You can alter like videos and images you can do. Data visualizations have some quick examples. Just so you can kind of see stuff made of p5. Here's an example.

B

Here's an example like a video shader. You can also make like an interactive movie poster just like this is you can do a lots, lots and lots stuff.

B

P5 was really inspiring to me because my background is in engineering and when I started when I first started learning how to code. I was writing like command line applications in Java and, like the whole thing was like really intimidating and scary to me and I felt like after I, like spent all this time, learning I like didn't actually know how to make anything and from ep5 was really inspiring. That, like I, could know very little and be able to make a lot so I loved it.

B

So much that I started my own open-source project, which was an in-browser code editor for p5. So the idea here is that, like you, really removed the barrier to entry for writing code or writing creative code- and you just like have the code on left preview on the right could just like open this website and get started. So this is what we're going to use to do some live coding.

B

So, let's so I'm just gonna open the web editor, and this is like the sketch that you get when you first open it I'm, so I'm just gonna hit play and we can see what we get and we just got this like kind of boring, gray square. But let's like break down what we've done here or what what it gives us so first important line we have here is create canvas, and this is creating like our drawing a surface where we're gonna draw all of our visual elements.

B

It just takes like a width and a height as arguments, so I could like make the speaker and then we'd have something bigger right and then we have a background, and this is just like painting a whole big color like one color over the whole canvas, and this argument that it takes is a color and this color can be like you can specify in a lot of different ways. So if we just have a number, then that's just a grayscale value right. So if we put in 0, it's gonna be black.

B

If we put in 255, it's gonna be white I'm if you're familiar with hexadecimal colors, that 255 is white. If you're used to like using colors on the web, like that, you can also make it a red, green, blue value right.

B

So man, if I just specify a red component and then, if I put in you, know, I, could put in a blue component and then it's gonna be like paint color, but yeah colors are just numbers and in fact like to our computer, everything is numbers, eight images text, video everything is just numbers, and that's really. One of the like exciting parts of creative coding is that we can like take these different numbers and like map them and interpret them and use them in different ways.

B

One other thing I want to note is that there are these two functions set up and draw, and these are being called behind the scenes by five piece by the p5 library and so setup is being called one time and draw is actually being called 60 times a second right. It's just like constantly being executed over and over again it's what computers are good at, and this is what allows us to make animations and and that kind of stuff.

B

So now what I want to do is I want to add, like an interactive element here and I just want to add, like a really simple one, so we can kind of understand it, and then we can like keep building on this, and this is how you might, if you're, just like starting working on it, creative coding project, how you might how you might just start. So what I want to do is I want to draw a circle at the mouse position.

B

So the way I do that is they use circle, function, yeah, that's that's right by p5 and then I can pass. It takes like a X&Y position and a size, so I'm just gonna put in the mouse x position. This is a variable, that's provided by p5. It just specified every go draw loop. Every time the draw loop gets run, it gets updated, I'm, just gonna put in a size, and we can like see. Oh you see, I have this in the wrong order. Background is getting drawn over it.

B

Put this in the right order, and now we can see our Sierra Circle right and if I move it around. I have the circle following me and you're like okay, Cassidy was so kind boring there with me. So now I think it gets slightly more interesting when we move background up to the setup, because then we have them were no longer like painting over the canvas every frame and then we're able to like see this trail of me moving the mouse, and you know actually it's actually like a bad.

B

It's like a bad drawing up and you kind of move the circle around faster and slower I mean you can like swing your eyes. You could like see this like three dimensional tubes filter or something it's kind of cool, but let's keep making this more interesting. What I do its I want to add some color to this and I want to have the color be affected by the mouse position right, I'm kind of doing like a one-to-one mapping here or I'm just mapping the mouse position to like the position of the circle.

B

We can get a little more more fancy. So, let's change the color at the circle, where that we do that is we use function, called fill just fill color I'm gonna, put in Mouse X for the red value and, let's gonna, put in maximum for the green and blue. Let's see what that looks like so, as I move the circle across the canvas, we see that all the way on the Left where the mouse x position is zero. It's like it.

B

We can see just the the green and blue components and then, as I move it to the right. It's white, because all of those colors some together is white. But one interesting thing to notice about this: is it actually changes to white like halfway through the canvas? And that's not quite what I want right like I want this to be like a nice gradient all across, and the reason that this is happening is because these two values, the position of the mouse and colors, have different ranges right.

B

The mouse position is a value between zero and the width of the canvas and colors are values between 0 and 255. So we need to need to take the mouse position and map it to the right range and the way that we do, that is. We use a function called map, I'm, saying what sort of map and I couldn't think of it. Red Dog equals Mac and so map takes five arguments, one of which is the input and then the input range and then the output range, so the minimum maximum and the minimum and maximum.

B

So this is going to be a value between 0 and rick. P5 also just gives us width of the canvas py. That's very nice to us gives us the things that we want and now as we move, oh I have to actually use this variable, rather than just define it and then it's we move the mouse across. You can see that now we have like a nice gradient and we can like enjoy it even more. Maybe I'll remove the stroke around it ooh and then we could like kind of draw around.

B

We got like a nice gradient, so that's fun, but now I want to like keep playing with this I want to like use a different shape here. um I happen to like using line and I want to slide. The line takes like a start and end position and I'm just gonna like offset the X, the X, the start act, I'm gonna make it like. You know, you'll see what I mean level with the expert a little bit offset to the left and right so I'm gonna be Mouse, X minus 50.

B

Now Y takes X, Y, X, Y, mouse, X, plus 50 Mouse, Y cool and we're not gonna, see anything because I removed the stroke and lines are an empty shape. They have no fill. They only have strokes, I actually just need to like get rid of my circle and then I need to specify a stroke color. If I just call stroke with a color, then I got a color. Now if I start drawing here, yeah I get like a line, but I'm not really I'm still not liking.

B

This variation in color I want to add a little bit more and so I'm going to have the mouse Y control the green green position. So then I'm going to change this to green brown. Make this mouse. Why make this type trait? Because Y is the value between 0 and height? Y positive is actually down takes some getting used to it's actually really common and like every everything digital I don't know, did a lot of creative coding frameworks, okay cool so now, I'm getting more more color variation.

B

That I, like it's kind of like a vapor wave. The colors and I could like play around with these color ranges and like see what I like I'm happy with this color range, so I'm gonna keep going it's kind of nice as I like move. My lying around I can see how like they overlap and make these like.

B

Textural shapes I'm liking this, but now what I want to do is I want to add some rotation to this line, to like have a little bit more to see like what other kind of shapes arise there, and so there's a Pew 5 function called rotate and it takes one argument: that's an angle and it takes by fault radians, which maybe is scary, but but unfortunately, in creative coding. You have to do a lot of I feel like you have to do.

B

A lot of maths like I need to work arranging visual stuff on the screen. It's like you have to do. Math and my students are always like. Oh no I thought we were avoiding math by going to art school I'm like hahaha you have to learn trigonometry. We have to remember trigger object, okay! Well as I'm like. Are you trying to use this rotate? It's like not really working. How I want like I, wanted it to rotate the shape but like as I have my mouse over here. It's like now.

B

It's position as well as you don't really understand, what's happening while I'm pretending that I don't understand something. I do understand something.

B

I have an example here to show you, so it's like offset in this weird way, and so what's happening is so here's an example where I'm rotating a rectangle at the origin I've actually translated the origin here to the centre of the screen and so we'll see if the second square, which is the blue one I've rotated it PI over four like the same amount, 45 degrees and we'll see that it rotates in place right when you rotate rotate something at the urge and it just rotates the shape.

B

But if you move the rectangle out from the axis, so here, I'm gonna I'm setting the x and y position, so the red one is the unrotated one, that's connected with the black axes and the blue one is the rotator one. You see how like it's, XY position gets moved and so that's what's happening in our inner sketch now, I. Don't want that to happen. I wanted to get rotated in place. So then what I need to do is translate the axes to the mouse, X and mouse Y position now, six.

B

That's why right and then what I need to do is then draw the line back at the origin. So then, I get rid of the zero makes this zero get rid of you get rid of you. Oh I got put a zero here, actually so cool.

B

What did I do wrong? What did I do wrong what's happening.

B

Cute Oh negative 52, 50 I was like the line, had nose, sighs, okay, cool so now, I have a rotated line. This is fun, but it's kind of just like the same thing that we were doing, but it's like tilted yeah, but now I want to now that I'm like able to rotate it. How I want I want to like have the rotation change with the mouse X and last by position.

B

So let's create a new variable called rotation when you use our trusty friend map again, what do you mass Y, which I know, is a value between 0 and height and let's map it to a value between 0 and pi. Pi is also just provided by p5.

B

So now I can get some squiggly shapes as I move my mouse and that's fun. I could try making this like 2 pi I'm, like honestly, if I'm like playing around with mapping it's something like this like I. Don't I'm not like thinking about this too hard I, just kind of like put in a random number and just like see what it looks like. It's really nice that, like you, can just write code. You know and then just like immediately see what it looks like and be like. Oh that looks good.

B

That looks bad and that's just really common in creative coding. Okay, like how this kind of had this like san shape, comics like a leaf or something cool so now, I could also try playing around with like the line size. I want the line to be breaker, 300 and then I kind of get like a totally different um different effect. So now, like I, don't like that every time I do this I have to like move my mouse around to test it.

B

I just want to hit play and I had the computer do the work for me, and so that's a really nice thing about I'm, using computers and you're, when you're trying to make visual art is that you can just hit play a bunch of times and like see what it generates and pick the ones that you pick the ones that you like. It's like you're laying the computer do the hard work doing what it's good at and you can also have it like have some randomness.

B

So every time you hit play, it does something different and then pick the good ones, and so the way that I want this, like the line position to move, is I. Have this like bouncing ball example and I want it to like just bounce around the canvas like this and rotate around and we'll kind of see what that looks like and because I already have this example like I'm. Not like you know it's all this is doing is really just like.

B

It has some x and y position and it's just updating the speed every it's updating the position with the speed and then when it hits the wall, it's just reversing the X speed or the Y speed. It's pretty. It's pretty simple and really like I. Don't want to think about how to do this again, like I'm, unlike I, know those code works so I'm just going to go back to my example and paste. It only thing to do is merge my setups, so I don't have to set up functions.

B

The one that comes second, is the one. That's gonna override okay, so now I have this and if I hit play they're, just gonna, be the things I haven't connected it at all. I also need my code. That's in my draw loop, that's doing all the updating, so it's gonna copy that go to my draw loop and now so this is my code. That's updating the position, and this is the code. That's making it bounce off the walls and, in the other example, I had the circle that had a diameter.

B

But my line has no diameter I just wanted to directly hit the wall, and so, in this case, D is zero. So I can just get rid of these these or set it to zero, where it makes sense. Okay, so now, if I hit play, it's still not working because I haven't actually connected it. So what I need to do is change this to X check every time. I have mouse, X or Mouse Y got to change it to Mouse X or change it to X or Y whoo.

B

Okay, so now I can just like let it hit play and just like see what happens. That's kind of fun I, really like how it looks if this speed it's a little bit slower or like that, let's see how that looks. Yeah and you have it slower. You get these more a patterns that I really like, because the the line is then closer together, yeah. This is fun.

B

This feels like this feels like math textbook energy would be like multivariate, oculus, third edition or something I, don't know cool, so I hope this has given you an example of how you might just like start from something really really simple and like iterate on an idea and just like play it around. If I hit play a bunch of times, I could get totally different things. Oh, this is cool like how it's just spinning in the center whoo I could like take a screenshot there and be like this is my art piece.

B

You can really generate cool ideas, but now I want to show you how to take like different types of inputs and how you might just like take some code that, maybe you don't know and just like put like just sort of like plug different things together and also work with libraries that you maybe don't know.

B

So what I want to do is I have this example, that is in one of the p5 examples and like where'd I put it, and what it is is that it's really simple synthesizers so like when I just press the mouse it like plays a sound and just changing the mouse position, changes the amplitude and frequency I'm like this is great I want it and then what I want to do is I want this synthesizer to be controlled by my face. What I want to happen?

B

It's like when I open my mouth I want to play a note. So how do I do that? Well, luckily, there's this JavaScript library called CLM tracker. That I know is really easy to use and really great doing face tracking. So I'm, like cool I, can kind of look at this documentation. I see it gives me a bunch of points for like where my mouth and eyes are and stuff I'm just going to need to figure out like the distance.

B

My mouth opens right: song I need to like measure the distance between 60 and 57 and I have some example code, and so here I have open the CDN, so I can just like copy it to my sketch. I can then put it in my HTML yeah, so this on the right is actually a hole. It's like a whole web page just running in an iframe here so yeah. That's how I add new files to my sketch and if I hit play nothing's gonna happen. Well, actually, I need to get my face.

B

Input right, like I'm, not getting my video in here. So what I need to do create function? Your variable called capture, I'm, actually gonna change. I want my the sizer to be a sawtooth instead, I think it's more annoying and I like that. So I'm gonna do capture equals, create capture. This is another function, probably by p5 that allows us to get video input and then I'm gonna do capture, got size, 400 225.

B

So now we can see there's my face and double, because you can also see my face over this thing and then what I want to do is you can see that it's like not on the canvas and that's because this actually creates a totally different video element on the screen. It's like it's like a literally a video tag. I'm doing this for triangle, HTML element, so I want to draw it to the canvas, and so let's first set my canvas to the same size and then I'm gonna do use the image function.

B

This draws an image to the p5 canvas at some position. All I have to do is pass the image or video capture 0 0, and then we see now I have two of me and I can like hide the other one capture, dot kind.

B

So now now, if I click on my face, I have the synthesizer here. I, don't want any of this text here. Okay, so now I have my face and then I need to get the points. In my face and I know, I'm gonna need to like somehow plug it into this amplitude and frequency right. um So now what I want to do is go to my face tracking thing and I want to just get the starter code. So I'm just gonna copy this, it's the initialization code, then I have my tracker.

B

I want to make this a global variable. I like can't even let a var exist in the world. I'm like a bar is evil. It must not exist so I initialize it emit and then I have a I need to connect my video input. This was just the video input from the example. So then I need to put capture a perfect captured, got me LT, because P 5 has its own wrapper for Dom elements.

B

Okay, so now, I have this and I'm still not gonna, see anything's I'm, not actually getting my face points so that I need to go here, and this is the line code I need. This is the friend that I need I'm going to come back here, go into my draw loop and have called this line, and then what I want to do is I want to iterate through all of the positions it gives me like.

B

However, my positions like 60 something and draw like little circles on all them, so we can like visualize those points, so I'm going to need to do. I have to put this in it, and this is just a thing that I know from running through this. A bunch of times is that positions doesn't get defined and maybe at least you have to just oh sorry, you have to do it like this, because I want to do positions dot for each and it was like people ignoring the errors. Like positions is not fine, but I'm.

B

Gonna save you I'm gonna, save you that okay, so yeah I have my for each loop and then I want to draw a circle each point. So first I need to set my fill. Color I'm just gonna make red, then I need to set I'm gonna, be no stroke and then I'm gonna draw a circle at the point, so I'm gonna do circle positions at zero positions at one.

B

So the way this positions array works is that it's an array of two-dimensional arrays where 0 as X and 1 is y, so I'm just gonna, be circle, 0, 0 5 and be real tiny. Of course, let's see, oh, if I think I spelled it wrong positions. Wow I made up a new variable, ok cool! So now, let's see if that working a has my face points.

B

So that's fun. So now I need two unlit allure. This three now I need to figure out what my mouse distance is, and so the way that I do, that is I, need points. 60 and 57. I need to like find the distance between them and I can't just subtract the Y components I need to like do some vector math like if my face is tilted, but luckily I can just let p5 do all the vector math for me and I don't have to think about it.

B

So what I need to do is create two variables mouth top and mouth bottom. This is going to be create vector. It's gonna be positioned at 60 0 at 61, it's bigger, so you can see it and then I'm gonna do the same thing but mouth bottom, but it's gonna be 57 57, then I just need to get the vector difference and magnitude of that difference. To get the distance of the mouth so I'm gonna do Const mouth fist, it's gonna be now top dot subtract. Now bottom got neg and this is all p5 functions.

B

This is all p5 vector stuff. So now what I want to do is like I need to figure out what the range is for my mouth, and this is God screenshot, that so what I need to do is just like see what range of values I get, and so this is. This is really common. When you're trying to work with some kind of input for in your creative coding projects, you've got to like see what your range looks like and usually notice, like print it to the console mouth yeah.

B

It's because Mouse dist is not defined now.

B

You, okay, so it seems like it's a number from like 3 to 17 I can't talk well, I'm I'm, trying that out.

B

Okay! So now we have our values. Oh, so I need to make this global and to do let or more global announced bist.

B

Don't need to print it out anymore and then I just need to connect it. So let's connect it to the amplitudes and then I can just do mouth now. Fist, which I know it's a value. What did I say it was between like 3 and 17 I'll make it 4 to 17. I'm gonna make this value between 0 and 0.5 in case it's like really loud: okay, okay, Mona, truth.

B

Nice and I have this like kind of stupid mount synthesizer, but it literally took me like 10 minutes to make, but it was pretty exciting, and this you know I, could make this much more exciting and like make the sound elements more exciting, I could use like a better base tracker. This is just the easiest one to use, but I could also maybe connect it to the frequency to good. Try. Let's try that we could do just do the same thing: Mouse fist, which is a number between 4 and 17.

B

So yeah, that's your mouth, synthesizer project, cool, I, hope, you've enjoyed learning how to make visual art and a weird mouth synthesizer um cool. So now, I want to go just sort of have some closing remarks here.

B

Well, I've sort of summarized my points about creative coding in the slide that you probably can't read. If you want to like screenshot it and take it, read it later, but really I've tried to cover just like some of the basic stuff you would do in creative coding and a lot of these principles apply to any creative coding environment, whether it be something like open frameworks or Maximus, P or Sonic, PI or shaders or whatever all all applies. I want to encourage you to learn p5, which I love so much.

B

This is a screenshot from a YouTube channel called the coding tram, just talked by a person named Dan Schiffman who's on the on the board of frosting foundation and makes really amazing toriel's. But there's tons of other ways to learn like you can read, there's examples on the p5 website, so many books I also encourage you to join the p5 community, which just really inspires me every day. This is a picture from the p5 contributors conference, which happened this past august at carnegie. Mellon see here's just a really amazing opportunity to meet in person.

B

These are a lot of people who I like talked to you over the internet over github a lot as I'm working on the web editor. It's really awesome to see everyone in person I also encourage you to contribute. If you have wanted to contribute to open source tools, this is a great place to start it's a really friendly and open community and there's so many different ways to contribute. You can contribute code, but contributing to open source is not just writing code. You can do you can do documentation, you can teach you can create.

B

You can also just donate money if you want to donate money to processing foundation and keep this project going. P5 and the web editor highly recommend classy I just want to say thank you. So much for watching, if you want to follow me on Twitter or github, these are my handles and I hope you have learned something or loved watching me. Frantically livestream are frantically laughs good! Thank you. So much for watching and I hope you enjoy the rest of the conference.