GitHub GitHub Satellite 2020 - Workshops, 8 May 2020

Previous Meeting

⏯

youtube image

►

From YouTube: Finding security vulnerabilities in JavaScript with CodeQL - GitHub Satellite 2020

Description

CodeQL is GitHub's expressive language and engine for code analysis, which allows you to explore source code to find bugs and security vulnerabilities. During this beginner-friendly workshop, you will learn to write queries in CodeQL and find known security vulnerabilities in open source JavaScript projects.

GitHub Satellite: A community connected by code

On May 6th, we threw a free virtual event featuring developers working together on the world’s software, announcements from the GitHub team, and inspiring performances by artists who code.

More information: https://githubsatellite.com
Schedule: https://githubsatellite.com/schedule/

A

You, okay, come everyone to the session on finding secure abilities in JavaScript with QL.

A

My name is Aditya Schad I am a security architect in the professional services team here at github, I work mainly with our customers to help them deploy and successfully use, github very exciting security technology, including code scanning and code QL, which you will have seen many announcements about over the course of this conference.

A

A part of that work that I particularly enjoy is being able to teach our customers and also the public github security lab community about code, QL and writing queries in this language in order to find security, vulnerabilities so I'm pleased today to have you in this interactive workshop and to be presenting on focusing on finding vulnerabilities in JavaScript code using code. Ql I like to also extend a warm welcome to our facilitators.

A

We have with us many experts from the Cote or JavaScript team, Oscar felt house s been Andreasen and Eric Kristensen, as well as my colleague, Luke Artie, also on the professional services team, who delivered a workshop on cocoa for Java. Earlier in the day, which some of you may have been part.

A

In today's workshop, we will cover a brief idea of what Coach QL is. We will walk through what the technology does and a little bit of. What's going on under the hood, we will learn how to write code. Ql queries the basic structure of the language, the features that it exposes in order for us to analyze programs, and we will see how to use it to identify patterns in source code.

A

Then I will interactively guide you through the process of writing a query. We will focus on a well known, JavaScript, open source project known as bootstrap, and use it to find a known vulnerability in a historical version of the codebase. This session is designed for beginners no prior knowledge of code.

A

Ql is necessary and although the session is focused on JavaScript and some detailed knowledge of the code, QL libraries for JavaScript, the knowledge that you acquire here will also be transferable you'll, be able to use it to analyze other languages with code QL, as well as other projects written in similar languages. Not just this, please look at the link. That's present the slides to get the workshop documents. In particular, there is a file for the JavaScript workshop.

A

If you'd like to follow along with the hands-on session, then you will need to set up a cultural development environment. Follow the instructions at this link, follow the setup instructions and ask in the site channel if you have any questions as we get going, I'll spend some time delivering some content about Coach QL itself. So if you're still in the process of getting set up, now is a good time.

A

Just start I'll check in with our moderators periodically to see if there are questions of particular interest for the entire group, or maybe answers to those questions that we would like to share.

A

So, to begin with, what is code QL? It is an expressive language and engine for code analysis. The key principle is that we can treat code as data. We take the source code, we convert it into a database representation that contains facts about the program. Now that we have such a database. We have these facts about the program. What can we do with it? We can ask questions to this database. We can query it using an expressive and declarative query language.

A

We call QL to identify patterns in the code, so we've got some information about the source code we put in the forum database and we might query it to find out particular patterns that may exist. Code QL comes in the form, various forms of tooling there's an IDE ID extensions, such as what we will be using today and there's a CLI that allows us to both create databases and run queries.

A

So now that we're treating code is data, we've got some language that we can use to express queries about this data. What can you do with this? What's? Why goes through this effort? The primary use case is to find bugs and security vulnerabilities in source code developers are human. Unfortunately, this means that we make mistakes. We make many mistakes.

A

Sometimes we make the same mistake over and over again in the same place or in different places, and so it is useful to have a tool that can describe patterns in a power fully and automatically search for those patches that underlie those mistakes. It is a high level and expressive language, which makes it very easy to write. New analyses expand the scope of those analyses to find new patterns, as well as to refine them. So we can keep the results that we find precise without having a lot of noise.

A

Then we can codify this knowledge of what is a security vulnerability? What is a bug that I care about in my code? We can codify this knowledge in the form of a query. Now this is programmatic. It's clear, it's shareable with your team, and so we've got a way of encoding our knowledge as developers as security researchers, as security team members into the form of a program which can be automatically run in the form of like a repetitive code review. It can be shared. It's it's an actionable way to share security. Information.

A

Now I mentioned that we describe these queries in a new language, and you might quite recently ask why, should we have a new language? What's what's so different, that I have to invent something totally different to analyze? Are there enough languages already cook? You all has an interesting and fairly unique combination of features which make it particularly suited to writing these queries to analyze source code.

A

We'll look at some of those here to begin with, it is a logic, programming, language. What this means is that you described through logical conditions, a pattern that you want to find.

A

We can combine these conditions using certain logical operators and or and not, for example, to build up more complex logic about the patterns that we care about in the source code.

A

It is declared the order in which you describe these conditions does not matter, and there are no side effects. So in practice these features mean that you can focus on describing what it is you want to find and the code QL engine will figure out how to find it. This is kind of different from an imperative, more traditional programming language, where, to some extent you have to describe how to do the thing that you want the computer to do cook.

A

You else, query optimizer will take the query that you write at a high level and it will compile it down and optimize it into an efficient form that can be evaluated. So you don't need to think about how these results are being obtained from the source code. An interesting feature for such a Kuei language, particularly you've, seen other query languages like SQL before is that code QL is object oriented.

A

We will explain concretely what this looks like in a few more slides, but the overall benefits are similar to what you might expect for object, oriented languages. We can encapsulate together data and operations or behavior on that data, and we can use nice reusability features things like inheritance and composition.

A

We leverage this by creating rich standard libraries for code QL for each language that is supported by the technology. We have a set of libraries that provide a comprehensive representation of source code written in the target programming language, for example. In this workshop, we'll look at a few features of the standard library for JavaScript.

A

These libraries help us look in a simple way at different patterns in the source code written in those languages, but they also provide us more complicated machinery for doing common program analysis tasks, things like analyzing control flow or data flow, or the ranges of possible values in the program.

A

These libraries are open source you're. Welcome to see all the details of what goes on. We look at a sample of those here when we write Christ it.

A

Finally, the database I we look at is read only and the cultural query. Language is read only as well. This is also different from another Cray language like SQL. There are no options for adding new rows or modifying the database. The database represents the source code at a particular point of time, which is built in a single step. If you want a database for a code that has changed, then you can build a fresh database and then you can analyze using the same queries that you wrote on the new database instead.

A

Okay, so now, I've talked to you through this high-level idea of this is a nice language and it has these features. What does the quay look like? What does the program look like when it's written in code qeh?

A

Here's an example of what that might look like the basic syntax is hopefully familiar if you've seen something like SQL or the query languages. There is a query Clause. Usually it is this from where select pattern and there may be some inputs which bring in additional libraries.

A

Let's look at the Cray clause in some more detail and see if we can figure out what it's doing. The from clause declares a number of variables.

A

Each variable has a type and a name. For example, in first variable we have a type declaration if statement with a capitalized I, which is from the code QL standard library for job and the variables name is if statement lowercase I types in code QL represents sets of possible values. So the idea here is that, if statement, which is from the standard library, reference represents in the source code, the set of all if statements in the program, similarly block represents all blocks which are usually denoted by curly braces in the code.

A

Variables also talk about a set of possible values. What do these values start with? They start with all the values in that type, so you can think of. We start off with all the. If statements we start off with all the blocks, and then we might want to say something more about those variables we might want to describe some relationships between them. We might want to restrict the set of values that they might represent.

A

The where Clause allows us to specify those conditions that restrict the values of the variables and we can combine these logical conditions using different operators like and let's look at the condition here. We provide a condition that specifies this block. Variable is equal to the result of get then called on the. If statement this shows the object, nature, object-oriented nature of the language as well on the type. If statement we have an operation or a predicate called get then, which returns the then statement to then block, and here we do an equality assertion.

A

We state that the value in the variable block is equal to the then part of the. If statement, this is not an assignment unlike another programming language, but it asserts an equality it states as these two things must be the same, and we see that there is another condition combined using end, which says that the block must have no statements within it and by combining statements, these conditions we enforce that both then must hold true, because we've said: we've used the end key.

A

Finally, there is a select clause which describes the output of our query here. We select a particular element of the source code, in this case the if statement- and we select a string message. This is usually so that we can produce a nicely presented output to the user. You can think of this as producing a table of results. In this case, it will have two columns. The first column will be the if statements that we care about that have an empty, been block and the second column will be the string message so stepping back.

A

If we look at this query and read it kind of like it was in English, then we see that we are looking for. If statements which have an empty, then block the imports at the top. Allow us to use the standard library. In this case we can use something like import, Java or import JavaScript. You pull in the code, QL standard library, for the language, other imports are possible, but many features already provided by the standard library.

A

Let's look at some building blocks that are used when creating queries so that we have. We know the useful tools that we need when putting together our own.

A

The first building block is a predicate standard programming. Languages have functions or methods code, QL has predicates. These provide a way to encapsulate portions of logic in the program so that we can reuse them. You can think of them as, like a mini query, a mini from why I select they declare some variables. They define some conditions on those variables and effectively like a select clause. They produce a set of rows in a table, but the difference is that you can give it a name so that you can reuse this logic.

A

This table in other parts of your query,.

A

For example, we might want to write multiple queries that identify empty blocks from the program, or maybe we want to do some more complex reasoning when we know that a block is empty. We could do this, for example, by converting the logic of our previous query into a single predicate that describes the set of empty blocks. This is done using the keyword predicate to define the predicate and then providing a name. In this case, it's called is empty, a set of for variable declarations.

A

In this case there is one parameter which is of type block and is also named block. This is like the from clause in a query. The body of the predicate then describes some conditions on these parameters. In this case, we say that a block that we care about is empty if it has no statements within it, so we say that the predicate is empty holds or holds true, when there are no statements in the block.

A

Now that we have such a query a predicate, we could use it to replace the equivalent part of our query that we had before we could change our where clause. For example, to call this predicate so here we see a call to is empty and we pass in the den block of the if statement that we described in another variable.

A

So these two queries on the left and right are equivalent in the second query: we're just using a predicate to separate the logic that says when a block is empty and we use it in our where clause so that we find, if statements whose then LOC is it same, results.

A

The second feature of building block that we will talk about today. It's classes, classes, define new types and remember earlier, we said that types in Cody I'll describe sets of values, collections of possible values. We've already seen two classes from the standard library. If statement and block- and we could use classes from the library or we are free to define, are how do we use a class or how do we define classifier? We could use the class keyword to create a class and give it a name.

A

Then we need to provide a super type or a number of super types. All classes must must have at least one, and these super types define the initial set of values that we begin with in our class. In this case, for an empty block, we started with all values that are blocks.

A

Of course, it's not very interesting if you can only copy another class and you can't change the values that are in it, so we might want to restrict the values in the cross in some way. In order to do this, we write a characteristic predicate that is this section here. It looks a little bit like a constructor in most standard programming languages. It is true that it defines the membership of the class, but note that we are not creating new values. We are just talking about values that already exist in the database.

A

In this characteristic predicate we add the restriction saying that a block is only in the class empty block if it has no statements within it- and here we use a special variable this to describe the current value whose membership of the class we are considering. A value could belong to multiple possible sets. For example, you could be both an empty block and the block, and so it's possible for a value to have multiple types.

A

This class is pretty much equivalent to the predicate that we had previously. We just turned the predicate into the characteristic predicate here. The difference is how we use this in a quit. Now we could use the instance of keyword to assert that a certain value in this case, the den block of an if statement, is an instance of or belongs to a class. So here we have the same query. The meaning is the same.

A

We started by finding if statements and their den blocks- and we assert that the den block belongs to this set of empty blocks.

A

We could also phrase this another way. We could rewrite it to use another temporary variable, which is defined with the new type that we create. So here we could define the variable block of type empty block and then by providing this equality assertion by saying that the den block of the. If statement is exactly this variable block of type empty block, we insisted they must be the same, often with code keo. You will find that there are many equivalent ways to write the same query.

A

This is completely fine, there's, often no hard and fast rule for which one of these you should do so long as it is clear, readable and ideally, reusable and, of course, if you have questions about about style or about different ways to phrase a query during the course of this workshop, don't hesitate to ask us and we can advice.

A

So at this point, I think I've throw a lot of information at you and spoken quite a bit in in the abstract. So let's move to the hands-on part of the workshop. If you haven't already done so, please follow the link on the slide, get your coat ql development environment setup using the database that we provided for the bootstrap code in javascript. If you have any questions- and please do ask them on the stat channel, where our moderators will be able to help at this point, I'll pause for questions asked.

A

Are there any questions that you think should be shared with the audience.

B

No, it's not common questions. I think you carry on sounds.

A

A

Let's move now to the track, the workshop again I'm, hoping that you are set up with your development environment. It should look something like this: I have Visual Studio code open here. I have installed the code, ql extension for Visual Studio code. You can see it here, the extensions view, which is also where you can install it. If you don't already have it, I, have checked out the code. Ql starter workspace from github.

A

This contains a number of query folders to hold my custom queries and a copy of the code to our standard libraries over here. I.

A

Have open here, friends the same workshop document that is available in the repo that you have been linked to will be following this process as we go through the workshop I have downloaded unzipped and imported a database containing a copy of the vulnerable code of bootstrap that we will be analyzing during this workshop. Please make sure that you have it here and have it set as your current database a.

A

Few, a few basics about making sure that your setup make sure that you follow these instructions here. If you get stuck or you're unsure how to use the code ql' tools, there are documentation, links, there's good general documentation on the process of learning codes, UL and specific information about using it for JavaScript, there's, also pretty good documentation on using the vs code extension, which also has some good general tips for setting up Visual Studio code to verify that you've got your environment set up.

A

We can look at an example quai, which is present within these demo. Folders there's, usually an example that ul found you can find one in could ql custom queries javascript, I have it opened down here. I can run the query in two ways: I'm the right-click menu. I have the command code. Ql run query.

A

It should allow me to run the query. Similarly, if I bring up the command palette, this is control, shift, p or command shift p on Mac and you search for code QL run query. This will allow us to run the query in a similar manner when I run such a query. I will get tables of results.

A

Which looks something like this: some craze are so that their output can be nicely formatted, and so we will see something like this. Other queries will just produce a raw set of results which looks something like this and we can click on them to go to the relevant location. The source code.

A

To begin with, I'll talk about that we're trying to find the particular kind of security, vulnerability that we are interested in.

A

The project that we'll be looking at in this workshop is bootstrap a well known open-source JavaScript framework used for creating web applications.

A

Bootstrap relies on another library known as jQuery another extremely popular but somewhat old, open source, JavaScript library that aids with things like HTML, manipulation, event, handling and Ajax calls. Jquery is extensible. It supports modular plugins, where we can add functions that extend his capabilities and bootstrap makes extensive use of this plug-in mechanism.

A

However, at some point in the past, the jQuery plugins in bootstrap were implemented in an unsafe way. This could make the users of bootstrap vulnerable to a security vulnerability known as cross-site scripting.

A

This vulnerability, as many of you will be aware, arises when user controlled data or attacker controlled data might come into a web application and he rendered on a web page. An attacker might send malicious code generally as a browser side script to a different end use.

A

The document has a link to a pull request which fixed a number of such vulnerabilities in bootstrap, jQuery plugins, and each of these was assigned a CVE.

A

The core mistake that happened here that causes a vulnerability, looks something like this.

A

We have a call to the jQuery Dollar function. This function is powerful. It is capable of many things. It is capable, in particular, of processing input and interpreting it in different contexts such as HTML.

A

The intent here is that we have some options which are provided from a jQuery plugin, which are here in this variable named at options. We obtain one of the options in this case text, source selector and the plug-in uses that selector as a CSS selector. It uses it to decide which HTML elements to look at, and then it reads text from them.

A

That is the intention, but the problem in this example is that if you pass this into dollar, it will execute JavaScript code if the value of options dot text source selector, is a dangerous string, for example, if it is a HTML string of this form, then by passing an into dollar, we will get the evaluation of this javascript code, and so the values and options cannot always be trusted, and it is dangerous if we pass them in to a place like dollar, which is going to evaluate in security terminology.

A

Those plug-in options are sources of user input and the argument of dollar is a cross-site scripting sink one method of fixing. These kinds of vulnerabilities is to use a more specialized and safer function, something like calling dollar on it on the document and then using find on the input selector, rather than passing it directly to do so in this challenge, what we're going to look at is to use code.

A

Ql try lies a source code, a bootstrap we've, given you a database that was taken from before these vulnerabilities were patched and we're going to use that to help identify the vulnerabilities.

A

Are there any questions at this stage?

A

B

And yeah there was a bit of confusion about which repository you could find like the markdown file. In an example, query and I think we might have to clarify again that the repository people were asked to clone sure it's not the one that contains the workshop. That's.

A

Right we should explain there. There are two works. There are two repositories in question here: the one that we've linked you to is the workshop repository that contains this much this workshop file. It's called JavaScript MD in the in the satellite workshop repository. So if you're trying to follow the steps to follow along, please look at javascript on MD in the instructions that are provided within that document, we ask you to check out a repository repository called vs code code. Ql start.

A

That is the repository that contains these folders and a copy of the standard code. Your libraries, that's what I have here. Yours might look a little different. There is a couple of files here that I've added, so that I can present this information to you and the same screen, but you should have the same folder code, GL, custom, queries JavaScript with an example.

B

So people asking about where we combined slides.

A

We've not made the slides available yet, but the recordings that this workshop will be made available, I believe 24 hours after it's complete and I think it should be possible for us to make the slides available.

A

Great, thank you all for checking in so we've described a security problem. Hopefully the nature of what we're looking for is clear. What we will do now is we will go through the interactive part, this workshop and we will write queries to find this kind of vulnerability. The workshop is in several steps.

A

You can write one query per step or you can have a single query that you refine as we go along. I will take the approach of having a single query file that I keep modifying with each step. You'll see there is a hint which is expandable and provides useful classes and predicates in the coke ul standard library for JavaScript. That will be useful to you. When you write this query, you can explore these in the IDE using autocomplete.

A

Remember that control space will give you suggestions from the code QL standard libraries for what to write next and jump to definition can be done using ctrl-clicking or f12, which will take you to the definition of different parts of the standard library. These are all written in code, ql4, transparents,.

A

Each step does have a solution. What I will do at each step is I will pause and I will set a timer which looks something like this and we, you will see a timer at the border of my editor window. During that time, you will be able to try out the problem. For yourself start writing a query to tackle each step and after the timer is out, I will explain the solution and we will move forward.

A

To begin with, we'll need to add a new query file, so we can write craze in it. I can do so by adding and you felt here making sure that I'm in the JavaScript, folder and I can call something like unsafe, jQuery or unsafe dollar code.

A

To begin this query, I will need to import the cookie all standard library for JavaScript, so that I can use old libraries.

A

Let's look at the first part of this book. Sure I mentioned that calls to the jQuery dollar function are interesting because there are sinks of cross-site scripting vulnerabilities. Let's try and look for those calls now, but let's start even simpler, let's look just for all function. Call expressions, there's a couple of examples here is a call to a function called alert. This is a call to a function called say hello, so I'll give you a couple of minutes.

A

Fine write, a query that will find all function. Call expressions, remember that you can use the front keyword to declare variables and the Select keyword to choose the output of your query.

A

I'll give you a minutes to get started.

A

A

All right, if I expand the hint and also if I start typing the autocomplete and I look for something like a call. I find the class call expert which says that it is a function call expression. I will use that and declare a variable of this type, maybe I'll call it call and I will output that as my select cross.

A

Let me see if my file and run my query and we'll find all places in the code that are function, call expressions.

A

Will get quite.

C

A

Of these a few thousand of those- and we see different- calls to different functions. Okay.

A

Now we said we care about not just a call but about an argument of a call. So having identified these call expressions I like you to identify the expression that is used as the first argument of each call. There are some examples here if we have a call to something like alert, and the first thing in the brackets is the first argument: try doing this now, I'll give you a couple of minutes to get strong.

A

A

Let's look at how to find the first argument of each call. We could do this by adding another variable to our from clause. For example, I could declare another variable, which is an expression and I'll call it arc.

A

Now I need to say something about these multiple variables I've created I need to relate them in some way. I see that there is a predicate on call expert which is get argument, and that allows me to provide an index which is zero based, so I'll provide 0 to look for the first argument and I will equate that to my park. Having created this new variable, let me also output it. As the result of my query. Now, I've got a call I'm finding its argument at index here.

A

A

You know I find a bunch of results as an example. This one I have a call to a function called require and I find. This first argument, which is the string assert okay, so far, looks sensible. Now, I didn't want care about all calls. That would be far too many results. We have nearly 7,000 of them. We care only about calls to the function dollar, so the next step is refine. This may be forked clarity will will rename our variables I'd like to call them something like dollar call and dollar arc.

A

And, let's add the restriction that not only are the call and the argument released in this way, but also the call is only to a function named dollar. Do that.

A

A

As an aside I'm, seeing there are questions about different equivalent ways to write the same query. All these are completely fine. If you choose to omit the dollar org variable and you select this line instead directly as perfectly sometimes it's useful to create the variables because we might need to use them in a later query that we're writing. That's pretty much. The only reason.

A

A

A

We can add a restriction to our crate by the and keyword in addition to the condition we have here and let's see what we can say about dollar call. If I look at the autocomplete, I can look for the collie, which is another expression that denotes the function being called by the more I can look for the collie name, which directly gives me a string. This seems like what I want and I can equate this to the constant string dollar.

A

Incidentally, if you look in the command palette, there's a format command which will allow you to format your codes well, according to the style guide, I have it enabled automatically?

A

Let's run this query, this should bring down the number of results from all these calls, in their first arguments, to only those that reference a function whose name is dollar, and indeed, if I look at, maybe maybe this one I see there's a there's, a call. The Kali is called dollar and I've identified. The first argument.

A

Look sensible so far, but this seems a little bit unsatisfactory in that we've done this kind of string matching where we said, we've insisted that the call is exactly to a Kaali whose name is dollar now. What if I imported jQuery in some other way?

A

Are there other ways in JavaScript code of calling the jQuery function, probably and I don't want to have to handle all those different cases when I'm writing a query, because that's not the primary focus of what I'm doing so perhaps the code cure library can handle this for us and in fact, if we look around the code cure standard library for JavaScript, there is a built in predicate of the name jQuery which identifies or returns all references to dollar. So every place that function is referenced. We can identify with this jQuery predicate now.

A

This is gonna, be slightly more involved, so I'll explain the hint for you.

A

We can use jQuery to find references to dollar week, get a call on the result of that in order to find places where the function is called remember here we are interested in calls $2, and then we can get an argument in a similar way at a particular index. You will notice that you'll get a different result. Type. So I'll give you a couple of minutes to try this out and change your query, and then we can walk through what is different and what the new query looks like.

A

A

A

Just give this a try, I'm going to just clear out my wear clothes, so I can start fresh and right using the new API that I've asked you to use. So, let's start by calling this predicate jQuery, the help tells me it gets the data. Let me refer to the jQuery dollar function: okay, having called jQuery. What can I do on it? Can I get a call to jQuery I can get a function, call to Jake print we'd get a call now I, don't want the call I want the arguments.

A

So, let's see if I can get an argument, I can get an argument, but I don't want any argument. I want an argument at a particular index.

A

So, if I look at this help here, I see that I can get an argument at a particular index. I- and this is my dollar arc. I.

A

Don't need all the call here so for now, I'm just going to get rid of it.

A

But notice that the compiler is still unhappy, the types don't match we were previously asking for an expression. We have now claimed something of type node.

A

These aren't called dataflow nodes. They represent places in the program that may have a value, different kinds of parts of the syntax expressions parameters. Functions calls could all be associated with dataflow nodes. Now we're moving into a more sophisticated part of the code cure library, where we are reasoning not just about things that are written in the source code, but the possible values that they might have and the reason we found these is because this more general handling of finding all places that jQuery is referenced might involve the flow of information.

A

We might have some assignments through local variables. We might have other forms of indirection that mean that we have more indirect references to dollar, and so the dataflow library and dataflow nodes describe these parts of the source program that might have a value and let us reason about them in a more complex way. Now we could make this type check by converting back from the nodes into an expression like. So we have a node which is returned by each of these successive calls and we use as expert to turn it back into an expression.

A

That's one! We try this crate, alternatively, and perhaps more usefully for what we'll do later, we'll change the type of the variable that we declared to insist that it is a dataflow node from the dataflow library.

A

A

This, if you have come over from the web, Java will have also seen dataflow notes used in a similar context, where we reason about the flow of information to the program. We're going to use it in that same level of detail, but for now, and no prior knowledge I said, was necessary.

A

Dataflow nodes represent places in the program that may have a value, we'll see in a few moments how they can be made more expressive, how we can identify those same nodes for different parts of the program and what we couldn't do with them to think about where those values are going.

A

So, if I look here, I see that I've identified. Similarly the arguments of different kinds of call to dollar, in particular the first argument, which is the one that I care about it, doesn't matter how complex this argument is. It might be an entire function if it's being passed a dollar it's being identified by this query,.

A

So this is the first section and fyz, if you remember the security problem that we were talking about. These are the sinks of our vulnerability everywhere. That dollar is called and receives an argument. It might interpret that argument as HTML, and so this is considered dangerous if we can find attacker control data that might end up in this argument.

A

So let's keep this logic for now and I'll pause and see if there are any questions that you that are sufficiently general that we might want to share with the entire audience before we move on to looking for the source of this kind of untrusted data.

B

And I think we can continue here.

A

Sounds good and.

C

A

Course I imagine our moderators are keeping well on top of all the questions because they said to me so don't hesitate to ask him there. If you have questions about the details and what's going on here.

A

I'm going to use this logic again later so I think I'll just comment it out and I'm gonna start with a new part of a query that I can use to continue now. I mentioned jQuery plugins and we care about finding the options to those plugins which we think are controlled by the outside world by the attack. What is a plug-in definition? Look like they're, usually this kind of JavaScript code. Usually there is a reference to dollar FN and then one of its properties in this example is called copy.

A

Text is assigned and the value that's being assigned is a function. That's our plugin! That's something that comes from the outside row. Rather, that's something that extends the functionality of jQuery and its options parameter comes from the outside world. So now, let's focus on finding such plugins and their options.

A

Now you've already seen a nice way of finding references to the jQuery dollar function. I like you to now find all places in the code that access dollar and then they read a property called s n on dollar. If you like, a hint about how to use this in the library and expand the hint I'll, give you a couple of minutes to try this out yourself.

A

A

A

A

A

A

Okay, how might we do this I'll expand the hint we can, as we did in the previous career, that I've got commented out, will declare a new variable and keep it as a data flow note for now. I'll just call it N and we want it to be.

A

Something that reads: a property FN on dollar so well, I'll, say n is equal to something. Let's start with dollar. We know how to find those such as calling jQuery and what can I get on this now I will again lean heavily on the autocomplete and the library. To give me something useful, get a read: get a property read: I can get a read of a property.

A

Let's look at the documentation gets a read of any property on this note, I, don't care about any property I care only about the property FN, so I'll keep going. I can get a read of a named property on this node. With this with this predicate that looks like what I want, and so I will replace this parameter here. This argument, with the string that I care about which is FN.

A

Dola I find places in the code that dollar ends up where it's property FN is red. Let's run that a.

A

Lot of learning to use code well is learning to use its libraries, but you are not magically expected to know all the things that exist in the library, although we provide them as hints to you here, follow the same processes that we're seeing in order to use the autocomplete to guide. You use the documentation to tell you what is available on the types that you have in school. This looks promising. Let's look at a couple of these that looks like a reference to dollar FN.

A

They all seem to be the kind of pattern that I'm interested okay.

A

Now we found places that read the property dollar Devin. What do we care about? We wanted to see where, on dollar, dot FN another property, it could be called copy text, it could be called anything else is written and a value that's written is or a sign is a function.

A

So those are the jQuery plugins. Now it might be written. The code in JavaScript might be written this nice straightforward way. We might have a single line assignment, which does this neatly, but there's also a lot of possible variation. We might have intermediate assignments, you know you might store FN in one variable. You might store the function in another variable and then you assign them- and this is slightly annoying if you're trying to write all of these possibilities out manually in particular to handle them robustly. We need something more sophisticated.

A

This is called data flow analysis. Data flow analysis helps us answer. Questions like does this expression hold a value that might have originated from some other part of the program, now you've already seen dataflow nodes, which are individual places in the program that may take a value, and we've already started using some parts of the library that return dataflow nodes as a result.

A

Now, if you're from with how programs are structured, note that these are distinct from, what's called the abstract syntax tree nodes, which represent the direct structure of the program, this lets us model data flow and the information flows through the program in a more flexible way.

A

When we're talking about data flow, you can think of it as there being a graph. The nodes of this graph are the dataflow nodes places that might have a value that correspond to some part of the source code and the edges represent the flow of data between those elements like through an assignment. If there is a path between two nodes, that means that data might flow from one to the other. When the program is executed now code, q else, dataflow library is pretty expressive, we see nodes already.

A

In fact, if we look at the results of this, we see that we get a special kind of node called a source node, which is where a value might originate. There are many other kinds of nodes function, nodes refer to functions, called nodes or function, calls parameter, nodes are function, parameters and value. Nodes are anywhere that might have a value check out the dataflow documentation for a better description about the different possibilities.

A

Here, when we are looking for the flow of information in this manner within a single function or scope, we call this local dataflow analysis. This is generally more feasible to compute and trying to think about this. Draw the entire program, and the library here for JavaScript analysis has a lot of predicates that do like local dataflow for us. So we don't need to manually reason about reassignments and those kinds of indirection every single time. We write such a query and in fact, you've already used one of these predicates I sneakily kind of suggested.

A

You use it without telling you what was going on under the hood when we call get a property read this handles the case where dollar, when it's first imported, might flow to some other place where it gets used, and then we refer to the FN property on the use. So now what we care about, if I go back up here, we care about places where a property is assigned. The property is on the object, dollar, dot, FN and there's some value that is assigned to it.

A

That might flow into FN, expand the hint and again I'll give you a couple of minutes. You know the drill find a place where find a function that is assigned to the property.

A

C

A

How might we do this I'm going to look at the hint that I provided you to remind myself of the right parts of the library we've identified, FN the accesses to that property? Now we want some node, which is assigned to a property of FN, and we don't care about what that property is as long as it's some prop.

A

So we see that there is a predicate, get a property source and again, how would you discover this well search for things to do with properties and I, see that get a property source finds me a source. Note some other value whose value is stored in a property of this node I could identify the named property. If I only wanted to look for copy text, for example, but in this case I, don't care about the property name.

A

As long as there is one so I will call the more generic version of get a property source, so I'm changing the thing that I'm finding now I'm, not change I'm, not finding FN I'm, finding the value that gets assigned to it.

A

Rather, the value that gets assigned to one of its properties.

A

If you're curious about what this is doing, we can always go to the definition and see what it's, what is, how is it implemented? What's happening? We see something like the result flows. This is local data flow, a place that is a property right on the right hand, side. So a difference is a property. Right is an assignment, and we want to find some value that ends up on the right hand, side of such an assignment.

A

Let's look at the results that I've found now this would be we'll be different. I find these functions that are often called plugin and here I have not flagged it, but I here I see that this plug-in function gets assigned to the effects property of dollar defense. That's one of the examples that I care about I also find more boring things like this, where I'm just referencing the value here, that's gold!

A

Oh no, sorry! This is flight because it gets a sign down here to the FX property. So, similarly, if I click through these I'll find a whole number of jQuery plugins that are all assigned to some property of dollars on FN.

A

This is already pretty powerful, we're reasoning in quite a sophisticated way about the flow of information through the program in order to find these properties and their values. Now, there's one more step that we need before I before we find the thing I asked you to find, which is the source of untrusted data. Now we said that these are the not just the functions themselves which are plugins but they're options. Their options are the first parameter. Excuse me they are the last parameter of these plugins.

A

Most of the examples that we see here actually have only one parameter. It doesn't matter too much, but there are some examples where a plug-in might take multiple parameters and the options are always the last one.

A

So these are values that might come in from the outside world. They might be controlled by a user or by an attacker, and so the final task in this section is having done this work of identifying values that I get assigned to properties of FN, which are usually functions. We want to find make sure that they are functions and we want to find the last parameter of those functions now expand the hint for some useful information on how to find functions in those parameters and give it a go.

A

A

Okay, so we found these bad signed, two properties of FN, and we found that they are plugins they're often functions. The first thing that we can do is insist that they are always functions so I can use the type data flow function note. This is a subtype of node that is slightly more restricted. I can use this to say if this is the plugin, let me rename my variable for my own understanding.

A

So I know what this is referring to in Kokua and now, if I have such a function, note I might want to get something from it. What can I get from this plugins can I get its last parameter? Can I get a parameter at a particular index? I can get a parameter at a numbered index. I can get how many parameters there are. I could combine these to get the last parameter if I work out how many they are and then I subtract one and I pass it in here.

A

There's a convenience predicate that does that for me called get last parameter so I'll use that if I'm getting this parameter, I need to use the value in some way, so I'll assign it to a variable.

A

Let's see what kind of value it returns, it returns a parameter node, let's give it a variable name, something like plug in options and I'll declare a new variable for that parameter, node and I'll call it plug-in options.

A

And we might as well.

A

Let's see, I have two variables: a plugin which is a function and a plug-in options, which is a parameter. The plug-in itself is something that gets assigned to a property of dollar FN and the options are the last parameter. Such a function.

A

What do I have here, I have some of these functions that we identified before the shouldn't. This code should now be looking familiar and I can identify the prompt, the options pround.

A

Similarly, for all these others of them, okay, and if we look at this code a little bit, we can see that the option ends up, there's a few checks, but it ends up here and there's options and that is passed down in various other places. Here, yeah, it's passed into this class, which is constructed and they might go elsewhere in the program. So we think of this as a source of untrusted input.

A

We are able to identify the place where they come into the application, which is here and now we might be interested in saying where they end up, which is something we will look at in the next section.

A

Now, we've already we've written a nice more sophisticated piece of reasoning. The logic that we wrote in section 1 was mostly about the syntax of the source code. We looked for calls to dollar and their arguments and towards the end, we used a more general variation that could handle different ways that jQuery might be imported. Now, we've done something at the next level of complexity, we've identified property, reads and property rights using the library to handle the indirection for us, and we've put that together to find places where jQuery dot, F n dot.

A

Some property is assigned from a function and we find the first sorry the last parameter of that function, which is a plug-in option now I've. This means I, think I think this is a good time to again pause and check for questions. How are folks doing with this stage of the workshop.

B

So we're not getting a lot of questions right now. I.

B

Think it's safe to continue. Ok,.

A

So again, to recap: you know it's always useful to think about what are we doing in terms of looking at the code and what is the problem? We are trying to find it's useful to keep that perspective in mind. Otherwise we get stuck in the details and it's unclear. Why we're doing all this trying to find places that were cross-site scripting vulnerabilities, where a plug-in option to jQuery ended up in an argument? $2.00 now I say these words like ended up in or flowed to, and we've spoken about data flow analysis.

A

So this means that we should have some way of putting these two pieces of information. Together, we can find our sources. We can find our sinks and I want to know whether the untrusted data that comes from the plug-in options will reach when the program is executed. The dangerous called $2, similarly to what you've done here with property reads and property rights. This is a dataflow problem, but it's much larger in scope. The reason is that in my cross function, boundaries, the plugin options and the dollar call might be to in totally different functions.

A

If I look at one of these results here, I don't see an obvious way, an obvious place where it's being directly passed in $2.

A

So this is a global data flow problem. The code QL analysis is well-equipped to doing this. It does not try and compute the global flow of information all the time in all cases, because that is too much and computationally intractable, but we can provide using a nice API that it provides us. We can provide restrictions on this problem. We can describe sources and sinks and we can ask the library to find flow of information from the sources to the sinks.

A

Now what we will do is use a little bit of boilerplate that most queries that find these kinds of problems tend to use. You have it here in the workshop document and so I encourage you to copy over the template and begin filling it in. What I will do is I will write each section out myself and I will talk you through what the parts of the template me. So that is not totally obscure and just meaningless boilerplate.

A

What we are writing here is a path problem, quiz.

A

So we begin by having some metadata at the top, which describes the kind of the query. Let me say that is a path problem, which means that it might give us some nicely formatted output that has the flow of information. There are some other optional pieces of metadata to identify the query. You can add them if you wish, but they are not essential. Here we will. We have already imported the JavaScript library. We know that this is necessary.

A

The JavaScript library for those of you who came over from the Java workshop needed to import the data flow library, specifically that's already present when you import JavaScript, but we do want another piece of library that will allow us to visualize paths to the graph to the program. So that's what this import does.

A

Having done so, I will create a little bit of boilerplate to describe our data flow problem. I will define a new class we've spoken about code, cure classes, give it some name and it must have a base now. What I will use here is I will use something from the data flow library. This is a module known as team tracking we've spoken about. The flow of data, keen tracking is a slightly more general version of that which also tracks minor modifications that don't affect the tainted nature of the data.

A

If you concatenate some strings and you started with something that was untrusted, the result is probably still untrusted. The team tracking is will handle all these for us. We extend this configuration cross now. If we go to this configuration cross, we can see how it is defined and some documentation about its meaning. It has two predicates that are most interesting, which we will define here. Remember that predicates are named reusable pieces of logic and they can be associated with costs.

A

The two predicates that we care about here are is source and is sink. These describe the problems, the patterns and code that are specific to our vulnerability. We know what the sources are in our problem. We know what the sinks are in our problem and so we're going to fill these in I'll copy over this.

A

The template I'll need to do something here. I'll also need to do something similar for the predicate is sync, maybe I'll call it sync, so I don't confuse myself, but first of all, this is a class. Usually it is useful to define the values in the class. It must have a characteristic predicate as much of the time for these configuration classes. The name is not particularly meaningful. It doesn't matter what you do here. We can call it configure just for simplicity,.

A

Doesn't matter what this is as long as it's fairly unique so now I had this configuration, I have two predicates is source and is sync and in fact, I'll use the boilerplate here from from the template to start filling in a body for these.

A

We'll fill in the configuration in a few moments time, but I'll show you first what it looks like when you run a query that has this kind of pattern again feel free to copy this over from the template. A query will now look something like this. We find we use a configuration that we have defined. We described a source node and the sync node a sync node, and we see that there is the flow of information from the source to the sink.

A

Having done so and identified that there is a flow of information between that, we can identify, we choose what the output of this query is. Here we choose the sink as the main location of the results, the thing that we click on and get you, but we also output pairs of sources and sync, which will be used to produce a nice formatted result we'll see what this looks like when we run the query and I have a message: there's a string, that's associated with the query result.

A

Now the next tasks field we've already done kind of the heavy lifting of writing this query. We know what our sources are, because we described them in section two: they are the options. Parameters of jQuery plug-in functions, so I'd like you now to complete the is source predicate by taking the query that we wrote in section two which I got commented out down here and converting it and putting that same logic here into the is source predicate. The hint will tell you some standard ways to convert.

A

We can take the variables of the from clause and put them in this exists declaration, and we can take the where Clause conditions and put it in the body down here.

A

By equating some results to this source parameter, this is like selecting that parameter that value as the output of this predicate so I'll give you a couple of minutes again to translate from one to the other.

A

Fill in the definition of the predicate is source using the query. The Regis wrote for section two which handles jQuery plugins and their options.

A

A

A

A

A

Let's see what this might look like now, we have logic Frost it already I'll copy over my variable declarations. First from the front claws, I'll put them into the variable declarations. Can the exists block exist, says that I declare some variables like local variables. I states and properties of them that I want to be true, and the whole thing will hold true if those properties are satisfied. Now, what did I say about the plug-in and the plug-in option? I had them in the where clause down here, I.

A

Said that the plug-in was an access to jQuery and it's FN property and a value that got assigned to that property and its last parameter was the plug-in options. Now, what did I say was a source of untrusted input. It was exactly the plug-in options and I need enhanced to add this additional condition. Now this checks, this also looks a little bit more verbose than I, strictly need it to be, because I see that there is some repetition here. I could just say that plug-in options are my source.

A

So we've just taken the same logic we had from section two, but put it into this- is source predicate if I want to verify. This is doing the right thing. There is another mode of running a query or a single predicate. This is called quick evaluation. Click on the predicate name, right, click, Hill, QL, quick evaluation. This will evaluate only that predicate as though it were its own query, and so this is a great debugging trick.

A

I can't do this here, because I compiler errors elsewhere, my good for now I will just make that pass and try again.

A

Evaluating pieces of your logic as you go along is what we always do when writing queries and it's the best way to make sure that you've got the right results. You find the right patterns and you don't have surprising or missing results as Ecola, easier to debug a smaller piece of logic than to debug, a large crate. That's doing on to too many things at once. The evaluate the isn't just have a quick look at the results. I see that I'm still flagging these options as I cared about.

A

So it seems like I've to preserve the logic of my query.

A

Now, I'd like you to do a similar transformation for the is sync predicate. This time we had the solution in our section. 1 I'll get rid of this because we or used it. If you look at the crate that we wrote in section 1, where we found the arguments of jQuery dollar calls we'd like to translate this into our async predicate, so take a couple of minutes to do a similar transformation here. Your boil page might look something like this. We might not need any temporary variables, because our query here is much simpler.

A

A

A

A

You someone asked me in the last session about the pipe operator and I say: there's a question about that too. This is not or this is you can think of this as something like such that it separates the variable declaration from the conditions that we insist on holding.

A

You, okay, so I, like I, asked you to transfer curry. We had from section 1 into this form of this is sink predicate here. We care about dollar arc. The argument of a jQuery called I can do it something like this here. My dollar Arg is exactly the sink, so I start to a jQuery I find a call. I find this argument at the first index and I can do the same process of debugging and quick, evaluate it to make sure that I'm still on the right track and haven't changed the meaning of these quarries.

A

A

A

It looks similarly like ice continued to find these dollar arguments.

A

Now we put together, we've got a source predicate that describes our sources of untrusted input. We have an async predicate that describes places that do dangerous execution. Let's find the flow of information between them. We've already written all the boilerplate. Let's run the query.

A

Praise the due date of flow analysis we'll need to do more computation than just syntactic matching, and so the first time you run this. There is more computation involved. As you run these repeatedly, you find that it will hit a number of caches and your results will be found. Quick.

A

A

A

Now, I have some results in this cream. If I look at them, I can click through to find the sinks, which is what I chose as the primary location, which are these places where dollar is evaluated on some options. Now, that's pretty interesting personalities is not clear to me where, where the sources, but because we've done this path, problem format, we have this powerful tool where we can explore the flow of information through the program.

A

Through these flow paths, we can start the source which finds us something like this: the option which is the source we can see where it goes through the program through multiple assignments through different levels of indirection. We see that is passed here into this FX constructor, which is here. We can see where it goes from there into a new object, and we see that that new object and in particular its target property, is passed into the dollar function.

A

So this means that if an attacker controls option dot target and that down here be happily executed by jQuery function and interpreted as HTML, possibly in turn executing some JavaScript, we can explore the other results in a similar manner by starting at the source and tracing the flow, through the sink to verify that they are genuine results.

A

This is useful for us as people knowledgeable about security as researchers or product security team members to understand whether these are valuable, vulnerabilities or not, and if there are false positives in fact, I think one of these might be then the value of code ql is that it's almost infinitely customizable. We can use the knowledge that we gain in order to add more limitations to our krei, which allow us to prevent false positives and the results that we find.

A

That is outer scope for this workshop, but we already have here a pretty powerful quai that finds a good number of genuine security vulnerabilities, including the ones that were found and reported and fixed in bootstrap, and it has a scope for customization further to make it even more precise or maybe find a wider set of possible vulnerabilities. But in just a short period of time. We've described some interesting logic in the source code and put together a nearly production-ready query that can find genuine security.

A

Vulnerabilities I, encourage you to run, try to look at the different definitions of the different classes and predict as every views to better familiarize yourself with what's going on under the hood.

A

We wrote a Kuei here from scratch, but this query is now part of our standard set of security. Queries thanks to the team who are here as your moderators, and so you can see the the query here in our open source repository at this link, and you can see the results on a vulnerable version of bootstrap which we're using in this database on the open source, showcase, LGG, I'm, calm,.

A

So next steps that you can consider as you take away from this workshop, there's good tutorials and documentation on other aspects of code QL in particular. A good starting point might be the tutorial on a dataflow analysis for JavaScript in typescript. Some of the concepts will now be familiar to you, since you have seen dataflow nodes and how to write a flow query, but that will go into it in some more detail.

A

Relatedly there's in you capture the five challenge on the get up security lab website that is focused on Java, but you now kind of have an unfair advantage because you've been trained in how to find vulnerabilities with code URL or, if you're not interested in competing for the prize just yet. There are a number of older capture-the-flag challenges that can help you improve your skills. Some are for C++ and some are for JavaScript and in fact the JavaScript CTF is the basis of this workshop, which was originally written by esbit.

A

Who is one of our moderators here today? So you can thank you for this wonderful source material that we have a whole bunch of other materials. We have a cultural course on learning lab. Hopefully there will be some more soon and you can explore the get up security lab research blog to see other exciting vulnerabilities that have been discovered with code QL and the methodology for doing that. Finally, it's always worth checking out the open source, queries and libraries.

A

You've got copies of them now on your machines, because you've been using them in this workshop, but you can always explore them to find the other queries read the documentation, and should you be so motivated, you are welcome to contribute you crazy. There are links and descriptions on what kind of contributions we welcome and how to prepare your submission of a security crane.

A

So with that we're up to time we'll be around in the slack channel to answer some more questions, I'm also willing to take some now if there are common questions that have been surfaced in the chat and I hope that these this and for those who attended both both sessions have been useful.

A

These moderators are any questions that you think we should surface in the chat that are generally useful.

B

There someone asking about where to find a workshop at MD, which.

B

A

At MD, which I've been using here in the workshop, is the same as JavaScript MD in the workshop repository. So if you click through the workshop repository, which is at the top of the suck channel- and you can you can look at the files in that repo there's one for the JavaScript, the Java workshop from this morning and one for the JavaScript workshop just now,.

A

And these materials are publics, you kind of walk through the entire process, with all the hints and solutions available to you.

B

Okay, thanks I think we can take the rest of the questions in slack sounds.

A

Good so again, thank you all for your time and and yeah don't hesitate to follow up with more questions. As you continue, your journey of learning about culture.