Cloud Native Computing Foundation Kubernetes AI Day North America 2021, 30 Oct 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Defending Against Adversarial Model Attacks Using Kubeflow - Animesh Singh & Andrew Butler, IBM

Description

Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Defending Against Adversarial Model Attacks Using Kubeflow - Animesh Singh & Andrew Butler, IBM

The application of AI algorithms in domains such as self-driving cars, facial recognition, and hiring holds great promise. At the same time, it raises legitimate concerns about AI algorithms robustness against adversarial attacks. Widespread adoption of AI algorithms where the predictions are hidden or obscured from the trained eye of the subject expert, opportunities for a malicious actor to take advantage of the AI algorithms grow considerably, necessitating the addition of adversarial robustness training and checking. To protect against and mitigate the damages caused by these malicious actors, this talk will examine how to build a pipeline that’s robust against adversarial attacks by leveraging Kubeflow Pipelines and integration with LFAI Adversarial Robustness Toolbox (ART). Additionally we will show how to test a machine learning model's adversarial robustness in production on Kubeflow Serving, by virtue of Payload logging (KNative eventing) and ART.

A

Hello and welcome everyone uh to kubecon, 2021, um north america and good morning, good evening, good afternoon, uh from wherever you are watching this. Hopefully you are enjoying and are keeping yourself safe. Myself is animation, I'm the cto for watson, data and ai, open technology, and with me I have my colleague andrew butler, and you do you want to introduce yourself.

B

Yeah, I'm andrew butler, I'm working for an open source team in and around cube flow, uh integrating trusted, ai and kfb techno projects.

A

Great, this is our topic for the day, defending against adversarial model attacks using chill flow right. Obviously, q flow is the conduit we are at cube.

A

Con and q flow is our machine learning and ai platform right built for kubernetes, uh but then you know, in addition to that, you need other tools and technologies, and uh you know in general uh what are the tools uh algorithms metrics scores, except for to use, so this stock is going to be focused in those areas now, in general uh business you know about ibm, ibm does have a history of tech for social good right.

A

If you are not aware right, there were thousands of ibm's who actually worked very closely with nasa and participated in the first moon landing right. We have been doing a lot of research and contributing to the field in human genome, sequencing offline with call for code efforts, ibm has been very, very active in areas like infectious disease response or climate change, uh working jointly with united nations health cross and others right. So there is a lot of work.

A

Ibm has been doing in the field, which is you know in general, for the largest social good and carrying forward that tradition. We have also been working responsibly to bring trust and transparency into a.I. Now. What do we mean right? What is our vision for trusted? Ai? uh Essentially, you know uh we look at this world from the pillars of these four pillars: robustness, fairness, explainability and lineage. Robustness means okay. Can anybody tamper with it? Did anyone tamper with it fairness? Is it fair, unbiased, ethical explainability?

A

Can you explain what your ai is doing and then last, but not the least lineage right? Can we trace back the lineage and have an audit trail when we translate that back to open source projects, um the projects which we have in the corresponding areas, for example in robustness, we call it robustness, 360 or r, there's one of the first projects which we moved out in this particular space into open source, and then you know followed with fairness, ai fairness, 360, which was followed by explainability 360 and of late.

A

You know we have already released something called ai fact: sheets 360, which essentially is focused on. uh You know the standardization of creating lineage for your ai life cycle uh now moving forward, something like fairness.

A

So if you're looking for tools and techniques on which you can measure bias, detect bias and if you do find bias either in your training data or your models, and you want to mitigate it yeah, if ns360 is the tool to go right, it has around 70 plus metrics on which it can detect bias and then more than 10 algorithms, which you can use then to mitigate bias in your ai models.

A

Similarly, you know we have ai explainability 360, which has a lot of proprietary algorithms from ibm research to explain model predictions both at a global level as well as you know, at the local level, and explain not only models right, but also you know. Data sets and data sets features uh one of the things which we also did when we moved and after we moved these tools in open sources, we essentially move them in open governance, because we firmly believe just open source is not enough.

A

Now this committee has essentially you know, with uh the input from a lot of these participating member companies, has come up with eight principles, what it means for ai to be trusted. So these are, you know: reproducibility robustness, equality, privacy, explainability, accountability, transparency and security. Today, our focus is on this particular area, which is security.

A

So let's talk about security right now. If we have followed, you know um we are living in this unprecedented time and over the course of last year right we have seen the the state of uh security attacks increasing quite a lot right. There have been many ransomware attacks. um A lot of these attacks have actually prompted if we noticed you know, just lost one president and calling you know the ceos of all these companies.

A

Major tech companies, including you know our ceo from ibm r, when kreshna was there in terms of you, know, figuring out and forming a task force how to handle these security and ransomware attacks. Now the the other part of it is. You know uh when we actually look at the state of security for ai, what we find out that in general, you know, the awareness of risk is low right or you know there is a low understanding of what ai security means, so that translates in general right that security posture is close to zero.

A

Now, if you look at, you know uh what adversarial threats to ai can do now. This is probably one of the worst case scenarios like now. This is a uh when we actually launched address cylinder business toolbox right. This is one of those scenarios we we prepared for. We had notebooks around it. Like you know, these are physical attacks.

A

Where stop sign images right in this case either you know, can be intentionally uh perturbed, or you know over a course, a period of time, because you know because of physical weather, wear and tear these top sign. Images are not as as accurate as they should be right. An air model, in this case a self-driving car can essentially you know, not detect a stop sign, and you can imagine the consequences can be terrifying right.

A

So this, essentially, you know, highlights the the importance and and to the extent that which you know adversarial threats to ai uh can cause the damage so to handle all these kind of different attacks uh ibm launched a toolkit called adversarial, robustness, toolbox or art. We will be using the word art quite a lot moving forward in this session, so that refers to this toolkit at the cellular business toolbox.

A

It's essentially a python library which works across different kind of frameworks, whether it's the deep learning frameworks like tensorflow, keras fighters, mxnet, etcetera or you know, machine learning frameworks like cyclical and hd boost cad boost. The goal is, you know, to actually evaluate different, certified, verified machine learning models and applications right against adversarial threats.

A

So uh you know it provides many algorithms to to actually detect whether your models are vulnerable to adversarial attacks and if they are found vulnerable to adversarial tax, then it also gives you algorithms to to defend against those attacks right and it works across different kind of data types like image tables, audio video, etc.

A

Okay, so let's uh look at the toolkit a bit, uh so you can go and look at this demo at art dash demo. I tried my bluemix.net so very quickly. This is uh the website. If you want to go and try it on your own, and essentially you know, it just gives you a way to try out art in a very quick format. For example, this is a siamese cat and if you see the model is 92 percent, confident that this is a calcium scat.

A

Now, if I choose an attack, let's say projected gradient descent, but which is one of the attack methods which art provides even with a low strength of that attack. The model's confidence has reduced to four percent. If you see them right, and this is just low now, if I change this attack to medium now, the model actually thinks this is a basketball.

A

Now, if you see behind the scenes, uh if you want to look at the code like we are essentially you know using the projected gradient descent method from from the our toolkit right and launching an attack on that particular model.

A

Okay, so now that the model thinks it's a basketball right and it's very vulnerable, traditional attacks, how can you reduce and- and uh you know, mitigate that so then you know what also provides you defense, algorithms right so, for example, something like spatial smoothing which essentially decreases the the pixels in this case on this image, so the attack surface is becoming less right.

A

So if we, you know, introduce this defense algorithm at a low strength, the model is back to 76 percent, confident right that it is a cat great if you make it medium 92, that's where we started right. So, as you can see, um that's how you can use the both. You know red team attack methods, as well as blue team, different methods which are provides you to to actually implement okay. uh Let's go back to this.

A

So we covered this. If you were interested here right so uh very quickly, I think we are at your phone. There are other sections around qfloor. uh If you haven't heard of qfloor right, it's a tool which is focused on the end-to-end air life cycle. It gives you tools to create your models and distributor training on your models.

A

You uh launch, you know, use hyper, parameter and then use neural architecture search right. So that's a toolkit called cated in that, then it also gives you a platform to deploy your models and not only deploy your models in production, but also monitor them for things like drift anomaly right so whole end-to-end platform uh comes as part of the queue flow, starting from you know very beginning, which is launch integrated, notebooks and start creating your models right now.

A

Two significant projects in that space, uh where we uh invest heavily our queue flow pipelines and kf serving right, so keyframe pipelines and care serving essentially are the two projects where we are investing heavily in that particular space and contribute, and when we look at q4 pipelines, it's essentially uh you know a platform and a toolkit which gives you capability to create end-to-end machine learning pipelines uh by encapsulating each of these steps inside a container. A sequence of these containers, which you can orchestrate together, becomes a pipeline right.

A

But even though you know everything is running in containers, you are manipulating humanities objects, like secrets volumes, etc. There is a python interface, so for data scientists, you can program this whole pipeline using python right and if you don't want, you don't need to deal with the kubernetes constructs right. So that's the great part about it and that's made what made it really popular.

A

Okay, so that's kevlar pipelines, and then there is another project which I mentioned right. We are heavily invested in called care serving which was founded jointly by google selled in ibm, bloomberg and microsoft. It's part of the queue flow and sure we focus on 80 of the use cases right and we provide serverless machine learning inferencing with capabilities around candidate rollouts and also you know, generating model explanations as well, as you know, plugging in your own pre-processing and post-processing code right.

A

Okay. So with that, I will pass on to and you to uh talk about adversarial robustness, toolbox, integration into the queue flow specifically to the pipelines and the serving project over to you.

B

Andrew sure, thanks anamesh, so what we're going to be looking at specifically is inside of each of the components uh so kubeflow pipelines and kf serving and so we're going to take a look at the specific adversary, robustness toolbox component inside of kubeflow pipelines and exactly what it does, the inputs that you would give to it and the output that you would expect after you're done with it. So at the beginning, you'll have some inputs for this component and specific parameters focused around the individual components, use case of the trusted ai toolbox.

B

So this is where we get to see the fgsm attack epsilon, which is a hyper parameter for the fast gradient sine method which is used to get and check the robustness inside of the model that we've just created so for this component, which uses the fast gradient sine method.

B

Basically, what happens is a number of samples are given to the originally trained model, and then the model interprets how it would classify those samples and then the fast gradient sine method will compute the loss and the sine gradient of the loss and try and maximize that loss in order to get an image that makes your model mispredict and for output parameters. We have the model accuracy on test data, so in the original test set that you have set up.

B

What is the accuracy for that, and that gives you sort of a milestone to do is to see how well it your model performs against the adversarial samples which is next in the model accuracy on adversarial samples which says: hey we've now you know, perturbed your images and came up with these adversarial samples.

B

How does your model do against that and compare it back to the baseline of and you, then you can compare it back to the baseline of the original test data and just to give a little bit more information on how your model is performing as well.

B

You get the confidence reduced on correctly acid, classified adversarial samples, so if your model is able to correctly classify the samples after they have been changed, what was the amount of confidence that was reduced for like how confident was your model before and how confident is your model now and how has that changed? And then the last one is the average perturbation for misclassified.

B

So if you were able to get the model to misclassify, how far did the fgsm have to go in order to trick your model and we'll look at this in a demo as well? So first we have a training step that could be any training stuff that you use to train a model and you can be removed and placed in you know it's portable.

B

You can take it whatever training step that you have whatever model you have as long as you change the parameters to agree with it, and so this is what we had seen before the parameters, and then we also have this model fairness check as well, which is part of the ai fairness 360 component inside of trusted, uh ai and keyful pipelines.

B

So, let's create a run here and we'll choose an experiment, one that we've already have set up trusted ai and we need to specify the namespace that we're going to use, uh in our case we're going to use the keyflow user example com namespace, and then you can see that we are actually giving actual data to our. uh These are the defaults, but this is the actual parameter choices that we use so 0.2 for the attack epsilon.

B

We want this model class file, a three-layer cnn, and you can do this for all, as well as the component for the fairness.

B

The parameters for that that we need to specify so which of the labels is a favorable label.

B

When we want to identify when we want to look at bias, we have to say which is a good outcome and which is a bad outcome and then also which groups we believe will be biased against and which groups may be. The bias may be in favor of so this will start impending and then eventually it will show running, and then it will execute successfully and we'll see this uh walk through. These are executing fairly quickly because they're, cached, and so the reason for that is.

B

This takes a very long amount of time to run so it's much easier if we cash in and just show it running through and collecting from cash. But the basic idea here is that this step will train as we've seen before, and then these two will, once your model has been trained, check the robustness and then check the fairness.

B

So in this example, we see the inputs that we have it's getting from the original that we specified and then also something similar to what we had before model accuracy on test data, all the things that we talked about previously and we can see they're very similar to what we had before and the robustness status uh remains false and so quickly we can look at the fairness uh and see we have a few things here from the ending the classification accuracy so similar to the robustness. It shows the original classification accuracy and then.

A

A bunch of metrics.

B

On how biased it believes your model performs, and now we're going to take a look at kf survey, so we've also implemented all the trusted ai components into kf. Serving so to give you an example of this, we have implemented the square attack method for art inside of camp serving and just to give a concrete example. We have an mnist image on the left and we see that it shows a one and then it will add through the square attack method, a mask that looks something like that and when those are combined.

B

Hopefully, if it performs correctly the square track method, then you will get a example that makes your model mispredicted and so how these actually get deployed is the original spec.

B

So everything above the explainer portion is standard for deploying just a normal predictor without this added explainability, and so it's very simple to just tag this explainer spec on afterwards and specify you want the adversary, robustness, toolbox, explainer.

B

So let's take a look at the demo for this, then. What we're going to look at specifically is that mnist example that we had previously- and so this is all on the kf serving uh repository that you could go and look at and we'll provide link at the bottom of the presentation.

B

But basically, what we're going to do here is we've already set up our inference service, so it's ready to be queried both a predictor and an explainer and we'll show that and then basically, what we're going to do is we're going to send a request to this explainer to try and get our model in this classifier like it does in this image here as well. So this is the kind of mask that we would expect to see.

B

Let's look at our pods here, see that we have them up and running and so yep. Here we go. We have a pod for our explainer and a pod for our predictor and then let's look that our inference service is ready to be queried yep. You can see that it's ready here and you can see the url that you would want to bring at I've already had it set up here.

B

So we're just calling the script that will uh implement the authentication and also provide a way to visualize the results that we get back from our model, explainer and so that'll run for a couple seconds and then give us back a similar example here. So you can see you know to the average eye.

B

You know an average person would be able to look at this image and tell that it's a three, but our model has actually predicted to be a two. This is all through the inference service and, like we said easy to prop up and push open. All you have to do is apply the ammo width that we've specified and shown.

A

B

And you can get these examples that are able to fool your model into misclassifying. These are the links that we have for the demos that we've gone and ways to reach us afterwards. Thanks.

A

Thanks a lot andrew, that's great! So hopefully you know you all uh enjoyed the session and also you know, appreciated the fact that trusted ai and and in general, in the context of custody, the security of ai is is something which is becoming very critical.

A

The time to act on it is now uh we don't have time to delay and given all these different regulations, whether it's the existing gdpr or the new upcoming regulations, uh uh you know which, which the european union is already you know uh very close to forming, um regardless of whichever way we look at right, the need of the hour is now right.

A

So hopefully you know you all join us in this mission to make ai more secure, more robust and- and you know much stronger so and if you need any questions or if you have any questions, and you need more clarifications about these projects definitely reach out to us. We would love to have a chat with you that, thanks again and thanks for joining us.