Cloud Native Computing Foundation Kubernetes AI Day EU 2021, 14 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Stand up for ethical AI! How to detect and mitigate AI bias using K... Andrew Butler & Animesh Singh

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Stand up for ethical AI! How to detect and mitigate AI bias using Kubeflow - Andrew Butler & Animesh Singh, IBM

The application of AI algorithms in domains such as criminal justice, credit scoring, and hiring holds great promise. At the same time, it raises legitimate concerns about algorithmic fairness. There’s a growing demand for fairness, accountability, and transparency in ML systems. It is important to note that training data isn’t the only source of possible bias and adversarial contamination. It can also be introduced through inappropriate data handling or model selection, or even incorrect algorithm design. To address this, this talk will examine how to build a pipeline that’s open, transparent, secure, fair, and that fully integrates into the AI lifecycle by leveraging Kubeflow Pipelines. Then the correct method to deploy machine learning models in production and collect payloads to enable inbuilt bias checkers using KFServing will be detailed.

A

Hello and welcome.

B

A

um To this kubernetes ai day, which is running as part of the kubecon um virtual event this year, and hopefully you know uh we can all meet in person next year. It's always exciting to be part of cubecon and giving uh you know talks but also you know listening to a lot of uh exciting talks which are happening.

A

uh So what are we here for so myself? uh I'm animation, I'm the chief architect for data and ai, open source platform and with me I have andrew, and do you want to introduce yourself.

B

Yeah sure I work mainly on kubeflow integrating some of the trusted ai projects into kubeflow and then some auxiliary kubeflow, kf tecton stuff as well.

A

Great so so we are part of this team right and today, here we are uh to talk about. You know how to stand up for ethical, ai and what we mean is. Essentially you know uh with the proliferation of ai systems and more and more models being deployed in production right.

A

We want to make sure, as we are building ai right, we are bringing and building it in a trusted manner, and what it means is that you know we have techniques for bias, detection and medication or being able to explain your model predictions or you know being able to detect adversarial attacks being generated on your models right.

A

So that's the focus of this talk and then you know we're going to talk in the context of q flow right, as, as we are aware, it's a very popular open source project, uh so both andrew and I work for a group in ibm called cody center for open source data and ai technologies.

A

What you see is a very nice picture of our san jose silicon valley, lab. We have a cricket pitch as well. If you are into that right, we can have tournaments, uh but it's an amazing place right, uh surrounded by green hills and a large part of our team is centered there. But definitely you know, the team is very globally distributed across u.s, as well as other parts of the globe, and we've contributed to open source projects.

A

You know across the entrance life cycle right so, for example, you know we are the second largest contributor to q flow. uh We contribute heavily to tensorflow and pytorch uh spark. We have our own open source projects like eli or data asset exchange model asset exchange, so essentially work on strategic, open source projects, which are you know, uh being consumed by our products as well as take you know the innovations we are uh doing inside and move it in open source like some of the trusted ai projects, which we are going to talk about today.

A

Okay- um and you might be wondering you know why this topic- and why now right? So I think we are all living in this. You know unprecedented time in history right. We are all in the midst of this global health crisis, and hopefully you know we are now all seen light at the end of the channel, but still quite a long way to go right and, as part of this uh within ibm right, we have had our our own brainstorming and and discussions around. You know how we can help.

A

uh You know make uh the situation a bit better right. We have been also focusing heavily in terms of bringing trust and transparency into ai right, but does it really matter? uh I think it matters a lot more uh in the times we are in right. Even you know, a google search, simple google search can show you that you know even at the time of this, this panda making crisis. What we have seen is you know uh the trusted.

A

Ai techniques are all the more relevant right uh because when you are actually for example, at this point right, we are doing vaccine distributions. How do you make sure there is no bias in that distribution system right uh things like you know, being able to get tested right? How do you make sure that you know the people who are being approved and being approached? They are not being done and approved in a manner where we are discriminating based on sex or race, or you know based on the nationality of someone.

A

So how do we handle it right? So within ibm right we have our own uh vision for trusted ai, but you know we are also making sure that this is something which we are aligning with the industry. We are taking feedback, always expanding it, but we when we began and- and at this point right, our focus- has been around four pillars of trusted air right, what we call robustness, fairness, explainability and lineage right. So what do we mean by uh these different things? Robustness means that can anybody tamper with your ai systems or air models?

A

Fairness means that is it fair? Is it not biased uh towards a particular or race or gender or a nationality? Can you explain your moral predictions right? What it is doing if it is making these decisions which are impacting my life? Can it explain its decision last, but not the least right lineage? Can I trace back and and make sure that you know if a model is giving life-changing decisions uh for the stakeholders?

A

uh We can trace back all the way like on what data center was it trained on? What were the hyper parameters used, uh and you know what version of a particular framework was used so that you know you can create accountability where it is needed right. So having right. Lineage is very, very important.

A

um Just to give you an example right since 2008 right, nearly every rsd in a broad county in florida right uh was being um assigned a criminal sentence which was based on you know if they are likely to reoffend.

A

It was based on uh not points compass, algorithm, right and and based on research, and you can trace the link here right, which was found that data set was found to be heavily biased towards non-caucasians right. The other thing is explainability right now. Explainability is not only important for someone who is consuming the models. Obviously it's the most important for that particular person right.

A

So if I'm being denied admission to a university or if I'm being uh given a loan- or you know I'm being accepted into a company or denied into acceptance into a company, the model should be able to explain to me right very, very critical, because these are life-changing decisions for me right, I cannot buy a house, I cannot get into a company cannot get into university now. She'll, explain why you did that. But, more importantly, I'm not the only stakeholder right. There are people in the middle right.

A

So there are government regulators, people who run compliance and safety. They need to be convinced that, over a period of time, what the model is doing, it's able to explain itself why it is doing and why it is leaning towards those certain uh predictions right and last, but not the least right. When we talk about adversarial a. What do we mean right? A very simple example could be like you are depositing a check of hundred and fifty one dollars.

A

Let's say right, uh but a hacker could you know, adversely modify your image and, and instead of you know, 151 like uh it to the machine. It appears as 757, because it's very easy to fool a machine learning system right. An adversary generate an image which can make a one look like a seven for a machine to understand. Now that's undetectable to the human eye, so the person you know, uh instead of who gave you the check, instead of having 151 dollars debited from his account he has now.

A

You know 757 dollars gone right, um that's you know more in the financial industry. But if you look at like you know, the impact of this can be life-changing and more scary in examples like this, for example, you know a lot of the self-driving vehicles and are being trained over. Stop signs images right now either. If these stop sign images are adversarially modified, more explicitly or implicitly, you know over a period of time because of the wear and tear right. These images are, are not great uh or you know they have lost their actual accuracy.

A

Then you know the way the ai model will learn to detect. Them uh will be less accurate right. So that might mean that your vehicle, the self-driving vehicle, actually running over a stop sign right and you can definitely imagine the consequences of a behavior like this.

A

So uh we talked about the four pillars of ai and you know gave you some quick examples why we think you know they are really really critical in real world and to handle that right, ibm actually created four projects and then moved it in open source right, so projects around uh adversarial uh robustness for both you know being able to detect if the models are robust and being able to mitigate them against adversarial attacks. That project is called adversarial, robustness, 360.

A

uh for fairness. We have launched a project called ai fairness. 360. again, you know this is around bias, detection and not only detection, but you know being able to mitigate bias in your ai models as well. Yeah explainability 360 is essentially, you know, being able to explain uh ai predictions right across for different stakeholders right and last, but not the least right uh project which we have not yet moved the code right. But a lot of the the research papers and the materials are already being made.

A

Public is around a fact sheets, which is around you know creating this lineage and governance model so that you can trace back the origin.

B

A

A model right right from the data set on which it started so adversarial or robustness. 360., as I mentioned right, it's a project to actually uh allows you to rapidly craft and uh launch attacks on your models. If they are found, you know vulnerable, you can have uh defense mechanisms and defense algorithms, which you can implement. uh There are quite a bit of different kinds of you, know, evasion, attacks uh and defense and detection methods. You can assign metric score robustness metrics to different kind of models, uh so definitely try it out.

A

It's one of the most popular projects which we have out there right. We are also working jointly with darpa uh in terms of advancing the two tools and the techniques and the algorithms which are available here.

A

The ifn is 360. um a library to actually detect bias across different metrics. So at this point we support around 70 plus fairness metrics across a life cycle right, so what we call uh pre-processing in processing and post processing phase. So that means you know. Even before you start creating your model, you should be able to point this tool at your data set and be able to detect.

A

If there is bias in your data set once you have, you know trained your model, you should be able to then validate that model right, which is in processing phase and last, but not the least right. Once you have deployed your model right, you should be able to. Then you know, monitor the model, predictions and say and figure out right if they're, biased or not right.

A

So you can work across this life cycle right and not only this right so based on these 70 plus fairness metrics, if you are able to detect bias, there are 10 plus bias mitigation, algorithms, which are provided as part of this, which essentially you can take, and then you know mitigate bias across the ai life cycle and last, but not the least right, ai explainability 360.. As I mentioned right, uh there is no one explanation which fits everyone. There is the end consumer who is directly impacted by the model predictions.

A

But then there are you know, regulators and then people make sure you know things are compliant who are in the middle? There are next gen of developers who are retraining these black box models and positioning it towards a customized use case or towards the customized industry. Plus, you want explanations at different levels right, so you want to be able to also explain data sets and the features of the data sets. What do they mean and then, when you are looking at you know, some of them are very simple models.

A

Some of them are black box models, so in case of black box models, you might want to have a surrogate model right, which is learning based on the explanations and creating a parallel model right which which, over a period of time as it is seeing the basic explanations right. It's creating its own, robust explanation model. Now, apart from moving these projects in open source right.

A

So as as a mission of trusted area right, as I mentioned, like we created these projects and we realized right, these techniques mean propriety, doesn't make sense, so they need to be an open source, but not only in open source. The whole ip of these projects need to be in a neutral place under a neutral governance model right, so move these projects to linux foundation, ai and they are now part of the linux foundation- ai hosted in a neutral place.

A

Hopefully that you know most of you have either used or using q flow, or you know, are trying to floor, probably have heard about geoflow right. So it's an end-to-end machine learning platform right which goes across the whole ai lifecycle right. So you can essentially build a model using chip flow notebooks. You can, then, you know run distributed.

B

Training on those models using.

A

Different libraries like tensorflow pytorch xgboost, we have mpi based algorithms, like horror, word, etc, inbuilt as well. uh You can use hyper parameter, optimization engine like captive, which also gives you capabilities around neural architecture service.

A

So if you need more auto ai capabilities, and then you know, you can deploy your models in production right, so uh using uh q4 server so end to end spectrum and tying all these things together is pipelines or machine learning pipelines which allows you to automate this whole end-to-end life cycle and not only allows you to automate but also allows you to change the lineage right so I'll briefly go into them.

A

uh So the two projects right where we focused uh from the perspective of trusted ai, uh are essentially you know, pipelines and serving right and- and the reason you know for focusing on these projects was twofold. One like when we talk about pipelines.

A

What we want to do is you know, make sure that, as we are running through these ai life cycles, there are inbuilt capabilities into these pipelines, which can then help detect uh things like bias. Or you know, if the models which are being trained, are they vulnerable to adversarial attacks, etc? So they can help. You manage the whole life cycle and ensure that you know uh they're able to detect if the models are being built in an ethical and trusted manner and serving which is care serving right.

A

Can then, once your models are finally rolled out and deployed in production? Can then help you know to.

B

See if your models in real world.

A

Are actually adhering to these trust, ai principles and you can monitor and create metrics right, so hence the importance of these two projects uh so very quickly on pipelines right, it's the most popular project in the pipeline, uh qflo umbrella uh targeted towards data scientists. So there is a python dsl, so allows you to craft pipelines in a programmatic way.

A

uh Each of these steps in the pipeline is called a run, which essentially is a backed by a container which essentially has the code right, which is doing that specific part right and- and it's very, very popular in in the queue flow community right because for data scientists, they can now use a python way of running machine learning pipelines on kubernetes, and they can also leverage underlying kubernetes constructs like volume, secrets and other things, but also um you know for for operational folks right they can actually go.

A

There is a very rich dashboard which shows you the streaming logs in real time, etc, and you can, you know, look at the real time logs and debug, and not only this right. There is a strong lineage and governance right, so you can track back like once. Your models are produced on what data set. It was trained on what were the hyper parameters used? What was the version of tensorflow so very, very robust, set of capabilities in this right and what we have done is right essentially uh created some pipeline components.

A

As I was mentioning right pipelines, components are the building blocks which essentially allows you to run, and you can change these components around adversarial detection or you know, bias detection plug it into your own workloads, and then you know identify as part of that whether your workloads are- or you know, adhering to the trusted ai principles, and then you can detect if there are bias, etcetera being found.

A

So with that, I will pass on to andrew to talk a bit about queue, flow pipelines and trusted aoi and show you. You know how we can leverage this in practice. So I handed you over to you so basically we're.

B

Going to take a look at for first uh keep flow pipelines and see some of the components in action in that, so we have to set up a model, fairness check which works with aif 360 and then an adversarial robustness evaluation with uh the art tool that we have uh and the first training step we're gonna do is on a gender classification, so takes images and tries to classify a gender based off of that. So the way you would run this through is click creating run and then specifying some parameters that you need for this.

B

You might specify the attack epsilon for the fast gradient sine method parameter our favorable label, which in af360 needs to know what is a favorable label, uh what they would like to see and then an unfavorable label what they would not like to see, and then you can also specif specify which groups you should check the bias for. So in this case, um we've chosen of race uh with the label zero and the unprivileged group is race of uh label four. So then we just can run this through.

B

This experiment takes a while, so we won't watch it go through the whole way, but it'll just start up like this, and then some nodes will start to show up and run through, but we already have a pre-set up, one that is executed already, so we can kind of look at what has happened so first, let's look at the training step, see what training logs we can get.

B

We can see all this is going on, so we can see for the failure. Condition uh is evaluated to fall, so it hasn't failed yet and then the success condition is also false, so it hasn't succeeded yet either and it'll continue to check that until uh one shows up true- and in our case it succeeded, so we've succeeded in training it and then it'll store all the training parameters so that we can uh go back into the model. Fairness check and adversarial, robustness, evaluation and kind of take a look at what we've got there.

B

So here we see some of the metrics uh there's a bunch of metrics calculated we're just going to look at one for time constraints so for the disparate impact. um The way that the disparate impact works is that it takes the probability of a favorable outcome for the unprivileged group and divides it by the probability of a favorable outcome for the privileged group, so values of less than one generally indicate bias towards the privileged group and uh values close to 1 10. To show that there's no bias.

B

So generally we go by the four-fifths rule, which means anything less than 80 shows some sort of bias, and so we have a 77 percent here uh which would indicate that likely.

B

There is some bias here and we would need to go and mitigate it um in some way and then, let's take a look at the robustness evaluation see how that went. So the model accuracy on test data in general was 85 and then on our adversarial samples. It comes out to only 15.

B

So this is not very good. It indicates that we would need to do more robust training to kind of deal with adversarials and to go a little bit deeper into that. You can also get the average perturbation on misclassified samples, so small values show that your model did not perform well. At all, and in fact very little perturbation will cause a misclassification which is not good, and if this label is very high, it shows that it took a lot to uh mess with your model and get it to mispredict.

B

So you would prefer higher numbers so yeah. In this case, you would find that likely. um You need to both check on the bias and to check on robustness in your model, training step, and so you could go and rerun this training step, perhaps with new data that you've created um based off of the model, fairness and adversary, roughness data that you created in both of these steps um so jumping back in.

A

Thanks andrew, that was great right, so I think in the interest of the time we will probably you know, speed up uh some of the rest of the slides right. So I think I gave a quick background around kf serving right. So it's it's a solution in qfloor community for production model serving right. So it gives you a serverless machine learning, inference capability built on top of k, native correct, and then you know there are capabilities around model explanations under which we have built our rest of the uh trusted ai uh umbrella right.

A

So as part of that right so andrew has been working heavily in terms of integrating uh some of these projects uh into kf serving including you know, explainability 360 or any iphone s360, uh and you know we had the first version all done and delivered as part of the kf serving v0.5.

A

So if you're interested, you can definitely try it out a lot of times, it's not possible to to do these analysis on a single transaction right, so you cannot just based on a single transaction, say that the model is biased or not right. So, in those kind of scenarios right, you need robust, payload logging capabilities. What that means is that you know you're able to capture the requests which are coming to models, the responses which are going back from the models collect curate, and then you know over a period of time.

A

Once you have that data, then you can run more advanced analysis, whether he over the course of last 100 predictions. The model is biased right or you know, 80 of these inputs, which are coming there, you know adversely modified and model, is getting full right so to do more advanced analytics like this, you need payload lobby logging, which we have enabled uh using. You know uh the cloud events standardized protocol, which is a standard which is being developed by the cncf community.

A

B

So, and you can.

A

Now talk a bit about these capabilities as part of the kf survey.

B

Yeah, so what we've implemented in uh kf, serving so far for the ax is using a method called locally interpretable model, explainability lime and the idea is you take an image on the left like an mnist image, and the explainer will highlight pixels that it finds are highly indicative of a certain classification so, and you can change certain parameters inside of lime to get different, uh highlighted pixels.

B

If you want something that is like the pixels that are most indicative of a classification, then you would raise a parameter called minimum weight and then the next project ever so robustn's toolbox, is using the square attack method.

B

So you take an image.

B

You add some sort of perturbation or noise and you'll be given with another image which is very similar to the first, but makes your model misclassify or at least is the goal of the art tool um and in most cases, that'll just be small perturbations that look identical to the human eye for aif 360.

B

The idea is to collect a bunch of payloads, as we mentioned, and then through the payload logs. We can create bias. Metrics based off of the large collection of instances and outputs, and so there's a number of specifications that you need to have here like we saw before in the kfp example: there's the favorable label, unfavorable label, privileged groups and emphasis groups, and this just lets.

B

You know what is a favorable outcome and what are the biased groups that we may need to look out for and so the same deal with the spec here the left is the same as you would deploy a normal predictor and then the right is all the exp extra commands and information that you may need in the explainer to give you something valuable and so the flow of the af360 is you send your prediction requests to a model? Then the model sends their logs to some sort of persistent storage.

B

In our case, uh when we show the demo, it will be a message jumper and then, when the bias detection request comes through all the logs, the person's storage will be sent to the f explainer, which will then be able to calculate metrics based off of this logs.

B

So we'll show that here.

B

um So this is housed in the kf serving repo. You can just go to the samples and look for it.

B

The basic idea is to deploy the model, uh the bias, the predictor and the explainer here, and the message jumper we've already have that set up so that uh it doesn't take a lot of time to spin them up, but because they are fairly large images, at least for some of them yeah. So there they are and then the idea is to connect to it and send simulation prediction uh requests. So I have that here so we'll just simulate sending multiple requests just to pull some logs into the message jumpers.

B

uh You can see we have about six requests, and this is the response, as we expect to see a few predictions and both the requests and the response will get logged and then the idea here is that we need to create a json from the logs.

B

uh So, let's just take a look at what that looks like it's created this data.json, which is just a json uh instances which are all the logs that we've sent to it, there's quite a few, because we added some information before just for added information without taking up too much time and then the outputs here are all the prediction: responses uh that we got and so now that we have those we can go back and we can actually query our explainer to check uh what bias do we see.

B

So there we go, and that's pretty quick so now that we have that, um let's look at the disparate impact, so we can have a kind of comparison to the kfp. um So again, the spirit impact is measuring, favorable outcomes for privileged groups and unprivileged groups, the ratio. So this is much lower than the fourth fist rule. It's actually 0.5.

B

So we would say that the model that we have deployed in kf serving is very biased and we what we can do since we have the payload logs for this is we can go back and we can see what the biased examples were. And then we can check if our model behaves differently when we change those examples to make them the privileged class. And if they do, then we can save new data.

B

For when we would like to retrain and keep the same examples before but say that those examples should be marked with a favorable label rather than the unfavorable laben label that they're giving, and this will expand our data set and make it a little bit better, because we know that if it is given to the privileged class, they are getting acceptable results.

B

So now, we'll hop back in.

A

With that right, we are also doing a lot of great work and advancing a lot of the principles of trusted ai as part of the linux foundation. Ai trusted ai committee.

A

uh We meet, you know, uh monthly, uh where you know a lot of the uh partners like you know the folks from the companies which are listed there, or you know folks, from microsoft and others right. We come together and talk uh both about the principles. What should define the principles of trusted? Ai but also you know, technical solutions like we discussed today right. So, if you're interested in this definitely you know join this.

A

It's an exciting place where a lot of us, you know people who are actually interested in advancing this field are getting together and bringing uh technologies and tools and use cases together as part of this linux foundation, ai trusted ai here are, uh you know some of the links uh you in terms of you know some of the github repos or the design document right. We service the cab serving payload logging plus, you know, if you do need to reach out, like you know, or or twitter or linkedin urls are listed here.

A

So definitely, you know, try and reach back to us. If you have any questions comments, feedback, we would love to get your feedback and and make sure that you know you're using these capabilities, advancing these capabilities and other issues which you find uh or your suggestions, uh we would like to incorporate and advance these uh projects together right. So so, in the end you know I would like to round up by saying. We definitely want to stand up for ethical ai.

A

We are doing quite a bit of efforts right both as well as uh through ibm, but also you know, uh linux foundation. Ai has been a huge partner that you flow community has been a huge partner, and we would like to invite all of you right into this mission in terms of bringing ethical and trusted ai to the masses and and building you know, platforms which are essentially, you know making this core part of their dna.

A

Thank you and thanks andrew yeah,.