ONNX June 2022 Community Meetup, 13 Jul 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: How to reconcile AI and privacy

Description

AI is revolutionizing many fields from healthcare to biometrics these recent years. However due to security and privacy concerns, data is still being siloed and not shared enough due to the fear of data exposure and IP leakage. Confidential Computing is a recent technology that enables end-to-end encryption when analyzing sensitive data. By leveraging Confidential Computing, data owners can share their data to AI companies, for instance to train or consume an AI model, without ever risking their data being stolen, leaked or used for any other purpose, as data remains protected even when shared to third parties. This talk aims to introduce the high level principles of Confidential Computing and how it can be used to deploy privacy friendly AI models. We will present BlindAI (https://github.com/mithril-security/blindai), an AI deployment solution, serving ONNX models with privacy guarantees, and see how it can be used to unlock confidential medical document analysis in the Cloud, or facial recognition with privacy guarantees.

Daniel Huynh is the CEO of Mithril Security. He is a graduate from Ecole Polytechnique with a specialization in AI and data science. He worked at Microsoft on Privacy Enhancing Technologies under the office of the CTO of Microsoft France. He has written articles on Homomorphic Encryptions with the CKKS explained series (https://blog.openmined.org/ckks-explained-part-1-simple-encoding-and-decoding/). He is now focusing on Confidential Computing at Mithril Security and has written extensive articles on the topic: https://blog.mithrilsecurity.io/

A

Thanks for welcoming me today, so I have done england, the ceo of metro security, and I will show you how we can deploy uh onyx models with privacy guarantees, um the connection yeah.

A

Did you take control at the top? uh Where is oh yeah thanks?

A

uh Okay? uh Does it work yeah? um Let me get used to it.

A

Okay, yes, uh so the issue we address at metro security is the fact that we might want to deploy air models, especially in a cloud setup. For instance, imagine you want to use ai to screen patients using deep, conventional networks, but the issue we face is that when we want sensitive data to be analyzed by other parties, we need to trust them for data, especially in cloud scenarios. We need to trust both the ai companies that will deploy the model, but also the cloud providers, and sometimes for regulation, privacy and security reasons.

A

This can be, you know, an obstacle for the adoption of ai, and so you might think what is the issue if, for example, I use stateless isn't that enough, but we see that we see that the problem is at the last mile during analysis.

A

So, for instance, we can imagine that this hospital wants to use this covet screening as a service. So they get the data locally on the device they can encrypt it then what happens is they will send it through regular channels, for instance the gls channel and the data it can be sent there and it will be decrypted.

A

So this is like regular stuff, but this is a problem at a specific moment, because we needed to decrypt the data. The data is available both to the cloud provider and the ai provider, and it is because, because at the last mile the data is accessible, any malicious insider can dump the data get access to it and therefore the data owner has no longer guarantees that his data is safe because at a specific time, when data needed to be analyzed by the ai it is exposed and because we have no control of that.

A

This is a huge issue to trust uh external parties, with the data for ai training on deployments. So that's why, at metro security, you will develop blind ai, which is an open source and secure solution to deploy onyx models inside what we call secure enclaves which are solutions that leverage hardware to protect data.

A

So what is the difference? Data will be decreated inside this enclave, but the idea is that this enclave, which are hardware-based technologies, provide isolation and encryption of the memory content of the enclave, and so any dumping attempt for the outside will will abort.

A

But while data is still inside the enclave, you can do whatever you want, including applying, for instance, an onyx model. So this way you can analyze that inside the sql environment, but it's not accessible to the outside and, for instance, here you can get a result.

A

You can then re-encrypt the data and send it back to the user so that they can benefit from it without having to trust a third party with their data in clear, but the best part is that they did not need to deploy the solution locally. They could resort to an external cloud solution, so it's very easy for them to be onboarded and for the air provider. It's very easy to maintain and sell a solution. So how does it work in practice of architecture?

A

The idea is that we have an onyx engine. We are using a tract which is a rust based onyx engine initially for embedded devices, but modified it to make it work inside a secure enclave and then provide some networking abilities for data to be analyzed and then wrap everything inside the docker image and then well. Several python, several cloud sdks to securely consume a remote model.

A

The idea that we, by leveraging the onyx uh format, we make it easy to uh deploy ai models with progress guarantees you can easily deploy or talk your images, for instance, in azure, using a confidential vm. Then, with our python sdk, you can securely upload your rx model. For instance, here we put a distilbert model inside of this and finally, with the same python sdk, you can securely send data to be analyzed without the third party having access to your data and really by leveraging the onyx expressivity.

A

You can see that it's quite smooth and resembles existing solutions and because onyx is able to cover a lot of models. Currently, thanks to its expressivity, we are able to cover a lot of use cases, for instance, baggage screening at airports and performance. For that we help a client use a euro model.

A

We also help other people use nlp to analyze medical documents to save times for um hospitals, in that case, there's a birth model, and finally, we can also cover, for instance, facial recognition, because our model also covers I mean loader also covers resonance model, and so what? What do you see in terms of different speed when you are inside a long claim? So here I show you different models that are quite used in practice and the influence time with our inference engine in rust, outside the enclave and the influence time inside the enclave.

A

So you can see that there is an added, it's a latency, but it's uh it's, let's say reasonable, given the added protection provided- and we can discuss this later, but also uh one thing I did mention that we are actually able to put huge models inside uh enclaves and more than testing, for instance, with a gpt model with three billion networks, and it does work under a second. So it's very promising to enable people to consume air models that are that will not.

A

They will not be able to deploy on-premise but still send data and have protection guarantees on their data.

A

And finally, uh the good thing with using uh onix is that you need a minimal code base to apply ai models and control it to having you know, a generic python solution by using just an inference runtime, you can have a little code base. That is better for security reasons and also you can reinforce each operators to make them uh what we call satchel resistance, which means that we try to avoid leakage to the outside, even though we're using a secure enclave.

A

So uh I am done yeah. So if you are interested by our solution, you can contact us and you can also have a look at our github. We also shouldn't work from end to end and we have various uh scenarios that we can cover.

A

uh I think I've left some time to for questions, but if not, I can also dig deeper into the technical security details or I can stop you.