ONNX June 2022 Community Meetup, 13 Jul 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Onnx-mlir: an MLIR-based Compiler for ONNX Models - The Latest Status

Description

Onnx-mlir is an open source compiler implemented using the Multi-Level Intermediate Representation (MLIR) infrastructure recently integrated in the LLVM project. It compiles ONNX models into native code for CPUs as well as specialized accelerators. It is able to compile models for many platforms including x86 (Linux/Windows/macOS), Power (Linux) and z/Architecture (Linux and z/OS). Onnx-mlir is a subproject inside the ONNX ecosystem and has attracted many contributions from IBM, Microsoft, Facebook, Arm and Universities since its incubation in 2019. In this talk, we will show the latest status of the project by providing the project overview as well as the latest features.

Tung D. Le is a researcher at IBM Research - Tokyo. He got Ph.D. from National Institute of Informatics, Japan in 2016 with major of systematic program transformation. His interest includes systematic methods to program transformation, high performance computing and compilers for AI. He is an ACM Senior Member

A

Good morning, good afternoon, good evening, everyone- so my name is uh dong lin from ibm research. So today I'm going to talk about our updates on honest mria, so which is a compiler for unleash the model. So it's a suppressed insight on it. So you can see it's linked here on the screen.

A

So, let's start with uh what is the honest, mri wire? So it's a compiler to compile an honest model to an optimized binary for different architecture or machine, so to obtain an optimized binary. We use mri and our vm so why llvm is well known for low-level optimization and of course generation.

A

So mria is an emerging infrastructure for high level, optimization and, of course, transformation. So in that sense uh the compiler is totally based on open source infrastructure. So we have been developing the project since 2019, so it's quite a young but an active project.

A

So, on average we have about 10 to 20 uh commits per week over the last year, and the number of unique visits during the first two weeks of june was quite large, and uh I think uh half of them got on the reference history, because the postage has attracted interest of some big companies such as ibm, microsoft, facebook and others.

A

So we have several goals uh when design uh developing online compiler. So, firstly, we would like to have to provide a reference on this dialect in mair, so which is an interface to bring on it to the mlyr world. So as a compiler. So we expect that it is convenient to write optimization not only for our cpu, but also for new uh custom accelerator from high level to low level optimization, so it should be easy to deploy so for end user.

A

We support four main language, such as python c c, plus and java for developers it's easy to integrate with other mli based compiler.

A

Finally, we expect that the compiler will be carefully tested, so not only by unit test, but also the whole model test on different architecture, a different operating system and programming, a language.

A

So this special how the compiler has been tested and deployed in practice so on the front base of the github. So we have a live report for different environment. So to see whether the current commit is a bus or not. So we also report the status of the honest model zoo so so far our compiler can compile and run more than 10 100 models, because the compiler has been deployed in ibm watson, machine learning for zos, so it has been used for online scoring service and pento ml.

A

An open soft model serving framework is using onix mri as one of the the infinite engines so in this year, so we newly introduce a framework for supporting custom accelerator.

A

As this frameworks allow us to quickly add a new accelerator to the compiler, so we can choose which unit operators will be uploaded to accelerator and each accelerator can have a special optimization specific to that accelerator.

A

So uh it means that in our framework, accelerator are blockable and it can be turned on and turned off easily.

A

So as a proof of concept, we now support ibm on-chip io accelerator, which has been introduced in ibm c16 announced recently.

A

Let's look at a little bit more on how optimization are deployed in on this emir, so from top to bottom. So an honest model is a gradually lowered into intermittent representation and finally, uh a binary is a produ. So at each uh intermediate representation we have a multiple dialect uh for cpu and accelerator.

A

So some dialogue, uh built-in dialogue from mlyr, are some newly introduced in onyx mria for cpu and accelerator.

A

So at the high level we have a high level optimization, such as a shape, influence or graph level optimization and at other level we have optimization for our cpu operation and accelerator operation.

A

So some optimization can be applied at the multiple level, such as the constant protein or canonicalization.

A

And finally, we would like to show how accelerators have speed up the influence for a practical model. That is a credit card approach. Detection.

A

This model is used for a real-time detection of portugal and transaction, so we did experiment on ibm z16 compared to running on the cpu as a baseline. So using accelerator is about 11 times faster and we cannot disclose more in detail so because it is related to an ibm product. So, but uh what we would like to show here is the accelerator framework in the compiler works. Indeed, in practice.

A

So, in summary, so we believe that, on this uh miya compiler is the convenient to do the optimization and to support a new accelerator so as a uh an open source project. So we would like to have uh contribute contribution from a community uh to grow it. So in the near future, we continue to optimize the important operator like a convolution, pooling, reduction, assessor and well. We will support our niche machine learning operator. So now we are focusing on a deep learning operator, but we still have a lot of work for only a machine operator.

A

Finally, we also want to support the other accelerator, such as the gpu. Finally, we in this thank all the contributors who have been contributing or will contribute to honest emil yeah. Thank you.