ONNX October 2021 Community Meetup, 21 Oct 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 005 ONNX 20211021 Huang ONNX and Ascend CANN inference interoperability for better performance

Description

LF AI & Data Day - ONNX Community Day, October 21, 2021

Ascend CANN and ONNX : inference interoperability for better performance

Speaker: Zhipeng Huang (Huawei)

A

Hi everyone I'm howard huang from huawei and today I'm going to talk about the sunken and onyx how we use onyx for very high performance inference tasks.

A

So, first of all, just uh a brief memory. Lane of uh what is open source participation in onyx.

A

We actually have been working with onyx community, uh with the china workshop and uh from the very start dating back almost four years ago, and we have been participating at each and every onex workshop, whether it's on online or offline, uh as many as possible in the community. We have spirit headed the ash working group initiative. We proposed uh the onyx edge runtime architecture, um so, starting last year, uh mine sport mindsport, which is our latest deep learning, uh training and inference framework, uh has has been using onyx in a large scale.

A

And today I'm gonna talk about our hardware support uh for onyx, and this is uh what also this pr is about.

A

So ascent can is an ai heterogeneous computing architecture which supports users to very quickly develop ai applications on ascend computing hardware platform, so can install for compute architecture for neural networks.

A

A key feature for ken is that it has a multiple layer of programming interfaces, so this multiple level of abstraction will enable developer to develop ai application more efficiently.

A

So, as you can see from the figure on the left hand, um in canon, we have the acl for the overall api. We we use ecl for the unified application, programming, interface and lower down. We have the unified neural network graph construction interfaces, which is air as a standardized intermediate representation, and we have a very high performance, compute engine and operator libraries, and um we have a a base service layer which provide capabilities like the drivers, virtualization communications and so forth.

A

So uh in our latest release the can 5.0 we currently have supported more than 140 onyx inference models, and hopefully that number will reach to more than 200 by the end of the year.

A

um Can 5.0 support offset 8 to 13 with the offset element as the the key offset we can support and more than 90 of the ops will be supported on camp by the end of the year and from figure you can see um from the 5.0 version uh to the earlier 3.0 version. We have a dramatic increase of performance and you are more than welcome to check out our moto zoo and check out the onyx models. We have been building uh upon a cent chem.

A

So here are some future thoughts. uh There are a couple of pinpoints that we have identified during our line of work, and hopefully the community will address these issues in the near future.

A

For example, we found out that the pipe force, nlp and audi models expert to onyx is still very difficult.

A

The trace mechanism doesn't support loop and if and the torch script is really underutilized in the whole process of exporting to onyx models, and also we have spotted there's a 35 to 40 percent failure rate when we export pi torch model to onyx models.

A

um The seventh uh problem we identified is that the the iteration of onyx offset is very fast. uh On the one hand, it's a good thing. It represents the fast development pace of on its community, but it also creates difficulties for hardware engineers, like the can developers uh to do the onyx adaptation work.

A

Okay here uh here is the end of my topic. uh You are more than welcome to check out a zen. Can we have repos on gt and also on gita? Please also feel free to reach out to me if you've got any questions. Thank you very much.

A