ONNX October 2021 Community Meetup, 21 Oct 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 006 ONNX 20211021 Kuah ONNX and OneAPI for xPU

Description

Event: LF AI & Data Day - ONNX Community Meeting, October 21, 2021

Talk Title: Intel® OneAPI software stack: ONNX Support for xPU hardware

Speaker: Kiefer Kuah (Intel)

A

So uh good morning, everybody um in this talk we'll introduce what is one api and uh what does it have to do with onyx.

A

Okay, so uh I guess to answer the question of what is one api? Maybe I'll just give a description of the problem that we want to solve with one api uh intel built several xpus the x can represent c or g or v, and um and and this is to meet the requirements of different types of apps and workloads.

A

uh The different architectures of these xpus present a challenge to developers. I mean it's great to have all these different devices and you know their strengths for different apps, but with these many heterogeneous devices developing for them is challenging and and to develop them your code, the apps optimally, it's it's even more challenging, right um and and also the other thing is not just xps, but with every new generation of these xpu's there'll be new instructions and new technology.

A

That means that you know, if you want to keep updating um your apps or your uh freight workflows, to be able to use these new technology.

A

You have to constantly be updating your code, so development cost time and effort will will grow very quickly. uh So one api was conceived to alleviate some of this cost and effort. It won't completely remove you know 100, all of that, but it will. It will lower that cost and the effort and time needed to develop code for each of this xpu.

A

So it is a unified api where you just uh ideally write once your code once and be able to deploy your apps to multiple devices and for the new technology that comes up in in the new um new generation of devices that uh intel will release.

A

uh So I'm in in one api. uh Various components is sort of a suite of tools of libraries, um and I will you know, highlight a few, a few of them, which are more relevant to deep learning frameworks.

A

uh One of them is um the one api d, p c, plus plus uh it's sort of the programming model or programming extension to be able to do data parallel programming in c plus, plus the other one is dpls, so that's the corresponding library for the uh library, that's also uh written for parallel code, so library for it's per it's, it's sort of like the stl, but uh for parallel programming, uh the other one that is relevant, I think to ml apps and ml workloads are, is the uh one dnn, so it it's a library that is um written that has primitives that support the different apps that are found in deep learning, topologies, deep learning, graphs such as convolution and mathematics.

A

These are highly optimized for kernels um and another one. I guess the last one I'll highlight is the one ccl it provides. Primitives for communication patterns that occur in deep learning applications so that this can be used to support scale up for platforms with multiple one api devices or scale up for clusters with multiple computer nodes.

A

um So I'll drill down into more details about 1d and n, because that's what is, I think, very relevant to onyx and onyx runtime 1dnn libraries is a collection of optimized, primitives or ops use in executing deep learning graphs, and we think that this library can improve developers, productivity and enhance the performance of deep learning frameworks.

A

This library supports key data type formats that are used in deep learning, such as fp16 fp32, vfloat16 and int8, and it implements a variety of operations. uh Computationally, computationally, intensive and prevalent in dl graphs and such as convolution and matrix multiplication intel has added deep learning instructions such as uh dl boost in the cascade cpus.

A

One dnn has implemented ops like convolution and matrix multiplication. Using these new instructions, it will also support the amx instructions that will be found in the sapphire rapids cpu.

A

So apart from cpu, one dn also supports gpus, and another thing that one dnn does is is that it can reorder data into blocked formats that are more optimal for the different architecture of our different expertise.

A

um The key, I think, the key objective of onyx, as I understand it, is to you know, offer interoperability and high web hardware abstraction.

A

That means, if you have one in our nx model, you should be able to run the model on a variety of hardware. This goal is very well aligned with one api.

A

One api is abstract away: uh that complexity of programming to not just one xpu, but potentially several xpus, so intel has built uh dl xl acceleration technology into our cpu into our gpu and we'll continue to do so in the future to run onyx models using these accelerators require writing code in, in the run time, for these accelerators.

A

And one dnn uh did part of that work for us we have to integrate and what we have to do is integrate the library into runtime and, in our case, we're integrating that into the onyx runtime.

A

So we have done sorry, so this is sort of an ongoing work and we have done some of that. Some of the features have been added. Some more features we added into the one dnn runtime execution provider, if onyx, rent in onyx runtime so last year, that was support for 32-bit fp323 um floating-point data type. It was supporting inference. It was supporting a convolutional network as well as cpu. It did not have gpu support at that time.

A

We added the gpu support and currently we're also adding support for nlp, basically ops in the execution provider for nlp transformer models and we're also getting support for training. Since onyx is beginning to support some training ops as well, and we're also beginning to support integrate data types and potentially other data types as well in the future.

A

So that's where, where we're at today and we'll continue to develop, features and optimize the one dnn ep in the front time, that's the end of the time! Thank you.

A

Thank you for any questions for kefir.