.Net Foundation .Net Community Standups, 23 Sep 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Machine Learning Community Standup - Sept 23rd 2020 - Blazor WASM & ML.NET with .NET 5

Description

Learn how you can use .NET 5 and ML.NET to add machine learning to Blazor WASM apps.

Community Links: https://www.theurlist.com/mlnet-standup-2020-09-23

Featuring: Bart Czernicki (@bartczernicki)

A

A

All right welcome folks, thanks again for joining us for another stream of the machine. Learning community stand up. um I'm luis, I am part of the microsoft docs team and I work on email.net folks want to introduce yourselves yeah and I'm bree, I'm on the dot net team. uh The pm for ml.net.

B

And I'm jake, I'm uh dev lead for the ml.net tooling good morning. Good afternoon my name is bartimke. I am a principal detective architect for machine intelligence.

A

Awesome thanks for joining us uh bart, so today we're going to be talking about blazer, uh specifically blazer web assembly bar he spent you know, took the time to come on and talk to us a little bit about that. We know that folks uh in the ml.net community they've been asking for this. I know I've been really excited about getting this uh working and now that uh.net 5 rc1 is out.

A

It seems like this is finally a scenario: that's going to be enabled so we'll be talking a little bit more about that uh throughout the stream. uh But in the meantime, let's go into some community links and contributions.

A

So, to start off uh there is this uh organization pass which is sort of like a virtual user group, and uh unfortunately this is uh right before our standup, so it happened literally, it probably just finished um and it was basically a workshop on introduction.net. So it's really great to see um other you know. Organizations and user groups um outside of this uh community stand up sort of evangelizing and sort of promoting email.net.

A

uh So this is great. uh We understand that there's gonna be a recording of this, so whenever we get, you know uh our hands on the link to that uh we'll be sure to share with you folks so uh stay tuned for that um we had our fat here. Who is an mvp and he wrote this blog post talking about beyond sentiment, analysis and how you can use object, detection with ml.net.

B

A

Who may not be familiar um the object, detection scenario with model builder uh ellen's training uh object, section models with uh with model builder uh leveraging azure machine learning um was released uh in the in the previous release right. So you folks can go ahead and give this a try. If you you download, train your own custom object, detection models right, um so he basically just goes into how you might want to go about.

A

You know basically, uh training an object, detection model in model builder and walks you through the process. So this is really nice um and if you folks, you know, take a look at this, uh you can sort of follow along with yourselves and train your own custom, objective action models, speaking of uh object, detection or just in general, uh different methods of training.

A

With model builder, we have bruno again who was kind enough to put model builder through the ringer and essentially use different methods of training models, whether that's just using the cpu, uh the gpu and using the azure right.

A

So if you want to train the cloud- uh and he basically just documents his experience here- training uh different types of models on these different environments- all right so make sure to check that out again for folks who, uh our portuguese speaking audience, davi ramos put together this blog post, which is basically talking about models, how you can deploy models in production via api using docker. So he basically takes you so through what the process is of containerizing uh an application or a web api. That would expose your mobile.net models via docker.

A

Now, if docker is not particularly the method of deployment that you're really interested in, or maybe you have a different scenario that is more event driven uh pravin put together this really nice guide uh that talks about end-to-end, serverless, deep neural networks and how you can deploy them with azure functions and again it's going to be using ml.net uh to to basically uh deploy to an azure function. So again you.

B

Have those post.

A

Scenarios, whether you want to deploy to docker or whether you want to deploy to uh azure functions or some sort of serverless environment, you know you can basically leverage whichever one of those best fits you this week is ignite, and uh hopefully you folks have been following along.

A

There's been a lot of really interesting announcements, uh and one of them is actually related to what we'll be talking about today and it's basically now you're able to use azure static, web apps with net and blazer um so before, right, so so static web apps, if you are not aware, is this service that you can basically use uh azure storage to put your files right?

A

So if you have a static site like a blazer web assembly website or application, you can basically just put your files in azure storage and expose them as a website as a static website through azure static web apps. Now, prior to this, my understanding is that you could only deploy the static files right and with the new update. What you're going to be able to do is sort of similar to if folks are familiar with the jam stack right, javascript, uh apis and markdown for generating uh basically static websites.

A

uh This is a similar use use case, except now, you're going to be using full sort. Of.Net azure functions uh type of stack right, so it's really nice that this scenario is enabled. So now, not only are you deploying uh the the static assets of your website or application, but you're also deploying uh the apis as, along with um you know, your your backend that you're gonna need all in one package. So that's pretty nice uh talking about deployments and basically operationalizing those deployments.

A

Alexander here uh did a talk. I believe it was at a user group on ml ops tool that he's been working on and he continues to work on and, if you're interested in checking it out learning about learning a little bit more about machine learning, operations or ml ops uh feel free to check this out now.

A

uh Bart's gonna be talking a little bit more about this, but I thought it was really nice in terms of uh helping you conceptualize part, some of the applications that bart has built, and this uh in the architecture center here of docs.microsoft.com you're gonna be able to find uh one of the applications that bars put together, leveraging ml.net and blazer. In this case it's a blazer server, um but you can imagine that now that the blazer was scenario is is available.

A

You can potentially um you know re-architect that in a way that best suits that scenario. So that's that's it in terms of community contributions. But if we move on to what's new within ml.net uh one of the nice things that just happened was uh ml.net 1.5.2 was released right so a few days ago um and for 1.5.1 uh it introduced a regression that has since been fixed at 1.5.2.

A

So if you folks haven't upgraded to 1.5.1, we highly encourage to just skip 1.51 and just move to 1.5.2.

A

um That being said, in addition to fixing this regression and introduce a few uh bug fixes as well as some additional support for onyx types, so make sure to check that out. uh 1.5.2 and last week we or not last week, but on the last stream we had the folks from the uh psysharp team come on and talk to us a little bit about tensorflow.net.

B

And the different.

A

Projects that they're working on and.

A

uh Tensorflow.Net actually recently released an upgrade to version uh to the newest version of uh tensorflow, so currently there's some investigation and some work being done on how ml.net can leverage this uh sort of upgrade of tensorflow.net to basically improve and upgrade uh ml.net. So yep, that's that now with that, let's turn over to bart and his demos.

B

Awesome, thank you, lewis. That's lots of cool activity there on the community stuff. It's pretty pretty impressive. uh Can you guys hear me? Okay, sweet, okay. So what I figured uh uh thank you for the introduction um around um the uh the workbench and I figured. What I could do is uh how uh why why why I'm so excited about um blazer and uh webassembly, specifically um I'm a um old silverlight developer.

B

So you know 2009 2010 were a couple books about serverlight and business intelligence and how you can do an analytics and advanced analytics on the client and, if you remember, silverlay had you know was able to do.

B

Some of these things run on the phone run, run the browser, but it required a plug-in framework, and you know because of the different tech stacks that have emerged and some of the technologies you know silverlight and flash kind of where kind of went it went their own way and what's really exciting about this now with laser web assembly, is that you can bring some of those patterns back and offload essentially compute down into the the local edge being the browser.

B

It's really interesting about what we're going to cover today is now we can do before you could do some compute. You can do some basic applications, but now you can do probabilistic programming leveraging machine learning.net, and that is what we're going to be talking about with net5 rc1, which I'm going to be showing as well as machine learning.net and blazer technologies.

B

So that's that's. What's super interesting a little bit about my background, in addition to the uh the machine learning I do at microsoft, so what I figured to do is I'll. Do a quick demo, maybe of what is available now, um if you, if you're in.net, core 3x or net 5, um follow this link, akms slash baseball ml workbench and you want to play along follow along as I kind of demo. Some of this stuff is this. This is what we're looking at right now. So let me maybe make this little.

B

Just like here so um if.

A

B

This link you'll be taken into this application here, which lewis was showing the architecture of, and basically this is a hosted application using asp.net core on uh using our azure app service, and this is has uh using, is using a blazer, server-side technology and is embedding machine learning.net models inside in memory here. So what you can do here is, I have a quick demo, I'm not going to spend too much time on this one, but I want to kind of give a lay of the land what's available now.

B

This is using historical baseball data for the last 140 plus years and doing a classification of baseball players careers, whether they made the hall of fame or or could be introduced into the ballot. So basically, if you're inducted to the hall of fame or the probability of being on the ballot itself. So you can pick pick your favorite player here. You can search pete rose, for example.

B

You know had a very, very uh great career. You can see his hall of fame probability and the ballot probability are very high and you can do very quick what-if analysis around what what uh you know. What is happening, uh what his career could have looked like if he played less years or what his career could have looked like if he played more years and what's really interesting about this- is that this feels like a like a client-side application.

B

It's very, very responsive ml.net is doing inference on the server in real time, but it's pushing down that inference down using signalr or websocket connectivity. So there is a server component that is required here um and then.net 5. What has happened is the the blazer technology has and especially on the web assembly side has moved over from the modern run time to including the the base class lab.

B

The core libraries are now based on.net standard, so this allows uh this allowed has allowed some um compatibility to be uh to be exploited using the compatibility that ml.net is written on and now isn't more in sync, with the compatibility with laser web assembly.

B

So laser web assembly is a little different than server side, where the entire compute and execution is that, as is done for you on the client side.

B

So, whereas this compute, as I slide to move each tick over and each probability has- is changing in real time, this is done on the server and this is pushed down to the client to the web browser here in chrome and what what what blazer web assembly allows you to do now is basically mimic this.

B

But now the compute is down is completely done on the client, so there is no server requirements that have to be uh have been processed, and so now you can do the full execution, the rendering the javascript dom interoperability, as well as the machine learning models can be loaded in memory on the browser and they can be executed and referred directly on the browser and on the edge. So you can have complete offline offline applications without having to make api calls websocket connections calling to azure functions, calling into remote storage.

B

It could be all pushed down to the local client. So what I'm going to do is maybe show a demo of the demo of the client application and show how it's it's it works and then go go over the architectures and the code as well.

A

So so before you get into that bar, I just wanted to ask you, so you were mentioned that you're pushing everything down into the client. So what you're saying there is, as opposed to sort of the the link that I showed of deploying to azure static websites, you're saying because it's all static assets, you can you really don't need connectivity yep to any sort of server? You can just bring everything down and even use it in offline scenarios? Is that exactly.

B

Right exactly right exactly right, you, because, because of those scenarios, the entire application, it can be cached locally and I'm going to cover uh a neat little feature that you get in a blazer web assembly called pwa, which is uh progressive web applications and that allows you to essentially make the application look and feel almost like if it's a mobile application like an android or an iphone application to a client and I'll, actually show a quick demo when, uh when we show this that you can actually turn off offline mode and you can see, there's actually kills all the traffic between the browser, tab and the network, and you can.

B

You can see that the full application will still still continue to run very cool, so uh moving on this is available. This is available now and louis. I mentioned there's links out there. If you go here info resources, I'm not gonna spend too much time here. There's if you want to build this application, the architecture, the documentation, the source code, uh there's docker containers I have available here. So if you want to get immerse yourself into this server side, technology uh feel free to do so and just to be clear.

B

This is what's available, currently in.net core and then long and ltos release, and this is what's available. Obviously, a blazer server continues to be supported in a first class product functionality in dot net 5 as well. So now we're going to move on to a demo in here called prediction: furnace test blazerwasm. So if you're following on below this is going to be uh right here so mlnet and blazer.azureedge.net and as louise was mentioned earlier, this is a this is actually hitting the azure static website.

B

So we're going to be doing I'm going to walk you through a demo that is a blazer web assembly uh with net5rc1, with machine learning.net deployed to an azure static website. So there is no server anymore. Now your this endpoint here is essentially a cdn endpoint that is pointing to a static set of files pointing to a cloud cloud location.

B

So um a lot of links here so feel free to to peruse on, as you will source code demos. Additional related demos are all available here, different ways of deploying it static website app server. So I have demo links and source code available to to to to so you can see the different nuances and how the different things work. So it's actually interesting here. What I'm going to do is actually click. This link here probability test and this probability test actually goes in and loads 30 000 rows into memory dynamically from a csv file.

B

So if you're a data scientist or you work with, you know, um you know excel notebooks or you know, jumper notebooks are python, you're, probably familiar with loading data from a csv. It's exactly what we're doing here and we what's actually also happening, is I'm actually loading several different algorithms and models that have been built. So these are models that have been built with these specific algorithms.

B

You can think of them. As you know, experts as a as a end user, you can think of it as going to a doctor and getting opinion one or opinion two opinion three and seeing how how these different experts disagree or agree and if you're a net developer.

B

You think of these as data structures, so data scientists operate on algorithms and models. um You know as you're net developer you're operating on data structures, yeah, hash tables, linked lists, uh arrays, there's differences between them, they've taken data; they do something with the data and explore expose that to uh uh into your software same thing. Here we have these different scary, looking names, you know fast forwards, fast trees generally uh generalized additive models, but essentially what these are. These are data scientist data structures that expose do something with the data.

B

Do something interesting with it and expose some functionality with it now with the functionality. What what a model does is it in this case is a classifier, so it basically classifies um the career of mike trout this case into yes or no. This person can be on the ballot or not in the ballot. In addition to some of these, algor algorithms also support the uh exposing of probabilities.

B

So you get this number, so we can tell you how certain I am that mike trout is going to be on the pro in the hall of fame, not just saying he will be or not saying he won't be, but also you know, maybe in combination he's really. We are really really sure, or you know what we're really uncertain that he might be.

B

So, let's take a look, though real quickly, um so if I click on any of these different algorithms, I'm loading a model and doing the probabilities in real time and basically, if mike trout's going to be on the ballot or also inducted into a hall of fame. In addition to I'm also rendering a nice looking chart here, let me zoom in a little bit- and this is all using uh svg, scalable, vector, graphics and basically plotting every single one of these algorithm outputs.

B

These probabilities for every single one of these algorithms, whether the person like my child, made the hall of fame ballot or is inducted as well as I'm drawing the line through the different probabilities of the of the of the outputs. So, for example, let's walk through this, so mike trout has played nine years for every single of the nine years. I get a probability for the model selected as well as the ancillary models, and basically I draw a line through the probability. So year two probability was less than ten percent.

B

You know probability of seven for making the on the on being on the ballot was around. You know: seventy eight percent, conversely, being inducted to hall of fame year, seven was around. You know, 30 percent, for these different models and I draw I have a range chart that that defines the the real realistic range of probabilities for the different models. Now what's really interesting here is that this is completely done in the client.

B

This is all you know. You can see how quick this application performs. um You know very, very it's very responsive and the best part is this is doing you know every single one of these data points that you're seeing every single point on this chart. Every single line point every single one of these items in the table is 100 generated in real time using machine learning.net dynamically.

B

There's no caching here, there's no, I'm not not cheating a little, I'm not shooting where, where I'm you know, you know loading everything in the memory and giving it a probabilities and kind of reading from a table or or.

A

B

From the csv file itself, this is dynamically as I, as I click each different player. I just play with it yourself.

B

This all works and it works pretty quickly now going back to luis's point earlier, if I hit inspect here and I hit network and I hit and I hit offline, so I'm going to take my entire browser offline now. So if I go back here and go back to probability test- and I continue working with this- you can see the application continues running. There is zero connect, there's zero network bandwidth going back and forth.

B

This is this is entirely pushed down onto the webassembly or virtual machine inside the browser and inside that virtual machine is what blazer web assembly is executing inside, and this is all net machine, learning.net and um and uh what's called and the data itself, that's actually loading the memory. So this is really neat now, as you can see, the application is responsive and run in um you know if it's fluent, so it actually works on the phone.

B

I actually have my two phones here, my iphone 8, my iphone 6., so my iphone 6 from 2014, which is what six years now uh the iphone 6 from six years now, can still run this application and run it pretty. Okay and obviously iphone 8 does better with it. You can check check, feel free to check it out. You can actually see the application running uh pretty pretty smoothly and when it's pretty pretty nicely.

B

So let me um actually uh uh okay, take a step back here and load this application in edge and I'll show you why I'm doing that in a second, so I'm gonna load this application in edge and I'm gonna show you another really nice feature of the of this. um Not only can you run everything on the client you can actually because these applications look and feel, like you know, essentially, you know local applications. The idea here is why not make why not take the next step?

B

Why not not just not just show looking at it run a browser but try to make it feel more as if it's running locally inside the uh inside the os? This is uh if you're, uh if you're browsing on an iphone uh on I o uh on uh using safari or you're, using uh chromium based browser, so basically chrome or edge. You will get this nice little link here that says install app. So actually I can take this application.

B

That's been running inside 100 on the on the local edge and actually install it so hit install, and this application now is running. You can see what happened. This actually is the progressive web, app extension that is available in blazer web assembly, so this took the entire application um cached it locally inside and still using browser, but it removed the and removed the um the uh the toolbar and removed the kind of look and feel of the browser. Kind of looks like it looks: gonna feel like a standalone application.

B

It's still um is responsive still works completely offline. If I, you know disable my wi-fi, which I won't, it would still work, and this actually looks and feels like a factline application that I'm running this is, and it's running, machine learning, dot net completely locally. All the models are running locally here and then what's not also cool is uh you can see I'm on a mac, so I'm cross browser.

B

So this is you know, as you mentioned, machine learning.net and dotnet five cross platform cross browser across you know, support uh diff, different tooling, uh across the different uh ecosystems. I get this link here. I can actually drag and drop this link and install the application down in my taskbar. So I actually have done this earlier because it is lm here. You click it and I'm launching the application, as as if it was a backline application.

B

This is, you know completely running inside um still running inside the um uh web assembly, uh a virtual machine, um but it is running it looks and feels like a like a uh you know, fat client application and does not look and feel like it's a browser application and you kind of removed, stripped out that component out of it.

A

So so let me ask you uh uh really quickly, so this is all really really awesome by the way, and you know this is truly the scenario right that that uh customers can enable, when doing uh you know when basically leveraging laser web assembly and webassembly in general.

A

But uh you know I'm thinking how much work is it for me to enable this right or is it? Is it really easy like what's what's.

B

The effort like.

A

B

And it is literally uh check box to get to get this experience uh this uh this this um this pwa experience. It is literally when you uh do. You know new project and you select the uh blazer web assembly uh template there. You're gonna have a checkbox in there where it says, enable pwa, progressive web apps and you just check it and we actually bring down and inside the template. We actually enable the manifest and the service worker uh which measuring and then some icons.

B

So you could that you can change and basically, that manifest file that service worker and the icons, those three components together enable the the ability to make this a progressive web app and turn a regular blazer application into a progressive web, app application. So literally a checkbox or if you wanted to you, can manually just copy, I think it's four or five files and reference them and then you're ready to go so this experience is, um you know, almost out of the box. If you will inside the tooling of visual studio.

A

Very, very cool, um so we have. We have actually a few questions here in the chat um one kind of going back to. I guess all the components that you're showing the graphs and- and uh I guess the the tables- maybe um I'm not sure, curious drive if you want to clarify that. uh Please do some chat, but the question is: are these components, part of and I'm gonna guess, what's sort of blurred out there is ml.net? Is that right, curious right?

A

B

Are these components, part of ml.net, so these components here? This is what's the beauty of of using something like blazer. um Basically it's all html and javascript uh here, so I'm um this is um just html and css. That's all this is. uh We do have partners uh that actually provide components out of the box. So there's I think, four or five components.

B

If you go into the blazer site, I don't want to start listing them because I might miss one and I don't actually feel uh you know, leave anybody out, but there's uh you know, uh there's our. We have partners that actually can provide components out here.

B

This demo is using straight up, vanilla, html and css to make it look nice and pretty styled if you want, but we do have if you want to, if you want to make it an easier button where you want to drag and drop a or or put a, you know, control on our table or button, and you just want to bind it to a data set or combine into an action. uh We have partners that do that as well.

A

Very cool and then uh another question that we have is, uh if you're, bringing all this locally, um how large is the whole package uh in the browser? Yes, that's.

B

A great question: yeah, that's a great question, so if you look at it completely all up, let me see if I can.

B

The entire package is 16.5 megabytes now, 16.5 megabytes might sound like a lot, but that includes a few things uh that includes every single one of the assemblies. That's that's referenced here. It also includes the entire full dotnet dependent framework and also includes my code, so it includes all those three things down to 16.5 megabytes, and this is- and this also includes my data, my models and everything else right. So this is the complete application and if you, you can actually see how quickly it loads.

B

uh It sounds like a lot, um but you can actually, you can actually see how quickly it loads by you know, clearing out and hitting the edge again. um Obviously these are um the blazer team and that net five team is working uh have to make these smaller. So these these will have one will not only get smaller but more compressed and more efficient. So this application, the specific application, is 16.5, but I'm obviously loading. You know a bunch of data. I have a bunch of, I think a couple dozen models in here.

B

I also have you know um you know um you know uh svg. I have the javascript references, so so I'm probably not the best application to get to the size of what blazer is because applications can be much smaller. I'm just referencing a lot of additional stuff that probably doesn't need to be referenced in a traditional, smaller application.

A

Yeah and curious drive actually uh points something out uh here, which is this 16 megabytes are only a one-time download once it's loaded.

B

Up right, it's.

A

There right it doesn't matter how many times.

B

Exactly right, so blazer web assembly is really nice is that service worker that I talked about essentially is a javascript file and basically uh each time you load a blazer web web application and progressive web. That matter, it reaches out into the server and goes hey. What's changed what's different and then it actually brings it down. Actually, when I was preparing for the demos today, I actually had everything deployed into my azure cdn edges.

B

I had everything I had these applications installed and several different computers, just as backups just get something it goes boom and I push down update and I click. I click this launch. This relaunched this and actually what's what pwa was able to do, is reach out to the server find out what's different and bring out bring not only the delta and I actually didn't have to reinstall the application or wait a long time. I just figured it figured that all kind of like automatically figured all that stuff out just. I think.

A

Very cool, very cool, um and I guess you know if it's all right with you. Maybe we can take one more question related to getting everything locally absolutely um so uh this is actually a really good question, uh one time poker. uh Are there security issues when it comes to dumping? The code on the client yep.

B

Yeah, basically, all those rules apply. I mean I won't take too much to answer the question, but anything if you think about like right, if you're writing a phone application right, is it an application like where um you know you want to? um You know, have have a a light footprint on the ui and basically call call a bunch of services through apis and because you know, there's some security or regulations that need it or some ip that you don't want out there in those certain cases.

B

You would not do this, however, if there's, if, if there's, if there is anything not non really, you know bring down machine learning models and loca and anything local you're, taking a risk you're, taking a risk of putting your code down on there and there's ways, obviously, to mitigate that there's obfuscation techniques. Obviously our partners that, uh as mentioned earlier, that bring down their ip and and their charting pla their their charting work and things like that.

B

There's there's ways to mitigate that and we have those those those have been around for a while, there's still existing blazer, but obviously that's a kind of like a middle-of-the-road kind of answer. So things if you want super security, and you don't want things on there, because you don't want things to get. You know how you, for example, you wouldn't put a password generator down to the client right.

B

That would be kind of silly right, because that you're just asking for somebody to get to somebody's hacking right that you would never do that on the client, no matter what, um even even if even if it was encrypted and super secure. So and then obviously, then we have these mechanisms inside.net, which allow you to obfuscate.

B

The code- and you know, have security concerns and being able to pass down licenses and obviously the other opposite way is when you know, when there's nothing, sp, very specific and very or or you know, you're not worried about your ip or security wise, then you obviously have more more freedom to do to do and bring down these um objects down locally.

B

So not a great answer, but I know it kind of depends. You can actually see some of that some of that in action. If you download some of our partners and you go, you know, new blazer net was an application download one of the charting packages or data visualization packages for our partners. You can see some of the security mechanisms in action actually you'll fire it up and go wait. Why is this licensing license thing popping up? Why is this key? Oh, let me look at their code and see. Let's see if I can.

B

You know uh you know the you know fire up eldas and try to get in there. You can't right so there's some there's some there's, definitely techniques to secure your code and make it make it um you know make it make it safe so and same same application same same thing as way, if you're doing a react, application, angular application, those exact those same security patterns, best practices apply here, no different.

A

Right and and just to kind of extend the definition here of code right. You should definitely treat your models as if they were code, because.

B

A

Are coming down as static assets and because you know people can expect them uh you should you should be treating those just like you. Would uh your.

B

Code yep great point exactly.

B

So um I can show, maybe in a couple architecture, documents and walk through some code just to see if that's, okay, all right. So um this is on my github. So um this presentation, I'm not gonna, walk through the slides, but if you're interested you can kind of see more details of how this was built and more information.

B

um This is the um I think it's a little bigger. Maybe this is what um the workbench look like when deployed into azure app service, so you'll have my mobile application web browser. If you're following along you hit it, you made a websocket connection with via signalr and basically you know, there's app, there's instrumentation and things like that, but this ran on a server. This is running on the server pushing down communication through signalr.

B

You can see blazer and ml.net they're running in memory and the machine learning models themselves and the data decision analysis all is running inside the server. It figures out what the probabilities are figures out: the divided data. It figures out how to make the you know the the the the the uh the card you know different colors. It renders the delta of the html pushes it down and tells the blazer runtime via javascript to say hey. This is what your state was. This is what it looks like now, and it makes it look pretty.

B

Basically, that's what the blazer server side looks like with machine learning. So what? What does it look like with webassembly so now with assembly? We're introducing a couple things we're introducing the the different deployment um option, using deployment deploying into uh webassembly uh and runtime, as well as dot net five and we're hosting on static site, not a dynamic site? So now there's no compute on on the on the static site. Now, how does this look like on a on a wasm option? Same thing, you know you have a mobile web browser.

B

You have network local communication, so obviously, initially it could be network. It could be completely local if you have a pwa or you have a application fully fully cached. It is actually using the web webassembly virtual machine inside the browser and the entire is doing the entire uh execution of code framework machine learning models. Your prediction analysis running the blazer uh interop between svg and javascript, and everything is inside the web browser. So everything here there's no! You know um in this scenario.

B

In a specific example, there is no callbacks to us api there's no server. um There is all the compute is a 100 inside your your local web browser and what this makes for cost purposes. This is a fraction of any cost. A a static website. Deployment like this is roughly about two bucks a month, the cheapest version that we have on app service. If you did, this would be about 10 bucks and realistically you probably want to go at 40 50.

B

So you can see that you're talking about you know at least a 5x to even higher savings by deploying your application as a static site. Now we say static now we're using the term. That's a microsoft term. Azure static websites, azure web app web apps, but you obviously have free options. You know you can potentially do this using github pages. Github pages has static websites now imagine being able to have a static website that you know in a term is static, but essentially has dynamic components into it.

B

By basically being able to do machine learning live so yeah. If you're familiar. If you're a data, scientist and you're familiar with jupiter notebooks, you know, jupyter notebooks are nice, you have uh you have some text, you have code, you have outputs your visualizations, but they're largely static. You need to you need to fire up. um You know jupiter jupiter server, you know, put it put somewhere where jupiter can run and run the compute, whether it's local or um or on you on a server. But you need some kind of you need.

B

You need that server component to run and to make it dynamic. What's really nice about ml.net and blaze or wasm is now you can actually have these documentation sites or or you know, simulation sites or interactive? um You know interactive kind of small small information portals like new york times. Does it does it does a great job once that once allows you to go, you go to new york times and you have an interactive site and it's all you know you can play with the data.

B

You can and see how it works, but it's basically it is a it's a news article, but it's interactive blazer and ml.net that can facilitate that and dramatically lower the costs here.

B

um One more thing to touch on a little bit here uh because I think, there's a question earlier: how I'm doing the the the the rendering of everything all html and css for the look and feel and the graphics are done by d3, js and svg. So after every after blazer is done, processing the machine learning models, the predictions, the outputs and and running with the data, and I I wanna I'm ready to render I I do a fire and forget method inside blazer and call in uh d3 js and render the application.

B

So it looks you know, aesthetically pleasing. So this way it's an svg is obviously super fast runs on mobile lightweight. um You know super popular data visualization framework and it allows it's perfect for something like machine learning.net where you want. You want to have a little bit more um power on how your visuals look like. So that's kind of the difference between what you what it was available now in net core 3x, with laser server and dot net 5. Laser server obviously works on dynam5 as well and what's available now, and how?

B

How would you potentially approach this problem in net five rc one? Basically, you can now take your code and I could literally copy and paste the code from here from my previous application. Put it in here uh make sure there's no, you know, obviously, no no, no signalr references and things like that, but I literally literally this would be a great majority, would be copying and pasting code. I can have my laser server application actually running inside a web assembly.

B

I actually have a couple of different demonstrations and links um in in the in the presentation here and in the github where actually show net hosted, non-docket hosted it's literally. If you look at the code same javascript, the same css same html, same c, sharp with a few tweaks, it's almost identical what you can do when you could do now. So I think this is pretty neat.

B

um Maybe I'll talk about some of the gotchas. uh This is uh slide seven. My presentation- and this is some of the uh checklists and making sure what you when you have obviously.net five is in release candidate. So it is um you know it's still. It's not. um You know general availability, yet I think it goes ga in a couple months. um I think november is the time frame uh when we go live, but you obviously we're a race candidate. So you do need in order to replicate this yourselves.

B

You do need the least candidate built. um You know I'm using ml.net 1.52, but I tested it with the previous versions and it works fine, so some kind of semi-recent version of ml.net I'm not sure what it is but semi-recent.

B

um If you have uh the visual studio 2019 it the tooling, is included in in 16.8 preview 3. So you do need the preview 3 tooling. From I think september.

B

15Th was the release for the uh tooling for visual studio 2019, the supports.net 5rc one that has all the uh integrations with uh nuget and all- and you know blazerbasm and things like that, and the big thing here is to and I'll go through the code in a second is basically, you need to put the um ml.net nuget packages in the shared library and reference them inside the project and that's kind of a minor work around uh for for some kind of uh it's for a cpu warning directive that you get that prevents kind of a clean, build and um that's something that that uh hopefully, will be fixed soon.

B

But basically, uh when you create an application, found laser rasm, don't add your ml.net assembly directly into the application, create a shared assembly, put them in there and then reference it I'll show you how that kind of looks like currently ml.net inference works in five blazer, so training automl is currently not supported yet, but you know you can take your models that have been pre-built, whether you, however, you built them and put them, and you can now take them and do inference on so basically, inference is being able to now take a model, not training it but being able to take an existing model and then passing new data and getting prediction results out static websites and asp.net hosting deployment supported here, as you saw the demos it works, fine with with ml.net and also pwa's offline.

B

Caching works with ml.net as well. So that's those those those two techniques here that uh luis was mentioning and, as I demoed as well, are fully supported and for a blazer server. The the gotchas here are basically, if you wanted a completely offline application, if you want, if you want static website deployment, that is not you. Don't want to do laser server, uh but let's blaze, the webassembly where you want. You want to kind of reduce cost dramatically and basically um deploy uh to a static website that so blazer or someone's technology.

B

Here, the the list of items to kind of be uh uh wary up until we go into ga in a couple months where this list will hopefully be completely mitigated and what's available in server and currently as well.

A

And, and maybe something to add on to the blazer server- is that the pwa scenario is also uh possible right, but the difference is you're still dependent on that server. That.

B

Yeah exactly right so you're you're, it's it's not off, not completely offline, you're, still going to be communicating back and forth right, exactly right, yeah. So so it's kind of it does give you some of the benefits for pwa. It gives you the benefit of the application, look and feel like it's. You know installed it's not running inside the browser. There's caching, except so the application is large. It can be. You know, pre-cached, um and also, um and uh and also the when you launch it for the next time. It's pretty quick, very responsive.

B

So there's there's no install. If you want there's no waiting so yeah, you do get some benefits of the uh of a server-side uh uh uh blazer, but you still don't get the full complete offline capabilities. If you wanted the application to be completely offline, um it's actually interesting because um one of the patterns I actually used.

B

um This is like 10 years ago from silverlight when I was to bring down historical baseball data, and I actually would take the the assemb individual assemblies and I chunk the assemblies back into different years, and I would have 150 assemblies for each year of data and as the as the user was dynamically, bringing them in you could dynamically download them and bring the assemblies down into the client, so very cool patterns that you can do right now that you can actually have a massive amount of data just available sitting on there on on a static website and just waiting for you to consume and as the user is using, it basically being able to.

B

You know virtually hot load it down to the client, and I think that's that's those those patterns are just super cool right and that you bring down things like that, so yeah, absolutely so mix and match uh hybrid, completely offline. Or you know I love, I love wasm, but I want to call it function right. You can do it that way too.

B

So a few different ways to kind of approach it and let me show you, maybe I'm working on time a little bit, but let me show you how this kind of looks like in code. So, if you download my code, this is you know we're cross-platform, so I'm using visual studio for mac. So this is, uh you know, on the mac, full visual studio, fully supported. I can build this, you know, and you know it's what I can run. I have my blazer wasm host assembly and my doc shared assembly.

B

So, as mentioned, that is where um it gets a little bigger. This is where my.

B

This is where my nuget package goes, so this is where my microsoft.ml my fast free my light gbm any any other related microsoft.

A

B

Packages they would go inside the shared assembly inside my main blazer application. I do not have microsoft ml reference. If I reference microsoft ml, it will actually give me a warning. It will prevent the build and things like that unless you do some work arounds, which are keeping it simple.

B

This is a nice way to you know, put put your ml.net assemblies inside the uh inside a standalone application, dot shared reference it and then uh and reference it here, and then, basically, you get the goodness of all all the microsoft ml stuff passed through implicitly into your host application. And if I look at my code here, if I look at you know, for example, uh pages probability.

B

Test- and you can see you know- I am using you know um and perform predictions- I'm using I'm using all doing, calling essentially microsoft ml through through the different uh methods here and and that's the way, to kind of to kind of work around the current limitation of not being able to directly reference microsoft, mail inside uh directly inside the blaze application. But in.

A

Part, it's kind of nice right because you get to sort of decouple both of those things and your machine learning components, get to change and and be updated independently of your uh blazer and web application.

B

Absolutely- and it's actually funny to say that, because if you're going onto my github when I have a hosted version and then non-hosted version, I actually just literally take the shared portable I'll call it that's not it's not I'm not using the right word here, but taking the that assembly and basically being link it to another project. So I can take this and link it to some completely somewhere else, and it just works it. It honestly should be a best practice.

B

uh But if you do find you, you uh file a new project out of the box, you just get. You know a one one one cs project and then every you you, you initially start going. Okay, let me do add machine learning.net in there, uh whereas, if you add the second assembly, if you start thinking about building a bigger application, uh this is where this sort of happened, but absolutely you're right. That's a standard best practice.

B

Everyone do this anyway, but if you're doing a kind of hacking up a demo, um you might run into that issue um and um a couple of couple of new notes here. So you can see how how's this brought in. Basically, I uh take this uh and inside the um project. I have a reference, so I'm, basically I'm just adding reference. So no big deal here right, adding reference any one. Other thing you need to do inside here is in the imports that razer.

B

Basically, what this does is you can think of imports that razer as uh where all my using statements are kind of going uh that are kind of shared across all the different pages and components inside a blazer, and you can see right here. I added microsoft.ml microsoft.ml and then without even having a reference directly in a project that implicit reference through that shared assembly allows me to actually directly reference microsoft.mml inside here.

B

So that's the kind of like the work around that you might run into, uh but otherwise um I think you asked the question earlier. How easy is it to get started? I answered the question from the pwa perspective, but from a um from the um guess, getting adding ml.net is that's it new project.

B

Laser web assembly create a shared assembly. Put your microsoft email down in there. You know find find a find. Some of the greater community examples that we.

A

Have out there.

B

And yeah, just you know copy and paste the code and it will work inference stuff um you know, should should give you a positive result there and you know. Obviously I have my examples here and I'm sure there's gonna be others coming, because you know just pretty cool scenario, so you'll be able to pretty quickly uh work with that.

B

um I didn't want to mention. Maybe a couple things here.

B

uh Where is this okay? um This is where the javascript, that is where the um the rendering of the chart happens. So this is basically a fire and forget where, after everything is done on the machine learning side, it gives me all probabilities, I'm ready to render I'm calling out to d3.js via blazervasm.

B

So, basically anything you can do in javascript. You can extend that out there. So if you already have a great fantastic code that you have in angular or react or javascript that you you're happy with the user interface, that's great, you can bring in blazer webassembly and machine learning.net and extend your application and fuse it further. I'm already have an investment in javascript. I already might already have an investment. I don't want to rewrite everything in blazer or or in c sharp you don't have to.

B

This is the way I would start applica a new application, but if you already have an existing investment, you can attach essentially um your your a new blazer uh blazer, a webassembly component and then interrupt with javascript called javascript to blazer or from blazer calling to javascript so that that's really cool right. So now you can have a javascript application that you already have add adam laser component and call into machine learning, inference methods right, so your ml.net models can now be and do inference using javascript. So.

A

Yeah, it's really a really cool call out, because uh you know some people tend to think like oh uh blazer, so I never have to write javascript.

B

A

um That might be true. It may be true that you can do away with a lot of your javascript, um but it's not an either or it's more like you know, an and type scenario where you use javascript, where you need it, uh and you don't worry where you don't right. So it's whatever level of comfort you have uh when building your application.

B

Yeah yeah exactly right and obviously you know at microsoft. We know we're we're very oss friendly. um You know we're very uh we're friendly with open source. Obviously ml.net is open source right that those kind of interaction- interoperability just makes sense. um It's funny. You say that about the application. If you go into the ml workbench application, I went all the way onto the other xtreme on that one I actually did.

B

I was specifically challenging myself to say not to write a single line of javascript and you can do that and if you want to see a 100 c, blazer application that looks, I I think looks pretty decent with zero javascript code, obviously not just javascript inside blazer and now you know, implicitly, you have to do references but that I had to write. That's custom, javascript! That's that's you! You have option if you want to be if you're particular and you don't want a single line javascript you can do that.

B

But obviously most applications are something like this right. Where you have a mix and match of this this stuff, um some things for the future. I wasn't able to write some slides, but I figured a couple minutes, um but one really cool scenario here is I'll. Give a cloud to lucien.net. So lucien.net is an information. Retrieval engine allows you to do search. You know, guided navigation uh querying. You know you can think think of it as a persistent store that you can do information retrieval uh uh all.

B

On top of the current application that you I demoed right now, it kind of does everything brute force loads, a gigantic csv file on the memory. I read it and I put it in the memory right in. In reality, this application can be made like 10 times faster using listening, and I started playing with it. Lucian.Net once again supports that the standard can run on the client, now imagine being able to do machine learning on top of an information retrieval on top of that on the client.

B

So now you can do very cool, very cool, uh very cool things and obviously that that file that we're talking about the questions around how big is it six point? Sixteen point, eight megabytes. Well, that's gonna shrink because the inverted index is very efficient in compressing data down into.

B

I would imagine 80 percent smaller than that than what it is so.

A

Very, very cool, um so we're running low on time here. So maybe we can get to some questions um absolutely so you were mentioning this pattern that you used to use with silver light of hot loading uh assemblies, and things like that and sky color here asks a question: uh is there a also lazy loading for any packages clls not to load everything at once, or is everything bundled at once?.

B

uh That's a that's a good question. I probably should not be answering that question because I might say something: let's take that offline because I'm not a yeah, not net.net g and I don't want to say anything, I'm on the ml side. So, let's, uh let's get that question answered the.

A

Yeah and the other thing that I will say, uh sky color x and for any folks, I see there's a few blazer questions um while we're showing blazer. uh We are not blazer experts, but there is the asp.net core community stand up which takes place, so uh you can certainly ask those types of questions on there. um We'll try to get to some of them, um but again we're not. None of us on here are blazer experts yep, um let's see so char fudine.

A

I guess this is kind of like a a blazer question as well, but um one thing I still cannot understand: blazer is only for spa or single page applications. Is it for replacing javascript or any fronted frameworks? uh Wasn't downloading. Very heavy data, especially dlls may have security issues, and I think we addressed some of these, but maybe we.

B

A

Clarify some of that.

B

Yeah, I would not, I would not. uh You know, table uh blazer as a spot. Okay, if that's what's being advertised, it is.

A

B

Service another framework, another uh something a tool in your tool belt you can, you can obviously blaze is going to have a habits, have its um place where it makes sense for for you to build applications, it's not going to be everything. Everything is going to be built on blazer from now on, or all spas have to be built on blazer. It is just an alternative. You know we have millions and millions and millions of dotnet dot net um dot net developers who are comfortable working with c sharpener.net.

B

You know who actually saw with machine learning.net being able to do interesting things with it, not just through you know, simple application, but infuse it with ml.net that uh in memory this is the kind of stuff that this these type of scenarios applications allows. There's I wouldn't say I wouldn't pitch the whole um blazer into one one specific thing, but those those security scenarios. I definitely would check out some of the the best practices there. We have. We have a documentation on blazer security.

B

I would definitely you know, check that out to make sure that whatever security or concerns you have is addressed inside um server, side or wasm.

A

Yeah uh yeah, that's I would you put it quite nicely there. um We have another question from brush scale technology private limited here- um and we kind of addressed this, but it would be good to kind of go over that and maybe talk about you know uh other scenarios right. So what about data model size, 100, megs, 100, mags, downloaded in client side.

B

Yeah, so I haven't tested um all the models, but you can see all these are all my ml models. I have brought down here, so it's not like it's not like I'm doing one little model and it's kind of working. It's you know, there's there's a lot here. So um the one thing that you do have to worry about when you do do that kind of stuff, is that obviously you know is is um you know, is how performance is going to be, and the other thing is also um blazer.

B

Wasm right now is currently, uh if you're trying to do a mobile or support like you know, devices that are much smaller. That still run this. You want to be careful on that, because you will uh on some of the mobile browsers. They will actually throw us a stack trace if it's too large, I don't know where, where the, where the limit is, but obviously you know having 100 megabyte models or 200 megabyte bottles, you know you might be pushing the capabilities of where laser wasn't can go. If that makes sense. So I have not.

B

I haven't tried anything, but obviously I have a bunch of models here. I'm not sure what the size of these are but they're. You know there's several of them and they're. You know.

A

B

Even if one is in 100.

A

Megabytes right, perhaps uh the collection of them adds.

B

A

To what's coming down to the client, um but again you're only using what you need uh when you're, when you're performing your uh probability, calculations.

B

uh And such okay and and then in this application, you saw how fast it was responsive. Every single one of these was loaded into memory and every single one of these had a prediction engine associated with it, and it was still performing right. I wasn't I wasn't. I was in context switching between each one, one at a time, but all of these were loaded and memory dynamically and blazer rosin was able to handle it.

A

Very cool, so yep yeah all right, so I think that pretty much brings us to the top of the hour here. Thanks bart for joining us. This was really really interesting. I you know I look forward to the space continue to um you know, evolve and for you, folks, in the chat and the community to to build stuff with this, and let us know you know what works.

A

If you get the training scenario, working uh we'd love to hear about it right so yeah thanks a lot and uh we'll be back in two weeks, so until then uh feel free to send us anything cool that you build, send it to us. So we wanna highlight you and promote you and uh bart before we go what's the best way to get in contact with you.

B

So I have the I guess pro and con of having a very unique name. So if you look me up bart trenitsky and type in microsoft or bartending machine learning, I'm yeah, you should be able to find me um and I have all the links posted. All the code is updated. All the presentation is all there.

B

So if you find me on twitter or on github you'll, be able to find this stuff, I'm trying to make it as easy as possible for everybody to find it, and I am a resource, feel free to uh feel free to contact me uh reach out and I'm glad to help. um You know I work on the partner side, so if any partners are listing, uh big or small, we're glad to help out as well, so anything related to mlai laser.net works. We're excited to help awesome.

A

Thanks a lot again for joining us, and thank you all for tuning in we'll see you in two weeks take care bye. Everybody.