.Net Foundation ML.NET, 23 Sep 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Build a ML model for GitHub Issue classification | ML.NET [5 of 8]

Description

In the world of ML.NET there is a high-volume of interaction with GitHub. This video will teach you how to classify incoming GitHub issues into one of the many tags (labels) using a multi-class classification algorithm.

Follow Pranav Rastogi: https://twitter.com/rustd

Watch the entire series here: https://aka.ms/dotnet3-mlnet
Download Visual Studio 2019 here: https://aka.ms/VisualStudio-dotnetconf19
Download .NET Core 3.0 here: https://aka.ms/Dotnet3

Learn more about ML .NET:
Multiclass classification with ML.NET: https://docs.microsoft.com/en-us/dotnet/machine-learning/tutorials/github-issue-classification?&WT.mc_id=Educationalmlnet-c9-niner
What is Model Builder and how does it work? https://docs.microsoft.com/en-us/dotnet/machine-learning/automate-training-with-model-builder?&WT.mc_id=Educationalmlnet-c9-niner
ML.NET Model Builder: https://dotnet.microsoft.com/apps/machinelearning-ai/ml-dotnet/model-builder?&WT.mc_id=Educationalmlnet-c9-niner
Get started with ML.NET: https://dotnet.microsoft.com/learn/ml-dotnet/get-started-tutorial/intro?&WT.mc_id=Educationalmlnet-c9-niner
GitHub Repo ML .NET: https://github.com/dotnet/machinelearning
GitHub Repo ML.NET Samples: https://github.com/dotnet/machinelearning-samples/blob/master/README.md

Follow us:
 Twitter: https://twitter.com/dotnet
Facebook: https://www.facebook.com/Dotnet/
Blogs: https://devblogs.microsoft.com/dotnet/?WT.mc_id=Educationalmlnet-c9-niner
Shows and Podcasts: https://channel9.msdn.com/Shows/On-NET/?&WT.mc_id=Educationalmlnet-c9-niner

Useful Links:
Learn .NET: https://dotnet.microsoft.com/learn?&WT.mc_id=Educationalmlnet-c9-niner
.NET Community: https://dotnet.microsoft.com/platform/community?&WT.mc_id=Educationalmlnet-c9-niner
.NET Documentation: https://docs.microsoft.com/en-us/dotnet/?&WT.mc_id=Educationalmlnet-c9-niner

A

In this video, we will look at how to build.

A

A custom machine learning model for github issue, classification.

A

Let's look at an example of what is github issue classification.

A

So imagine that you are an owner of a github repo where you have your source code over there and users are entering issues.

A

A common problem is what tags to.

A

Apply on those issues that users are opening.

A

So in this case, we'll build a custom machine learning model for classifying issues into different tags based on what was the title of the issue and what was the description.

A

These two inputs are known as features which are going to be used to train the machine. Learning model., the output or the column to be predicted is going to be.

A

The label or the tag that you want to apply for this particular issue..

A

This is a common problem that lots of developers have.

A

Where they have lots of repos and it's hard to tag them.

A

So we'll build a custom machine learning model for this scenario.

A

Github issue classification is an example of.

A

A multi-class classification machine learning task.

A

Where you can classify an issue into one of the many categories.

A

In this case,, you want to classify this issue into a,, b, or c,.

A

Which means that you want to classify this bug, whether it belongs to system web,, asp, net, or security, or infrastructure, performance.

A

So you can choose one of the many labels that you want to apply. This particular issue to.

A

So, let's look at how we can build.

A

A custom machine learning model for github issue, classification.

A

I'm going to switch over to my application, which is a console app.

A

I'm going to build a custom machine learning, model.

A

I have already downloaded modelbuilder, which is a ui tool to.

A

Build machine learning models and I'm going to launch it by right-clicking at "machine learning".

A

I can choose from any of the scenarios.

A

so in this case,, since I'm doing issue classification, where I.

A

Classify data into three plus categories.

A

So I'm going to choose this template..

A

The second step in the machine learning process is.

A

Around loading, your data and choosing what column you want to predict..

A

So in this case, I can choose file or sql server, and I can upload.

A

A file which is my training, dataset.

A

So what I've done is I've exported all my issues from github where I've taken their area,.

A

Title, description, and I'm going to use that to train my machine learning, model.

A

So you can see some data we're given a title,.

A

Given a description,, it belongs to area system, net, area infrastructure..

A

So this is the column that are used to predict.

A

So I'll choose area as the column to predict or the output column,.

A

Or the label column,, which is going to be result of calling this machine learning model on..

A

Now that I've loaded the data and chosen my column to predict,.

A

The next step in this process is to train the model.

A

The longer I train the model for,, the better model, I'll get., so in this case, I'm going to choose 100 seconds and start training..

A

What happens here is modelbuilder.

A

Uses automated machine learning underneath the covers,.

A

And automated machine learning explores different models with various settings.

A

And it evaluates the performance of each of these models. To give me the best model for my scenario.

A

So, as a developer,, I don't have to worry about learning the intrinsics.

A

Of machine learning and how to finetune model performance.

A

So, let's see what kind of performance would be good for this scenario.

A

So far, automated machine learning has found.

A

A best model average perception ova.

A

With the accuracy of 71 percent.,.

A

When we see the state of finalizing model,, that means that model training has finished and now,.

A

The model is being generated.

A

Once it finishes,, the training process has.

A

Completed and I can go ahead and evaluate the model.

A

In this case, I get a summary of what was the machine learning task,, what was my column to predict, and the best model and the best accuracy, and other models explored..

A

So in this case,, our training model was only able to find one model within a specified timeframe,.

A

So it's only one model., but if I were to increase the timeframe,, then it'll find other models as well..

A

The last step in the machine learning workflow is around consuming the model..

A

So here I can click on the "add projects" button to add projects for consuming the model.

A

Also, I can learn how the model was trained., so in this case, modelbuilder.cs has.

A

The code for how the model was actually trained.

A

I'm going to add a reference to the class library to consume this model. In my console, app.

A

So let's go ahead and add a reference to the class library.

A

This allows me to use the classes from that project and I'm.

A

Going to use this code to consume them all in my application.

A

I'm going to add it to programcs.

A

Since I added the references to the class library,.

A

I'm going to add this using statement.

A

To consume the classes from the library.

A

Then,, I'm going to add my input data, which is my.

A

Input.Title, to figure out what title,.

A

What category does this belong to., so this is asp.net..

A

I can say input.description.

A

So this is my description and then I'm going to.

A

Output, the results of calling the model on my sample dataset.

A

So I'm going to say result.prediction.

A

Then,, I'm going to run this application.

A

What happens when you run this application?

A

Is the ml context gets loaded., your model gets loaded., the prediction engine gets created, and then you can call the predict function on your sample. Data.

A

So in this case, for my sample input of this asp.net,.

A

The machine learning model predicted the output label to be area-system.data.

A

So I can quickly sort of users to play around with other input types like.

A

Title and description to figure out what category they belong to you.. In this video, we looked at how to build.

A

A custom machine learning model for github issue, classification.

A

In the next video, we will look at how to predict taxi fares.