.Net Foundation On .NET, 24 Jul 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Detecting unwanted content using machine-assisted content moderation

Description

For any website or service that accepts content from users, it's important to have that material reviewed to filter out any offensive content. The Content Moderator platform provides us with a set of APIs for monitoring possible offensive, undesirable, and risky content.
In this episode, Sanjeev Jagtap (@SanjeevJagtap) joins us to discus how we can leverage the Content Moderator platform and its collection of APIs to help detect potential unwanted content in our .NET applications.
00:47 - How does CNTK relate to the Content Moderator?
02:36 - What are some use cases for Content Moderator?
05:30 - Overview of the Content Moderator platform?
010:15 - Image moderation demo
12:50 - Text moderation demo
15:22 - Review API overview
17:40 - How to you get started with Content Moderator?
Useful Links:
Content Moderator homepage: https://azure.microsoft.com/en-us/services/cognitive-services/content-moderator/?WT.mc_id=ondotnet-channel9-cephilli
Content Moderator documentation: https://docs.microsoft.com/en-us/azure/cognitive-services/content-moderator/overview?WT.mc_id=docs-twitter-cephilli
Content Moderator SDK and Samples: https://docs.microsoft.com/en-us/azure/cognitive-services/content-moderator/sdk-and-samples?WT.mc_id=ondotnet-channel9-cephilli
Machine-assisted content moderation at scale webinar: https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fevent.on24.com%2FutilApp%2Fdownload%3Fpath%3Dhttp%3A%2F%2Fevent.on24.com%2Fevent%2F15%2F94%2F84%2F6%2Frt%2F1_A881AF79CB182952585CFBF641A90900.mp4&data=04%7C01%7Cv-pacook%40microsoft.com%7C1687b222c4764728231408d579a0108d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636548650741791573%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1&sdata=ZLdYE46bZB%2BgsY48YA1EpzqKy1T%2Ba1DPJ3psXyez3%2Bg%3D&reserved=0
Create a Free Account (Azure): https://aka.ms/azft-net

A

Today, on on.net show we're going to be learning about content moderator,, which uses machine learning to moderate content at scale.

A

Take a look. hey, welcome.

A

To another episode of the on.net show.

A

today we're going to be talking about that content. Moderator., I'm your host, rich lander and here's sanjeev.

A

So,, how about you introduce yourself and then we'll get into the topic. sure, richard., I'm sanjeev jagtap.

B

And I'm a product manager for all content, moderator.

B

a content moderator is one of the cognitive services product, that provides machine-assisted content moderation using.

B

Both api, as well as human review tools for images,, text, and videos.

A

Okay. so,, I often hear this term cognitive toolkit or cntk.

A

We talk about that at microsoft. All the time.

A

How does content moderator relate to the cognitive toolkit?.

B

Yes. so, and I think we'll go to the block diagram a little bit. Later.

A

B

But we will see, there are layers of technology being used, right. so, at the bottom, most layer,, there's scanning of content, really is being done with classifiers, and which are based on toolkit lets, say cntk for example.

A

Okay., so that's where. so, at the bottom layer. You.

B

Have the machine, learning or deep learning based classifiers. on that is the engineering layer,, the web service layer, and in our case,? We have to human in the loop review solution as well on top of that.

A

So, this one in azure, we have some very raw products.

A

I don't mean raw in a negative way,.

B

But storage is more.

A

Fundamental is a better web product.

A

This sounds like it's more in the finished product side of things.

B

Yes. there's an end-to-end scenario,.

A

That you guys have and you've covered a bunch of it.

A

B

That is correct. so,. We do that,, we go broad.

B

We like to go broad and then we go deep.

A

B

What we want to do is make it as easy as possible for developers to use this technology..

B

It's literally three lines of code, that you can send an image, and get all kinds of insights from it, in terms of offensiveness, and content, moderation, perspective., that's right.!

B

We want to make it a finished api,, and then we also want to let the human in the loop tool, because we know, that it's when the ai augments the human in the loop, decision making process, that's when you get the best results,, especially for the more nuanced cases.

B

yeah, humans are actually useful.. I've heard this.

A

That's exactly right. okay. so,, it looks like you got a little bit of content,, so maybe we should go through that.

B

Okay. sure,, we should do that. yeah. so,. The first thing. I guess the first question I get is really who are your customers, right? for sure.? The easy answer is that, pretty much. Anyone in the world who wants to keep bad content out from their properties, websites,.

B

And protect their users and their brand,, they need content. Moderation., so,, yes, online marketplaces are obvious because they need to moderate their product catalogs coming in., I mean, there are tons of content coming in from the suppliers and vendors.

B

social media. Messaging platforms are big users of our technology. enterprises, right, with documents which are employees, or these are enterprises with big products that consumers are using,. Designers are using., think of the assets being created around time.

B

gaming companies,, who are doing the same.

B

Then last but not least, is the content. Moderation service, providers. remember,. These guys have been keeping the internet clean, for a number of years, now. so,. They are ripe to use ai to do that. Work.

B

And reduce the harmful effects of bad content on people.

A

So,, that's the other interesting article.

A

I see. is this. Like "net nanny"? is that,. I might be dating myself there,.

A

What's an example of a content?.

B

Moderation, service, provider?, yeah. These are people,. You heard about stories all the time.. These are businesses who are taking in content from companies like a social media website like a facebook, or twitter,, and so on,, and and have human moderation, team sitting or looking at the stuff trying to filter out.

B

but essentially, anywhere, where you have.

A

User generated content coming in at scale,.

A

exactly.. That's where you need a service like that.

B

Exactly. okay. so,. The problems we are solving for them, is the brand and customer risk,, of course, to their companies., then also when they want to apply the ai and machine learning.

B

But it's a huge investment for these companies.

B

plastic tools, content moderation hasn't gotten the attention of software companies like other verticals have.

B

So,, what we do is-. so we're coming to the rescue?.

A

B

That's what I hear you saying. of course,, because it's a hard problem. first of all,, it's a really hard problem.. The difficulty is increasing, every year,.

B

People find new ways of creating toxic bad content on the internet..

B

You want to keep your family's employees, user safe.

B

So,, it's a problem. That's growing like crazy., thanks to the internet, explosion.

A

Yeah. well,, it seems like it's actually two parts I think. one is the safety part, and the other one is,.

A

There is just a huge amount of negative content,.

A

And even if you weren't offended by it,, you don't want to wade through it, either, or you want a signal to noise ratio to be.

A

yes. to be good.. Yes. yeah.

B

Yeah, what I'd like to say is content moderator not only allows you to keep bad,, actually explicit, bad, content out of your stuff, and create a good experience,, but you wouldn't let other ai insight.

B

so,, think of other cognitive services, who are creating this phenomenal insights for customers like, the nice thing,. If it keeps the bad stuff out,, it makes it easier for the good insights to come. Out.

A

B

Okay. so, here's a quick overview of what this platform is all about..

B

We call it a platform because we have a set of apis and services. so,. Customers can right now sign up for our moderation, apis from azure and photo dna is a service that is specifically about child abuse, material exploitation..

B

We won't talk about that today., but we also offer moderation on images,, text, and videos.

B

For images,, you can moderate for adult racy content or you can detect faces, and you can also detect text within images.

A

B

That theme of adult racy and offensive you'll notice it goes across image,, text, and videos.

A

I can see that.

B

Except that text, of course, has api element where which is your personally identifiable information.

B

right, and profanity filtering. right. so,, I'm guessing the difference.

A

Between adult and racy is like nudity versus swimsuits. is that?.

B

That is correct. okay., that's pretty much it. yeah. now, okay, makes sense.. Then remember,. You can be calling these apis.

B

But if you imagine millions of content units being passed and you're getting those insights and some decisions are easy.

B

right?. If they're, clearly above your threshold of what you can allow,, then block it,. If not, and then you let it go..

B

But what happens in that nuance in that middle range,, where you kind of not sure? the scale at which this content is getting scanned, a lot of cases.?

B

That's why you have stories of human moderators sitting in far flung offices and looking over this content.

B

So,, what we did was we built in a review api layer on that. that is really a bridge between the scanning. That's going on, the machine assistance scanning, and the human in the loop solution. at the top there,. You could have your teams taking final decisions on the content, and that's the-..

B

What we do is, to kind of give you visibility into the decisions,, we'll pass all the data back to your system through an called back end point and so on.

B

But what we focus on is completing the loop, finishing the loop from apis to human, the loop and giving you the final results., because we believe it's when ai augments humans,.

B

That's when you get the best decisions.

A

Right. so,, I see with these different content types..

A

Those are the content types that you typically do see on the internet. that is exactly.

B

Text, image, video,.

A

That's kind of it.

B

Text and images. yes, exactly. exactly. so,. We wanted to make sure, again,.

B

The breadth is important,, because what you do in one customers, as customers, I don't want to go to three different vendors or three different companies and then are having to integrate all of these services.

A

That sounds like a pian. so,. It's a one, stop solution for all media, formats., okay., yeah. so,. Let's dig.

B

Deeper into the apis now. so,, this line.

A

So, now we're getting more to the developer side of the equation.

B

Exactly, yes. so,, we looked at the product overview and now this is what developers will see..

B

Even this is a very high level view of the template on the block. right. so, developers will sign up on azure. all right..

B

They will use an api endpoint to call our servers.

B

They will have an api key that they have signed up from azure..

B

They will input their content into the image, text, and videos, certain settings,, but the output is always a json formatted, text, formatted output, to right with certain labels,, scores, and insights, and that's a general instruction for all our apis.

A

Yeah, I mean, so far,. This is almost as simple as using azure blob, storage.

A

B

What you just described., I like to say, think of this as your little coding function,, where one function, procedural call in the internet.

A

Yeah, exactly., you can see the content and get the insights back.

B

So,, this is image api.

B

Again, you will notice. The only difference is that you are parsing an image, right, to the api, and then evaluating for adult or racy faces., but these inputs are image-specific now, and the output that you're getting is adult or racy,, true or false scores faces and matched with the image list..

B

This is interesting because sometimes a lot of the content is actually,.

B

Sometimes it's less unique content,, but mostly it's recirculated or reshared a lot number of times.. Sometimes companies know that these images will appear,.

B

This content will appear,, but we are okay with that. so,. You don't want to scan again and again. right.

A

So,, that's what we allow companies to do: here.

A

now, with the faces,. Do you just detect the fact that there are these faces of these particular coordinates in the image?.

A

Or can you have a database,?

A

Could I upload a bunch of pictures and I have a database?.

A

Sam just uploading, my photo collection and I want to know.

A

Which particular pictures from my photo collection have pictures of me. right.? Does that service? Do that or is it just saying there are faces.

B

Exactly. so,, it is just saying there are faces.

B

A

What you want do is from a moderation, perspective.

B

Again, think bi moderation. so,, just the knowledge that there are faces in here is only valid.

A

B

But that's a shout out to our face api,, which is the other cognitive service.

A

That we want to call. okay.. Does it do something more similar to what I just said? to get face, recognition, face matching and.

B

All of that good stuff will come from our face. Api.

A

B

So,, that's an example: here, I'll, usually the demo,, it's a small demo., but you can already see the format of the json sample extract. Here.

B

Right. so, can we go play the demo here?.

A

Yeah,, let's do it. okay.

B

So I'll open up my visual studio,, that's already open.

B

This is an image of moderation,, a sample code.

B

If we go through this code, very quickly,, all it is doing is it is picking up certain sample files and you will notice the sample files are right. Here.

B

the sample file, I'm using, is a simple text file with links to these sample images.

B

Right. so,: this code is really taking.

B

For that folder and file- and it's calling moderate images. now, the moderate image, predictably enough- it just really iterates through those five images by calling those others from that notepad file..

B

This in turn is calling this moderating image. Function.

B

And here this is where we're calling the actual rest api, the url.

B

right. using and parsing the image url and index to build the url.

B

But really, and then finally, getting the insights back and that's the response. You get back. so, it's little one line of code that you parse your image and you get the insights back that you want. okay.

A

That's the beauty of it, right. makes sense. so,. Let's quickly run this here.

B

I think I saw some opportunities.

A

To use async 08 in that code.

B

Oh sure., yes, of course. we'll do that afterwards. yeah. yeah. so,, as you can tell, right.

B

It's starts moderating it and you'll see the response. There.

B

so, just to give an example,.

B

If I click on this one image, here,, let's click on this, right?,.

A

Okay., this is the image.. I've got some swimsuits.

B

Yeah, exactly. so,: let's.

B

Take a look at the output of this image: here., let's see whether we got any raciness.

B

Sure enough,, as you can see classified, was true because the score was beyond the threshold of the racy.

B

So,, what you have here are these scores, and then you have the boolean corresponding values. okay.. These booleans values are set to true or false, depending on whether they exceeds the internal threshold.

A

Okay., you guys are on the threshold.

B

We are on the threshold, but then our customers can also, of course, have your own custom threshold.

A

B

They can switch to their logic based on their own scores that they see. so,. The idea is that you sign up for content moderator,.

B

Perhaps scan your data and to get a really good idea of what kind of insights your content is generating because the reality is, every business is going to be slightly different and will have certain behaviors.

A

Make sense. okay. so,: that was the image thing and then we.

B

Have the text moderation.

B

So, here again,, the idea is the same idea, except that you're now checking text for classification.

B

And address the offensive pii, then again, matching against custom terms that you want to match specifically.

B

right. so,. This is the kind of a sample output that you get from the api.

B

Where the match tones, like the clap tone, is built in profanity.

B

so,, what we do is customers, though, when they sign up for the service, they're, also getting a built in database of profanities and bad terms in 100, plus languages.

B

right. so,. That's what this is matching against., but they can also upload their own custom terms and they don't have the profanities that can be very business, specific to., that's what some customers are using that way as well.

A

Okay. makes sense. yes. so,. If I quickly go.

B

Into visual studio, here, yeah, go ahead.

A

You guys must have quite the unit test, suite.

B

Yeah. so,: this is again here: the sample files are.

B

This is the input dot text..

B

I've got three simple text: messages, all right.

B

You can probably guess what we're looking for in those.

B

So,, if I run our program, here.yeah.

B

So, now it is reading the text and saving the response. so,. Let's go back.

A

I'm curious whether is going to do with a spelling, mistake.

B

So,, let's find out. right. so, yeah,.

B

The word lazy was misspell here..

B

Let's take a look at the first output.

B

I notice this is the original text, right.

A

Oh, we got some auto correct, going on there.

B

Then there was auto corrected,, the spelling was corrected to the right spelling. in addition to that, all right.

B

We had, what else was useful here?.

B

That was the auto correction. I guess.

B

yeah, the auto-correction feature coming in here.. There was no pii detected here. right?. There was nothing offensive about this and so on.

B

Yeah., let me go back to the input dot text here.

B

So now,, let's look at the output,, the second output or the second text.

B

So, this was the original text, right?

B

so, it shouldn't have in this case,.

B

There were some pii detected.

B

potential, pii, right? email, addresses, phone number., oh,, there's a phone number, right? category us., then in terms of classification, it was all good..

B

There was nothing offensive about it, but there was a pii that was detected here.

A

Okay. so,, we should probably switch to,.

A

I think you're we're going to show the human tool.

A

Yes, okay., you should probably switch to that.

B

Yeah. okay. so,. Let me hit back.

B

So,, this is a good starting point.

B

For human review, tool, right? so,, imagine moderation is completed. now,. What do you do?? What do you do with the moderated output, right? so,? We allow block or you can review the nuanced cases.

B

Now, for nuanced cases,, normally companies will outsource the human moderation to outside companies,.

A

To out bodies- or they have will have some in-house tools.

B

What you can do is either you can continue to use your own system and your own team, or you can use our already built and review tool that we provide with a service. there's no extra charge for it,.

B

It's included in your subscription. makes sense. so, what this talk about, like everything, else,.

B

All our tools and products have any api that work as a front end, so it's easy to integrate. so,. Let's talk about the review api here..

B

What I'm talking about here is the review operation here., all it does,. It will create a review in the human review, tool.

B

so,. What's the usefulness?, the usefulness is that you could be using your own machine learning api from any other company or your own built in technology or your own logic. For that matter,.

B

It doesn't matter what it is,, but you can still use that human in the loop tool by creating a review. all it takes. The input is really just a metadata.

B

Of tags that you want to assign to your tool under the tool, entry.

A

Right. so, you're, saying you couldn't use a human with this tool or you can just use this as a cue for your own ai system or whatever..

B

Data labeling for example., yeah., yeah. so,, all of those use cases here. now,. We do have the job api, job operation as well.

B

now,, that's where it comes because we'll use it as a one stop solution, because if you call this job operation with your content and you give it to your review team,, this is a new pattern which you'll notice because remember the human moderation teams are working here and you would have sub teams there.

B

so,, that's an extra pattern., but this will moderate your content by calling in the underlying api, such as image or text, and then it will evaluate a workflow..

B

A workflow is something that sits in the review tool that you can define, which are your custom thresholds and criteria and depending on the workflow,, it will create a review or not. all right?.

A

Okay. makes sense. so, I've two questions.

A

One is,: is there a nuget package for this or you just call the rest on point directly?

A

then,? How do you sign up for the services within azure portal?.

B

Yes. so, the sample that I was showing you, the demos today, they're, all using the directory, the rest api.

A

Okay. so, .net., but we also have.

B

A .net sdk as well. all right. so, in fact,. I have the whole,. If I open up my- so,.

B

These are the samples, .net samples and where you can go to find these are right. Here.

B

I thought I had the page open.

B

so,. Let me go back.

A

B

So,, if you click the "sdk" option, you'll, find there's a .net sdk right here that you can use.

B

so, yes,. You can download and you get to your visual studio and use the .net sdk or you could call the rest api.. It's totally up to you.

A

Okay. yeah,, the rest api are fairly simple.

B

A

But obviously,, if use it in your git packages, probably the best experience I take it.

B

That is right. of course. yes. in terms of starting, well,. How do you sign up for this? so,? The easiest way is to click the "get started" button,.

B

It takes you to our online review tool where you can sign up, right,, using msa or using a local user login.

B

Then, you can essentially "stay".. Let me do "stay".

B

You will be taken into the tool where you can define a team or you can just get started and you can try image and text moderation right, here.,.

A

Really? so,, you could just upload an image.

B

You don't have to write any code to experience the moderation features and experience the workflow right here.. We even provide you sample images which are used here right, now.

A

Okay. right., similarly for text as well.

B

So,, you can use text, and so let me use the default text and I submit it and you will.

B

Notice, it will now review the text and there is api highlighted.

B

Nice. so,: that's how it works. here too, you could actually do the same, which is maybe you have an image sitting on your hard drive somewhere and you can try that. here. Let me try image, right, and you can choose your file or something.

B

all right. anyway, okay.

A

Not a powerpoint.

B

But yeah, you can put an image and take a look at the inside here. in terms of starting out,. You can either go from here from the review tool or, if you just want to start with the api directly, right,, you don't have to deal with the human.

B

In the loop process, here.

B

then,, the easiest way is to press the "quick start" and the quick start page is really good in terms of giving you three options, start with a review tool or subscribe on azure, in the azure portal.

B

okay. right, so,. There is explanation or you can also start actually.

B

what we're finding is,. We have flow connectors for content, moderator., so,, a lot of mvps and a lot of consultants and a lot of.

B

Even our own microsoft field, personnel.

B

Are finding flow to be.

A

A really good way to get started and show.

B

Something to the customers and we have templates available here.

A

Okay. so,, it's kind of a more visual way to do. It?.

B

Yeah, exactly. so,, you have all kinds of options.

B

sign up for the review: tool, start with the apis and then move onto the review tool or just start flow..

A

Okay, nice., okay, well, thanks for stopping by today.. This is great.

A

I can totally see where this fits in and probably a bunch of .net developers are the ones who use this on their app.

B

We should get, now it's.

A

Definitely the time to get feedback.

A

sounds good.. Thank you. Richard.

B

It was a pleasure coming to here. okay. thank you.

A

Well, thanks everyone for watching another episode of on.net and today was about content moderator with sanjeev. thanks for watching.