Internet Engineering Task Force 98, 27 Mar 2017

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: IETF98-NMRG-20170327-1710

Description

NMRG meeting session at IETF98
2017/03/27 1710

A

A

A

B

B

Hello welcome everyone I'm pleased to welcome you for this. Forty second and imagine meeting the of it for today meeting is next steps in autonomic networking it's a one-hour session, so it will be a big shot that we have today to presentation first, not well, which are all aware, but please do not.

B

This well have quick overview of the agenda for today. I'll take a few minutes just to give a status on the research group activity and then I will introduce the first speech of today self-driving networks from 280 Kampala networks, and then we will have a second presentation by a person nombre on the autonomic that working response rate respected and the remaining time will be allocated to an open discussion, try to identify the gaps, challenges and define a roadmap for next step in atomic networking.

B

So a quick research group status, we have a couple of active IDs that I've gone through a researcher class called beginning of this year, and this has passed and is currently ready for IRS you video channel to IRS year after this. Maybe then we have also a second draft on IP fix an emerging occasion. This draft was in our ESG complete review and ask come to that outcome. Is that it needs to go through the IETF review and I NC approval, so we have a bit of work. It push.

B

Is draft forward important information, the next meetings for the energy, the next one will be organized together with the I Triple E IM conference. You could be in Lisbon second pecans may. We will have to say shin there. The first one will be on Tuesday 8 of 9 of 9, which is the mainland introduction to the rumors trying to recruit new participant to dinner. An emoji and I will have a second session, which will be a technical session for energy, and the one option is to be a part of the discussion of today.

B

We plan to have also a meeting in Prague dedicated to a workshop on measurement based network management, which is a series of workshop organized together with energy and the possibly based on the feedback, also an additional session on another topic.

B

We also plan to have an emoji meeting in ihe f100 in Singapore and not this year, so the pics remain to be defined, but there is also some people suggesting to have a full day workshop in addition to the IHF meeting on a futon driven networking. So we will try to post more information on this purpose workshop as it as it progresses final slide for the research group status.

B

We will continue to develop and focus the research group on a set of key topics. This is a ongoing work and we tend to share the research arena. We are bringing together with Alessandro as soon as possible, but we need your input and feedback. We have posted some elements and got last year on the on the mailing list. So we appreciate your feedback on that and, of course, we will try to build upon this the outcome of today's meeting.

B

So that's the end for that. I would like to ask you to come and this presentation on self-driving networks.

C

Thanks and thanks for inviting me here, we.

D

C

Okay, so this is kind of a a vision, an idea that don't ask me how to actually do this. Quite yet. I have some ideas around this, but I do want I mean I've, been talking about this for almost 18 months now and I want to get beyond talking about it to thinking about how we might realize this so next slide, so um I made a presentation at the mpls World Congress a year ago. So this is a first public presentation about self-driving networks and I.

C

The way I think about this is I draw an analogy with the sub driving car and there you know there was a vision which was DARPA Grand Challenge, which happened in 2004 to where we are today where everyone is trying to get into the act, and so the question is what what kind of trigger would it take for us to start building this and beyond that.

C

You know: when can we actually see this in action so next slide. So this is the self-driving car journey. You might have a kick to it in 2004 DARPA issued a grand challenge. Saying: can you drive build a sub driving car? They basically wanted the car to drive itself with no human intervention, 240 kilometers on a deserted stretch of sorry on a stretch of freeway in the Southern California desert, but they they did make sure that there is no other one, no other cars there.

C

So only the robotic cars and it's a very simple, very well contained problem, but the first time around in 2004. Nobody could do it, they reget the child in 2005, and five teams actually succeeded, but and then they went on from that to the Google car that you see there are way more I. Guess is a better term today. So in this, in that period of ten years you have a fully autonomous car. So this car doesn't have a pedal, doesn't have a steering wheel. It really has to drive itself now.

C

Then, of course, along the way they took Alexis and put a bunch of stuff on it and show you how the Lexus could work, but they do say that hey, you need a driver and in case the robotic car the driver robotic driver could not do it. The human has to take over with this car the human. The only thing that human can do is open the window and jump from what I understand, I'm not actually driven in this next.

C

So the thing is: what would the sub driving network do, and so the first step is, you accept guidance from a human operator. You are now that you have this autonomic network or a sub driving network. He can be anywhere or she and you know, in a hammock at the beach drinking a mai tai, that's perfectly fine, but the rest.

C

Everything else will be done by the machine by the robot and we talk a lot about zero touch, provisioning and you said at the beginning, autonomous networks, and we have another group called anima, but I want to go beyond all that. Yes, you do have to. You know figure out what the pieces are put them together and self configure, but you also have to detect that something change you have to say: hey a customer came on board. You have to provision that customer. You have to monitor that.

C

If the customers service is not up to snuff, you've got to do something if a router crashes, if a link goes down, you don't figure out what to do so. Every think that a human operator does should be done by the same. So that's that's the idea of a self-driving network. So it's not any one. Piece of this is not any one small part of the network. It's the entire network being completely manually, run completely automatically run next.

C

So to do that again, drawing an analogy with cars. I think that these five basic technologies that are needed, the first is telemetry. The second is multi-dimensional views, so having independent telemetry is fine, but you really want to put them together and start you're getting more insight from your network. The third is automation and that's fairly clear. The fourth is decorative intent, and this is now a very young common buzzword, much better than the SDN buzzword. By the way, I should point out that self-driving network is another expansion of SDN.

C

So if you don't like the first one, you can use this. One who is carefully chosen yes, but decorative intent is, is really important and I think we put our fingers too deeply into networks and we should be able to step away and let the network do its thing and the last year's decision-making and decision-making can be ruled by scent or can be machine learning base. But ultimately, I have all this information great. If, if that information is just sitting there and doing nothing, it's not helpful.

C

We need to get to the point where we can make a decision based on this, and ideally the machine will do that, will make the decision on its own and can use a rule-based system or can use machine learning or can use a combination and the Google self-driving car uses a combination.

C

So next so I'm going to start with telemetry on cars and there's a usual thing is the spirometer? The gas case as the tire pressure sensors there's a tachometer which I didn't put up here. These are all helpful to you. Don't give you some idea about how you should be driving. Maybe you should change gears. You can hear the engine is no working too hard or you can look at the tachometer or you look at the speedometer and say maybe I should speed up anyway.

C

I should slow down more recently, people put a radar so that you can have this smart cruise control. So not only will it maintain the speed, but if the car in front of you slowing down l slow down today, I'll just trying to well not.

E

C

A few years ago, we're just trying to the power in front of you so now for parking, mrs. cameras for all kinds of things, like you know, finding out who hit you. These are all to help a human drive. But when Google said I want the car to drive itself, they asked what what kind of telemetry do do, I not have, which would be really useful, and so they put this thing called a lighter. The lighter is eighty thousand dollars that they put this on a lexus, which is seventy thousand dollars.

C

So you had you had the slider, you had dual cameras for stereo imaging. You had a very powerful pc, all this other equipment they put on the car for sharp driving with 150 thousand dollars, but the lidar was the biggest most expensive part, and for those who are curious, you can google, you can search for google or way mo uber and lidar and interesting stuff will pop up. Oh, the reason I put this up here is when we talk about telemetry, we're talking about telemetry.

C

That will help a human operator run the network more efficiently, but I think we also need to talk about what is it we're not thinking about if we really want the network to run itself the same way that Google sort of came up with the lighter, so here's I think where we are today in part where we could be essentially n stuff Isaiah thanks, you have out here or I'm, not in the box that ok I'll get back in the box quickly, you're having here a collector, and you have here a network element and basically, rather than doing SNMP queries all the time you should.

C

You should be able to tell the box. This is a kind of information. I want and will come streaming to you in real time. So you you don't want to be polling for this, and so you want the telemetry to be really efficient, very efficiently encoded and you want it to be coming at you all the time. So you have a sense of what's going on and I think this is a very critical part. Today we get information in parts in places that we we sing they're useful. We dump them into our database.

C

We then take the disc and bury it somewhere deep in the ground, so that if someone asks for it is there, but we don't use it again and I am being facetious virtual. We need to do much more. The second thing we need is this multi-dimensional views, which is the next slide, which is not just you know, get this piece of data here and get some other piece of Gator there and get some surface of data somewhere else, but be able to correlate these. So you can correlate across layers of the network.

C

You can correlate across geographies. You can correlate what's happening. It appeared with what's happening inside your network. You can do time based analysis trending, so you know, what's normal, you know. What's the baseline and you know what's maybe not Arnold, and so that gives you a hint. Maybe I should look more into why the traffic change or why this user is talking to the other user way. He's never talk to that person before.

C

So it's not enough to get the data it's important to pour that data into a data make which is an expression that people use where you can correlate these things, and you can get much deeper insight from from the data that you have. The third is automation, and so this is an example. I have to slightly sanitize. This, hopefully have done a good job, but this is an example of a device network element at the bottom. You have the hardware you have the chassis and the line cards about that.

C

You have an OS and above that you have a number of things and networking is bringing to our os's and our software things that are typically seen in servers for server automation. So you have ansible puppet chef, salt you've got python script and Ruby scripts and then, of course, we have we've been moving in the direction of net pump or restaurant. So you can, you can operate the box remotely. You can operate the box from a more centralized thing. So when everyone says CLI is such a horrible thing?

C

Well, you don't have to use your life or your CL I. Think this be and in town not half call from within the box, but all of this allows you to to start automating and understanding how, to you know, upgrade a whole bunch of devices all together or make the same type of change on a number of devices instead of having to log into each one. So automation is really important again the analogy with cars.

C

If you look at the google subtyping car the way more car, it's an electric car, so it doesn't have to worry about years and stuff, there's a number of things that become much simpler, but even if you didn't have that, if you had to have a physical petal moving thing so sort of a foot that was changing gears for you, it'd be much more complicated, then if you could just tap into an API and make the car stop or change gears or whatever, so automation is really important.

C

The next is this decorative statement of intent, so in the car, if you run have a pedal, if you don't have a steering wheel, what are you going to do? Well, you have a little dashboard and you can say I want to go to the airport: hey, I'm in a hurry, take the fastest time or I'm a cheapskate so take the path. Doesn't that doesn't have any tolls or, like me, I'm going to leave I'm always like great I've got range anxiety, so find a way that will have the least use of battery.

C

But whatever I mean you sort of say this is what I want and you might give some hints about how to get there. Rather than giving all the details of you, you should take 101 in the area, so I would take 101 and I'd be in the left lane, except for this place, where I know I need to go to the right lane. I should exit here, I should get on there. All those details.

C

You don't have to worry about anymore, but of course you can go even better and say: hey car just talked to my phone they'll. Tell you what my appointment next appointment is and just take me there so I mean the this whole thing about intent.

C

You can get higher and higher level in hand, so I've talked at I think with the SD NRG a few years ago about I, don't think with an MIP about sdn as a compiler, where you essentially give a high-level decorative statement of what it is you want to do and then it will compile it. It process it and say: okay device, one device to device three, here's what you need to do today.

C

We you just get on the on the device and start typing at the CLI, so we're focused on how not why or what. But we really should get to the watch and say this is what I want, how you do it I? Don't care that your job and the last technology is this decision making. So you have this idea of either rule based learning or machine learning, rule based learning you put down some simple rules. If this, then that and and then you- you say- oh that didn't quite cover it well put. Another condition.

C

Is this and that and as you keep putting these conditions on, each one is very, very simple, but very very quickly. It becomes very complex for handle in principle, is also very deterministic, so you could read this the set of if this, and that and say I know what this is doing or what we'll do, but it is very, very cumbersome in machine learning, you say I'm going to walk you through something I'm, going to show you how I, how I would do it and I'll give you a million such examples.

C

Then you can learn, and so your account or Alan Turing. This is the essence of artificial intelligence. It can become very creative and machine learning has become really really powerful. Part of that is, we have much more data today and part of it is. We have much more much more powerful computers. We have a few better algorithms with the algorithms that are actually the least of it.

C

So it is a very frustrated. Learn complex behavior. The problem is we don't know how it got there and we don't know what I do so. Typically, you pair the machine running with rules, so you say, drive stay within the white lines and do all that, but you'll put a rule thing. Do not kill a human if you can avoid it and, of course, there's a whole bunch of questions about if you do have a kill, a human which one would it be, the passenger, the little girl here or the old man there.

C

It's like you know, and so now there they actually have courses in ethics of machine learning, no jobs. This is yours, okay, so these are the technologies I also see sort of like they have five levels of autonomy for self-driving cars.

C

You can kind of put up five levels of self-driving for networks. You have completely manual stuff, which is I, think where we are mostly. We do do some automation. If you're gathering all this data, you can start crunching on it and visualizing it at least have a sense of what's going on in your network where the hotspots, how does the hotspots move around during the day during the week? um If you go further, you can analyze this and start getting some deeper insights.

C

You can make maybe some predictions about where you would have to add capacity in three months or ways where things are happening. The next is recommendations, so, if you think of this as a closed-loop system, I learn stuff from from the network, I operate on it and then I make a decision. Rather than make the decision yourself up for the machine, you could tell a human. This is what I would do and then the human could say you know based on their experience yeah.

C

This is a good thing to do or not you, you don't have enough data I should I should take over, but at the end we want to get to it's totally self driving and- and so this is what they call level 5 autonomy for cards anyway. So how do we get? This kicked off? Well, if we take the model of the Grand, Challenge I would propose a networking and challenge, and thank you let's do this fine. So this is a very, very high level architecture of how this might work, and this is the closed loop.

C

That I was talking about. So you get data from the network, you analyze it you make it to sit or you go through a decision module and you make some take some action back in the network. If you do this, then you're constantly optimizing so based on what your intent or what your high level decorative statement is. I'm, going to constantly go through this loop or a new customer came on board new subscriber came on board. What do I need to do? I? Did it monitor yeah?

C

That person is happy, nope they're not happy make some changes and you keep going um I do want to stop there for a second, because I think this is where there's a lot of work for us to do. If you're gathering a lot of information, this telemetry has to be really efficient.

C

We need standardized data models and, to a large extent we try to do telemetry for human consumption. You got to stop doing that. We okay, so we really need to move on to you know putting this data in a form at the machines can understand and can do the analysis most efficiently and then they can present it. In a format that's useful to humans, also for the actions we need to say: how do we interact with the box? What kind of things do we do?

C

Okay, next, so here keep checking through. So the grand challenge will be here is what we want to do: self-driving network, with what our definition, I don't have a prize. I will figure that out and the result would be to free up people to work at a higher level to do service level definitions to do new service design.

C

22, maybe learn how to treat the AI to work better. The result would also be a much more agile infrastructure which can anticipate things because it has this prediction engine, and maybe the most important, is faster, intelligent response to security breaches, and so here's an example, but pick your favorite little thing and say: can we go build up the things that I want you to think about? Not just the technology? But what is the impact? And what is what are the possibilities? What can we do differently? I mean completely differently from what we do today.

C

If we had a self-driving network and the next few slides have some examples of that, but this is really where you need to think about things. So, for example, there's a huge skill set change. I'm, an ospf, geek and I know how to set metrics, not important, but I need to be able to create a new service or tweak the air, and so on. You can read them.

C

The next is a sort of crazy idea that for the next, so I will tell you that the olympics for 2020 the planning side in 2015 and this kind of five-year cycle planning for a big event is probably not the right way to do it. It would be great if you could wake up and say: oh the Olympics are tomorrow order a few things. They will come on a self-driving truck. Will you know they'll set themselves down self-organized do what they need to do and then go away. So that's!

C

Basically it I think we have a compelling vision, I think so. There's an economic imperative to make operates really really much lower. There is an efficiency imperative. There is an agility imperative. The security imperative, so we need to get the word custom yeah.

B

Thank you, karoke I think we have a few typos question. It's important.

C

F

Hi Jahi, yes, at the end, so I'm not really familiar with that some kind of meal to that and try to understand what is the difference between your proposal relation to other technologies like automatic networks? What is really new in here I just try to understand here. So what.

C

I'm thinking of and I I have to apologize that I haven't followed economic networks, I mean I randomly get into the animal working group or this working group. You know once in a while or research group. What I'm looking at is a completely completely self-driven network, which means that everything that human operators do in the NOC. You know whether in the central office in the pop you know in the core.

C

All of this should just happen automatically and at some level human operators are still needed, but they're going to define things at a much higher level. They're going to say here are the application that mattered to me here. The customer subscribers that matter to me here the peers that matter to me, here's my resources. How would you organize yourself to maximize happiness or whatever, given this finite resources?

C

G

Yeah, this is a small portion and from the model you have sown, how much have you already implemented or experimented with this? Do you have any gets better or something like real thing.

C

So what I started playing with is looking at bandwidth and trying to come up with a time series model for them. The problem is.

E

I, don't have enough bandwidth.

C

Right I need data for six months or a year. Here's two things I want to do with the bandwidth. One is to make. If you do a time series analysis they'll tell you: these are patterns. Okay, there's some! You know, errors from the pattern and here's the long-term trend. So from the long-term plan. I can say for this. You know for LA to New York traffic I will have to upgrade some some ports in six months or three months or whatever. So I can do some predictive analysis.

C

The second thing you can do is to say this is normal traffic on monday at at 10am, and if I know that the traffic are you on this port at on monday, at ten a.m. is between 3.5 and 4.5, gig, plus or minus.

C

Okay, sub driving networks would not would not do that with.

B

C

The the the rules would prevent them. So if I know the the traffic at a certain time would be between this range and the traffic is significantly higher or lower. It would tell me you need to go look at that, and this is data that's available today. But people don't do that kind of analysis. So that's just a front part. I do want to do other things like do syslog analysis, using machine learning and say what is the root cause so, instead of doing root cause bye-bye rule-based systems which is typically how they were done?

C

I think some people are trying to do it differently. I want to use a syslog. Sorry I want to use machine learning to analyze this log data and say this is what happened. Yeah.

G

Rated to that, you have mentioned time series analysis on machine learning. Have you considered all the kinds so, for example, expert systems that can be even smarter than any machine learning algorithm? Neither actually are, from my point of view, are actually really silly come comparison with any expert system that fits they sped from a human during some your own period of time and after that you leave the system running on itself. So I'm.

C

Not at all against any technique, so really I don't want to get into. You know how to build this I. The first step is to say, let's all seriously, build, not just a an autonomic system, Europe, let's take all of our networks and make them sub driving. Now how you get there, you know I think we will find as we do it. We need different technologies.

C

Expert systems have a problem that you have to have humans. You'll, give you a lot of guys pitch them, but but but so is machine learning, because machine line runs typically on label data and you need humans to label the data. So there is no ideal way of doing it, but I think we really say: I mean when DARPA made the challenge, they didn't say, use this technology. They said just the car should drive by itself, Evan even.

G

We humans do not learn my ourselves. Somebody teaches yes, yes, yes, I. I will.

D

Stop it? Ok! Ok, ok! Definitely.

C

Work to discuss yeah.

H

Then rotate in T interesting, the vision, um I think the analogy is pretty pretty appropriate, though, and I'm thinking that technically.

H

Just like with the autonomous car, there are probably networks that are pretty self-sufficient, it all comes down to legacy scale and, and these are the kinds of challenges, the reason you don't see. Cars on the road today or they're being pushed out from the road like yesterday was just bored the day before uber cancel, though their experiments, because they got a crash. Why? Because, usually the humans are the ones that making the the accidents so extending your analogy, usually is.

H

The problem is with your legacy system that may may have interference with your learning and self self-driving network um legacy. It's a big issue because you got investments in there and and and and I think what you're just putting on this vision together, bits and pieces. Are there? Maybe we're not using them efficiently? Maybe we're not using them coherently, but they are there. So what? What is the deterrent? The deterrent is what I just said: scale and and and and hybrid legacies.

C

Yeah yeah yeah. Well, yes, and no today, there's a lot of human operation and it's not only because legacy.

C

The question I think is just as for the car, when I, when I went from a clutch based system to automatic transmission, I made it easier for a human to drive, but the car still doesn't drive itself it's when you start thinking about what does it take for the car to drive itself? Yes, I have these bits and pieces, but I really need to have that thing. I do not want to have a human involved. Sure I think you started looking at things very different. We.

H

Also have a vision: we we, we made it public, it's called project, indigo, yeah,.

E

H

Pretty much along the lines of what you just presented, cool, that's good,.

I

Well, I got a couple of reflections. One is when it comes to making whatever the technology you're using I do when you want to make this learn one of the problems that they say we have set it to face. Think some experiments who have made is about data or the availability.

I

No, it's not only about the availability of data that is complicated to it and it's very complicated to share and to convince the people at on the data to share, but is about the fact that when you have a lot of raw data without tagging without wobbling, this basically is having nothing. I mean what more than nothing but.

C

I

Then, but what we have decided to start in inside the our lab is something well, even if you have read about the immortality, is it something that we call the mouth work the abs that you have something which you can play with things that well, since you don't care, because I like like mice, do whatever you want to do with the idea of, on the one hand, generating on data sets under control conditions and second generating noise, but know is that we know that this is a.

I

Is it now noise that we can combine we're making an experiment of this? We are trying to get to port on this for you and everyone here. If we we can talk about trying to make this evolved, I mean, is, that is a let's say, a smart learning environments. So to say right.

I

This is one thing said: second, is a is because I when I was younger than now, not that not so long ago, I made my PhD thesis on fuzzy logic and well lo and on because I know I found that there is an ideal combination of one fancies, the possibility of machine learning and the possibility of having human language having it is intelligible.

I

What you find well again, I, don't know if you have consider it because as a something that they ceased really powerful is not so in fashion right now, but I find it extremely powerful because it's this combination of the ability of to learn and the ability to express what has been learned in something that is close to human language. Okay,.

C

um Thanks for comments, I will say two things: one is I went to my first conference in machine learning and yes, it was machine. It's nips, neural information, processing, symptoms, yeah in Barcelona not so long ago and I do want to go two more, but the problem there is they want to polish a diamond.

C

Even you know finer and I, don't want an algorithm, that's like ten percent faster and twenty percent, more accurate or no whatever, but there are some other things that are doing that are really interesting, which is multimodal learning using different techniques and using different types of data. So, if I'm trying to understand what you're saying but I only listen to the voice, I'll have a certain accuracy, but if I'm also missing or seeing your face.

I

C

Do better and how can I apply those techniques in in the network? So I get data from the routers I get some data from about traffic. How can I put that together and make a better influence about the network so and I'm not wedded to machine learning? It's just that machine learning is in the news right now and she very very powerful you're right that it's really really hard to get lots and lots of label data.

C

There are techniques for semi-supervised learning, and so you have some label data, and then you try to extrapolate so, but that is the biggest problem, but I think the second problem is for us networking guys we collect some amount of data. Sometimes we collect lots of data, but we don't use it as a patient as we could so I think we even start looking at that and start understanding. Our networks better would be better, but if you are willing to share your data, yes, we should talk. This sharing, sharing the environment for.

I

Generating ball.

D

Okay: okay, okay, yes, small.

C

J

Yes, hey Lou, un-fun ebay, so um since ya go and you I'll mention the background I wasn't intend to, but there was a common health expert system, knowledge based system compared with the machine learning the more deep learning stuff. So my master was expert system. My PhD was in your networks, so the neural networks with the deep learning it's much more capable. So you don't I'll, rely on from getting all the knowledge from the expert pulling the knowledge from experts difficult. So that was long long ago.

J

Reason years is all networking and now what we're doing is I think I'm. Very that's what we're doing actually, including the bandwidth your mention of things. We actually have the time theory data as Diego mention the data is now completely clean, is not complete right.

J

Some portion, maybe not there- some equipment thing I, cannot support this and the data is, you know, SNMP, it's very heavy and everybody has a lot of data sitting you know under, but it's hard to process is hard to do it's because when I first ruling saying okay, let me look the whole day. Does that? Oh, that's! So much! Let me look one more cluster. Oh that's too much. Let me look one device. Let me look 24 hours, it's way beyond the mailing point.

J

So the data points there's a lot of data point, because you continuous collect that and also the SMP is cumbersome. It's a wave, pooling pool model right. So all these things, the telemetry, is why important thing so the point I resonator I want to make today what proven as the in the past, who go far to skill with the AI machine learning. It was machine speed, not there I'm big data, not there. Today we have big data, but we're not mature with the technology and the Machine speed is not a problem.

J

The algorithm today, it's as similar as 20 some years ago, so that algorithm is not a problem. The first thing to attacker. We also out: let's get the data on the AI machine learning. It's not that you, you need to go lower layer right first, you say how do I process these data? Are these the right way to collect it? This data, the telemetry, the the memory processing? All these things are are the important things, so you go down to the level. How do we? You know ultimate many things before we can really drive it.

J

I mean I, would say a self-driving network in the sense more difficult than self-driving car, because but there's no, that same safety but self-driving heart. What you need to do you have a sensor. You've got all the pictures in front of you and around you a pack of you right. You have a navigation with how you were to go. The network is distributed that one is limited space to worry about to process right, and we know the networking is not like.

J

When we do speech recognition, you got labeled a date all over you can buy them. You can whatever right image. All these are available now working, no label the data, so you mentioned supervised and supervise the learning. Our various things need to know combined together, but we can use certain, um maybe not exactly math, right to figure out things. Basically, what we're trying to do is to learn the pattern, because everything at the end it be long before we do a- is a pattern. Recognition right, you do. All these things is Madame.

J

We want to establish the baseline pattern. Then you see, this is a tag pattern. This is false.

B

Through the clip I'm that are.

J

Very interesting.

B

About to be up still, some presentation to do: yeah, I'm, not just a conclusion about your comment. My.

J

Comment, this is the right direction and that we have a lot of layer work to do before we worry about which algorithm choose.

C

J

Collect the data get the right data it infrastructure. Thank.

B

You thank you again. Karakura T thanks a lot, so we got nice decision. Yes, please.

K

Don't come in that one remark: if you want together I, listen to what the same here, we have to change our mind regarding a non networks operation, because what I try to say that the network know everything. So if you're looking about Carl, for instance, the switches, if lecture today the singles, we are debugging network, for instance, just if something is not working, we are singing pink for the management station is like if the car is not driving correctly.

K

Somebody sent from the central office off I, don't know for Google a packet to the car and see if the quiet car is not. You know, driving in the right way. So what I to say that the switches, the network and networking, know everything, and we, if we can get more information for this devices, if you, if Pakatan, if you have the gash in the service, the switch, you know it the switch in order to do a pocket for sure. So we have to think differently. If you want to get to this, you know that's.

C

The lighter yes.

K

That's the writer exactly this is this: is the lighter not the management, a thank.

B

You ok, we have a presentation, steel and I'm, not sure we would have time to discuss after that that I try to move.

L

On already already.

B

Part of the objective of this discussion.

L

So good afternoon my name is Jefferson Aubrey and I. My job is easier than the from the previous speaker, because I'm going to talk about the past, so I'm going to talk a little bit about a retrospective about net autonomic networking and it's not I, know it's an ambitious title for the presentation, but this it's more like the retrospective, considering the efforts on the IDF oops delta line is earlier introduction out of networking at the end emergy the animal working group in the outlook next slide.

L

So it's a unknown set of properties of that we expect in autonomic system and it's sometimes it's easier to say, which are which are the properties that are expected them to say what is an autonomic system, but usually it is. It is expected to be automatic adaptive in our in order to define the other or some different set of properties that are used so, for example, have the self chop or the maid k, and there are others, but for our concern.

L

We think that an autonomic networking system isn't a the application of this kind of properties in the network lifecycle. So, for example, we can talk about the installation, the commissioning the operating like the monetary and the this kind of jobs in a network operation.

L

So as usually it's necessary to have some some reputation in the autonomic network insistent, it's usually used some kind of a closed-loop control. So this is a simple sketch from the 4k Oh system, which is a influent autonomic network management system and the the important thing on in the figures that it has a two different components: the autonomic manager, which collects information from the network and also it's changing. For example, the configuration of autonomic of managed resource in the managed resource which provide information for the manager next slide.

L

So the autonomic networking was the subject of several research projects and the investigations in the last few years in the last decade. So, for example, there is an umf again and others, and also considering the work in the IETF. There are some efforts and, for example, you can talk about some efforts in super homenet as the energy and serg in the itrs. So there are lots of interesting, and also- and this is what brings me for this presentation- the autonomic networking investigations is usually addressed by the network management community.

L

So, for example, we can say about works that are published in IM norms, CNS em. So at this point we can say that form a network management perspective, autonomic networking something of is a subject of interesting in this regards. There are lots of of meetings of NMR G that are focused on autonomic networking management, so the first one was in Vancouver huh a bit faster. Ok, so there are the first one in Vancouver.

L

The second one in London X lied the third one in Toronto, and also the reason that a meeting which was held in huge Janeiro in the cns m 2014, which is also there, are presentations on autonomic networking excite in these presentations.

L

There are lots of words considering the definitions of autonomic networking terms, and this leads to two drafts, and that are now rfcs, which are a set of design, goes in no goals for autonomic networking, which is the RFC 75 75, and also the the gap analysis of autonomic network, which leads to RFC 75 76, and these two works. These two documents, leaders to the you can bath, which is an important calm of the in mr g work.

L

It's above for the use case of an autonomic networking, and this is our the docs that appeared in his in this bath, which are from background not enough networking, some use cases and some works on the solution. Space slide.

L

So after that the aneema working group was shorted and in this this text is from the from the charter of the group. We have a definition of the outcome of the animal working group as a system of autonomic functions that carry out the intentions of the network operated without the need for detailed low level management.

L

Thought of individual devices, which is I, think it's constant with the the the the slides that the previous speaker showed us, and also the goal is, as inserted in the group is not have an a complete solution for photon networking, but I specification of a minimum set of reusable components to improve they maintain the operation of networks in the focus in the Indy working group is to have is, is to to work with professional management networks.

L

Ok, so there is in the Charter for items for protocols to be development, and also there are three works that are developing such protocols and for the discovery and the negotiation of autonomic nodes. We have the grasp from Ryan, which I think it's here, no there's also the bootstrapping of a trust infrastructure and which is addressed by the briefs, T and also the development of an autonomic control plane, which permits the communication of the autonomic device, which is known by the acct okay.

L

So as there is a limited set of functions that are defined in the Charter, because it's important as a NIMBY shows goal, it's important not to boil the ocean, but to have a more focusing specifically for interesting in the feasible points on photo/nam key networking.

L

There are some work, which is out of a scope that was presented in the UK and both and after that, in the witch, for example, can talk about polishing, tent use cases or autonomic service agents, and this in the charter of the animal working group is stated that this work can be is encouraged as in g video submissions or in emerging submissions. So it's recognized that as outcome of energy and energy is a good place to work with interesting work of autonomic networking budget, which is outside the scope of the working group.

L

So right now there are some in chart at work that remains in an email waiting for new faces or the recharter of the group. So, for example, that I wrote some coordination, the intent format in distribution, and there are also other works that are from the you can both that are still in the energy and focus there's this work about submit annotation or facility relations, which I one of the atoms of this of this director, which is in our working group.

L

Let's go by the way so finishing anything that I'm me really quick this, but the definitions and the goals in the gap. Analysis of the autonomic Network probably need more consideration considering the IETF, and we think that maybe the MRG is a possible a good home for this discussion. This kind of analysis. So, for example, there is an internet draft about the network network in definitions, which is this. It should be complementary to the RFC, 75 75 and the he.

L

It is only state, maybe- and he said before so I'm, one of the authors of this draft, but we're looking for new contributors. So if you want to work on that, you were welcomed in there are some other point that maybe also can be, can use an mr g as a home, for example, machine learning nuh, as I said before, as presented before.

L

It's also something which is really important, considering self self star or autonomic systems, and besides that, we think that maybe the definitions of autonomic networking, our previous from the the wave of network, machining learning so probably some cutters, some some room for investigation, try to compatible eyes both areas, there's also the works on intense, which I know it's kind of a controversial topic, but currently it's out of scope of a nema and, I think also it's out of scope of super.

L

So maybe, if you were interested in working in writing down something about intense NMR, GE can be possibly our home for that. Also.

L

The programmability of networks, for example, considering sdn in an effigy, also are an interesting input for autonomic systems and also we know that there are lots of interesting in interest in this kind of network approach.

L

We can see as, for example, the interesting ESD energy in FV RG working groups, research groups and the works on that also can be done. Considering, for example, management aspects in the NMR G, because.

B

L

Ok, so, as my last words to the point of new technologies is typical, time-consuming and labor-intensive tasks so and I'm saying that, because it's important to to concentrate the energy to do that- and maybe the energy can do this kind of stuff, you know like a place that different works. Considering some part, different portions of autonomic topics can be, can be dried together in an MLG.

B

Thanks a lot to a person, we already finishing this session that I would like to take a couple of minutes just to reflect on what we have discussed today and I really liked the presentation in the discussion. We had I think a piece on my notes. I only try a set of key points of discussion. We came back regularly on that I mean the version which start what should be the end point of the surviving network, auto networks.

B

We have ongoing work either in an energy animal working group and other groups even outside of a ETF I RTF. So I think that still some interesting topics that you could tackle in this in this research group. What I would like to invite? You is ready to express. We don't have time right now, but maybe you can go to the mic, that what will be the first steps we should take in the research group to say: do we start a new document which we can sub sub set of topics?

B

We can identify what action concrete actions we can take really try to bring this work forward, not just to have this discussion that, while also follow out of this discussion, so I don't know if anyone needs to propose something or to be the next step, how you want to work.

I

Well, taking jargon that Jefferson has named a couple of things that some answers on how, in involved with I, think I think that the point here is that we have several main places in which we have many touch points, and there are a lot of fussy songs in which you can end one side on the other.

I

Probably will not be the bad idea to start. Why try to mmm identify gaps and challenges and the anguish which are the directions to move in. I mean in a document that we can. We can discuss home rather than sharing a DSP consistent and a 12 I can agree with, for example, what you mentioned is about the programmable networks. I mean the proneural networks in general. What do you have any fee? On the one hand you have whatever is coalesced yang, I mean at the end of the like. A software networks in general is.

I

This is something that whenever we move that there will be this position, which whether this is.

I

Software based networks, whatever all we're talking about server software based network management, is a little bit if you think in terms in the same terms, and when you have an operating system, you have the applications running and then you have the other things that are managing the system that are applications at the end. Well, drawing the line is challenging anyway, but it would be interesting to have some kind of document to discuss on refractory.

I

Let's call it challenging so or research directions or whatever, but the lug nuts and I would be happy to be a to be part of a team vil thanks legal.

C

Cubic impella um I would be really interested in two things. One is I talk about standardized data models, but I think it's very important to have data models to play with them and be not to have a standard necessarily but to have an evolving standard.

C

As we understand the data better I know that other people who work with telemetry have said all this is a great data model and then they realize it's not the best friend and they have to keep changing it so that sort of fits better in an RG rather than I ietf, and the second thing is: can we have a repository of data? Data is very hard to find.

C

People are scared about sharing data, but if we can have a repository of data, then people can say: let me use that data and try some experiment that I haven't tried before. So there are the thing called cago with google just part by the way. So you know there are repositories of pictures of you know, and so, if you want to recognize a cat rather than downloading a million things from the internet, you can download this set of data and run your algorithm and say: are you better? Are you more accurate? Whatever? So?

C

Can we sorry yeah? So if we could find repository or we could build repository of data network oriented data that people could then say, let me try correlating this and that or let me try something else, because that's the foundation and I should repeat: I, don't want to do machine learning. I want to stop driving network machine learning. I think will get me there, but I'm open to anything else. The goal is important and then the technology comes afterwards. Thank.

B

Security for the suggestion, additional final comments already a bit over time.

M

Then I'll make it fast, so John, Stroessner, Huawei I'll be happy to work with diego and others. On this, I think there's a couple of very important things that we need to work on, or else marketing will try and define it for us and we'll be stuck with trying to implement it and as an architect, I really dislike that. um The first is intent. Everyone is talking about it. It was everywhere at the mpls World, Congress and.

M

It's very very misunderstood, um so I'm also the orchestration area director for the em EF and we're working on a definition and solution, and perhaps we can share that here in NM r g and get a wider view on this and uptake. um The second is, unlike ready: I actually do like machine learning, I.

M

Couldn't resist sorry, Caretti yeah, um you know that I forget the person from ebay, June or Julie or what? Yes do you want? Oh okay, sorry I've got a bad hearing, um but you know one of the interesting things in deep learning is that that distinguishes it from prior types of learning?

M

Is that it doesn't have to be homogeneous right so, as I build my layers of machine learning, I'm free to use different abstractions and different mechanisms from layer, x2, layer, X plus 1, which you can think of as allowing different algorithms to provide fine-tuned parts or pieces of the aggregate hole right I mean this is true for any type of ensemble based learning.

M

But more importantly, none of this is going to work, not just without a standardized data model, because you'll never get a standardized data model, it's physically impossible right, because you've got to take into account the limitations of protocols and languages and formats from these different data sources. Right I mean so instead of that, I actually think you need a standardized information model and maybe not in a homogeneous information model, but rather fragments that can be used. Much like you use ontology to assemble learning systems that serve as the base for different domains.

M

So this is a working group just.

B

Stop it, but we graduation, because we have been through information model and models and data motives a lot of time fee all the time yep. What you're suggesting is a moral of this work, more link to the Martian learning I mean mining.

A

B

Of an information model that will be useful for the data representation for motion on so.

M

Info Cal, which Jefferson kindly referenced. Thank you. um We used models to represent facts and ontology zand first-order logic, to add meaning to those facts, so that you could then build a semantic, a grid or semantic Network that described the impact of what a particular data more series of data meant on the rest of the system.

M

In order to do that, we had to be able to normalize things like IP based data with audio data with pressure-sensitive plates, with all these things that probably weren't even on an IP stack, and so it doesn't matter if it's JSON or protobuf or fubar as a data encoding. Who cares?

M

If you fundamentally cannot agree on what a customer is or what an alarm is you got a much bigger problem and that's why you need an information model and I know: that's not popular in the IETF and tough I mean pay me now or pay me later.

B

Thanks a lot everyone I would like to create for this evening, we will follow up on the mailing list, I think from the minutes. We're trying also to extract some set of action and I really would like to see that the volunteers to to work on the situation of documents. We followed thanks a lot order for your time and attention. Thank you. Bye.

A