Internet Engineering Task Force IAB, 9 Nov 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: COVID-19 Network Impacts Workshop, 2020-11-09

Description

Recording from Day 1 of the Internet Architecture Board's COVID-19 Network Impacts Workshop.

Workshop home page: https://www.iab.org/activities/workshops/covid-19-network-impacts-workshop-2020/

A

Including a fantastic typing.

A

A

A

B

I guess you don't really see this lights in a big window right.

C

Yeah they're, taking up a pretty small portion of your screen.

B

Yeah, that's a little bit annoying. I thought it would.

C

Only definitely better thanks: okay.

A

So do we have a couple of people who volunteered to help take notes in the pad, or do we want to try and arm twist for that now.

B

Yes, uh my approach was to just ask everybody to share their notes with us later on, but um you know if people want to have common note taking in the in the notepad. That's probably isn't even easier, so.

B

I don't know, should we should we try to find one leap.

B

D

I think it would be more helpful to just take collaborative notes in the ether pad, um because if we don't actively engage in the note-taking, it probably won't happen is my experience or you know, trying to pull everybody's individual notes in in different formats.

D

That's my suggestion.

B

So I don't think we need detailed minutes for this meeting and we will have an um ib report um published afterwards at some point.

A

Yeah, so I think the good to put in in the ether pad as notes is stuff that would go in that report, yeah so or at least a reminder to somebody to go and check this recording.

B

So I guess you have an eye on it. Steven.

A

uh Sure good, if other people do that too, I'm not good at taking out, because I always think I'll, remember stuff and then forget is.

E

There a link to either pad it's in the chat, I'll post it there again: okay, yeah the chat. History is not a thing.

B

Okay, but even it's five minutes past, uh I would suggest that we just start. uh I think people have like most people are here. I don't hear the joining noise anymore. So, let's start so welcome everybody to the uh ib workshop on um covet 19 network impact.

B

um This is the first session we have today. We have two more sessions on wednesday on friday, um thanks everybody for joining, um and we will go right on the next slide. Yes, so um this is uh also part of the itf, so uh the node wall applies here. However, you know if you have something to share which needs other another framework. We can. You can indicate that and we can see what we can do, but otherwise you should assume the noteworld applies, and you can see this also as a contribution to the ietf.

B

Logistics, we have a web page which has some information about the workshop, which you might have seen because you might have submitted something. um It has also all the papers on there um that we have accepted accepted. Thank everybody for submitting a paper.

B

um We actually received a few more papers than on the on the which which have not been published or then are published on the on the web page, because we only published like real, full position papers on there. That was just like a short statement of interest or whatever we didn't publish it, um but I think the papers we got cover a really nice broad spectrum of input, and we are happy to have you all here.

B

We also created a github repo. The github reaper currently has mainly the slides from today. um It also has already an mb file for the report, so if you guys already want to put some note, there start working on the report. That's also helpful, um then we're today here on webex. We will use the same web accession on wednesday and friday.

B

The session is currently recorded and the intention is to publish the session the recordings later and also use them potentially for note tagging looking up stuff. So if you have any concerns, if you want to share something that should not be published, we can actually stop recording. You have to indicate that um but other otherwise. We would just assume that we can publish um the recordings later if you're not indicating.

B

Otherwise, um uh you should turn on your video when you're speaking or presenting, not sure if you want to keep on your video, the otherwise the whole time we have a lot of people in here. I don't think it's a should be a network issue. If the the tools are smart enough by now, but like feel free to turn it off anytime, make sure you're muted when you're, not speaking, you know that can be messy otherwise and for the queue we will try to use.

B

The plus q minus q scheme in the chat, um so yari is helping me watching the chat for the queue and I hope we can manage as well. um The goals have to have the very interactive discussion here, um but we are too many people to just speak. I think so. We need the scheme and then on. Here is also the link to the easy pet that um already published in the chat. um Somebody is not muted in making noise so that person didn't listen.

B

Well, thanks, um so etherpad uh we have the etherpad for taking notes, but if you can also put your name and affiliation there, then we have a little bit of record. Who was here. That would be very helpful.

B

Any questions on the logistics so far.

B

Okay, then, let's talk about why we're here, so the scope of this workshop is really to talk about what we have seen in the in the crisis. That's actually still ongoing, but mainly also in the in the first peak of the crisis, and what was the impact on networking.

B

uh I think we have all seen many reports um how quickly the the traffic went up in march and april, because everybody moved to home work from home and there was big shift and also different applications that were used, especially video conferencing, media applications had a big increase um and all over, we saw that, actually things work. Well, we know that some operators managed to upgrade their um capacity quickly. They have been like some small hiccups. We also saw in the u.

B

There was a request, for example, for netflix to actually reduce the video rate and lower the traffic load there and these kind of things. um So this workshop is really to figure out a little bit more. What happened? What have we learned and what do we need to improve in future um from a from a networks operator and service provider?

B

Point of view, but also from an architectural point of view, what's in the technology that maybe needs changes or what's set up in a way um to survive those crisis very well that we should make sure we maintain.

B

So it's really about figuring out what had happened, figuring out, what was behind the scene about network management and capacity uh um extensions, and also um you know what are uh what what are the general impacts of these traffic shifts um to our um architecture and potentially in any kind of future effects, and a little bit about these three points. We also set up our gender, so our session today will focus on measurements and observations.

B

We have a bunch of little small, a few slides presentations from people who send us papers about measurements.

B

So this is really to get everybody on the same page, get some knowledge about what happened, get some data and have a discussion about this. The session on wednesday will focus more on operational issues, um so we, what did the operators do? What was their experience and what can we learn from it and then on friday we will be a little more future looking and we also have hopefully some time for other topics. People want to discuss that didn't find a place in the earlier sessions.

C

We have cullen on there. Thank you thanks. I was just thinking at the um end of monday and wednesday. Can we just reserve a little bit of time to stop back to stop our normal discussion and just make sure that we sort of outline what we want to talk about the next day, so we can sort of guide those agendas a little bit more.

B

Yeah, that's a really good idea: okay, thanks.

B

Okay, any more questions on the scope and the plan for the meeting for the next days.

B

Okay, then, let's start with today's topic, um so I looked at all the measurement papers uh which you can find on the iv webpage. I also look at the slides I received already, um and the overall observations isn't that surprising right we had like this huge increase in traffic in march and april, uh we have observed changes in traffic pattern because we have different kind of applications. People are using because um of the different way they are work and they they live. We have especially this very strong increase in video conferencing.

B

We have seen based on measurements. We can also see that we had that there have been reactions from operators upgrading their capacity more quickly than they would usually do.

B

We had got also some measurement paper showing that there are some problems about last my congestion and latency and some networks, depending on actually how the infrastructure is set up. um So this is just like the very high level view and we kind of dive today, a little bit more into the data and and looking at the details and figuring out did we actually measure the right thing and do we know everything we know or did I miss?

B

Did we miss anything here and as such, we also have these presentations, which I tried to structure a little bit uh in this way and there's a christie in the queue.

F

Hi, thank you and yeah. I just wanted to sorry. You mentioned. If there's anything you missed in the high level observations and just in terms of a shift on traffic pattern, and then I think we should really acknowledge the shift in um cyber attacks and the security aspect of what we saw in terms of the shift in traffic patterns. So I just wanted to mention that as a as a high level observation before we go into more detail here, thanks.

B

Thank you. That's uh that's a nice contribution. We don't have a measurement paper about this, so um but that's something definitely something we can further discuss.

B

Okay, so on the measurement papers, um you know, I consider the order we want to go through this and there are different ways to do that, but that's the order I choose. um I would like to start with oliver um on the on a paper which he called the lockdown effect because they have various measurements from different kind of networks, and this shows a really nice high level overview. I'm getting some data at the observations I just um stated.

B

Then we have this little block where we get a little bit more into specific traffic patterns in campus networks and mobile networks, and then we have amp papers looking more at interconnects at different points um and then at the end we talk about this last my congestion problem that I mentioned. So that's the plan.

B

Any kind of agenda.

F

B

B

I can also quickly slip a flip on the next slide, um so there's a slide with potential questions we want to address later on. um Of course, um your own questions can go here as well. You can look at the slides in github if you want to double check this, but let's look at the presentations first and with that we can start with oliver and I'm will try to find the slides.

B

It should be this slide set- and I increase this again a little bit for you.

B

Oliver are you there and ready.

G

All right thanks mia, then uh so, thanks for the introduction, my name is oliver hoelfeld and, on behalf of all the offers, I'm going to present our paper on the lockdown effect. So what we intended to do was we wanted to take a very broad perspective on what is actually happening in the internet when the entire world shifts their behavior. So we have a couple of vantage points, an isp internet exchange point and an educational network, and we want to look at all of this broad set of networks wow.

G

How um did they actually cope with the traffic change and what was actually happening in terms of measurable observations? So next slide please.

G

So we have a lot of vantage points, so a lot of data to crunch. That means we have a lot of data crunchers that are involved in in this project and looking at the different perspectives. So next slide, please.

G

So what kind of vantage points do we actually have? So we have we're looking at three internet exchange points, so one big internet exchange point in central europe, uh one in southern europe and one uh internet exchange point in the us east coast. So that's kind of the core networking perspective where a lot of the networks meet, then we are also looking at a big internet service provider and here to get the residential customer perspective.

G

So all the dsl customers that have now uh have to work from home and then we're also looking at a big educational network in the madrid region that interconnects more than or 16 different research institutions, students, faculty members and so on. So next slide to give you in a view on how this traffic evolution is kind of developing. Here you see a plot that starts in january last year and it goes up until the end of june this year.

G

And what you see here on the left portion of the of the plot is more like the general increase in traffic that we have so as time progresses. We're more utilizing our networks.

G

However, when the initial responses and lockdowns were announced. So this is the part in the plot where it goes from lay ddy, shaded background to the gray shaded background. So that's um in march this year, when the the government's imposed lockdown restrictions, we have a big jump in traffic uh in the order of 20 to 30 percent, and this is just because people change their behavior uh and because of changes in life. So that's a big um additional set of traffic that these networks needed to handle.

G

So this is what we see at this internet service providers. Now we have a couple of more vantage points, so if you go to the next slide, we see how this looks like for the internet exchange point, and here we see exactly the same behavior for every of the three internet exchange points that we're looking at.

G

So we have a more or less gradual increase in the traffic evolution and then once these lockdowns were announced and and the restrictions were imposed, then we have a big increase in in traffic in the order of 20 to 30 percent in every vintage point that we're looking at. If you go to the next slide, then we can have a look and how this looks for from the perspective of a mobile operator, and here, as we would expect, we see a slightly different um pictures.

G

So once the walk the lockdowns are in place, um the the traffic is slightly decreasing. So that's the general um observations that we can see here in terms of how the traffic level is rising.

G

So next slide, please what are the takeaways that we can take from all of these measurements that we did so people's life changes, and this leads to the advent of new traffic patterns and most notably, what we observe- and this is in line with like many of the um other works and operators perspectives that have been shared. Meanwhile, is that the difference between workday and weekends vanishes?

G

So all the uh all the work days becomes a bit more like weekends, so the difference here, kind of vanishes the traffic composition, changes and in terms of traffic composition. Here we see shifts so, for example, all the applications that we use for remote work, education, vpn, video conferencing- you mentioned it in the beginning. All of these applications see an increase in traffic.

G

I showed you in the beginning the plot of the general evolution of the traffic increase, and this is what is mostly reported and you mention it in the beginning. This is what we mostly talk about: uh networks increase in traffic. This is, however, not the case for all the networks.

G

So, for example, if you look at education in the networks, if you send people at home, they cannot use the networks anymore, oh well, they use it for different kind of services, so they need now streaming, meaning that the um the traffic symmetry and the in and outs out ratios they completely change.

G

And then, of course, if students are sent away from the campuses, then these campus networks are not utilized anymore, so they kind of see a decrease in traffic and for the educational networks here that we looked into, we saw an up to 55 um decrease in traffic.

G

um The other thing is that we see here is that most of the people when they look into internet traffic, they see they look at hyper giant, so the big cdns and cloud providers, and so on. What we noticed is that, however, only focusing on these hyper joints is not sufficient, because we saw a lot of increase in these non-hypergiant networks, but in general what we find here is that, once the human patterns and their behavior changes, this is directly reflected in internet traffic patterns. That operators need to account for so next slide.

G

G

So what we saw here is that if we just look into the traffic volume, then we saw an increase of about 15 to 30 percent with just a few days only so that happened rather quickly and if you now consider that network's, usually provisioned for about a 30 percent increase per year. um This is pretty substantial in terms of like how did the networks cope with that mostly pretty well, because they had sufficient idle capacity provisioned, but there's.

G

Another effect that happened in here is that the impact on the peak traffic was rather limited, so most of the traffic shift um they uh they they occurred in non-peak hours, which means that the valleys get filled, but the increase in the traffic peaks is not as substantial as the the actual shift in the traffic.

G

The other observations that we made here was look when we looked into port capacities and how did networks deal with provisioning. Their links was that the track, the the most of the networks actually reacted rather quickly to the additional need for capacity, and they were very quick in provisioning additional capacity.

G

So, for example, here the central european rxp reports in capacity increase of about uh one and a half terabits here. So what can we take away from this? Is the networks um could deal with this sudden increase in traffic and these um shifts in demand um pretty good if um they had sufficient idle capacity provision and they had like a very quick reaction times in terms of like provisioning, new links um and and adding capacity.

G

So this is why, for uh like these kind of bigger networks that we looked into, um the the impact of the pandemic was measurable, but it didn't really hit these networks.

G

So with that, I'm at the end, um thank you and I think we go to the next presentation before we jump into questions.

B

uh We can have some questions if you have clarifying questions or whatever um next presentation will look a little bit more deeply into especially campus networks, so that fits here directly.

B

But we can also see if somebody has a question or already wants to comment on something.

A

Yeah, I have a quick question on. I think the second slide, with graphs no slide numbers uh the next one yeah. So there's a downturn on the the right hand side there there's a there's one line that returns back down to the the normal. If you like to say just go: okay, what was happening there.

G

um With that, I would like to hand over to chris or someone from the dkx.

H

Yeah, so thanks ollie, um unfortunately, I'm currently not able to to view the exact uh graph so which one is it only. So I have my browser. I don't see the sharp presentation so.

G

um That's the uh the second second plot, with the the second slide, with the ixp plots, and here it's so the dip that you're referring to is the south european.

G

B

So, let's actually, let's move on for now. Maybe you can check this out and let us know and later on.

F

I guess it's also.

I

B

J

J

Question: okay, so here I mentioned that the capacity is uh can be increased well uh because of the planning in advance. But you know that in the in the process of the kobe 19, because I think the capacity may be not a very big problem. But you know we know that the experience of the users may be not the same as euro.

J

I'm not sure if there are some data to evaluate evaluate the experience of the user against field.

B

Yeah, so we do have a more presentations talking about some places like japan, where there was actually much more congestion and higher latency.

B

So let's maybe keep that for later discussion.

J

B

Okay, I would just uh in this case, move on uh and we have martino next and let me find those slides.

K

B

Hello, we can hear.

B

K

Hello, hello, everybody. First of all, thank you for all the organizers for the nice initiative. I will present something about our paper about campus campus traffic during the pandemic, as also oliver was mentioning.

K

The kovid pandemic had an impact also on the campus networks, and very often we see a decrease in traffic, but not always, and in our network in politecnico di torino, we see an increase of traffic due to some specific situations. If you go to the next guide,.

K

Okay, thank you. Yes, in our paper, we collected some uh some data from our campus pretty technically torino. Some passive data using uh a network rob running a passive meter, called this data that we developed here here in pre-technical literino, and we compared what happens in our university with respect to other large universities in italy. Here on these graphs, we see how the traffic pattern changed from before the lockdown and during the lockdown.

K

Here we have the daily pattern on the top y-axis. We see the incoming traffic pattern, while on the bottom y-axis, we see the outgoing traffic pattern. If we focus on the incoming traffic, this is uh mainly what people do while browsing, and we see that in all the three universities we see a very large decrease of incoming traffic due to the lockdown, because people are not are no longer there in the university they do not browse, they do not download content.

K

Instead, if you look at the outgoing traffic, which is basically the traffic served by uh the servers of the of the university, we see that in polytechnic and university, uh the outgoing traffic is almost remain the same. But in polytechnic, which is the left image. We see that the outgoing traffic really exploded. This is the red line and the reason uh is because polytechnic adopted an in-house teaching infrastructure.

K

So, basically, all the lectures are hosted are distributed by servers in cell polytechnically train rather on the cloud, and this makes the outgoing traffic to explode uh in the next slide, and we have some uh details about that. In general, we observed uh different traffic modifications due to due to copy the pandemic.

K

First of all, we can mention that we noticed a large increase in remote access, so in vpns and remote desktop connection for the internal resources of our university, because professors, but also technicians and administrative staff, were forced to connect from outside.

K

We also testified a large increase in the use of microsoft teams because in our university, microsoft teams is used by faculties to communicate with students for administrative stuff for storage and these things and finally, the most important traffic change uh was due to, as I said, to the remote learning infrastructure that we set up in polytechnic, which is based on a bbb big blue button, which is an open source application for uh for remote, remote teaching, which is based on webrtc.

K

In turn, it uses rtp traffic to distribute the live lectures to the students. This had a large impact on the campus traffic. If you see the bottom left image, we had like 600 virtual classrooms per day and 15 000 users connected every day. Also. What is important here is that these uh traffic changes are persistent, so they are not only during the first days of the lockdown, but they lasted all the lock down longer. So if you see the figure here in the bottom right, this is the incoming outgoing traffic pattern.

K

We see that during the look down the outgoing traffic really exploded while the incoming traffic dropped, but this is persistent so also now, during the second semester, we have a a huge amount of outgoing traffic due to remote lectures.

K

This traffic even increased because we have all the new students, because all the classrooms are now working remotely, and so uh we have like two gigabit per second of outgoing, mainly traffic.

K

If we go to the next slide, we can also uh see how this modification created some different traffic pattern in terms of daily and holy pattern.

K

So here we have a breakdown of the traffic due to remote teaching, so uh outgoing traffic, we have live classrooms uh which are based on rtp traffic. They are live, so this is udprtp and they happen only during the weeks and during the weeks from 8 30 to 7 p.m.

K

We can observe up to a half gigabit per second of outgoing traffic due to multiple classrooms, but there are. There is also a large traffic due to uh due to the on-demand content that students download, because students can for sure download, also the uh the classrooms offline, uh for example uh during the weekend.

K

In fact, we noticed that uh during saturday and sunday we have almost uh half gigabit per second of downloads of teaching material, and this is uh huge because, because before they look down, we didn't observe such large traffic during the weekend, and so this is a very hard modification.

K

This traffic pattern also has another characteristic, because lectures are scheduled to start every 90 minutes, and so every 90 minutes the students connect and start watching the classroom, and this triggers new tcp and udp flows that are born coordinately and even if the traffic in terms of packet per second is not doesn't have this peaks.

K

The the number of new tcp and udp flows exhibit a great peak during the beginning of this uh of the classrooms, and this can be a problem, for example, for firewalls for some middle box for passive meters for all these kind of network, epic equipments the need to have a per flow state inside.

K

So this can can be a problem for some network equipments. If we go on the next slide, we can finally study the performance that students enjoyed while uh using our teaching facilities during the lockdown. uh Here we first focus on the speed the download speed of the teaching material, which happens over https over tcp.

K

There is some difference across operators because we notice that uh fixed and fiber operators are typically much faster than mobile and low cost operators. This is the figure here on the left on the top. We see that, for example, fastweb was much faster than a free mobile, which is iliad, which is a mobile operator. Also.

K

We noticed that at some point, all the traffic uh from italian students going from italy to our university was outed to the france and then coming back to polytechnic, and this was a clear performance penalty that students were enjoying and, on the contrary, we didn't notice a great uh great differences across italian regions. Here. This is the graph on the bottom left a bit throughput.

K

Yes, there are some variability of the throughput between regions, but not as much as we expected, and this was rather interesting because also in italy, there is a large debate about the digital divide, for example all of uh rural areas, of isla of the islands in italy. We have uh sardinia and sicily. There is a huge debate about uh if the internet is lower or faster in these islands, and actually we did not measure such a different performance across regions and finally, we can say something about live classrooms which happen over rtp uh in udp.

K

uh Since this is not a batch download, it has a fixed bit rate at which the content is generated by the teacher, and we didn't notice any problem due to the network during the lockdown actually also because the bitrate is rather low. The video is very sedum above 200 kilobits per second, because we set up a low video b30 to avoid congestion and also audio is typically at 40 kilobits per second. So this almost never saturated the network of the students and we didn't we there.

K

No big problems were reported by the students during the lockdown, and yes, this was the slide in general. Yes, we can say that, uh from the campus point of view, it has been a large effort to set up this. uh This teaching infrastructure.

K

This uh clearly stressed the campus network to have such a large amount of outgoing traffic, which kind of really modified the pattern uh rather than before the lockdown. Before they look down, we had incoming traffic and not much of going traffic. Now we have almost uh all outgoing traffic, but in general uh the network uh managed to to carry packets, and that's all thank you.

B

Thank you very much for sharing, especially this last slide, because there's a lot of discussion already ongoing uh about how to measure user experience um before we take oliver. I have a quick question um so in the previous slide um you talked about that. The number of tcp connections would uh challenge like fire firewalls and these kind of things. So did you see any outages there and did you had to upgrade those firewalls or did you do anything there.

K

Regarding firewall, I don't know if they had problems the technicians but for our passive probe. That would be. That is running this, which is a custom software custom, passive meter. At some point, we had to increase the number of cores because it is a multi-core multi-threaded. We had two cores. We had to upgrade it to four cores because the traffic increased too much so, but I I'm pretty sure that the network administrator had some other problem.

B

Okay, thank you. So then we have oliver.

L

Yeah, I have a question regarding slide number two, um where you compare the basically the incoming or versus the outgoing traffic before and after the lockdown, so, um and especially for b and c. So, basically, not the universities where you investigated, uh I guess most of the time. So why don't you so? Why is there so little difference in the um in the outgoing traffic um after the lockdown, so in the lower part of b and c? So why is.

K

It longer actually, I think that this is because uh the outgoing traffic is mainly due to the servers, so the teaching portals- and so this pattern didn't change too much, because these are students going there to download the slides to download teaching material, and this maybe has been rather rather stable.

K

Also, I don't know which is the: how is organized infrastructure of these two universities? How much content is on the cloud and how much content is on premises. So I know thanks.

B

Okay, any more questions.

B

Okay, as I said, we have some discussion on going on the chat so check that out. We will pick that up at the very end.

B

Definitely we uh move on with uh looking into mobile networks a little bit more and we have andra here. I.

B

B

M

Go ahead: okay, great thanks! So much yeah! Thank you so much for for having us today. um So I have also diego perino joining from from telefonica today.

M

um So yeah, um I'm just going to quickly introduce to you a bit of our analysis on the implications of this pandemic on the traffic of a mobile network operator in the uk.

M

So specifically we're talking about um all two in the uk, so this is joint work with telephonica research university, carlos the third country in telefonica in the uk. So specifically, we looked at how the pandemic comfort unfolded in its early weeks within the uk, and you know how all the government issue measures to actually tackle the epidemic, impacted the mobility of the people and, implicitly the way in which they actually used the mobile network services.

M

So the period we focused on captures about 10 weeks from the end of february until around mid may of earlier of this year. So, within this period we actually captured a few interesting moments when the world health organization declared the coronavirus outbreak of global pandemic, as well as all the follow-up measures the uk government took to um you know, impose social distancing, including a nationwide lockdown. They declared on the 23rd of march uh next slide, please so I'm going to start with the takeaways.

M

So in our analysis, we actually found that you know in an effort to tackle the pandemic. um The uk nationwide mobility decreased by 60 after lockdown, compared to the last week of february of 2020, um so actually people did stay at home.

M

At the same time, their usage of the mobile network also decreased and found a drop of approximately 20 percent in downing data traffic after knockdown, compared to earlier of this year, london in particular, so many of its residents actually relocate elsewhere, um showing a 10 drop in residence after lockdown or around lockdown uh in a steep, decreasing downlink data traffic, especially in central areas of london.

M

uh However, you know the pandemic brought voice conversation back in the center of our attention, so people did started, calling their relatives more and their close ones more, and we found a surge of about 140 around lockdown compared to february uh next slide list.

M

So our analysis put together, um you know a rich data set that we actually collected from from o2, including a general mobility, signaling information data set from devices in the network from mobility management entity and radio network performance metrics. So we capture users mobility only by checking which are the radio towers to which they actually connect. So I must specify that this data is always anonymized and aggregated and we don't ever access user gps information um from these data feeds.

M

You know we complement with third-party information from, for example, the office for national statistics, and we derive different mobility patterns at the user level, including the home location of the user and a bunch of mobility metrics. So one of these mobility meters, for example, is the user gyration, which tells us about the area in which a user moves only based on the radio towers to which the user actually connects.

M

So next slide, please.

M

So this is where we actually see the evolution in gyration. So, based on all this information, we are actually able to observe how the mobility of people changed in the uk um in these early weeks of the pandemic. So what you see here is the average variation in gyration compared to the nationwide average in february, across our data set of about um 16 million users, so we captured the variation from the 24th of february until um 30th of march in this plot, so each bar that you see here corresponds to a different day.

M

So what we observe is that oval dependent was declared in early march, and you know social distancing was immediately encouraged. um The mobility patterns of people did not change immediately, so this actually dropped dramatically once the government actually issued the standard order later in march. So we note that after the 23rd of march, um the gyration drops by at least 60 percent compared to earlier in february.

M

So the main takeaway here is that the only effective measure to actually convince people to social distance was actually the countrywide lockdown next slide. Please so then we asked ourselves: what did all these changes actually meant for for the mobile network so to understand the impact on the actual network? We looked here at six uh different indicators. You know performance and gear kpis, so we looked at downlink uploading data volume, downlink active users, downlink averages or throughput sales resource utilization and the total number of users in the rate yourself.

M

So we only report on the y-axis, the delta variations of each of each of these kpis compared to the nationwide average in the last week of february of 2020 prior to the pandemic. So we show here only average values across one week so on the x axis. Basically, each of these plots each points corresponds to one week in twenty, so we look both at country wide, but also specific regions, namely we look at um phytoplankton counties which we mark on the map below the legend there.

M

In terms of the data traffic patterns, we noted that the evolution at the uk levels of the countrywide is similar to that at the metropolitan county for more or less for the five different counties. We capture, in particular the data traffic volume dropped by 20 or more while the number of download active losers also decreased together with the cell research civilization in some sense, uh maybe unintuitively. However, the throughput also dropped by 10 percent and we put that likely because um of applications. Rather, but you know this is just conjecture.

M

um We also see that london actually made a discordant note from the rest of the areas with a steep decrease in the number of users and both in the uplink and the download data volumes. So we confirm later with our analysis, that this is likely due to the temporary 10 relocation of london residents who didn't remain actually in their homes after lockdown.

M

So next slide, please right. So we know that we not only looked at um all this analysis based uh broken down for metropolitan county. So at this um you know geographic units, but also we looked at the analysis on geodemographic clusters and I'm not sure how much I have, but I'll quickly explain what these are so every 10 years the uk runs a census and with the data from the census, they actually classify output areas in different clusters.

M

They call them output clusters and what you see here in the legend, um they are the largest clusters that we can. We can use and you see them in the plot map. In the left side, you actually see how all these clusters uh map to different geographic areas. So then, instead of like looking within in the limits of a specific area, we actually looked at this demographic and try to understand how the performance of the network maps with these trading rough clusters.

M

So what we immediately see here, the first thing that jumps in our faces is you know the the discordant that london makes with uh with with the rest of um of the different demographic clusters. So the blue one is the cosmopolitans they usually map um in in central london areas and, as you will see in the next slide, we actually found that a lot of the people who living in central areas in in london um are going to move away.

M

As I said before, in the previous slides, the previous slide. In the initial slide of three takeaways, the voice traffic actually uh surged, we actually found uh 140 percent in value in terms of the increase in the total volume. However, this system supportability a strain on the network that actually interconnects the different operators um that was quickly fixed, and this was basically showing the rapid response of the network operators and the service providers, restoring the the quality of the surface. So next slide.

M

um So, finally, I show you the temporary location of the residuals from london, um so in this matrix, what you have on the oligarchical columns are different days and the rows represent um different um cities within the greater the region of london within inner london. So yeah see how people from london moved to other cities around london. um So you immediately the first weeks which are the weekend day weekend pattern patterns which um disappear.

M

Now, if we focus only on the mind of the remark, you see that the color goes slightly um towards the lighter blue, which basically means that ten percent, we see a decrease of ten percent in the number of uh inner environment residents who actually are present living in london after week, 13 of the the lockdown, and then we see that just prior to the actual lockdown being imposed on the 23rd, which was the monday.

M

You actually see a bunch of um movement happening um that corresponds to people moving residents moving from inner london to, for example, areas like hampshire or kent just on the weekend before the 23rd.

M

So next time. This is just um an update, as you probably know, and you've seen the news. There is a new lockdown being post now in in the uk, um so they started with like tier love, towns and now they've entered like a full uh locked down like a but milord and then before in march. um So we looked only at how mobility changed across these last nine months. Just like this.

M

So, if you think about the I'm, not sure how well you can read this, it's not the best um yeah the easiest, uh but this is just the evolution in gyration. The upper part is just um the same bar plot that you saw initially, but now it goes until the end of september and I'm seeking. So you do see that across the summer you know this um social distancing, kind of um diminished considerably and now we're as we go into the second one.

M

This the generation starts to decrease um and probably will go down to the levels that we see in the first level. You know check different areas as well um at the month aggregated per month, and this is what you see in the bottom part. So you know the steep decreased in march and then a slight relaxation of um you know that the step by step in the months towards the summer, all of us that looks you know normal sort of in terms of mobility of people.

M

And then you know we go to september and this season's decreased and we're we're analyzing this to see how this affects the neural conflicts.

M

So I think this is it.

B

Thank you very much um so yeah. I think this is like my two high-level uh takeover messages is that first, um the situation looks slightly different on mobile networks than maybe for other networks, um and also that this is a lot of interesting data about um people. Mobility, that's very helpful as well.

B

um You you mentioned very um briefly that um also mobile operators had to react and extend their capacity, but I guess it's like I mean like you, cannot put up new base stations just now right, so do you have any insights what they did and how that worked? They did deploy.

M

5G- and I confirmed it- does not cause code, they don't know, that's a joke and we're deploying actually 5g uh across these this month, and they were especially in london. They were extending, and uh probably we've seen the news. The press releases that ultimately they were deploying um capacity, especially within london. Around the areas from outer london. They focused um the issues they had in terms of capacity were more towards um the inter peninsula voice traffic that was short-lived.

M

They were from march, like the 23rd immediately so the first week before lockdown um and the week after lockdown, so it was the effect of um a natural disaster that you observe when people just pick up the phones and they start calling- um and you see that, even if you know in huge, you know um earthquakes or or natural disasters, we've seen, we saw exactly the same uh pattern and this this matched that, but actually what was interesting was that even after the the week after immediately after the lockdown, the force traffic stayed, you know at um basically was was constant with an increase of about 50 compared to february.

M

So it kind of remained at very high levels before, depending again.

B

We have yari in the queue.

I

Yes, uh thank you for this. This was really interesting data and a really interesting presentation. I particularly liked that we went into the details behind the big numbers, like the demographics and uh people moving back and forth between places, and uh it's good to comment on one thing that we at ericsson had or smart colleagues have messaged um traffic in different countries, and they saw a lot of variation. One of the possible reasons for that variation is how much sort of fixed network capacity or or users there are in a country so, for instance, india.

I

They rely quite a lot on the mobile networks, so so doing the pandemic traffic. There grew a lot in those networks um and- and I wonder if that might be an additional detail behind the big numbers that one should perhaps or somebody should look at. um You know whether people have their their fixed networks and whether that effect, how much traffic there are sending or not.

I

So there might be more factors to look at them. I don't know if you have any of the this data on your datasets.

M

So I think I think it would be great, and this is something that we've been struggling with quite a lot to try to understand the. Why, behind what we see right, so one immediate um assumption is that okay, the traffic decreased in terms of mobile network usage, so it you know the waterbed effect should have been seen for for the fixed um traffic for the fixed network traffic. However, you know we cannot. We don't have that sort of information to check for the same users. um So we're not very.

M

We didn't know how to verify this, so we left it at. You know the conjecture level, so we could only assume that that's what happened, um but then it would be very interesting to be able to put together, for example, nice in the uk, with the mobile operator and check. You know in the same demographics, how you know if actually this water bed actually happened, you know the 20 drop and the increase of proportional increase in the isp traffic.

I

Right: okay, thank you.

B

Sure, thank you thanks. um If we don't have any more questions right now, we would go into a break um given we are already behind time when we have some more talks.

B

I first of all need to ask everybody who's, presenting next to speed up a little bit and be as brief as possible, because we also want to have some time for presentation left for discussion left, but I would suggest that you really only go quickly into a five-minute break, so everybody can grab a glass of water or coffee or whatever, and then we restart with the next presentation.

B

And stephen says in the chat that you of course, can keep discussing in the chat, um and we will also monitor the chat and try to make sure that points are captured in the report. Okay, see you in five minutes.

B

Okay, um how to chat up everybody's back I'm here! Five minutes are over, um so we will simply kind of start again. um I noticed uh that I actually didn't introduce myself and uh webex is nicely hiding my name.

B

um So just for the record, my name is mia kulibin, I'm an ib member and I would also like to quickly introduce um those people from the iab who helped putting this program together and reviewing these the papers um that stephen farrell, uh who has been already talking and you've, seen his video at the beginning, there's also yari ako, which has also asked some questions already, and then there was also ben campo, which is here who is here today but hasn't been active yet so you're the next one to ask a question, um and then we have also um colin jennings here from the iab who will share the session tomorrow, but he can introduce itself tomorrow.

B

I guess, um okay with that. um Oh sorry, I forgot colin colin perkins was also helping reviewing papers, uh who's, also who's the irtf chair. Sorry, um and with that I believe, because jason just turned his video on you're presenting perfect so then go ahead.

N

Yup thanks so um thank you. My name is jason livinggood, I'm an engineer at comcast in ift in the united states and I'll be presenting today with my colleague, nick feenster, from the university of chicago next slide. Please and we'll certainly go through this quickly. These are the folks that we are collaborating with um next slide.

N

So the high level here is that we were interested in.

N

M

N

Looking at both long-term trends um in uh interconnect, data and some near-term isp observations, and so the primary data sources.

M

N

M

Put in the general bucket of.

B

Nick, I think you're not muted.

N

Thanks so the um uh the primary data source which nick will go into momentarily is from the interconnection measurement program was kicked off in 2016 and is still running today, and essentially this is comprised of sampled, netflix or ipfix data for all of the interconnect interfaces across multiple isps, and so you know timestamp the region where the interconnect is and then anonymous peer ids.

N

So the data set that nick and his team have doesn't identify a specific peer name, just a random anonymous identifier, of course, exercise p, total diets, ingress and egress and capacity so nick's going to dive into this and then we'll come back to me. Nick next.

N

N

Nick, you might be on mute, go ahead.

A

Nick sent emails saying he might, he might have to pop into some other internet of some other meeting as well.

B

He's here, but we and he seems to be talking, but we can't hear him got.

N

It all right! Well, I will I will jump into this then um and we'll come back to him in a second. So the key findings here is that um I think similar with the other presentations that we saw. We saw an enormous increase in utilization in a very short period of time and at the same time, new capacity was added in a very brief period as well. So we see, of course, a steady increase in capacity in the interconnect measurement program of capacity, particularly from mid 2018 but beginning with covid.

N

We saw nearly twice the rate of 2019 editions, so in a very short period of time. All of these interconnect interfaces were having a lot of new capacity added that growth continued, not just in this initial first quarter and a first quarter time frame, but through the second and third quarter. So it has continued and um uh you know the uh let's see if we go to the next slide, we can see a bit more.

N

So nick uh are you available now I see you're you've stopped talking available to speak to this, or should we come back to in a second.

O

N

Go ahead hit this line if you could and then I'll do that.

N

N

Nick you seem to be muted.

O

Can you hear me now, okay, good, so in this slide, we're.

P

Basically, looking.

O

At peer link utilization in particular, how much traffic um goes across the links from a particular isp in the upstream or downstream direction, and then comparing that to uh between two periods, january 1st, 2020 and then september, 1st 2020, and what you can see basically on the diagonal uh is, is that um uh the diagonal shows, basically, that uh would be like uh equal ratios up and down, uh respectively, uh from those two time periods.

O

If it's above the line, we basically would see an increase, and if it's below the line, we see a decrease in each point is appear, and so what you can see is basically that traffic volumes and ratios change significantly between those two um uh between those two time periods for specific peers.

O

um This is uh uh data that we looked at uh both from the isp perspective, as well as from the internet interconnection measurement project. You go to the next next slide.

N

Great, so just to wrap up um in addition to the long-term imp data set that nick analyzed, you know just some individual observations. Looking at all of the detailed comcast interconnect data, we definitely saw about a third increase in that march april time frame and it happened, as other presenters said, really in a matter of days, so that was a year's worth of growth. In a few days, which was astounding, when we look at our access network to date, certainly a peat.

N

You know in that march april time frame went down due to seasonality in the summertime, has come back up and now uh with return to school in the august september time frame, but uh over that period of time we've seen downstream consumption or usage in the access network. Up about 13 and upstream up about 36 percent, largely driven by things like video conferencing, um we saw a huge increase in the amount of access network augments.

N

So you know 400 plus percent um in many weeks, which was big, um 500 plus uh augments to our core network. Those are primarily at the edge so think of the interconnect. Links we've done through this period as well. This isn't really interconnect related, but about 700 to 800 000 tests per day to cable modems in the network, just to measure qoe, in addition to traditional capacity measurements um and notably so within our interconnect link types.

N

One of them is settlement, free interconnect and typically those would grow at about 15 a year, so a little bit slower than the norm. But those went up. 37 percent over this period and some of the notable per partner ones were interesting. So it seemed like our interconnect. Growth were heavily driven by specific partners, so it wasn't across the board that everyone was increasing, but that particular transit networks, content, delivery networks or web conferencing providers just saw amazing growth.

N

So in one example, you know one peer more than doubled, 115, basically in four to seven days in a week's period, um and then we saw other platforms up 3900, which was just mind-blowing and uh more of the norm. Was you know one to two hundred percent, which is crazy to say so that is it um and for more information on the next slide. You can obviously click on the paper. There are more insights there and we're happy to take questions later on. Thank.

B

You thanks, um as you are mentioning um the speed test here and and qe, um so you increased your test volume here. Did you also noticed actually more mqe problems and do you have any kind of data or like rough numbers for that.

N

Yeah I haven't, um we didn't share that in this paper, um but there's another paper I'll I'll post it in the chat uh paste it in the chat.

N

Excuse me, um and so it was kind of good timing that we had this test come out um in early february um right before the pandemic, so we had a little bit of beforehand volume and uh what we ended up seeing is certainly in some cases in that early time frame an increase in latency a decrease in the percent of advertised of speed, but in all cases um we we still were above 100 of advertised.

N

So you know we had that extra buffer as one of the earlier presentations said of capacity and it ended up getting consumed. You know we ran close to where we we, you know, thought that was like the danger zone and then very quickly added capacity back up to re. You know recreate that buffer. If you will um so, we felt pretty good about that and and that correlates with a lot of data sources like ripe atlas and their latency changes in their data and so on.

N

So it seemed to work out well, um and you know we we felt lucky that we had that extra provision capacity and you know, certainly it makes my job easier and our team's job easier with our finance folks, who have always said. Why do we maintain all this extra capacity? That's not being used? Well, it just got used very quickly and that's why we have it so it was. It was interesting.

B

So did you mainly measure um throughput and and delay, or did you also try to look in in like more application metrics which are close to application performance.

N

So our test did not look at application: specific metrics. um They looked at upstream and downstream throughput latency by itself and then latency under load and I'll. Just give a quick preview on latency under load.

N

It's a thing, and it was a big thing in the pandemic, both um on the land side, which would primarily be the wi-fi land and then on the upstream network link, and um we actually had just through an accident of where software releases ended up being one particular device model, one made by one manufacturer, another a different manufacturer, different chipsets in them. So slightly different software.

N

On one of these we had basically a an aqm turned on and then the other it wasn't quite ready yet so for a window of time we had an exact exact same make. You know, exact same model. Excuse me slightly different chipsets and manufacturers, one with aqm one without aqm, with very interesting differences of latency under load. So that's in the process of getting written up now to share more information, but it's a big deal and I think there's a lot more to be done to study the wi-fi land side of things.

N

We've seen a lot of significant issues. There.

B

Very interesting we have jared in the cube.

D

Hi jason, uh I'm curious. I know a lot of people either upgraded their data. You know kind of upgrade their speeds uh and also, actually, I know several people who who entirely switch technologies during uh you know, as as a result of cable having much higher speeds. um I'm curious uh what you see. You know how much of this traffic was just organic traffic growth and how much of it was additional subscriber base.

D

um If you can comment on that, if you can't I understand, but.

N

I mean I haven't looked too specifically, but um you know off the cuff. I would say most of it is um is from existing subscribers just because the subscriber base is so large. You know it was like 24 or 5 million uh homes.

N

Before the pandemic, we have added a significant number of new customers, uh particularly with our what's called our internet essentials, uh tier, which is a 25 megs down three megs up from 9.99 a month for low income folks and that's added a huge huge number of people, um but even within the existing subscriber base. You know lots of people upgrading because they're all working from home. Much like me, I went to a gig.

B

Okay, we have one more question in the chat, I'm not sure ben. If you want to join and ask the question or if you should just like move the discussion to the chat and move on.

B

Maybe let's do that? um Okay and then next we have david clarke and we will look a little bit deeper into congestion.

B

P

Hi there uh I'll try to be quick on this, because I understand we're running late. This is a different perspective on some of the same stuff that uh jason and nick were talking about. Go to the next slide uh back around night uh 2016 uh keda and at ucsd at mit started measuring congestion on lynx, because at the time there were all these disputes between the big isps and the big content providers, there's a lot of congestion on the transit links, and now this is outside measurement.

P

uh Jason's talking about data, he has. We can't get the data from the routers we have to probe, and what we're doing here is a I'll very quickly. Try to explain the background work. Basically, every five minutes. We we probe a link. We send a ttl, expired packet near side, far side and what we look for is episodes of elevated latency, which tells us there's cueing on the link. So it's an indirect measure of congestion and we're probing every link. We can identify the data, I'm going to show you here's from comcast.

P

We have data from most of the large american isps and some overseas. Just so, you see what the data looks like go to the next slide.

P

This is a this is a picture from the bad old days when there was a lot of congestion on transit links because of disputes with netflix and- and this is the signature, we look for a daily pattern of elevated latency, which tells us that there's probably congestion.

P

There are lots of reasons why you could see elevated latency, blink changes or route changes or traffic management. So we look for this recurring pattern to give us confidence, we're actually finding congestion.

P

So we can look at this data and try to decide whether we've seen anything interesting uh when the pandemic started, because we've been collecting this dust for several years so go to the next slide. This is a this is a picture from comcast and notice just for fun. It goes all the way back to 2016 and what we're doing is we look at every link we can identify. We don't necessarily find all the links, but everything we can identify. In this case.

P

It's comcast peer uh con major content provider, cloud provider, transit providers- we don't we I'm not plotting here. They're customers and I took an arbitrary measure which is, is the link congested for more than a half hour a day, and I just add up the total number of links that that are congested.

P

I'm just lumping them all in one big basket, and so what you see is you know around 2017-2018 you get these huge spikes, which is where a lot of content providers were dumping, traffic onto congested links and then it goes away and you see 2019 it's pretty flat, and then you look at 2020 and what you can see I'm going to drill down.

P

But what you can see is in the first few months of 2020, you can see that more links are congested and then it sort of goes away again and then maybe there's something interesting right at the very end there we'll go. Look at that now. This is, of course, just looking at every link, but we can of course identify the autonomous system.

P

That's on the other side of every link and nick has to be, and nick and jason have to be a little circumspect, but you know we're probing from outside, so I can name names so so look at the next slide.

P

What I've plotted here is the I, the the interconnection partners that contributed to congestion and the first thing we need to get out of the way. If you look on the right side, there's this huge lump now, let me explain what I'm plotting here. I've organized the data into two week baskets and what I count up now for each interconnected party is the number of congested day links that there are in two weeks.

P

So so, if, if you found a number 14, if you look over on the right hand side there 14 says on the average one link was congested every day, because they're 14 days in two weeks and that huge lump on the right hand, side is is, is akamai and you might say well, what's wrong with akamai.

P

Well, the thing you got to remember in this context is that akamai has well over 50 links the I. The reason I don't know exactly is that we try to identify link aggregate groups, but we don't always necessarily quite get that right.

P

So so, basically, what you're seeing here is that there were two or three links from an akamai server into comcast that were congested and in fact we did confirm that and you notice it went up and then it went down, but if you, uh but but it actually doesn't surprise me that there might be technical reasons why two or three links couldn't be upgraded.

P

Obviously, since they have 40 or 50 there's, not a business issue here, if you look around march, you know 20 2003, you see, you see various things spike up, you see cloud flare, spike up and, and you see uh telenet, and then they go down again, which is exactly what we uh saw from other data now. The important thing is jason. Can tell you exactly about link capacity.

P

I can't measure link capacity, but when he talks about link capacity, we don't know whether the link went from 30 loaded to 70 loaded or it went from 60 loaded to it was trying to get to 110 loaded.

P

What we find here is actual congestion, and when I look at this picture- and I look off into january of september october, up to the beginning of november- you see little spikes there, but in general, what I think you see is exactly what people are saying, which is bits of congestion occurred here and there and people put in extra capacity just for the fun look at the next picture. I won't dwell on this. These pictures are all in the paper we sent in uh yeah go go to the skip over the discussion.

P

I already said that and uh again I went back to 2016 just so you could see uh who was congested back in the period when there were all these disputes and what you see is back, then the congestion were on youtube links, tata links and tt links. So the congestion moves from place to place depending on what the what the business issues are more than the technical issues. uh So there are graphs for the other isps major isps and the paper we sent in uh in general. The story is about the same.

P

You see some congestion occurring in march april timeframe and and then it goes away and in general what people are saying is yeah. We put an extra capacity. So what uh go ahead next slide? I think that's all I had yeah, that's all I say if you want to see some more more pictures go look at the paper there but, as I said for the u.s, they more or less look the same. So I'm done.

B

Thanks a lot, I specifically really like this methodology that you actually look at congestion and not only bandwidth. That's really helpful any questions.

B

Then I'm sure there's more on the chat. Thank you for now. We might pick this up later again sure and let's go to the next talk.

B

Next, we have ricky um looking a little bit closer at the index image to the cloud.

Q

Yep um hi everyone, I'm ricky from cada, so this project we are measuring instead of measuring performance from the edge. We do it another way around to measure performance from the cloud to ss network. Next nice breeze.

Q

So in this project um we, which is one of the nsf uh fund project uh for in particular, for uh setting up um uh measurement for measuring the intake of kovic, actually built a system called claps to measure to set up many vms, mainly in u.s regions, to perform frequent measurement to a number of speed test servers.

Q

We use three types of servers or clear, comcast and mlab ndt servers to measure the throughput in both direction and see how the cloud perform is performing during the pandemic.

Q

We start our measurement in may, so we kind of measure in the aftermath of the pandemic effect and see. How is the performance going while the people start moving again and also some re-lockdown events happen during the pandemic. So here we we are going to show some of our preliminary results that we have right now. So it's nice piece.

Q

So actually we find that the cloud is not as good as we thought. We definitely see evidence of congestion not only to use to small network, but also can find congestion events um in between cloud regions and that big access network. Like calls, we can definitely see a very significant throughput fluctuation, this kind of like a diurnal pattern recurring every day that we have the throughput jobs in a certain period of time.

Q

This is a download direction, which means it is from the isp to the cloud which probably caused by impact to like video conferencing. Application like uh like google could be like google hangout um in this uh in this traffic direction. ah So next one please.

Q

For small isp, sometimes it could be even worse. So these two, these two data shows from the updater of from aw as north carol regions to two small regional isp, one enough to surf north colorado area and the other one served up south colorado area and not the other one served north texas area. So those surface area are more suburban area and we can see we can see that the upload direction means that from the cloud to the isp is mainly served like video content and also download activities.

Q

um Definitely we can see suffering uh bandwidth, degradation and user is kind of suffering during like evening time and the other one. Actually, we serve a persistent low throughput except those bit like off-peak hours during this week.

Q

So next one peace.

Q

So, apart from daily variation, we also have a high level view of a data between months. So we pick the first week of each month to see the ever the median download throughput between some certain cloud region and isb, and we can find some downtrend in a certain car regions between car regions and certain ss isps like the left.

Q

One shows from the spectrum to azure central asian central one region, and we can see that the fuel fruit from uh like six six seven uh spectrum surfers actually decrease over time, and we also can see like a slight decrease from comcast server to aws ohio region between three months over three months time.

Q

So next one.

Q

So apart from access network, we also take a look on the educational lab network, because everyone move from like in-person learning to distant learning, also observe a downtrend in a certain area of cloud regions. We we have recovered three different universities in one of the gcp regions, which east east coast. um Although some, we saw some evidence that there's less user um in the in the campus, uh but we find that their food could help, show some downtrend during this read these three months.

Q

um We investigate like try to investigate on whether there's a rare change or they may have used more distant learning too. That may cause degradation next one.

Q

So to summary, actually uh we we use of throughput measurement or speed test measurement running from the cloud to different isp, and we find that actually the cloud we're still suffering some congestion events show. We showed some evidence of.

Q

Anti-Anti-Degradation because of like unusual last mile link is meant because of the performance between two big parties in network and we we observed some time trending throughput in the post lockdown period. um We have data until october and we will keep working on the analysis and see where the downtrend continues or is coming back something later and we make the data properly available. um It is not. I will still working on it now and we'll put it online later so in our next step.

Q

I'll be also happy to work with other people on corporating different data sets and see their finding csm findings with our data source, and we may have physically and diffusion tests in the later in the future.

Q

B

Thank you. Thank you very much. I think nick just answered my question in the chat that I wanted to ask now, because my question would have been have you looked at latency as well or only at throughput.

B

And nick said, and they have a similar paper where they provide also some insights into latency and they also see effects on latency. Can you confirm that or did you not look at that.

Q

Oh no, we have not looked into that.

B

Okay, yeah, but definitely um I think one piece of the puzzle to look at. Thank you. um Are there any further.

B

Questions: okay: um we have two more um brief presentations. We now um go into the part where we talk about last mile congestion. We get some data from japan and then we also get some more general data from ripe atlas also took looking at latency. Actually, um so next we have kenjiro.

B

Yes, hopefully, yes, let me find your slides.

R

I believe those ones yeah that looks good.

S

Yeah this one, okay, I'm going to talk about in a broadband traffic in japan, which is not so different from the you know, the other country. So next, please.

S

So this graph shows the you know the iijs broodman traffic since 2015., so the green line is the download so.

S

Until you know, last year we had a steady growth and in this year uh we have a search. So we had, you know the uh the first wave in february and then uh in april april, uh made time frame. We had state of emergency and then we had second wave in in the summer.

S

So we can see in the graph. You know the the increase of the traffic during the state of emergency and it goes down and it goes up again.

S

So the uh the, but if we look at the you, know overall trend, it seems not that far from the you know, original growth curve, so an express.

S

So these are details from february to june, and these are summarized in the next slide next slide, please.

S

So what we observed is, as uh oliver said, that the uh the first talk uh the weekday traffic became similar to weekend traffic and a state of emergency.

S

So the left foot is a weekday and the right produce weekend and the blue one is before, and red one is under the state of emergency and the green one is after the state of emergency, so the for weekday traffic. So during the state of emergency, the daytime traffic increased a lot and after the state of the emergency, it goes down again and the during the state of emergency. The traffic looked very similar to the weekend as people stayed home and next please.

S

And this one looks at the distribution of the traffic usage pi user. So the basically approach are the upper properties for upload and the lower proteins for download.

S

So the is the evolution of the distribution for the last 10 years.

S

So if we look at the longer term trend, so this year's increase- and this is measured in june- so uh after the state of emergency, so the the every. If we look at the evolution this year, the evolution is bigger than previous one, but it's also not so far from the previous trend.

S

If we look at the in the semilog scale, so the download volume, for example, uh evolved for the last 10 years by the by two orders of magnitude.

S

So the yet so you know 10 to the ninth- is a one gig one gigabytes per day. So this is, you know the. If we look at the mode uh people download three gigabytes per day.

S

Okay next slide, please.

S

So, in summary, so the overall macro level impact to the broadband traffic is not so big, so the traffic increased during the work hours, but it was still under the capacity and after the state of the emergency was lifted. It's slowly coming back to the original growth cup, and we have many reports in the talking about the minor issues, uh mostly with legacy infrastructure and the equipment.

S

So the in japan, the broadband boom started 20 years ago. So we do have legacy equipment like pboe. Alumni will talk about in the next talk and vdsl in apartments and old wi-fi at home, and we had some good luck uh because we didn't have any. You know real broke down in japan and we are still uh I'll start hearing me. Okay and uh we are doing okay so far, and also uh the olympic game was scheduled this summer this past summer, so the many infra capacity upgrade was scheduled early this year.

S

So the so many upgrades are, you know already scheduled this year and also for preparing olympics many big companies industries push for remote work, so people are prepared. So these are observations from iit services, but we have similar reports from other japanese isps. That's all.

B

Yeah- and here I would suggest we um just like go on to the next talk immediately and dive a little bit deeper into the latency measurements. Yes, and now I need to find that one this one.

T

Can you hear me.

B

Yes, we can hear you, but not sure what happened to your slides uh this one.

T

Would be very quick there.

R

T

Okay, so, uh and that's great, because kenjiro brought all the background um so in this work we're looking at the last my latency and oh, you can go to the next slide and so what we've done is we took a ripe at last stress route and we tried to estimate the queuing delay for the last mile in the truss route. So we.

T

We just look for what is the last private ipa we see and the first public ap and we're going to say this is the last night. Then we monitor those delays over time and for a lot of ascs that host ripe atlas, probes.

T

We are not really interested in in the delay for just like one prop, so we try to aggregate those results per yes, hoping to see if a common pattern could emerge- and this is what is shown in those figures here. So the uh x-axis here is the time of the day. It's all utc time, because those are in different. Those networks are in different countries and the y-axis is the aggregated queuing delay and the colors of the curve shows a different measurement period, so the blue, orange and green are in 2019 before kevin 19.

T

and the red one is uh during april 2020. So when a love network were impacted, so I guess it's not a big surprise. We see increase in latency for the last mile.

T

What all this network have in common is the we see increased delay during uh during the day, so all of these figures have a peak in the evening, so the top left one is in germany, so the utc is is smallest uh the time in germany, the bottom one are in north america, so the peak is on the left of the figure and the the top right.

T

One is in japan, um so they usually all have like a peak in the evening, but uh during covenanting we can see the the latency increase uh during that time.

T

I would say the two bottom one just have like a slight increase: the the most significant one here is in japan, so the top right, one where we seen a very severe last mile congestion um and uh in in our paper. So we uh just restate what uh angelo said like there is some problem with the legacy infrastructure. So using those results, we could uh point out that the problem we are coming from there and so next slide in in our paper. We try to uh correlate those results, those latency measure with uh throughput.

T

So in the paper we use cdn data, but here I have redone some of the experiments we're using mlab and dt data, and we found that the throughput drops are very correlated with the deal increase within so on the left. You have the throughput for ntt ocn here the measurement paired a bit different.

T

um This is to match what kenjiro said like in. In japan, we had a state of emergency, so the blue one is before that the orange one is during the state of emergency and the green one is after the uh the first wave.

T

So we can see again in the throughput that we had throughput drop during that time compared to what we had at the beginning of the year and uh like kanjiro said again, we see that after the first wave we have a bit higher shroop, and our guess is that this is uh thanks to the upgrade that was planned for the tokyo olympic and next slide. So that's that's all the the result I have here so we found overall, um the the number of of congested network we see using atlas is, in fact quite small.

T

There is we've observed only about 10 percent of the esn. We monitor with atlas that have usually have less mile congestion problem before queen 19 and this increased by 55 percent.

T

um So in japan we did a bit more in the result and and found that the the one of the codes was the legacy network usually accessed via pppoe.

T

We have other results I haven't shown here, but the uh using mlab or the cdn data we found that the mobile network is unaffected and that those two urls so the first one is the the paper with some more results and the second one is uh actually it's completely unrelated.

T

um It's it's some um measurement, we've done where we look a bit further in the tress routes. We are not looking only at the the last mile, but we are looking at uh delay between two networks.

T

uh So it's a bit more similar to what david clarke have presented before that's a dashboard we've done during the uh hackers on the right packaging, but well that brings something else for our discussion. If people want to discuss that, thank you. That's all. I have.

B

Thanks a lot um yeah, um we can check if we have any questions. There was like one question about: what's the what the unit of the x a y axis axis on the peris.

T

Oh yeah slide yeah, that's in yeah and yeah. Those values are very small, um the well in the paper. We show that uh because it's aggregated across like all the probes, um so it's and it's the median. So I didn't explain very well, but uh it's it's the median value we observe across all the probes in that network. So that means that um about 50 of the probes have more than that value.

T

um It's a bit hard to plot, like it's better to see the whole uh distribution.

B

T

B

I mean there's more data in the in the paper.

I

B

um So, basically, uh that's the end of the presentations and we only have little time left, but at least I would like to thank everybody who presented, because even so, I read the papers um seeing everything together, putting like the data somehow in context, I think, was very helpful. My big takeaway really is um that it's not enough to just like look at one angle. We need, like all the different data to actually understand. What's going on.

B

um The other point is that I I think we realized during the session that we actually do need more data, especially about quality of service or quality of experience for specific users for specific applications for specific service providers. That's all different.

B

There might be a regional differences that we didn't have enough data on and uh with that I will just quickly um turn back to my own slide set um and we can stare at this light for one more minute. um I think um I already kind of- or we already detected, that we potentially need more measurements, uh but we need some awesome more digging, um but I want to open the the question to everybody. You know: where do you see problems? Where should we look deeper and what else should we measure what's missing?

B

uh What's actually normal? What's, what's uh can do we understand? Well, what has changed? Do we understand well what the requirements from the society are, what the needed quality of preference quality of experience is, um what are the differences and then also um maybe this is then uh sliding that a little bit into the session tomorrow. We should think about what one, what what we want to discuss tomorrow and did we actually manage it well, or was it just luck, so the floor's open.

B

We have brian in the queue.

E

Cameron, hello, can you hear me.

B

And I should yeah, we can hear you. I should say the session is on wednesday and not tomorrow. So.

E

That's the first point um yeah, so I think one of the things that that struck me um from looking at all of these talks, which were great thank you to all the creators by the way, um was there a lot of sort of of common bits right, like so hey the internet kind of worked, at least.

E

If you look at the core um and a lot of expected bits like I guess, if you had said and looked at the at the japanese home broadband infrastructure two years ago and said by the way we're going to need to um uh you know triple the load on this network in three weeks. um I think. Actually, if you look at the investments that were made for the olympics, you can see that people knew that. That's where the problem was.

E

um The one thing that that kind of jumped out at me from from some of the talks about the the capacity ads is that um it was actually possible to add um capacity at. Like I mean there was what comcast had one that went up by a factor of 40 um during the the pandemic, but it was possible to actually do that.

E

I'd like to so, I think, jason made a comment that you know the bureaucracy kind of got out of the way, which is one thing but but it'd be interesting to look and see if there are any remaining gaps in how fast it takes out to roll out the physical infrastructure, and it seems like at the interconnection points, there isn't um any remaining um uh sort of barrier to physical infrastructure, but but that might be something that we can't actually answer with measurement is, like you know, what would the next big thing we'd have to do in terms of like capacity expansion be, and is that a technical, blocker or a non-technical blogger?

E

um I guess yeah.

B

As you're saying this, my question would also be like if, if we need another like 40 times capacity, is it still possible now or are we now it wouldn't matter.

E

Right because I mean like you can see that like over time, the previous stuff was clearly business limited right, because if you can, if you can do it twice as fast or 39 times as fast as you were before. Well then you just didn't need it before, um but yeah I'd like to take a little bit more into that um again. That's probably also a tomorrow question, but uh thanks yeah.

B

I mean like actually my assumption was that a lot of people just like took the the expansion that was planned somehow and did it earlier right. So there was already some kind of planning and there was already maybe um right things aligned right. It's not sure, that's possible again, very quickly.

E

B

That's also something we should understand tomorrow, maybe but.

E

There's one of the talks that said you know hey, we, we carry 30 additional capacity for 30 year on year increase and oh, we also needed 30 additional capacities that that feels like luck to me, but but it'd be good to sort of like take a more reasoned approach to that yeah.

B

Definitely tomorrow uh we had somebody else on the cube, but now I can't see it anymore.

F

B

U

First, I was wondering how well calibrated the data is about our background organic growth um back when it was really active in the ietf. It was typically 13 months doubling time and now I'm hearing 30 per year. um How well, is that? How stable is that? How well known is it?

U

How well documented is it, um and one of the approaches that was, I know, was used in the past when there's big surges was taking the point of view that it, an unexpected load, is sort of equivalent to a failure, and you have to reserve at least double capacity in order to be able to deal with failures and being allowed to eat into that reserve to deal with crises helps a lot.

B

Yeah so like do we actually still have this reserve right for failures. That's another question.

B

um Next, in the queue is yari.

I

Yes, we're trying to answer a couple of the questions or propose some partial answers, at least so I I think you're it is a conclusion that we need more measurements. I need to look at things like holistically, not just from one particular angle. I think that's exactly right and some areas where we, I don't think we have enough, is uh I, like the quality of experience, uh also application based measurements.

I

How much do we know about? uh I don't know conferencing platforms and such, and so um everything that we saw today was was really great and amazed that we have had those numbers but uh would be even better to have more um by the way I uh in my paper, I had some some material on quality of experience like ericsson had asked people around the world like how did they feel about the situation and so on so I'll post a link to that, but uh that that's a sort of a feeble attempt or simple thing.

I

I think we need much much more. um The other thing that, like you, you're asking this about uh luck or relevance, uh it's kind of like what is your expectation on the real. This kind of you wanna handle like can we do a pandemic and uh I don't know material heat that takes down 10 of the current infrastructure.

I

You know in some sense we were lucky because we did like it was a big big change in the society, but you know everything was sort of retained and nothing sort of initially broke down and so on so, but do you want to build for that? Do you want to pay for that?

I

That that I think, is the question mark, but one of the things that the ericsson research actually did observe is that, at least for the moment the consumers are really interested in uh reliability and resilience of networking that they they use or enjoy or purchase. So so, at the moment at everybody is interested in okay. You're gonna be able to deliver this thing for me going forward. Hopefully that can be oh. That sentiment can stay for a longer period of.

B

Time, yes, um totally agree. um Let's take oliver hoffeld's question or comment.

G

So I actually want to make a comment that goes a little bit into brian's direction. So to me the question is like on what level of granularity are we talking most of the results? Also, the results that we have shared are at the granularity of we take the isp or the ixp as as as a whole, and this level can be, or this picture can completely change. If you look into individual users, individual peering links and so on, and of course we have seen these cases where you know a smaller.

G

Smaller enterprises were kind of like not really planning for this pandemic to happen, and then, if they send all of their employees to home office, then um of course the um the small pipe that they have purchased. um It went from like a medium utilization to like a really high utilization, so, um of course they can have like a this can really have like an effect on them.

G

But if you look at the, if you take, do a statistical analysis on like the operator or the network as a whole, with like a large set of peering things that we see like um a kind of a different uh perspective for the majority of the cases, so I guess that's like one thing to discuss for the wednesday session is like the level of granularity um and how, like uh you know, how deep should we dig into like um individual users that really suffered um individual links that went to full utilization?

G

So these kind of things.

B

Yeah, definitely I mean like that was also. One thing is actually understanding what the problems are, understanding all the details, understanding um who suffers, but then the other thing is actually the big picture right is it? Is it the architecture? Is it the technology where the limitations are or is it a business question? um I think that's also a good point where we hopefully come to some insights. At the end of the workshop, um I still have uh colin in the key yeah.

C

Basically, out of time.

B

Okay, perfect: I.

C

Don't want to comment on this. I just want to tell people if you would like a less than you know, seven minutes or less lightning talk slot on the on wednesday to talk about one of the operational issues um send slides our request, either to me fluffy at cisco.com, or to I'll put my email in the chat or or to the program committee list, um and let's uh see if we can have some more time for discussion of this and that stuff on wednesday.

B

Yeah and I have a request, I would like to ask people to actually go to the note-taking pad that we have and and add comments there on things that.

F

B

Think we should measure we should do we should care about. So we don't all forget about it immediately, and we can look at that on wednesday, again um yeah and with that we're at the end of this session, um thanks everybody for joining thanks everybody for presenting thanks everybody for comments. We didn't have as much time for discussions as I was hoping for, but it was also a little bit expected to be honest, um but I hope you all enjoyed the chat session.

B

I did a lot and I think we're now well prepared to have discussions on wednesday, any last words from anybody else from the program committee.

B

Okay, then same time on wednesday same link. Thank you. Everybody and goodbye.

K

Thank you, miria and all the organizers.

R