Internet Engineering Task Force 106, 17 Nov 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: IETF106-IEPG-20191117-1000

Description

IEPG meeting session at IETF106
2019/11/17 1000

https://datatracker.ietf.org/meeting/106/proceedings/

A

B

Yeah hold on he's in the wrong.

C

B

Can I have a mic just by pushing all of the keys at once, I think right. Oh.

B

All right, well, okay, anyway, I can only do this one hand okay good morning. This is the IEP G at IETF 106 in Singapore. If you're not supposed to be in Singapore I'm. Sorry, you took the long flight if you're not supposed to be the IEP G and you're supposed to be at some other meeting, then you're in the wrong room. You should consider moving or you could stay, because it will probably be fun for those people that are presenting there's a microphone. Please don't please speak into the microphone.

B

There's a nice thing, we'll show you the pictures that are up there, so you don't have to do this. Okay, there's also a pointer.

B

Doesn't the clicky thing doesn't work? My laptop is misbehaving or something so you could at least point. We have five presenters five presentations. First up, Carlos, oh by the way, my name is Chris and Warren's. Not here he's busy with something else, hello, everybody.

B

D

Kick man you're.

D

Talking everybody actually thanks Chris for the time this is. This is just a short introduction. A short I would say: marketing presentation I want to introduce you a new rpki validator.

D

We call it fort because it made sense at the time and it says something about the stronger internet or something, but we just call it fort and we try to not think very much about what it means. So, there's a background to this there's a fort project and the four validator, the full project started as a initiative to promote route and security in the region. There's there was a call from the open, Technology Fund, which is a fund that finances technology projects having to do with freedom on the internet.

D

What I think it's something we all sympathize with and back at the time we were toying with the idea of developing a new replica, a validator, and we were in talk in discussions with the great guys from Nick Mexico about developing something and at some point, everything came together and we decided to move forward with this project, which has two components: it has a validator and it has something we call the tracking tool, which is a hopes to be when it's ready system or mechanism to keep track of routing incidents in the region and that it sort that is I mean the I would say social part of the open, Technology Fund project, which is that that is the part that interests them the most.

D

The idea is that money, the tracking tool, will provide information about writing incidents for non-technical people under the hope that it will help people realize whether their country government is PE. Whatever is actually you know, playing tricks on them next, please. So what I'm going to talk about today is the Arabic. A validator with a said is one of the two parts of the project. Next, please why another our PKA validator, you know lately it seems to be that everybody is developing a new epoch, a validator.

D

There is at least four projects that I can remember about, and I think it's good having diversity is good. Having programming language is. Diversity is good, and it's also worth mentioning that back at the time when we decided to move forward with this project. Well, the options about RPK validators were rather limited, so it may not send a lot of sense at the time to create a new one sex. Please so short description about. Fourth, what is implementing right now is repository validation and the RTL server.

D

There is no GUI, there's no web there's, no nothing. So it's very silly something you run, and hopefully you don't get anything on the console right, so it's rather an exciting. In that sense, it's written in C, which is you think, it's great I mean I, didn't know, people still read strong programming in C, I mean people, I thought everybody had moved to go and rust and Python, and you say that it compiles cleanly and runs on several Linux and BSD platforms. The license MIT so basically is almost.

D

You can do whatever you want with it and the road map, as it's done is. We are now at version. 1, 1 and the guys from Nick Mexico are planning a version 1.2 for the first quarter of 2020. The code is in github that we have the URL next, please, but it does. What does it support right now?

D

You can check the details on that URL there's a very nice table with all the LLC's and the percentage of implementation, but basically some of the things that I consider interesting in is validated it that support this learn the support they say some files from the RFC it supports validation, reconsider validates, goes faster. It supports policy qualifiers and has some support for BGP SEC objects. What is missing, notably, is a DB support and the HTTPS tiles hopefully coming in version 1.2 in a few months. Next, please, it has reasonable high test coverage.

D

This is some of the platforms where the tests have been run. I haven't tried, compiling it. Obviously, but I guess it's the same thing as always: it's very easy to install it on Divya no.12. Next one, please, the usual routine right. You install some dependencies. You unpack the Col configure, make make install and you're running I created a couple of docker images. You can try if you want next please, and there are two ways of running it. One is what the unique only documentation is called standalone.

D

What is basically, it will run validate a repository or a set of repositories, produce some CSV output and exit, no fancy web, no graphics, nor anything and the other way of running it is with a survey with the LTL survey enabled, and it will just stay there and the only thing you'll hear from it is where it finds errors and or some inconsistencies in reporters in the repositories. These are the basic options it has tons of options.

D

Some of them are quite interesting. For example, you can tune a little bit how strict the validation it performs it performs, which you think is kind of nice, because there are still some I would say, divergences in the implementations of the repositories. Next, please a few tests they run. This is memory consumption running on a standalone mode. These are the M dark. Blue is a penis repository, light blue slag neck and the sort of yellowish greenish, something color.

D

There is a Phoenix, they repository it's quite low, it's quite efficient in memory usage which is kind of nice and it's I would say natural by proud of it's been written in plain C text. Well, thank you. If you like the idea of having a new tool in your tool chest, please send us your box I'm, not promising. This is the great thing, the greatest thing since sliced bread nor the disciple free is probably full of bugs.

D

That is what we are mostly interested in hearing from you send us your bug reports and, if you liked it, please let us know. Thank you.

B

Questions that might please.

E

Randy Bush our consumer Jay.

E

Randy Bush our Cassandra EJ. um How do I deal with, inter instance, slurm? In other words, I, have a slurm file on my in my cop in San, Diego and I. Want you to run it in your pop in San Francisco? What's the trust model, that's.

D

A completely different discussion, I kind of agree with you, but I, don't know you have something red in your hair. What is it the light?

D

D

D

D

B

All right next up, yeah sure.

B

Tell me why you're.

F

F

Good morning my name is arianah from the VRS today, I'd like to introduce our research work to detect the TMS ik provocation, failure at the TRD server monitoring next, please! So, as you know, DNA SEC validation failure happens frequently, but the reliability of DNA six operation is a key of the nsx deployment, so so that next, please we made a research question: how can we detect validation, failure rapidly and efficiently?

F

Of course, active amazement is an easy way to detect DNA segmentation failure, but it causes high higher loader for more DNA SEC available domains so that we thought about how can we detect the NSX validation failure by the passive measurement? So we focused on the to see the change of curly patterns could be a good indicator at the monitoring of power. I'm, sorry, authoritative, nameservers, Rabia. Next, please so to us, detecting validation, failures with passive measurements.

F

The goal of our research is depicting chant, detecting change of clearly patterns at CCTV servers. At this moment we are the GP operators, so we are seeing JP DNS servers before and after failure when Colin tomahawk is using both octave measurement and passive measurement for the active measurement we are using ripe address for the DNS client and the else worried.

F

Dietas neighbor to the iPad rests approximately 500 at ISPs on edges, and also we prepared local, authoritative, DNS servers which provide both successful DNS, SEC validation and failed as DNS expeditions with different TTL settings and passive measurement. Vantage point is JP DNS servers. Next, please, this figure shows the measurement ababeel.

F

The measurement period for this research is equally 2018. This is also conjunction with the idea happened on April, 20, 80s and then passive measurement about age point is JP t JP, DNS servers and active measurement at the local, authoritative servers at the light bottom light shark is a local, also tilting Sabah provided Saudi Arabia of Jupiter by name and the targeted queries from light. A Taurus probe n is a DNS key and the DES records, and we use around thousand life matters probe and six hundreds of DNS SEC marketers at home.

F

Next, please so before we perform our measurement, we.

F

Search for the Public DNS, because we do not know the concrete behavior of Public DNS and we targeting the body heaters at ISPs on homes so that we sent light probe at was unique, queries to our associative servers and check the source address of the I curries, which paragate parties these robots, sent to Korea to our local service and check. If it is in the public, dns servers IP address or not so that we find, which type a terminal node uses the Public DNS.

F

We exclude the excluded those try Patras probes from our measurement next, please, and we also measure the TTL distribution in signed, JP, DNS, JP Zahn's prior to our measurement. So we can find four categories of the TTL settings for NS and DNS keys. This quadrant shots the classification of the lengths home, NS and DNS key details, so we don't know why this quadrant happens in Japan, but this is very, very typical to some industries next, please! So, according to these TTL settings, we set up the eight zones for this test next, please so.

F

The full of food zones, successful validation and for other force by a very odd validation because of the inconsistency between des and DNS key, and for these settings we use the fixed TTL for DS and a records next, please.

F

So this table shows the result of queries that we observed at the local, authoritative server, that is three laboratory and at UC when DNA segmentation failed them. Overall queries increased in for DNS, SEC related queries, so the left-hand side is a successful validated successful domain names and the right-hand side is a varied zones.

F

So you can see some several multi prides of the each resource record queries.

F

So next, please- and this is a result of queries monitored at the Jaypee dns. As you see, the dns expiration failure happens, DNS SEC related records, Khali's increased, but the attenuation is lower than the authoritative at local servers. So next, please, but you can see.

F

Dns key is better indicators because, especially for the longer TTL settings for okay 23 at the left hand, side and okay 43. At the left hand, side is a longer TTL and see the mg 23 on ng 43.

F

It is also the shorter detail, so the marriage price numbers is higher than the shorter TTL of DNS key it's about seven times, I Adam the successful a so so that we thought that the the increase meant of DNS key is in the could be our indicator next, please.

F

So this table shows the comparison with local. Also rotated that is heard rate was a third rail is rabbit and Jade JP ccTLD servers. So you can see the huge nation of of the bubble queries. So if we can see the local circuit of services, we see much more queries, but we still see some increase meant on the TLD servers, but it is suffering to say that the validation failure in minor domains are likely to difficult to detect next, please so.

F

The collision of our research is we conducted DNS, passive and active measurement to know clearly behaviors in DNS, SEC failure and DNS queries, increased a-team, ASIC, validation, failure, the automation of queries in DNS hierarchy is different as expected, but do you see the MSC difference is still a good metric to detect validation.

F

Failure next, please, so that we still continuing this work because we have still some unknown issues, so that was one of the issues difference in public and as a reservist, Public DNS and as a reservist, we will have different behaviors, the Public DNS of a very large and very good operation and the quantify of ccTLD that is top level or second treble was horribly attenuation, because I am a TD operator. So I am watching over monitoring.

F

The tlg servers mainly and TT react in various status and we had to have some longitudinal analysis of a part of GP DNS service traces.

F

We want to have a victim cause in real time detection method of theory disturb aside, because we are monitoring to two out of seven JP DNS servers almost real time, so that we can find which I used. Those two servers, as, on the vantage point, to find detect of DNS SEC failure. So I'm very welcome here any comments regarding to research work. Thank you.

B

G

Hi everyone, my name, is Ray Adams before I start on this topic, a disclaimer I work for ICANN and all I. All opinions expressed here are my own, not necessarily I cans. Here's! What I make that clear. I'm gonna talk about private space names. A lot of you are thinking now not again. Well again, this private name stuff. We talked about him to know we talked about dolls. Can we just move on well next slide.

H

G

You go, thank you.

G

So if you look at the solution space currently there are a lot of solutions out there. People suggest that you register your own name. Well, that's not really private! Before I go on. Let me explain a little quickly what I mean with private use. It's a it's a it's a name that you can use on your own private network, media corporation, be it your home network, etc, etc. Another argument, just don't don't don't do it.

G

I've often heard that, but that also really doesn't work used us internal or good, local or dot, alt or dot home, but I'm not really gonna talk about at home or home dot. Our plan and the problem is, there is no defined way of doing this, and the result is something you see on screen here. These are the results of.

G

Results of I, thi, I teach knives, identify technology, health indicators and, in short, is the stuff that we've observed at the L root or iMRS root server at the top, the little 13 and names that do not exist, that's being queried for create for, and you can see it all over the place. There's nothing wrong with these strings I'm, just saying that we observe that these things leak and they do not exist in the root zone and next slide. Please so I'm going to talk about a proposed solutions, place constraints.

G

So now that we understand the problem space here are some constraints in the solution space. We need a simple and concise BCP, the BCP that basically says. If you want to do this, this is your way of doing it and we need to have a label in there that has no semantic meaning what I mean with semantic meaning is like it shouldn't have a literal, meaning like internal or private or alt, or local or home, or anything for that matter.

G

Because the moment you choose private or internal folks have an argument when they say well that doesn't really cover my home network or my private news network and remember the X, my headers, that I talked about before right or in language text. You have X, there's enough semantic headers, they have no literal meaning, so we need something that has no semantic meaning, and we also another problem with these restraints. Are they to angular funny?

G

They don't work in German or in Dutch, and it also smart to choose a label that will never never be delegated and we also need to choose a label with label. I mean a top-level domain label, choose a label that may not require our C 67 61, this the exceptionally reserved list of top-level domains. So maybe we can just use a two character: ASCII domain next like this.

G

This is a potential political minefield to leather askew domains. So you can. You can kill me later in the Mike line, our C 1591 talks about domain name, system, structure and delegation. There's this infamous section of famous section that talks about country codes. The Ayana is nothing the business of deciding. What is what is not a country, and the next section talks about the ISO 3166 list and keep in mind that the next section, and actually the entire document does not talk about countries at all.

G

It talks about country names and it talks about entities, but most countries- and so I saw three one six six lists: they are basically the two character: top-level domains. That's right. It took two character country codes and if the next slide please and lists them all, I can go through them right now, it's a a a b, a c and I've counted them, one by one, they're, 676, I'm, sure, I'm. Sure, there's some mathematical tricks that you can use in order to do this faster, but there are 676 next slide.

G

Please and what he is: oh three one six six folks have done is they've categorized them. All of these individual elements fall into a category and I'm a little bit of color blind, so I hope, I'm gonna get this right. They can have here next slide. Yeah on the top left, you see a colored squirm. This is a a a color square. It basically means it's user assigns and this one's going to talk about in a minute. I'm gonna go quickly through the rest. You have the white background.

G

It's actually feasible, I know I'm, Valerie, perfect, that's visible! So the white background, like a bee, those haven't been assigned, yet they may be assigned in the future, but they haven't been assigned yet so the categories and the science and then we have exceptionally reserved like UK this yellow and EU.

G

This yellow small sidestep here I used to work for a company names and Namie nets and I used to work it for nine years and I knew UK is not gb, but I never knew the following, and this is only what I discovered in the last couple of months: ISO 3166 and, as you know, and as UK is exceptionally reserved and in the context of ISO 3166, it means that UK is not a country. Another country name is not assigned because ISO 3166 is assigning official country names to a two-letter code right.

G

This UK is not a science to the United Kingdom. Why not is because the ISO 3166 decided long ago to avoid generic terms like unites or Kingdom or Republic, if you're gonna stick with all these generic names like United, Kingdom and Republic, you're gonna, soon run out of huge case and ours to use in your to letter, doing so for the United Kingdom of Great Britain and Northern Ireland, the first distinguishing string is Great Britain and the official encoding for the country code.

G

For therefore, a country named United Kingdom, Great, Britain Northern Ireland is GB and not UK. So why is this exceptionally reserved? Sorry, yes, that's an exception.

G

Yes, so if you look at Russia, you know, let's move on these. The other codes is the green stuff that's officially assigned, and then you have some transitional e-reserves and in in in determine a division. I'm not gonna, go into that right now, but it's just for me. It's an interesting thing. Next slide, please. So these are the codes that I want to talk about the user design stuff a a qm2 Q Jets XA 2 XS jet set next light.

G

So what does the ISO standards say about these things about your construction of the alpha-2 codes? The ISO 3166 standard uses a combinations in the range, a b, TQ l, ra, t wz, and why eh-2-zed, why so d, iso standard talks about dead range? Of course, what did they say about the other 42, these user assigned elements?

G

They are not used in the standards they are not used, they will never be assigned, they will never be reserved, they are not used, and this is important in the context of what I want to talk about in a minute. So the standard goes on in 8.1 about special provisions. You should sometimes need to extend or to alter the use of country code elements for special purposes and in an 8, 1 2.

G

It says if users need code elements to present country names of countries, country names not included in this part, these series of letters, AAA, qmq, jet, etc, are available. This is guidance, it doesn't mean you can't use this for anything else and I'm gonna. Give you a few examples of how other standards are using these things. Next likely.

G

So are these things used to elsewhere? These user sign codes is o3 901 in the international standard, recording code and I'm not going to pretend I know anything about these things. I've just found them, so it reserved said set for direct registrants, independent of any country. Now the next one is quite interesting, mi. So for tu-16 codes for the representation of currencies, it reserves XA to exit for transactions and precious precious metals as their country. Independence. I'm gonna talk about this a little bit.

G

Us dollar, the official code for your Zola is USD and the US in USD comes from the three one: six six tenders, GBP, remember: GB UK! You don't have UK P if GBP the GB GB, please stems from the ISO three one. Six extend it. So, what's all this XA stuff in official standards, gold can be used. Sorry, nothing official standards in general gold can be used to pay for stuff right, its currency, in transactions, etc, etc.

G

There's no official code for that is X a you silver x, AG Bitcoin I'm, not kidding XB t, so these are used. These private space names this em. Sorry, these user sign names are used in all kinds of standards: iso 6-1 6-2 6-2 about excess for security, clear to Euro, clear, clear stream, etc away from the ISO standards. Next slide: please you have ICAO International Civil Aviation Organization, it reserves LSATs for UN travel documents and WIPO WIPO.

G

The worth is lecture, Property Organization uses a whole range of these private space and things, and it uses not for countries but basically for their own offices. So if you have a peasant's or an application from a certain con, you can use a country code, but if it goes through a certain Patent Office, you can actually use the code for that patent on this closer to home, to can't form the certificate authority in browser form, it reserves xx to signify location, not covered by ISO 3166 and even closer to her closer to home and RFC.

G

Five. Six, four, six well known to many of you text for arson identifying languages. This is when you use action, these language, taxi your browser, etc, cetera next slide. This is the text lifted from the standard. From that sorry from the BCP, it says, for example, the region subtext, a a set set and dosing ranges, etc, etc. You recognize these from before. They can be used to form a language, language stack and here's an example on the screen. So next slide please.

G

So, in conclusion, we now we now have what we I, because I speak for myself have shown you that alpha-2 user assigned codes, these two character codes. They are used as intended in various standards and therefore these codes will never be ISO, assigns or reserves.

G

It says so in the standard that implies, if I, follow, RC, 1591 and price, that these codes will never be delegated and since they never be delegated, we might may not and I say, may happy to do it if that's needed.

G

We may not need these to go through RC 67 61, where you, where you reserved them, such as the onion, etc, etc, and the cool thing just like these X Montag's right using these two character costs have no semantic meaning, which is a good thing next, like so I'm just gonna use this as a suggestion. I don't want to paint device yet too much right to use that said. I'm gonna use that zet as an example going forward.

G

Luckily, I only have one slide left so, but I'm not suggesting that we should use this now, I'm just suggesting that that is available for this purpose and next slide. So what about D in a sec? Well, we all know that Zed Zed doesn't actually exist and your validator will validate that. That said, that doesn't exist because it doesn't validate, can prove this. That's a good thing! I, don't want to be lied to right, at least not without approval.

G

Now some have suggested that Zed says or internal you should be delegated in the route and signs now I disagree if I as a validator want to be lied to I want to make that explicit. I want to have to configure it that people can lie to me. So does the internal both odds, don't onion or what not? If I want to allow that in my in my validator space, then I want to make it explicit.

G

I want to have a truss tank for that, a negative trust anchor now stubby works sure that bind works for that it all works, basically, so an even better if I wanted resigns his private space. I can now do that with a positive trust anchor as opposed to a negative trust anchor anyway. This is my my rent. My my little talk about using private space names. I will take this to DNA shops on on Thursday and I will bore you all once again with this stuff.

G

If you have any questions, let me know I'm available all day all week for discussions on this.

D

Just a quick comment: thank you for this and I think it makes a lot of sense. I've been using dot XY for a number of years. I didn't do the detailing research I, just read the Wikipedia article on our own I guess so: 3166 I, the other proposals. I mean dot. Internal is nice, but it's just too long I mean it's feels really unnatural to pinger. You know, TV, you know the ping TV dot intimal feels unnatural. It's people are not going to use that. So thank you very much and uh alright. We worry thank.

I

B

Nobody else, alright, thank you.

B

Job Snyder's,.

J

Good morning I a PG, my name is Jobe Snyder's I work for NCT and we're having random slides.

J

Next slide, please I, want to give a bit of a recap on recent rpki deployments highlights a regional internet registry policy updates that is of interest to this group, perhaps share some updates on an open source project and finally, conclude with an update about the best operating system on the planet.

J

Next slide, please so I have a server somewhere and with a bunch of people we're using that server to map outs, rpki deployments on the internet using a fairly simple method. We have two prefixes 1 as an RPG, I, invalids, prefix, the other is pellets and from those prefixes we sent ICMP messages to every ipv4 address on the internet and we compare which of the source addresses, gets replies.

J

Now in this image you see, bloom is IP. Four addresses that responded to both the valid and invalid source address. This means they are not performing. Rpki based, BGP, origin, validation and yellow is only responding to the valid source address, so they are in fact doing origin validation or a network between that server, and that IP address is doing origin validation. So this is a snapshot from August next place a month later next place and.

J

What you can see here is that rpki deployment is rapidly picking up. The largest networks on the planets are deploying it's. The smallest networks on the planets are deploying its route. Surfer operators are doing it. So that's very cool developments.

J

Next, please, now a update on rpki in context of a re are in the right community. Roughly a year ago, the IRR database was split into two parts, almost called ripe and the other is called ripe. Numb off and the splits occurred, because the ir objects in the ripe source were authenticated against the wishes of the resource holder, but there also was a bunch of data in the right eye arm.

J

That was never in fact checked against the wishes of the resource holder, so we call this non-operative and that nan afford if datasets cannot just be deleted, because we don't really know which parts of that datasets are still useful, which parts serve a purpose and which parts are detrimental to operations.

J

So the thinking was what, if we use this original addition procedure and apply it to IRR objects. In other words, if an IRR object is in conflict, we've published harpy guy are ways we can conclude that the ir object is describing a state that should be rejected and therefore this ir object can be deleted. Now the community came to consensus on this. The policy was ratified last week and out of the 65,000 ish objects in this data source roughly 900 are in conflict with published, are ways.

J

If you want to inspect this, you can download a small Python scripts, I've wrote and that will show you the data next slides. This is one of such examples, probably some time long before I was even walking this planet, a company created route objects' that describe state that is in conflict with our ways that entity published an entity did not have any way to delete these IR objects in the right database.

J

Until now, because now we can point at the ARCA carry our ways which signified that objects like these should not exist.

J

Next, please related to the policy that will be implemented in the right database is a effort called IR, Rd and IR D is becoming smart middleware between your prefix filter generation, clients and the various data sources that exists out there in the world next slide. Please.

J

The reason I already for exists is that legacy IR D was no longer extensible. There were some reliability issues with this demon, which meant that we just had to restart it every week, but the code was complicated, which meant that we could not add new features, specifically new security features and one of the cool things that is coming to IR D is a mechanism similar to what is happening in the ripe database.

J

The IR D for demon in its next release will have functionality to consume rpki data to suppress or deletes conflicting IR information, and this will happen on two levels, both in context of the IRD demon being a furtive quote-unquote supportive database and on the mirroring level, when this demon consumes data from other databases over nrt m. In both halves into this into the cache of this demon, origin validation will be applied, thereby significantly cleaning up the potential damage that the irr can bring next slide.

J

Please, the beauty of this mechanism is that it not only helps us clean up the past route objects that have been created in the last 20 years.

J

It provides us operators with a industry-standard mechanism to get rid of old route objects, but it also helps protect us going forward, because the moment you put in our Picaro us these demons will reject ir object, updates that are in conflict with the published arrays. So publishing our ways becomes more valuable, so to speak, because it helps you clean up things, but it also helps you protect going forward.

J

The timeline for these things is roughly spring. 2020 I expect the various evolved parties will provide more updates, as these timelines become more firm, but the good news in all of this is that it's not too far away. Next slide, please now onwards to open BSD, open BC is a research project where we experiment with things and amongst those things is RPG, I mean. Nowadays, you have to realize open.

J

Bc is not only very useful to create beautiful, slides, like I, have done here, but it also is a functional network operating system, meaning that if you install a software out of the box, you have a BGP implementation and OSPF implementation, decent MPLS tech, including layer, 3, VPN and pseudo wires and whatnot, and from that context it is very interesting to experiment with the notion of creating a network operating system that out-of-the-box can perform our API based original evasion.

J

So we set out to write a validator called RPG I'm clients, but this was not a trivial efforts.

J

Libre essays ohm, the cryptographic core that is used in open BSD, which is a fork of open SSL, did not have capabilities to handle CMS, so that had to be added to libre SSL and the developers were giving me a lot of for it because it turns out. Cms is not trivial to implement.

J

Furthermore, we had to implement our sync. This project is called open. Our sync and those three components have created a scenario in which we can have original Edition out of the box. Next slide. Here's a small screen capture the beach P demon is running from Chrome I am calling the RPG I clients that outputs a list of the VIPs into a text file format, that's suitable for open, Beach PD, and this allows the Beach PD daemon running on this box to do original edition and show us fellas were invalid States now.

J

My hope is that by implementing our PGI at this level in a network operating system, we inspire more commonly used network operating systems such as cisco, juniper, others to consider the deployment model where routers extruders themselves can perform the origin, validation, the RPK cache validation function in order to do original edition. I think this is one of many deployment scenarios. This is not necessarily the recommendation that will work for everybody, but I think there are some use. Cases specifically in smaller is piece where it could be beneficial to take a page from this approach.

J

Next slide, please. So everything is well with archives being deployed people like it software's out there. We are close to doing things out of the box, but it appears there's just one last outstanding issue, and this is the matter of the Aaron rpki talam.

J

It is a bit of an unfortunate situation that a organization that is supposed to operate in the best interests of its community, specifically its members, that they are not publicly distributing their public crypto keys and I'm, not sure what the immediate solution is other than to keep lobbying and keep applying pressure directly and indirectly to this organization to get them to release their public keys to the public. But this is a measurable issue on the Internet.

J

A double-digit percentage of networks performing RPI region foundation is not using the air Intel, because the air Intel cannot be included in open source distributions, such as open BSD. So we have to keep an eye out for this and whatever security mechanisms we design. Next, we should keep in mind that we should make it hard for people to make the public keys unavailable next slide, please. So, in summary, what I wanted to share with you from this slide deck? Is that rpki Israel? It's here?

J

It's being deployed and a lot of people are using it, so this means that we are now running in a maintenance mode where we have to keep our eyes and ears open towards the end users and the implementers of RPI suffer, and that's all for today, questions comments. Concerns remarks can be either shared through the microphone or emailed to me directly.

K

Oops still doji Telecom.

K

Really go for Luke, okay, still doji telecom I.

K

Usually have to do some nitpicking about the language used.

K

Precise use of language sometimes is actually helpful to avoid confusing people, even if the first use of precise language males may called confuse people who are used to imprecise use the thing, the thing where I would want to do when it picking right now with a perspective attacking your last topic, is you are giving one map about rpki deployment. I would change the headline for that to rpki or Ridge invalidation deployment, for there are.

K

There are many ways to deploy and use rpki, and there is at least one essential thing were a map like this would could be given and would be useful, which is what parts of the address space actually are covered by certificates and roars may be different and for attacking your last topic doing the maps showing are the deployment that is visible for you through the public available tools, as opposed to those two.

K

What is what is available with the secret ones? Could one of the nice things to push stuff I'll? Take it up with the team now.

J

We can render some additional essential associations and.

K

The last and the last word the last to your last words. You were saying.

K

Well, okay, V vTOLs should be public. The IETF defined standards are assuming a signal. Tell that would be public.

L

Hi, this is Andre from I, see I just wanted to say thank you for sponsoring the open source you use in production. I think it says a good good example for the community.

J

In the category of thank yous, it was Internet Society that helps extend libre SSL with CMS functions. Actually, there's tens of companies and individuals evolved in this project to get it going. The list is long and that's pretty awesome.

E

Really push energy Marcus I just want to say thank you.

J

Thank you for the kind words.

B

M

So the score is currently two BGP talks and two DNS talks. So DNS wins, because this is a number DNS talk next slide.

M

So the thing about the DNS I think BGP also shares. This is that these are distributed systems that spend a huge amount of time chattering to themselves.

M

So, in the case of BGP there's an event, you might announce a prefix, there might be a link state up down, and then this sort of ton of updates is ripple throughout the rest of the network and at times trying to understand why your router is kind of having a seizure and what was the original event that is causing. It is sometimes really hard to tell now. That's BGP- and you know, a fair deal of study has gone into that and we still don't understand that at least I don't DNS is about the same.

M

You might think that the DNS is kind of this transactional thing where your browser, your application, is given some kind of name. So it asks a question in the DNS. The DNS goes to hunt for the right, authoritative, server and out pops an answer right. So every question: that's all query that sitting inside the dns should be able to be mapped to some original event that the users driven. Nothing could be further than the truth.

M

The DNS is like artificial intelligence. It just talks to itself and at least 50% of all the queries that we see inside the DNS are. Basically, we have no idea why they're there or what's going on and- and this is something we were looking at when we're looking at an area called aggressive, insect caching, there's this kind of theory that DNS SEC is really really good.

M

But the really hard question is: what is it good for and if you're not allowed to answer Dane, it becomes a really really hard question, because if you can't use it for keying in TLS, what's it good? For, and one of the theories was that well, if it's not good for that so I, why don't we use it to actually change the way negative answers ocation in the DNS, because the easiest way to bring down the DNS or parts thereof in a highly targeted fashion? And this is no secret. It's basically random named attacks.

M

You set up a largely distributed system. You set up random labels across all of your little zombies, and you just hit the authoritative, nameserver Roy says: I'm not allowed to say how it's really easy. You just hit it because the random names ensure that caching doesn't work. All of those queries go to the authoritative servers and if you thority of servers, aren't well defended, they will melt now. What you really wanted was to get the recursive x' to come along and help you so the first time you asked even a random subdomain name.

M

It would go now, I'm, sorry that doesn't exist. The only way you can do that in theory was with signed, domain names and range based answers and whether it's in sick or in sex 3 makes no difference. You can actually manage to deflect these attacks in theory. Now that whole story and the answer is a different presentation. That's not it! That's not what we're talking about today, but what we were talking about and looking at was a really simple thing.

M

What happens when the answer is No, so we present to the user. A DNS name go and resolve this name and it's our zone, and we look at the number of queries and what queries happen when the name does not exist and the really odd thing was, the person only got the name once only once it's inside an online ad control, this the user only exercised that name once and only once. Why are there more queries? Next, so we control the end user and we control the authoritative server. There are random labels flying around.

M

So there is no caching. The thing is not signed as I recall, or maybe it is makes no difference and we're getting a whole bunch of queries from the same resolver from different resolvers. But this is weird: why are we getting so many queries next, on average, over 60 million names that were used in this experiment will get 142 million queries, in other words, no has to be said at least twice: no, no and a bit more, no 0.37 of a no and that's the average amount of time to say no. Now it's really great.

M

The DNS queries are free cuz. If they weren't, some will be making a massive amount of money cuz. You know this software is crap. Why is it doing? This is sort of an interesting question next, so the naive expectation is that no means no browsers should understand that asking three times for name that doesn't exist is kind of two times to many, but even out there in the bigger DNS, that's 1.3 7 times too many.

M

So is this just the fact that folk who write DNS resolvers are lunatics, they could well be I, don't know their source code is baroque, I have no idea. What's inside it, there could be strange and wonderful loops doing this or it could just be. We've created artificial intelligence, deep in the DNS and just haven't realized the stunning beauty that we have created, that the DNS truly is a neural network and it's assumed a life of its own next.

M

But there are some more prosaic reasons that sort of begs you down in what's going on and part of the I suppose. The theory is that at various parts in the application sack no one gives a about any other part of the application stack just do not care and happy. Eyeballs is one of these just do not care I'm going to solve my problem and tough. You get more queries. Well, that's what you're built for so, oddly enough happy eyeballs.

M

If you're running dual-stack, you double the number of queries instantly, because you don't wait for an answer from the first before you go to the second, you send off both an A and a quite a straightaway, normally within 20 milliseconds of each other, and in our case we saw 23 percent of these clients do two queries back-to-back.

M

Interestingly, and you know this is bizarre. 3% only asked for the quad-a now they're, not a v6, only end hosts. We know this. The ad was delivered over v4 we're having a v4 conversation. It has dual stack great, but it's asking for a quad a and not an a.

M

My feeling is that it's serializing not parallelizing, so it asks for the quad a doesn't ask for the a record it gets back in there. Next, a man goes, okay, the name doesn't exist. I will go away. 3% of the cases 74% only v4, which is about the same as the amount of v4 and v6 out there on the larger Internet.

M

So roughly expected next, so we can sort of start dividing things up a bit and we sort of say well, okay, let's just group these a in quad a queries together and if we group them together, we saw 74 million seventy-three point five million dns resolution events, but we still saw one hundred and forty two thousand four and three two million queries. So we've got our two point: three seven down to one point: nine three queries better, but the dns is still weird next.

M

Under half the time it just worked with either one query now that might have been one A and one quad a, but it was just one query: the other half of the time, 51 percent, that query got repeated and the average was two point, eight four. So there's a lot of queries going on next and the real question is: is that an average misleading?

M

Are there one or two resolvers that are doing an amplification factor of a million and everyone else being saying or is there sort of most people are sort of randomly kind of weird next, so we looked at the distribution.

M

Fourteen hundred and thirty queries I'm like what part of no is a problem. Fourteen hundred and thirty times, whoever wrote that software, you know good good Anya work about down. On the other side, a large amount of folk re queried one, so it was two queries, a smaller amount twice. Thirty. Two percent of queries, though, were three or more repeats. So, third of the time the DNS is going throb, throb, throb, throb throb, and the answer is always there.

M

Next domain same name same query type, just throb, throb throb next, so it's as UDP just completely suck so you're going to say. Yes, it does and I agree with you UDP sucks. Now you had a problem Jaret and just can't wait. Can it no.

N

It can't it just can't wait: hi Jared watch Akamai. So when you see these three queries in a row, do they have the same query idea and come from the same UDP port? It's.

M

The same query idea in the same just hang on a second Jared and you will start to see what's going on because it's more than you think that was your yeah. No, so there's no clues going on here does UDP sack. Oh yes, UDP sucks, but not he doesn't suck that bad and the the response was not big. It was only 603 bytes and if I can't get a 603 but DNS packets ripped through the internet, I, don't know what you're doing here.

M

You should be fixing your because, quite frankly, if I can't get that through I can't get anything through. So why are we seeing 51 percent of test generating two or more next, so we started looking at the repeat intervals Jared and we started looking at the time between successive queries from the original epoch time, and you know there are these really strong Peaks. You know next so who writes a piece of DNS resolving software with a 370 millisecond timer Open DNS.

M

Why did they use 370 milliseconds? Because that's a really aggressive, re time rate? Why who writes software with an 800 millisecond retime err.

M

Unbound, who writes one second, bind so a lot of what we see here. There are actually signatures of the resolver folk. Making arbitrary assumptions about the UDP timeout intervals that have no relationship to reality have nothing to do with anything else other than this number and oddly enough, they all use different numbers and what they actually cause is some degree of pathology of repeats, because these time intervals appear strongly in this wreak.

M

Weary signal, what's weird, is: does UDP suck that much that the original answer never got there or is the resolver simply on this mindless path of God or repeat the query? Don't care if I got an answer or not I just have to repeat this query: I'm a driven resolver, no clue next, so timers is one issue. The next is DNS SEC. It is a DNS SEC sign name, so it has an insect record.

M

Aha is validating the N SEC a factor and don't forget these are unique names and they're, so structured that they're, even in unique domains, so the validation is, has to be done every single time, so is DNS ik causing this. So ok. Let's add another test: let's go unsigned next Wow you have to get right or get right up here to slot the difference between the blue and the the red lines. Now, at the point of won't help.

M

Annex signed n X domains generate a lot more requir e's, going up over 5 6, 7, 8, 9, 10 and so on, and the reason why is basically because of the time required to validate every single time. The original recursive is going.

M

Look I'm, really bored about this I'm going to ask you again: cuz 370, milliseconds, says I've really got to ask you again and again and again and again and again so signing generates more load when the name doesn't exist because of validation next, and we can actually quantify this a little bit when it's signed the require ear. Aid is three point, one nine. On average we're unsigned it's two point: five one.

M

So around twelve percent of the time it's DNS SEC, that's causing the requir e's next, so okay, DNS SEC, happy eyeballs, twenty three percent DNS SEC, twelve percent. Next.

M

How many folk think DNS resolvers are just standalone boxes? Well, no one anymore, because if I ain't there, these weird things called load balances and they sit inside a whole. Bunch of you know single resolver engines, except when the load balancers actually become load amplifiers, as is a case from this Taiwanese network, where the front-end load balancer decided to send the same query to every single member of two distinct subnets. And you look at those query times. Every sort of twenty have David sorry.

M

Every hundred milliseconds I get another query from the same subnet, but a different engine common as hell. Whoever runs this stuff has no idea what they're doing and and that a huge number of these distributed. Dns resolver farms actually become DNS resolver amplifiers, just because you can thank you very much next. How common 70% of the requir e's come from these shared subnet prefixes. So it's not the same. Ip address it's a different IP address, it's a different query, but the original query is being forked out again and again and again across the entire subnet.

M

Whoever runs resolver farms, whoever runs yours, fire them, cuz they're. Obviously, at their job cuz, this is really really common. 70% of the requir e's come in from these kind of odd resolver farms. Next, so it's kind of wondering is this a no problem, I'm like when I say no, do you not believe me, whereas, if I give you an answer, go: oh yes, good, fine, there's no worse than yes! So next is the require rate, lower Wow.

M

If I give you an answer, you're more likely to believe it. But if you re query, if you get a real answer, you'll hammer the out of me so that 39 percent of the requir is in in X domain signed, become requir YZ, three point: one: nine queries per per name when it's an real answer: I and quad-a. Only 13 percent wreak wery much much better guys, but your average is five point. Eight one re queries. What the I'm, like you, don't even like a good answer. It's this sort of subset of lunatics.

M

That kind of go look I, don't care whether it's a no or a yes I'm, just gonna ask and ask and ask, and ask, because you know: I can so yeah whether it's a real answer or not a real answer. There is this phenomenal wreak weary rate down there. Next, so that's the same graph! You need to be right up there to see it, but what you see is a DNS. Sec sign name generates a lot more responses than a real name, but the real names when they require e re query very very hard next.

M

So we can kind of put it out there inside of a single graph. This is the one that I showed you before 42%. Do it once next happy eyeballs I'd, say about 10% additional load. Next I put DNS SEC at near 70% of the total load. Next NX domain itself signed an X domain, no vs., yes about 14%. Next, insane, stupidity down on the farm about 13% of queries and the next wow you guys are just inventive I have no idea. Why else you're doing this, but you know good on you.

M

So, yes, this is the DNS for you next, nothing more! Thank you.

N

Yeah heíd shared much again so if you know, if you can hit the backspace key or something Chris so here, I don't see this showing whether or not this looks to be dysfunctional, Network behavior of packet replication in the networks, because I've definitely seen that before especially with UDP, which is why I asked the prior question about query, ID and the port numbers and so I'm curious. It is that the reasons or is that sorry.

M

It could will be I couldn't think of any other deliberate pathologies that you know. I was smart enough to understand, but you've just uncovered one, and- and thank you, you know, replicated query packets, something subtle, though, if the replication is happening between the recursive and the authoritative I will see it if the replication is happening earlier. It's actually not obvious what the recursive is going to do when it starts to get back-to-back queries and there's already a query in process. Good engineering says duplicate queries get held in occassion till the answer comes back.

M

Reality says you know. All bets are off sure.

N

I mean in the case of a resolver farm I could easily see a subsequent query being load balance differently, yeah, but what I have specifically seen as I have seen Network pathologies, because people are either configured ports, panning or mirroring, or have active or passive taps in the middle for their own monitoring of the network that sometimes end up replicating packets, and it is especially harmful for UDP based protocols like this, which is why I would urge you to look at that further in part of them. That's.

M

A good point, thank you.

O

Jeff, as you certainly have plenty of interesting bugs here- and people probably fix a number of these things just by.

M

People in the room.

O

Created these bugs I just watched them. Do it I'm sure the one thing I'm here to respond to is is UDP that bad, so I live in BFD for obvious reasons, UDP isn't always that bad itself, but the ecosystem that consumes it can be.

O

No, so you can do your darkness to pass the packet all the way across the network and I'll get to some part of your ecosystem, whether it's a firewall of load, balance or host paths for VM pick your poison of choice and there's weird assumptions in the stack that it's okay to drop these things, because no, you know it's meant to be unreliable and things look it through and what that tends to have informed us as a protocol.

O

Is that being adaptive about your load rate is actually very important, and the graph here is showing about the various implementations that we're timing assumptions you know constants are almost always wrong. In some sense, an interesting line of inquiry might at some point be. Is there a way to allow dns to sort of actively signal back into the system? So here's how the cadence should look like that? Would get you a better response rate? You.

M

Know you know, that's a great question and, and had you said that about 10 years ago and the obvious answer is DNS over TCP. Why TCP? Because, instead of a preset timer that tells you there's a problem, there's an acknowledgment time. You actually understand round-trip times, and you know when you're missing a packet, you wait for twice the time, etc, etc. Tcp does a much better job of getting over packet pathologies because that's its job. Now it withstand a loss rate of up to about three or four percent.

M

That's why mobile networks work as distinct from don't so. This is a good idea, but ten years ago, when you said DNS over TCP you'd be covered in arrows, because obviously you're a heretic you're a pagan and you should be burned at some local stake these days. It's fashionable to say DNS over TLS and everything will be cured and life will be wonderful and if you're doing ultra trendy I'm going to mention the magic word, the magic word is Doh and it just makes everything better. So.

O

My heresies aside, my other big head there, whereas you know PGP, obviously, and spend an awful lot of time dealing with TCP pathologies. I think that TCP is the primary transport freudian, as we just simply compound a lot of very interesting problems.

O

I, don't really consider it part of the solution. Space could.

M

You ever believe of a future where BGP worked over UDP I.

O

Have some very entertaining slides about the history of BGP that has some pathologies about EGP that our elders would share about why fragmentation based protocols are not good for routing yeah.

C

General to clarifying questions so when you say yes, when you actually return some answer does client get the page. Oh I'm.

M

Not looking at that sorry I just never went that far. I really am just looking at the DNS query, logs at the authoritative server and I'm just looking for repeats because.

C

It's actually interesting is actually client not getting the answers at all. It keep repeating right.

M

So you know the wonderfulness of the dns is there are so many points where the queries could get replicated and you know it's in the load distributors, it's in the recursive engines, it's in the forwarding system.

M

You know let alone in the stubs, let alone in the application and in some cases the bizarre thing is your application might get an answer and go and get the webpage, but somewhere deep in DNS land, some engine's going frog, frog, frog, frog: frog, because you can we time everything and every DNS query has a timer forty, five percent of all the queries we see on our servers are more than two hours old, a visible percentage of more than three years old I'm like I, don't think it uses sitting there with the browser going.

M

You know, I pressed this three years ago. I'm really am waiting for that page, so the DNS does take it on a life of its own and the web is is, is not really relevant. Completion at the user level doesn't stop the pathology. Okay,.

C

And the second question: so this three percent, when they only ask for quad-a, you said you're sure they are not v6 only so you mean you initiated communication over before without.

D

C

P

I, that's tail. I came up with one request, but you've just added a second one in that I think it's probably best for all of us if we not refer to the throbbing DNS, but apart from that I'm curious. If, if you happen to repeat this experiment in the future, if you could also include the yes/no answer, that is a no data response both signed and unsigned to see how.

M

We've been wondering about refused surveil in X domain, no answer, because this phenomenal amount of ghost queries in the DNS that just don't seem to have an obvious reason and it's always been an issue. What responses encourage, reek, weary and what responses kill it flat and we're working on a theory at one point that said, NX domain is cache no, and the answer is well. No, it's not it sort of hit me again, so yeah I, don't know where the no data refused to fail or any other response code would have different response.

M

Behaviors right, it's worth.

P

A try, that's mostly what drives my curiosity.

Q

Yeah hi Jeff, Mark Smith, another thought I had was that these persistent queries, even when you give them a good answer, a load balancers doing liveness checking, so they might present a single IP address to their client. But then they have a pool of upstream servers and they likes reaching quickly and so on. I hide them for.

M

Various reasons and I think it's probably a mutual hatred. The folk who build load, balancers and the IETF have never met eye-to-eye. There is no standard. There is no spec, there is no behavior. This is a bit like the first weird and wonderful world of Nats. Everyone builds DNS load, balancers differently. They screw with I some hash, some hash, various parts. Some do you know and so on, and so because a lot of the recursive resolver families appear to sit behind these wacky load balancers. The end result is a set of queries that dooby-doo behave.

M

Oddly I mean Jared's question.

Q

Would possibly also explain this.

M

Q

Well, just yeah thanks for telling me you there I'll ask in in a few seconds Thanks.

R

Hi Nalini Elkins I wanted to go back to the do H and D ot question a bit if we wanted to see this kind of behavior using DNS over HTTP or DFT or whatever, where? Where would be the points at which you would try to measure this, because it's encrypted on the wire right.

M

You're, going down a completely different path and and a path that in the dns, is a path that doesn't have very many answers. The dns is incredibly opaque. Our attempts, misguided as they came, of putting fingerprints in queries otherwise known as client subnet, have been loudly met with accurate accusations of acute invasions of privacy, and so the dns is resistant to any kind of realistic measurement of the things you're talking about. We can't see that clearly, you can see either end, but the middle is sort of a mystery yeah.

R

Interesting. Thank you.

S

Robbo Stein couple couple of comments: one the time out issue with the ridiculously short retry intervals, yeah back in that you should probably expression the 80s we actually used. Retry intervals, like you know, two three seconds and stuff like that and as near as I can tell what changed it was essentially market pressure. You know it's basically an old version of the happy eyeballs problem, people whine when they don't get an instant response and the idea that your query global database it might take a little while well screw that I want an answer now.

S

Damn it and you know the vendors just kept cranking the numbers down until it got to the current insane levels, but I mean it was already insane by the mid-90s. What Microsoft is doing then and hasn't improved now.

M

You quite right and the problem was with UDP: there's no feedback right. You don't understand what you're driving in so it becomes a number given to you from on high use this number for no time outs, yeah the.

S

Other thing is, you know: I've not actually been to the DNS business for about a decade. In fact, the last time I was our working theory about a lot of the insane query that just like nothing, you can do will stop. It was people behind stupid-looking figured firewalls idiot with firewall, so they send lots of queries and never get a response, so they send more queries and they never turn anything off. They're like well, it isn't working so I'll. Add another thing and I'll do this and I'll do that?

S

Oh look, I'm, father, yeah response, it's good, don't touch it and it just stays like that forever. I, don't know how much of that you're seeing I have no idea how you'd even detect that, but that was our theory at the time it.

M

Certainly seems in Chrome in its tests of liveness, which it sends off random names towards the root. There is this attitude in other parts that the DNS is infinite, is fantastically engineered and and works exactly as planned. All the time and people could just throw queries in it. Whatever magnitude they want and they'll always be an answer.

M

I wish it was so you know, but the fact that it isn't creates this astonishing sort of amount of tension between what application folk believe the DNS is doing and what operators see it's doing and where DNS code then to sit in the middle going. What the hell should I write next time, how to defend ourselves and folk around our code. It's a tough one.

I

Aaron Falk alchemy, thank you for you talk Jeff the two quick points and a question. Thank you for sustaining my belief that there's a lot of weirdness in the internet, I'm glad to see that hasn't, changed and I. Think that you gave your the the reason in I think your first couple of sentences, which is that it's free right, there's like many of our tragedies of the Commons, there's really no reason not to keep trying. It would be interesting if there was a way to quantify the cost in your organization from the the extraneous requests.

I

The and my question is: are you doing any sort of longitudinal measurements? Are you seeing any trends here? Any periodicity is there? Is this changing over time? Are you planning on continuing measurements to gain that now knowledge, we've.

M

Been seeding, the DNS with unique DNS names for seven years now, and we see seven year old queries because the timestamp is inside the DNS name. So you know there are just old queries out there.

M

I always think the DNS is actually like long term storage that all you need to do is encode what you want inside a query and to sit there and listen, and sooner or later, you're gonna get re query and that's your read function, I, don't know it's getting worse or better, but certainly our perspective is we seed more names in our repeat rate, you know the ghost rate just keeps on inexorably growing at some point, we're going to have to walk away from the top-level names that we're using because they're so polluted the signal is getting lost inside the repeat noise.

M

We know what we're doing because the names have meaning, but your server, which have been running for years, which has a whole bunch of gross queries and you're. Seeing a thousand queries a minute is that nine hundred and ninety and one real? How do you know no one does the suspicion is that you are overwhelmed with your legacy and that real queries are actually extremely low, but they're masked out by this extraordinary property of the DNS to just simply memorize a repeat and recycle quantifying that difficult that it certainly seems bad one.

I

Question that occurred to me as I was looking at your your analysis is: is there the potential that some of this is malware traffic? That's using you know, DNS queries as some way of probing the state of the network, reach ability. We see.

M

A lot of log re players, so there is a massive business out there of capturing and then replaying and generally they are relatively benign, but we also see a whole bunch of other things where pathologies that are inexplicable there was a DNS resolver in Jordan and other one in Guatemala doing up to 20,000 queries per second for days of the same name, it's kind of I'm. Sorry, what exactly is going on here guys you know. So the combination is just an amazing pool of awfulness.

M

We've exhausted the line. Thank you very much. Thank.

T

B

See you in Vancouver.

H

B

Also, the slide, where should be available Susan from the website.