Rust Programming Language Rust Linz, 30 Aug 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: lychee - writing a fast, async link checker in Rust - Matthias Endler - Rust Linz August 2023

Description

For the last two years, I worked on lychee, a new link checker which supports Markdown, HTML, and more. It heavily uses async Rust and has a focus on performance. During the session, I will talk about the challenges of writing a link checker and how Rust was the ideal language for this project. The key takeaway is that simple things can get complex very quickly and that the Rust ecosystem allows building high-performance CLI tools with relative ease.

---

Matthias Endler is a Rust developer and Open Source maintainer. He built popular Rust crates like tinysearch, hyperjson, and lychee. He spoke at conferences such as FOSDEM in Brussels, Cod{e}motion in Amsterdam, and BrisTech in Bristol. He gave workshops at emBO++ in Bochum, RustBeltRust in Columbus, OH, and RustFest Barcelona. He is passionate about Idiomatic Rust and an avid blogger.

A

That's the lichi logo.

A

The slides are pink but you'll be in for 30 minutes of mystery, because it's actually a very, very tedious, very terrible topic. We talk about the web. Actually, we talk about the Brokenness of the web.

A

The talk is actually called, so you can write a link Checker in the weekend and I'm Matthias I'm, a Ross consultant from Dusseldorf and I have a list of static analysis tools. It's a long, long list, there's a lot of tools. I started with that list a couple years ago and as you can guess, a lot of these links keep breaking so I decided to do something about it and I wrote a link. Checker I tried to do it in a weekend.

A

What I had in mind was hey. Let's live code, it I had a YouTube channel called hello rust for a while, and this is part of it at the end of the week and I had something work workable and it checked my links. But of course I was not done.

A

I was kind of happy with the results. So here you can see how this thing works. It's supports various formats. You can tell it to dumplings all the links from a website. You can tell it to check all the links from readme.

A

You can store the output as Json and lots of other things.

A

It's a link, Checker very exciting.

A

Now, of course, if you ever wrote a project, you know that there's more work to be done. Than People expect this is a website from Daniel Stenberg. He is the author of curl. Probably everyone knows curl raise your hand if you know curl nice.

A

This guy also had the task of building something that everyone would use, and then everyone would tell him yeah. You can do it in a weekend, so he has an entire blog post about people that tell him all the time that it's such an easy thing to do so I could do it in less than 100 lines. I can do it in a three-day weekend, but what they forget is most languages use Curl underneath, for example, reviews PHP chances are used.

A

Girl lip curl to be accepted, I can build this in the weekend is what every developer says that has never built anything in a weekend. I really like that. Now you might be thinking why use rust.

A

Well, it turns out rust is an amazing language for writing a link Checker. It has native performance, it has Fearless concurrency, so you can play around with various things running concurrently.

A

At the same time, it has zero cost abstractions. So if you do it right, you don't pay for parsing, it's all in memory. You don't allocate. You have a minimal run time. That means you can deploy it everywhere again cross-platform and it's a self-contained binary. You put it somewhere and it just runs, but this is true for literally every single Ros project. I guess I mean for this. Specifically, we look for a very robust networking ecosystem.

A

It should never fail, and also these things we have a Tokyo runtime. Everyone knows what Tokyo is yeah, it's an asynchronous runtime for rust.

A

We have a production grade, browser engine called Servo from Mozilla, which we use and we also have a production grade. Quote-Unquote markdown parser from Google, and all of this is used by lychee, which is written in roster.

A

Now, I guess the task is pretty simple: we want to find broken links now. First I guess: we need to Define what a link is, but it's super simple: if I tell you to Envision a link, then probably everyone has something like this in mind: you're like yeah. This is what a link looks like. That's, because we're human we're really bad at parsing stuff, or we assume that it's pretty easy for machines.

A

That's why I want to do a game with you. I will show you something, and then you will tell me if it's a link or not. So if you think that this is a link, raise your hand, is this a valid link?

A

Yes, it is how about this one show hands.

A

Yes, it still is- or at least it depends so it doesn't have a scheme and if you type it into a browser, it will work because your browser, which is Firefox, of course, we'll know that you want to maybe check https example.com www by the way is just a subdomain. It's a convention.

A

How about this is that a valid link raise hands. If you think this is a valid link, we have a. We have a very bold one down there, but it's not actually because the scheme is missing.

A

What about this one? Yes, this is a valid link, because obviously it's a protocol relative link. That means, if you are on a website and you open the website with https example.com and you have slash slash example.com, then it will use https as well, and if you open the website with HTTP, it will use HTTP, and this is nice. If your website supports both, then you can just say use whatever protocol you used to connect with me.

A

Is this a valid link? Yes, it is how about this one.

A

Okay, okay, do you always lift your hand or is it just? Do you really believe this easily awesome? Yeah localhost is a valid example for this. What is localhost localhost is a convention. Where does it come from from your Etc hosts file? So if you put something into your Etc host that can resolve to example, then this will be valid now, of course, that's cheating, because obviously you would have to do that manually. Who does that?

A

But this exists.

A

So if you go to your open your browser and you type AI- probably have to type AI slash, then it will show a website, because this is a TLD website now, if he turns out, if you buy a domain and then you decide to host something on the route of this TLD, then your browser can resolve it and I guess. No one would have guessed that this is a valid link now I guess what we can establish is a link is everything or nothing, but at least it's always ASCII right unless.

A

You count emojis. Now it turns out that there are tlds which support emojis. One of them is WS.

A

Do you know what this results do? What do you go? If you open this in your browser, it's a nice marketing website.

A

Take a guess: what could it be? Oktoberfest, yeah, love beer. Well, someone at Budweiser was extremely smart to register that domain. I really like that idea, not that you could ever type it but I still like it now. How does it work in the background they use punicode? They translate it before they resolve the website. So technically it's not Unico. Technically, it's not emojis. It's ASCII, but still there's a way to resolve it same with umlaut domains through storage.80 I, don't know if that exists, but it should what about this one yeah, that's Arabic!

A

It says Arabic I, guess, and this is also valid, or at least it used to same idea of punicode there. It turns out there was someone that wrote a list of all the TLD with a records. This is what it's officially called and they just check regularly. If there's something resolvable on these tlds and Arabic is on the list, that's fun. If you want to parse links, because our task is to parse broken links and in order to find links, we have to know about all of these edge cases.

A

Okay, maybe you think I can still do it in a weekend, but then there's this other thing. This is a valid link.

A

I, don't want to go through all of the details here, but it turns out that you can do a lot with urls. It starts with a scheme username and then password and so on. So there's an entire blog post, I linked it below check it out. It's a fun read, but it can also be dangerous.

A

Why anyone see the broken link? Where is the exploit.

A

Basically, yeah something like that: I wasn't cheating, there's you can even see it there's a difference. You have to have very keen eyes, eagle eyes, where's my man in the back guy with the white T-shirt.

A

There we go. No, actually it's the slash. You see the Slash. The slash is different. It's a bit more tilted. First, one is the Divide sign. The second one is the normal path Jesus. So if you click on the first one, this will be an exploit and I mean. We all know that we shouldn't click on links that we don't understand, but I'm pretty sure that I could even trick a couple people from Dyna Trace to do that.

A

That's all because Google decided that zip is a nice domain name. Let's register that emails are easier mail at example.com. Everyone has an email address like that right, emails, email emails are easy right, right, well same problem. This could work if you decide to register an MX record on your TLD, similar to the a record that we saw before turns out that the dot TT has something like this. So you could have a mail address, like example, at TT. No one would ever send you emails like this.

A

That would be ridiculous and the list is long. It's terrible, but you know you can add, quotes you can add one two three one. Two three: whatever IP addresses everything goes, but one thing is for a certain. There is always the at we can count on the ad. So this is how we can discern between urls and emails unless medium decides that they want to have that in their path.

A

And then we royally screwed, because, as you know, slashes are fine in email addresses. How do you ever parse that? Okay? This is what we are dealing with lots of fun and we only covered the first part. We only covered the links part. We haven't even talked about Brokenness.

A

What is a broken link? So if you try to explain to another engineer hey, this is how you find a broken link on the web. You have this notion. This idea, how a broken link looks like you think you go to page and then it doesn't load, but in reality that's not true right, you get a 404. Maybe so it does slope provide some content. Maybe the simplest explanation would be. If it's a 404, then it's broken hope. Everyone gets that reference.

A

Easy if the status code is 404, it's broken now for those of you that have been on this Rodeo before it's, not true, there's 404, but there's also 403, there's four something something so you say nah in reality. Don't you want to check if a link is not returning a 200.?

A

Well, that's also not enough, because there's different 200 States as well, which some of them might be. Okay, you have two or three for example, which or two or six, which is no content, so you say everything that is not a 2xx status code.

A

Well, that's also not true. What about redirects? That's a 300 status code. Do you handle redirects?

A

Maybe then you say well, okay, everything that is above 400, then, but that's not true either, because what about five XX status codes?

A

Okay, five heavy XX would be client error, server errors, no client errors. Well, then, you might say also not enough, because maybe sometimes you try again and the link starts to work. So you have a temporal component to it. You want to retry a couple times, maybe unless it's a permanent error- and this is all great until you're at LinkedIn and it's a Monday morning and you decide to screw the world you're like ah come on what are story status codes anyway.

A

So you decide, my status code will be 999, so LinkedIn returns a 999 status code. If they don't want you to crawl, the page and I really hate LinkedIn for this LinkedIn is just weird, because a lot of people tell me that LinkedIn links are somehow broken and I need to tell them. Actually it's linkedin's problem. So there is an exclusion in lychee which just checks, if it's LinkedIn and then just does some special stuff.

A

The web itself is pretty weird YouTube sends you 200, but then the video is not found.

A

Now it turns out. The way to check YouTube links is to check the thumbnail, because when a thumbnail is gone, the video is gone now in Link in lychee. There is a check which says: is this: a YouTube link then check the thumbnail, but don't always check blatantly the YouTube thumbnail, because YouTube has a lot of pages which are not videos and you want to check them normally because they for some reason, work.

A

Work so to verify that here's, a curl request for a YouTube link and here's a similar color request for YouTube link, if you spot the difference, you're, pretty good, there's just one character at the end: Miner small queue or big queue and one video does exists, and the other does not.

A

But Twitter is like help. My beer I can do it even better because they decided that Twitter will cut off all the tweets from getting read by fetching a URL. You need to be authenticated and you need to have JavaScript there used to be a workaround which was to use the googlebot user agent because for SEO reasons they used allowed it, but also this doesn't work anymore and then there used to be a different alternative, called knitter.net which allowed you to open links without being logged in which is nice.

A

We used that for a while, but then it got even worse was a bit of a nightmare to be honest, and eventually they decided to close down and lock down, and now you have to have an account, no matter what and even nature is struggling.

A

That means we had to disable Twitter. Yes, it's very sad. These are the problems that you will encounter a lot. Does anyone know what that is? Dui link ever heard of a DUI link? Academia, probably some of you have studied Maybe.

A

You read the paper once and they sometimes have DUI links, because the idea is that they should never break.

A

It's like a unique identifier for a research paper and turns out they break, and someone sent me an email once and said: hey I, like your tool, but those links are broken turns out. These are DUI links. What can I do about it and I told him? Okay, nothing I can help you with. It seems like they are legitimately broken and in order to fix it, you would have to find out who maintains the UI server.

A

So the way UI works is every University or every Library can register to be a server of the UI links, and then they are responsible for their infrastructure. You would have to literally go to every single DOI instance and tell them that their survey is broken, which is exactly what he did.

A

So he went and he's still sending emails to people, for example, to wiley.com, who is one of the maintainers of DUI links, and he says in capital letters your DUI infrastructure is broken I, really like that, guy persistent awesome, we covered Brokenness, we covered links, we're done unless there are some edge cases and there are plenty of edge cases. How would you parse inputs? Let's look at this?

A

We all learned how to write HTML in a text- editor, I, guess unless you're as old as me and then probably use something from Microsoft back in the days.

A

But the beauty of the web is that you can start writing text and then see it on the web, and even if you make a mistake, for example, you forget the end angular bracket, it's gonna be fine, your browser understands it. That's a nice feature, but it's a nightmare to parse, because our browsers are extremely permissive.

A

They allow you to not only do this. They also allow you to end the tag with a completely different tank. It's still gonna be fine, and that is my arc. Nemesis browsing various inputs. This is an example from markdown someone just recently said this gets detected as a valid link. I hope you all see that it's not or it shouldn't be. That's a math equation inside markdown, well, parsing error and, if you're the maintainer of a link Checker, then this is your life yeah. This is me at 2am.

A

Looking at the issue.

A

Another problem that you will hit is too many requests turns out literally gets used a lot we get to that in a second and GitHub doesn't like literally being used so much because turns out. If you check a GitHub repository, you have a lot of GitHub links, so we Hammer the servers and we run into those too many request issues a lot.

A

What can you do about it? Well, if you want to be a good edition of the web, it's very easy. There's a standard for that. Every website, every API sends you headers. Those are rate limit. Headers, that's the ietf proposed standard.

A

Now what you should do is just parse. These headers wait for the given time, for example, in this case 600 seconds and then continue parsing easy. You have a hundred pages that you can request. You have 50 remaining and once you're done, you wait for 600 seconds easy.

A

Well, unless you read it and then you say, ah come on, ITF is not standardized. Yet it's a proposal. I will use my own standard. I will use x, minus rate limit or you are at GitHub, and you say now it should be lowercase and the reset time should actually be a Unix timestamp and then gitlab says. Yes, you were onto something here, I really like this idea. Actually it should be uppercase, though, let's add one more observed, header and also the reset time should be in milliseconds foreign or you are at X or Twitter.

A

And then you do everything lowercase again and you also use the same time format. So then your parsing becomes case. Sensitive well, I, wasn't sure about the standard. So I wanted to look it up. There's two more formats, Vimeo decides to use this state format and Akamai said I like it, but let's just change the date format again, but if you want to stick to the standard use the ietf there's a standard for it and I implemented the standard. That's the implementation turns out.

A

I was the first one to even write any library in any language that I found to parse rate limit headers for some reason, I wonder why I wonder why really? But you can't parse them. It's not integrated into literally yet but I plan to and if you want to look up the standard just go to the website. That's the official website.

A

Okay, well, what do you do if, for some reason, the page does not send you those headers, and it turns out many pages? Don't in this case, you can still use something like a small little algorithm for it.

A

It works like this. You have a bucket and there's X coins in it. In this case, there's six coins and every time you make a request, you take one token out and the bucket gets filled up at a fixed rate, and this is how you do rate limiting. That's the token bucket algorithm there's a couple Alternatives too, and this is pretty simple, but you would have to track every single website. So every single host would have to be tracked like this, which is really nice. But then, where do you start?

A

How many requests do you send with your first batch 1, 10 100? Do you ramp up? Do you not ramp up depends.

A

Bot detection is another problem, cloudflare hates crawling.

A

You need website fingerprinting to get around this Firefox decided to impersonate, uh actually curl decided to impersonate Firefox and mimic the TLs handshake, because this is how bot detection in cloudflare works and then, even if you cover all of this, you need to handle JavaScript because the majority of the web, only it works with JavaScript.

A

This is Google without JavaScript looks pretty legit you get search results. Google does a really great job here, kudos to the guys from Google amazing I, like that their websites work with without JavaScript, unless you use Google maps. Of course, then you need JavaScript, because they just keep you for a full.

A

Well, that's much better than what Twitter does or what Bing does. This is all that Bing loads. Ask me anything: you can ask it anything: it's just not going to answer I mean every company. Does it differently, I, really like how Facebook handles it honestly, because they just declare defeat I think this is the cleanest feed I ever had on Facebook, so maybe they that should be my normal way to browse Facebook nowadays without JavaScript I. Really like it, it's much better it loads fast.

A

So JavaScript is a bit of a problem. We went from rendering HTML uh with JavaScript, because HTML was too slow and then everything is in JS, but it's too slow, so we pre-render HTML and then we pre-render all HTML, but that becomes too slow. So we render some pages on requests and then suddenly we render HTML on request again. So we went full circle for some reason, sometimes unnecessary and definitely a problem for people like me that want to build tools.

A

Recursion. Let's say you want to check a website has million sub pages. When do you stop? You can set a threshold Maybe 100 Pages down you're gonna have to keep track of that.

A

Here's more what if the server never responds? When do you give up what, if the response never stops, do you ever stop like it sends you won't bite per second forever?

A

What if the server sends no Unicode? Can you handle non-unicode character?

A

Sets what if you read a one terabyte file, so lychee supports reading websites, but it also supports reading from disk and there are big files and if you read a one terabyte file or a one petabyte file at some point, you run out of ram, though you cannot parse the entire thing and then load the links from it, because you won't get to that point because you cannot allocate, and even if you can do that, what, if you read a one terabyte line, you cannot parse it line by line.

A

You have to yield links as you find them. What, if you read a one terabyte link? Well, that's another problem, and this is the famous iceberg: try to summarize everything that we talked about a couple other things as a lot of edge cases, a lot of edge cases.

A

I'm almost done this is the architecture.

A

It's a bit big for a CLI tool, yo on the top. You have a collector collector, collects links and the extractor extracts links and it can extract links from HTML from markdown and from Pure text and for HTML we use HTML5 ever, which is the engine from Servo, and we also use HTML5 gun, which is a bit more performant and state-based.

A

There's this thing on the right, which is a fragment cache, just real quick. If you don't know what fragments are. These are fragments they're, the things that are anchors or IDs. We call them fragments, that's the correct term, actually, those get parsed and they get stored in the cache, because if you find them on a different page, you want to make sure that the fragment exists same for markdown. Markdown can have fragments too and plain text cannot now you have a bunch of links and then you run it through the below pipeline.

A

You create a client, we use request and we use the GitHub API we remap. So that means you can do various remapping. For example, you have a production website and you have a stage website or like a local host website. You can remap from production to localhost and every link that you find will be remapped automatically.

A

We filter you can exclude Things based on regex and whatever we apply some quirks, the quirks that you saw for LinkedIn for YouTube and so on, and then we run a check and the check can either access the fragment, cache and see. If the fragment is there or use request or octocrap, which is our GitHub API client. Now you can set a GitHub token to get way more requests on GitHub and all of this gets retried a couple times with some logic.

A

I will get to that in a second well I guess we are talking about rust. So, of course we have to cover this part. Is it fast? Is it fast in comparison to what I tried to do a benchmark, very scientific? At the bottom, you see I check, 150 1 500 links Max, which is nothing, but the reason is not literally. The reason is other tools, because you see a couple zeros.

A

This is when the other tools stop and by stop I mean I, waited for 30 minutes and it just never came back with an answer. There are tools out there which cannot check 50 links, the the one that is sort of decent is called Lish. It's written in go and lichi is just slightly faster with that many links literally supports files where Lish does not, and it's also better if you have way way more links, but at the same time I wanted to make it comparable to the other tools.

A

So it is fast, and why is it fast? Well, because everything is async I talked to even yesterday asked him: hey should I show some Rust and he was like. Yes, you need to show some Rust and I told him. The grass code is boring, it's really boring, he said more rust. You need to show more rust. It's a rust made up, so I will show you some Rust code, but actually in reality it's really boring. This is maybe one of the more exciting parts, and even that is boring. What do we do here?

A

Magic? We use a stream. Everyone knows what a stream is: okay, we return an Impulse stream. That means we use infiltrate input. Trade is a way to statically, say at compile time that your return value will be of type stream.

A

Where item is result request and with that, the compiler can optimize that and we don't pay any runtime overhead, which is great, and we also don't have to tell it the type for leader type, because in reality, impulse stream is can be, can be very long. If you know about Futures, it can be tricky to write it down. Even now, we guarantee you to return a stream of requests and the way we do it is we parse contents and then for each content file. We do the thing below we send it to a worker.

A

There's this thing here, not this here, that's kind of the magic source part then unordered, there's a library for that.

A

I actually quite forgot, the name by now uh stream, what it's called enter stream or something like that, and they have a stream X extension trade and an extension trade is something that you can attach to any type to extend its functionality. It's like a battery pack or a jump jumper pack for your type and what it does is it attaches something to our iter method and then we can run part and unordered on it.

A

The non-parameter is the params for the workers and we just pass the input through it, and this way it spawns up a couple workers by default. It uses two times the number of cores on your machine, so more or less the optimal number with hyper threading, and then it runs the checking in the background on these worker threads. So they never block.

A

That means you can run a lot of links like sixty thousand seventy thousand, depending on your network connection at once, and you won't see any CPU usage really because that's the kernel just doing the work, so the the lychee process itself will be extremely bored. It's just sending all the requests to the network card and then it just waits for the results to come in I guess if there was anything cool about this project, maybe it might be this, maybe something that you can use for your own projects.

A

I, really like that Library Maybe, a small choke I, read that yesterday, I really like it Tokyo Futures I think await walk into a bar. They pay they leave they drink. They order. Thank you I'm, here all day,.

A

Even said, we need more rust, we need more rust, so I said: okay, let's, let's do one more example.

A

That's an extension trade. We have ourselves, it's called retryxt and it only has a single method, which is nice I, like my trades to be very small, and it only has a should retry method and it returns a pool. Why do we have that because lychee supports many libraries to check links and they have different ways to handle edge cases. So, for example, we support requests, we support hyper, we support octo, crap and whatever, for example.

A

Here is the implementation of retry X4, request response and what you can do with this trade is, you can say you have a response and then you can say, dot should retry and you can use it in a condition. You can say if response should retire and all the logic is hidden behind this extension. Trade so use extensions rate, extension, trades, I'm, a huge fan.

A

uh For example, we check the status and if the status is a server error, we retry and if it's a client error and it's not a request, timeout or it has too many requests, then don't retry else if it's a success, don't retry, of course or finally, if the status is request, timeout or request too many uh requests. That's when you should check for validity, and if this is the case, then retry yeah. If you don't get the logic, it's fine actually there's more.

A

This is the other part for checking the error, not the response from request. There's a couple things that we do here too check. If we ran into timeout and whatever yeah lichi is used by a lot of companies, you might not know by the number of stars.

A

It only has a thousand stars and Abu gave me the Thousand star just before this and I told him that I will mention his name. So thanks Abu, it is used by Google AWS, Azure hashicorp, cncf honeycomb, oh wesp, Super, Bass, mariadb lots of companies and it also tracks a lot of links. That's actually quite an old number I haven't checked in a while, but that's my recent estimate from the public repositories but I know a lot of people that use it in their own private thing.

A

Critical infrastructure is literally critical infrastructure, I, don't think so. There's more important problems, but I also want to raise awareness to this problem in open source. A lot of modern digital infrastructure depends on a lot of small developers like me that maintain these little things and they don't really get any compensation whatsoever.

A

Does anyone still know this page? That dates me even snotting nice, like 10 years younger than me, but you still know that you're a bit of a nerd. So this is the Million Dollar Homepage every pixel costs a dollar: that's a million pixels. Pretty smart idea was sold out real quick people paid for these links.

A

How many do you think are still available?

A

I was surprised, there's a website that checked that did some research on it amazing a lot. The green stuff is still available after so many years, though, maybe you don't need a link Checker after all, if that's the real problem like why, why do this work? But in reality you know how it goes. These are the valid links and a human would not think that these are the real pages that were there before. There's like parked Mains, weird blocked stuff cyber folks I really would like to know what they did, but yeah they are dead.

A

That's unfortunate. It turns out that 50 of all cited links in the Supreme Court are invalid. By now and 70 of cited links in academic, legal stormers and that's a huge, huge, huge problem because we use we lose knowledge. That is critical infrastructure for sure, and even the way back machine is struggling. The Wayback machine is a page that archives the web. You should donate, they do an amazing thing: they archived a 808 billion web pages and they describe what they do on their page and there's a link up here.

A

They compare themselves to the Library of Alexandria, which I think is a fair comparison. So I wanted to learn more about that clicked on the link.

A

Yeah, it's live. There's a website called perma.cc, it's sort of a research combination. You can apply there too, similar. As for DUI links. They allow you to host stuff there. They kind of yeah take snapshots of pages, but even that broken that's GitHub issue from the open source project from permo and it says, link to library Partners on website is dead, pretty bad and I guess this is the point. You are an open source developer. Lots of companies use your project. Maybe if you're lucky you get a star, but nothing else, I guess.

A

Fortunately, there's still Academia a year ago, some team from Switzerland reached out led by Professor Olaf Zimmerman.

A

They are from Switzerland and they asked me if I want to participate in the research project, oh Penny and Thomas, they uh did their bachelor's thesis on link checking and they use literally for their comparison, which is really great, got a lot of nice contributions from them. So big, big kudos, and if you think that you want to build something valuable from your open source project, I guess, then you have to take it into your own hands.

A

The idea that I have right now is to build something called a lychee server which you can host yourself and then you can connect all of your repositories or your websites and then send reports to it, and you get a dashboard of all the broken links, and this way you find out where the most broken links are, and maybe at the end, you can also fix them automatically with a pull request. That would be the idea yeah.

A

That's all I got. Thank you very much.