Microsoft 365 Community SharePoint, 8 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Avoid throttling in SharePoint Online by using RateLimit headers in API calls

Description

In this 26-minute developer focused demo, Bert Jansen guides viewers on how to regulate request traffic for optimized throughput. Demystifying SharePoint throttling. Start with terminology review - user and application throttling, resource units and quota, and RateLimit headers. Review throttling handler calls from Microsoft Graph, SharePoint REST, and SharePoint CSOM. Demo of sample allows you to test your app calls and decide if need to use RateLimit headers. Resources and answers to 6 typical questions. This PnP Community demo is taken from the weekly Microsoft 365 Platform Community call recorded on November 8, 2022.

Demo Presenter
• Bert Jansen (Microsoft) | @O365Bert

Supporting materials
• Sample - Using RateLimit headers to optimize the throttling behavior | aka.ms/sharepoint/throttling
• Documentation - Using RateLimit headers to optimize the throttling behavior | https://github.com/OneDrive/samples/blob/master/scenarios/throttling-ratelimit-handling/readme.md
• Repo Standard - Rate Limit fields for HTTP | https://github.com/ietf-wg-httpapi/ratelimit-headers
• Documentation - Best practices for discovering files and detecting changes at scale (Scan Guidance) | aka.ms/scanguidance

Learn more
• Microsoft 365 Unified Sample gallery - https://aka.ms/m365/samples
• Microsoft 365 Platform Community in YouTube - https://aka.ms/m365/videos
• Microsoft 365 Platform Community - http://aka.ms/m365/community

A

B

I'm, a senior service engineer inside the customer engineering organization in SharePoint and OneDrive and in my day-to-day role, I, do talk a lot with the isvs with customers and throttling is a common topic. It's like I think one out of two calls throttling somewhere pops up and we have to talk about it so before diving into the technical demo around weight limit, headers I would just want to step back and say and demystify SharePoint throttling a bit helping. You understand how it works. If that's already in the introduction really explained very well like.

B

Why do we need throttling? We need trottling to ensure that not one particular user one particular application is not jeopardizing uh stability for all the other uh tenants inside an environment. So because we share resources with many tenants. We have to have a fair use policy and prompting is our way to achieve that.

B

All right, so imagine you have a funnel, which you see here and your request comes in what will happen? The request comes in then. Your request for the most request is Titan application. So if you haven't actually the application and using that application, you make a call to a SharePoint, API you're on this bucket in this application bucket- and this applies to both delegated and application. Permissions request can also just have a user linked to it all about request.

B

If you use delegated commissions, your request is coming by an application, but you also know who the user is most common approach. Is this application model here and the first measure we have is number of resource units per app per minute, and this is a pretty common reason of throttling.

B

So what that means is that your application can execute a number of resource units per minute, and if it goes beyond the threshold, it will get throttled we'll explain later on what it means. There's research units. So but it's like a first measure going further. We have the similar measure, but then for like a day. So if your app is doing too much across a full day, then it can be throttled as well quite seldom, but it can happen.

B

So these are the common, uh the more common approaches for getting throttled on the left side for the user. Here, if your request is tied to a user, either being a delegate of missions or just being like a script expert part making a call to a rest endpoint, then we also measure how much use requests happen in a certain time window and if that goes beyond threshold that can lead to totaling as well again, quite rare, then.

B

Finally, there is some Farm level Spike protection in place just to protect the resources, very rare that you get throttled by this one. Eventually, your request comes here. We need to, and in most cases almost all cases is green. Your quest happens and you're good to go. You get your data back and you can issue the next request.

B

If you get throttled you're, not so happy, um but throttling is a common concept and it's easy to deal with you get either a 49 or 503 HTTP error. You catch those exceptions.

B

um There is a try after header telling you you have to wait for like X seconds 10 seconds 20 seconds, so you wait and you then just retry the same request again so and then you're good to go.

B

So this is how it works.

B

Now we use this those resource units. We talked about that previously visual students is our way to document what your application can do before it gets throttled and the number of resource units and application gets um is depending on the tenant size. So the larger Lieutenant is the more one. Particular application can do so. The 50k cannot get six thousand resource units a smaller one. You get 1200.

B

and it's per minute, so uh we have a bucket of one minute and all requests in that minute. If you don't exceed this quota, you're not throttled. If you exceed your totals.

B

uh Furthermore, those resource units, um they are translated to requests. So, like you, do a request and- and the request can be a system request, a graph request, a SharePoint trust request. All these requests uh are kind of eject and possibly throttled, though the graph requests are kind of more favored in this model.

B

So if there's a graph equivalent over a SharePoint rest or see some equivalent, then you should use that it's more favorable than consecrotting for the graph site, so we have actually defined all the calls and you can kind of translate your request into resource units. So if we do a Delta request with a token, it's just one resource unit, if you do expand the permissions that will count for five, because there's really expensive call for us overall um I, think investor a guidance is to just assume two resource units per requests.

B

So if your app does 1000 requests, that means 2000 research students on average. That also then applies to SharePoint system and rest. So that's an easy way to calculate that moving on to rate limitators, so you know if no learns when you get throttles and why you get throttled. No, no trolling is not a good thing.

B

Well, it's a good thing to protect the service, but you don't like any application, because when you get throttle to your application kind of gets to a stop and for that we've recently introduced A New Concept called great limit headers, so rate limit. Headers are response headers. So when you do request, you get response back. That response contains uh three new uh can contains to be clear: three new headers, a rate limit, Dash limits uh showing you uh what's the Your Capacity that you have and like the OnePlus 250 of 200200 for a small tenant.

B

If you have big tenants will be six thousand, so how much can can you consume?

B

uh Where are you how much is remaining so uh in this case just one thousand two hundred and it's 120 uh students remaining and finally the reset header. It will tell you like okay: when will this one minute time, window expire and I again start with a new set of 1200 uh resource units.

B

So we give you this information and then you can use this information to uh start slowing down. The idea is that, when you're about to get throttled, let's say you're at 90 of consumption of your resource units, then you maybe want to do a little pause So to avoid getting throttle altogether.

B

um One more thing is that those headers, if you start testing this and you you can, maybe you don't see those headers immediately appearing and that's accurate and normal, because those headers only appear when you are above 80 percent of your allocated quota. So if you don't see the headers you're all good, you can keep going if that starts to appear, and then you need to uh uh act on them now. You might wonder like okay with it. Why should it do all of this?

B

What's the the value of for me dealing with those great limit headers and for that um I've made this graph over here, where you see a number of requests over time, so let's take a a graph when you're, not using rate limit headers.

B

So your application, you launch application it gigs of like five threadsden threats, all those threats they start hitting SharePoint and, and you keep ramping up and everything goes well at a certain point. You get throttled and then you get to like more like a flat line. Your application gets fully throttled and it has to wait until all the struggles, kind of expire and you're good to go again. And then you see this ramping up again: Flat Line ramping up flat line. So you you really get those kind of stops in the application.

B

uh If you would use rate limit headers the green line, you get, you start at the same speed initially, but then, whereas the throttling would kind of get you a full stop weight limiters will slow you down a little bit, but you don't fully stop and then you pick up again because you're in the next minute again some small delay, because you're delaying some requests, full speed, delay and so on.

B

The good thing is that this gives you extra throughput um and the reason why is is because you avoid getting throttled and that just saves requests. If you get throttled and you try again, then you do two requests, not one. So you you do two requests against a particular quota that you have so the more you get throttled the the more you kind of feel worse than the situation.

B

So that's why using red limitators will give you more throughput in the same time window compared to not using them all right time for some demo I'm looking at the shed, if there's anything specific, faster, um then uh I can handle that, but I don't see anything. No.

A

B

A

Can follow up afterwards.

B

Yep yep all right so demo Studio over here and actually let me do one more thing: um I'm going to paste some links.

B

So uh in the chat you see a link to the SharePoint throttling page, which is the basic documentation and also to the sample. So you can the code I'm showing here you can just follow along and already look at the code there and then play around the sample to test. This is relatively simple. What it does is um it allows you to kick off a number of threads like five and decide whether you want to use weight limit headers, yes or no, so you can really experiment and play with it and and learn.

B

That's the goal behind.

B

The user application you need to have an actual application configured which is done here already- is in the documentation of the configure things. But this case we set up an application with a certificate because we're using application permissions I'm not going into details of that so just check out documentation to learn more and how to do that.

B

Then the key thing in the application is this HTTP client and totally Handler, now to explain that a bit better I'm going to go back over here and go to this slide. So in.net HTTP client is the core class that you use to make a HTTP request to call something to call SharePoint, for example, HP client model, there's a concept called delegating headless which allow you to kind of extend and plug in a functionality inside HTTP client.

B

So in our demo we built a delegated Handler called totalinghandler that will handle the throttling, but also will handle the rate limiting so using the rate limiter.cs class. So let's have a peek at both of these classes going back to the code. So here we have our HTTP client and you see that we are kind of passing in a totaling Handler going into a totaling Handler.

B

The main thing is that we have an overwrite of the send async. So when you make a request, you do call the send async method and on HP client, and in here we have a loop and in this Loop let me go to the bottom. We do uh the request, so we call SharePoint color rest API called graph API, we call an API, we get the response and if we're not throttled, then we're good, we return the response.

B

Otherwise we um clone the request and try again so because this is a loop. So this code shows you how to handle throttling and how do we try now? The key part of this demo is the rate limiting and for the rate, limiting. uh The thing is that when you do request a certain point in your response, you you see those weight limit, headers appearing, so your code needs to look at the responses coming in and store the information that comes in the response and then for the next request. You have to then validate like.

B

Can you make this request or should I wait? So the first part is looking at the response and possibly storing weight limit headers. Let's learn using this uh and update window methods.

B

So let's go over there and this method is fairly simple. It would check the response headers how those headers there so the limits remaining reset headers and if they're there they just start and that's it. So we have a threat Safeway to read those headers and store them so fairly basic.

B

B

In here before, we actually make the request, so we imagine that our last request gave us some red limited us and we store them. The next request you want to make. We want to look at this rate, limit headers and decide whether we can continue yes or no, and that's done using the weight, async method I get in the right limiter class, and this one you see us here.

B

Looking at the data we collected and then the site, whether we should wait have a delay or not in this case in the sample, you can tune the delay based upon the minimum capacity left. So what this means is that the whenever we have 10 percent of resource units left or less than 10, then we'll wait. If not, we continue, and this figure it may depend on your application. If you do lots of parallel threats, you might want to put this a bit lower.

B

If you have just two threads, you can maybe put in a five or even less good. um Going back over here. Explain those things so at HTTP client can be used to make a maximum graph call SharePoint rascal uh for season. It's a bit special there to make a season goal like you can share position, call using uh our system, libraries, all good, but to get the response and look at the response. Headers, uh you need to plug in a custom web request, executive Factory, which then can use those this HTTP client.

B

So this same HTTP client is used for making graph calls Rascals and season calls right. Let's go back to My Demo class and I will explain this. We explained that uh that and now here we are. We are starting a number of parallel threats uh and each thread does some work in this case in a loop. We are calling rest graph and season and then we just get a response back. So it's fairly simple application. Doing three calls now.

B

Let me just to start use it with rate limit the rate limit headers on five threads, so we press, F5 notification starts to run ah yes, yes, that's a little Annoying thingy there. Okay, let me make this bigger, so you can see more and what you see now is that the application started. We have those five threads that we kicked off. Each of those threads is doing calls uh doing a rest, call graphical and a season call and a certain point.

B

You will see this application getting rate limited us, they start to appear when we are at 80 mark and then, if we see but you'll then see how they are kind of processed and uh our application will wait. um So we have to wait like 20 seconds more because we have this one minute uh window um that we use for the resource unit checking- and here you see already like rate limit headers appearing- you see if you see some of the graph for rest, so it doesn't matter which API use.

B

You get the same rate limit address and you see the reset like. uh So we're still good we're not having to wait yet um because I configured this, like only wait once we are at the less than 10 percent, and we just starts to happen right now. So you see this graphic. Rest is the later four seconds and another season request delayed for eight seconds, so you see how it works. So now we're kind of slowing down a little bit and once this slow down this is done once we're in a new minute.

B

uh We can kind of speed up again and then slow down again, and this is how this kind of works, with rate limitators now I'm, going to stop this application and let's try again um using without using weight, limit letters and see what will be the difference so same application. But now we just ignore regular materials, pressing a five sure, let's zoom.

A

B

A little bit so it starts to say you get the same speed in the beginning. Application starts to issue the SharePoint, the rest season, Max of graph calls and, at a certain point, we'll see a different Behavior.

B

Looking at the.

B

It's too many messages to quickly see the questions. If there are questions.

B

Any questions that I need to uh address.

A

I think people are answering the questions. There's a good chat on going on on the implementation, and people are being impressed on on saying this live. This is.

B

Really cool, okay.

A

B

Thanks um we're getting close to the one minute time window. Remember that we added one minute and we're pushing hard, and now you see throttling happening now you might say yeah whatever the thing is good. Well, I'll just wait, but look at the wait times, 30 seconds, 39, 40, 43 42.

B

So when you get throttled, you also tend to see like uh longer wait times. They will try after header is a giving telling you to wait longer, which really means this application gets to like a full stop uh and once the minute is kind of over and all the wait times are gone will kick off again and we start going, but this is not so not really that efficient compared to using the rate limit headers.

B

So that's why we have those headers, no point in keeping this running uh I'm gonna close it looking at the time. Okay, we have a couple of minutes. Let's go back to slides and let's go to some q, a common questions that that we get when talking to customers and I, see this other exceptions possible. The answer is no even first party uses the same logic and the same rules for most parts: um the user agent string. What about that thing?

B

Because we initially said, like you, have the special user agent and we can give you exceptions a long time ago. It was today no acceptance anymore user agent, not required, but if you specify it it's fine, because it helps support and kind of, investigate and understand which application, which version of an application is making a particular request. So if you have it in there, just leave it in it's good.

B

An interesting one can I use multiple Azure applications. Yeah you. You just told me that uh one application gets so many resource units. What if my application just just uses five Azure the applications and load balances across those? Is that fine? The answer is: no, that's not fine, um simply because you will start to exhaust the complete tenant and that will impact all applications inside the tenant.

B

So if you do this Behavior, you will see that the things start to fall down and then your customer will be happy because multiple applications will get heavily throttled or even blocked at a certain point. If this keeps on going so definitely a no-go.

B

Should I use a service user with delegated permissions versus application, permissions and again the answer. There is no bad ID, um because a service user is a user and remember that we had that left column. When you had the funnel. We also check how many requests the user does and if you use a service user Bureau likely to kind of run over those limits and get throttled.

B

So if you're a big, your scenario is this is a back-end scenario. um Then you just use the application permissions and not the service user concept um last ones.

B

My app needs to read all contents day up to date, so I have an application that needs to monitor all files in all libraries across all sites in my tenant can I. Do that? Yes, that's possible.

B

We have a concept called scan guidance which explains how to do it, but it comes down to the fact that you do initial kind of crawling of the content that you need and then you use subscriptions and Delta queries to update your copy of the content. So you do scan everything just once, and then you do Delta updates so and this using this pattern, you can really uh process a lot of content in your application, while not getting too much throttling and the last one load testing. Is this a good idea like many of these?

B

The answer is no, please don't do that because you will get throttled and your test results. Don't make much sense because you get troubles and here are the resources that they already talked about and share them in the chat window without.

A

Before you close, there was two questions which I wanted to ask um is the the is all of this automatic in the PMP, Powershell and PMP core SDK was the specific questions in uh so, um and what about craft sdks is somebody it's all of this? Taking into account in those.

B

A very good question um for the the PNP side in in the Commerce Decay. This is implemented but not turned on by default. So there's a configuration option allowing you to just turn it on. Then you can say like I want to uh apply waiting when we are at 10 capacity left or 15. So you can choose that like in this demo, so that's there in core SDK. The framework and Powershell is not owned by default and not there yet so.

B

There's lots of work to do to kind of integrate that fully into that's the case, and the same applies for the uh for the graphics case. So today they don't officially uh have this built in. You can start to add this yourself, but it's not natively built into the sdks today.

A

Cool really really cool and awesome. Demo um I shared this sample uh link to the the sample in the chat already we'll put it in the summary blog post and all of that so and a great work on the sample as well. The documentation on that is awesome, uh it's for everybody to understand, what's available and how things work. So thank you. Thank you. Thank.

B