Fetching Activities within Rate Limit

[calebpanza]

Hello!

I'm writing an application that requires storing information about new activities in realtime as they come in from athletes. I have been successful in getting the creation of new activities sent to my server through the web hook. New Activities do not include any metadata about the new activity, so I have to follow up with additional calls to fetch the data.

I've explored the following 2 approaches but both have their problems:

Fetch By Activity Id: the most straight forward, I could hit `/activities/{{ id }}` for every activity that comes from the web hook. This would be the most expensive against my daily rate limit, though.

Fetch By Activity Date Range: in order to avoid hitting rate limits, I could hit `/athlete/activities` and run against a date range to fetch multiple activities at once. This is great cause it allows me to "batch" requests, although the `event_time` that comes back from the web hook does not accurately reflect the activity event time (this is especially problematic when syncing with datasources like Apple Fitness where there could be a delay in the import)

How do I properly go about fetching activities for athletes in realtime while still sitting within the rate limits of the API?

[calebpanza]

This is immensely helpful! Thank you so, so much!

While this solution touches on the short term rate limits, my understanding of the rate limiting was that there is also a per-day limit which would not allow for my application to exceed 2000 calls per day.

Am I understanding that limitation accurately? If so, how do you handle this?

[ActivityFix]

Correct, there's also a daily limit. I handle it the same way as the 15m limit, the only difference being waiting until the 24h period resets.

Ideally your limits are set up such that you don't hit the daily limit.

[calebpanza]

Are the daily limits per user or total? Cause if it's 2000 calls per user, per day, I'm golden! But 2000 total per day literally only gives me a total of 2000 users that I could handle input once per day..?

[ActivityFix]

All limits are total across all users. As your app grows you should be able to request increased limits to match your usage.

[ActivityFix]

Ideally your API rate limits will be scaled to match the number of authorized athletes and number of API calls your app is making. That said, you can never rely on that as activities aren't evenly distributed throughout the day and some more rare occurrences like Strava being down could result in a large burst of activity notifications once things come back online.

Back to your original question, this is how I handle everything with my apps

• When a webhook notification comes in I save the webhook data into a queue of pending activities.
• I have a backend processor running which pulls from that queue (in order, oldest to newest) and fetches activity data by ID. It passes that data to my main app routines to do any required processing.
  • All Strava API calls return headers with your rate limits and current usage. I use those to determine whether I have enough calls left to process another activity. If I'm at my API limit, wait until the next 15 minute increment for it to reset before resuming processing.

If your app has a user-interface which can use up API calls be sure to reserve enough of your limit to allow for normal functioning of the app. E.g. if you have 100 calls/15 minutes, stop processing webhook calls once you hit a 15m usage of 85 (or whatever number makes sense) so that your main user interface continues to work. The same goes for the daily limit.

---

With a system like this you will distribute periods of heavy activity over time so you don't suddenly exceed your rate limits. If you suddenly get a large burst of activities which would exceed your rate limit, processing just gets delayed some time until you have API calls available again.

The only disadvantage is that if you reserve some API calls for the UI, you won't be as efficient as possible since there may be a few unused calls which were reserved and could have been used for activity processing. But if you get to that point you probably need your rate limits increased anyway.

---

Edit:

To illustrate, this is the last 12 hours of API usage for my app (from https://www.strava.com/settings/api). You can see that it has periods of higher and lower activity throughout the day but nothing is being rate limited. At 15:30 UTC a large burst of activities came in. The 3 consecutive bars of the same height are my backend processor hitting the rate limit I set and pausing until the next 15 minute period. It had to do this 3 times in a row to deal with the large number of pending activities. Once that burst of activities was dealt with, things went back to normal. The key thing to notice is that those 3 bars are all the same height.

Without the backend processor rate limiting things, it would have been one massive spike 3x the height and all of the API calls beyond my limit would have failed. Because of the rate limiting I described above, I never went over my API limits and everything continued to function normally with no downtime. The only thing a user might have seen is a slightly longer processing time on their activities during that 45 minute window.