Erroneous Challenge Leaderboards

[mercerkg]

Update from Strava

July 2, 2024:

Hello! Thanks to all who have commented and provided feedback. We're listening, and we recently made an update that will improve the accuracy of Challenge leaderboards that contain walk and hike activities.

We know leaderboard integrity matters and we will continue to iterate on improvements.

Could I also flag up a (possible) issue with the challenge leaderboards?

Some of the challenges seem to have implausible achievements that do not tally with profile data. For example, in the February run climb challenge there are several athletes with tens of thousands of metres of climbing at the top of the leaderboard. These figures don't correlate at all with the activities recorded in open profiles and I thought I would flag it up.

[Jane · Accepted solution]

Hello! Thanks to all who have commented and provided feedback. We're listening, and we recently made an update that will improve the accuracy of Challenge leaderboards that contain walk and hike activities.

We know leaderboard integrity matters and we will continue to iterate on improvements.

[WS_from_A]

Can you ensure that essentially impossible achievements do not appear in the challenge statistics?
such as the 30 km walking challenge in June. number 1 has walked 13,400 km. that is impossible. Numbers 2 to 10 have also walked more than 1000 km. I think there is only one person who has really achieved that. The rest is all noise. So the statistics are completely wrong, and it's easy to see.

[anchskier]

@Bryant - this issue is an ongoing problem with every walking/running challenge.  Right now, the top 258 spots in the 5k "running" challenge are all sub 4 min/mile pace.  A vast majority of these are "walks" that are all sorts of messed up.  I can't for the life of me figure out why you haven't implemented at least SOME kind of screening of walk activities.  Anything listed as a walk with a pace of faster than 5 min/mile should be automatically flagged.  I could write an excel formula that would be 100% effective for that in less than 60 seconds, so not sure why Strava can't seem to figure it out.  I just flagged one today that was over 7,000 miles because someone had recorded an international plane flight, yet Strava didn't seem to see anything wrong with it..... smh. 

[Bryant]

Hey @WS_from_A,

Thank you for bringing this to our attention.

We'll be sure to clean this up. If you notice any other challenges with incorrect leaderboards please submit a support ticket and we will be happy to investigate further.

[mercerkg]

It is April 1st today and the April run climb leaderboard says someone climbed 58,800m today… 😳

[Noradavis]

 Some of the challenges seem to have implausible achievements that do not tally with profile data. For example, in the February run climb ...The Challenges leaderboard is our in-app “scoreboard” that tracks the names and points of Challenge participants. To browse the   clickercounter.org     Challenge leaderboard: Tap ...Add members to a group or team at work. Let people choose their own practices OR choose what practices the group will track together (as admin).  Is there somewhere I can see Leaderboards for challenges? Right now in the DPS Threat challenge I have 82 and am around #5000 ...You can read more about segmented leaderboards here. Challenge Leaderboard. Keep track of all the teams within the challenge.

[Silentvoyager]

> If you are noticing invalid activities appearing on Challenge or segment leaderboards, please use the activity flag tool. Flagging an activity will remove it from leaderboards.

Flagging an activity isn't possible most of the time because the offending users on top of challenge leaderboards often have private accounts.

For example, looking at March 2023 Running Climbing challenge, the top person has 64,444 meters of vertical gain in just nine days. That is nearly an Everest mountain elevation every day. Of course, that isn't humanly possible. Of course, the account is private.

> We do have a system in place to catch and block invalid efforts on Segment and Challenge leaderboards

That doesn't seem to be the case in vertical challenges. For example, the second place in the March 2023 Running Climbing challenge is a person who got 142,493 ft of vertical gain in one short 3 mile run on a beach. Most people don't get even near that number in a year of running. Looking at the run details, the elevation gain comes from a large number of nearly instant elevation changes where the elevation goes up and down by 1640 ft. I guess that was a result of water or sweat going into the barometer sensor. That is somewhat common.

Erroneous elevation changes like that should be easy to detect. The world record in vertical K is just under 30 minutes for a vertical km. Anything more than 2km of vertical gain per hour should probably be suspicious. Then Strava could do automatic elevation correction and compare that to the recorded elevation gain. If the numbers are substantially different then Strava could just apply the elevation correction and the problem would be solved. Or perhaps Strava could look through the stream of elevation changes and detect impossible large instant changes in elevation and, again, apply the elevation correction to fix the issue.

[anchskier]

@Jane - A big problem with that is that many of the offenders have their profiles private so nobody can see the activities to be able to flag them.  For challenges that are based on total stats, such as the climbing and total distance challenges, you can't link directly to a single activity to flag.  You can only link to the profile, and if the profile is private, there is no way to verify of their activities are legitimate or to flag any activities.  For these, we really just have to rely on the automatic strava filters to auto-flag bad activities, but as we all know, that function is terrible and misses a vast majority of the bad activities.  I am still seeing people post "runs" of hundreds of miles/hour in the running challenge leaderboards yet they don't get auto-flagged.  If Strava could improve their filtering, it would save a lot of hassle of other users and the strava support staff since there would be fewer support requests.

[mercerkg]

Please could I say that "offenders" indicates some intent or (at least) gross negligence. There are (as described below) open accounts that show no evidence of a record of climbing everest every day (due to deliberately faked activities or a device malfunction).

There's a thread elsewhere on the community talking about faked activities (gpx uploaded to win prizes) but the activity I can see suggests that (in some cases at least) the profile data looks completely legit but the run climb achievement bears no resemblance.

Declaration of Interest: I have no skin in the game of the top of leaderboards; for me it is for admiration of real achievements of my friends and others in the Strava community that I will never meet.