Hide Impossible Activity Records From Leader Boards

[TrueDisciple]

Whether on purpose or by accident, many times there are activities on the big leader boards that are physically impossible. 

An example would be running faster than 3 minutes per mile. Or recording a bicycle ride while riding in a car. 

Sometimes this is accidental, because a person forgot to turn off the recording and then drove in their car. Other times, it's on purpose when a cheater tries to game the system. 

This needs to be done for segments, as well as for overall activities. There might be an activity with a normal overall speed, but one or more segments were recorded at an impossible speed. Or there might be an activity with no segments, but it shows up on the leader boards as an impossibly fast activity. 

There are users who periodically report these super-fast activities, but this takes up a lot of time for users. And also for the support staff who review the reported activities. 

This could be automated. If a user uploads an activity that is physically impossible, such as being a lot faster than world record speed for that sport, the app would say something like: "Something is wrong with this activity. It will not be counted for segments or other leader boards."

This would really improve the user experience when looking at leader boards and segments, and reduce user frustration when cheaters attempt to win all the leader boards. 

[Soren]

Hi @TrueDisciple thanks so much for this suggestion. We do have a functionality that runs on the backend called "Automatic Flagging aka Themis" that effectively checks activities for anomalies. This depends on the Sport Type, but runs for instance are automatically flagged when a portion of the activity exceeds the current 400m World Record. For rides, it depends on the speed combined with a certain value for the grade of the hill. We have algorithms in place for this --  admittedly, they aren't perfect and don't catch every erroneous activity, but we are constantly improving them and appreciate your feedback on this important subject. Our longterm goals are to incorporate machine learning into this mechanism.

[anchskier]

@Soren - What is in place currently definitely leaves a lot to be desired as far as auto-flagging of activities goes.  Hopefully it will continue to improve and the machine learning will go beyond.  I am constantly flagging totally unrealistic activities, mostly biking and running.  Just looking at what you mentioned regarding the threshold for the running activitys, how about adding in additional benchmarks to trigger auto-flagging?  In addition to the 400m world record mark, how about applying the world record 1-mile time to any activity of 5k or more and maybe the world record 5k pace could be applied to any activity over 10k.  I see a lot of running paces, mostly from bad treadmill run calibration, that show paces of under 4 min/mile.  It may not trip the 400m world record pace, but definitely something people could not realistically hold for 5k or more.  Just refer to the 5k and 10k running challenge leaderboards on any given morning for examples.  

[Soren]

Thanks for the feedback and suggestion to extend the benchmarks for further distances. We are aware of the flaws in the current system and understand that there's an onus on our athletes and support team to "tidy up the leaderboards", which is not an ideal experience. I will gladly relay this information to our product team!

[Pakete207]

Hi Team, hi @Soren ,

I was planning to spend some days in my parent's home and decided to check the segments around.. It's insane what you can find over there. I certainly would think that an automatic detection of these should be in place (being almost in 2023..) and of course, I'm glad to read that there's something in place (checking against current 400m WR), but I think it is not working.

Just checked the current 400m WR, and it's a pace of 1:47min/Km.. Just checking the closest segment to where my parents live, I can see 8 people who should be medallist in the olympics. Having in mind, that my city is not specially famous for being a criddle of olympians, and the segment is an uphill (4,2%) segment,  I would expect these to be automatically removed (some of them are there since 2018..).

You can't rely solely on the community for the job of flagging (honestly, talking to other strava users, they are churned, and have the feeling that flagging is worhtless).

I would expect certain thresholds, based on distance and speed, even I would compare the effort with the regular performance of the runner or even require HR measurments to appear in the leaderboard.

Let's be honest, most of us came to strava because of the segments and the competition around them, but when you see unrealistics KOMs you lose bit by bit the interest in the platform.

Thanks for hearing the rant, and hoping to see some 21st century technology applied for that in the near future.

Have a great entry in 2023.

[j1mreeves]

If implemented it would by default deter any cheater types from attempting to game the system in the first place. Activities to start with could be running /cycling as these are probably the most likely ones that are cheated /accidentally left running. 

[Soren]

We hear you, thanks for the comments @Pakete207 and @j1mreeves  -- the automatic flagging doesn't work for older activities and wasn't implemented retroactively, so it's unfortunately still an issue for those cases.

[TrueDisciple]

@Soren 

I'm glad to hear that this has been implemented, at least at some level. Another thing that would help massively is to allow flagging in the app. I believe it's currently only possible to flag on the Strava.com website, which is not ideal. 

I also agree with the others, I see a lot of sub-4-minute mile times in running, which is just not realistic for most people. Even 1-minute miles....

I know it's a lot of work to implement it retroactively, but even taking out historical runs longer than 1 mile and faster than a 3-minute mile, would help a lot.

Maybe instead of deleting the activity, it can just be excluded from the CR's and leaderboards.

I doubt that most athletes even try to get on the leaderboards any more, they're so full of erroneous data.

[Soren]

@TrueDisciple thanks for your comment! There's an existing idea suggesting enabling flagging in the mobile app that you can vote for. 

---

@TrueDisciple wrote:

Maybe instead of deleting the activity, it can just be excluded from the CR's and leaderboards.

---

Flagging will never delete an activity, but partial (automatic) flagging as well as community-driven flagging via web will remove activities' segment efforts from public leaderboards.

[TrueDisciple]

@Soren thanks for that suggestion, I voted for it! 

Do you think I should also start a new post about removing leaderboard entries retroactively if they're way too fast? Or is that too similar to this post? 

[Soren]

Great question @TrueDisciple -- I think it's worth creating a new idea suggestion for that request!

[Sander]

Hello all,
When I cycle through my ridden segments, I regularly come across KOMs that can't be right at all. KOM times with speeds of more than 100 km/h, for example. Because the cyclist already had his GPS turned on when he drove his car to the starting point of his cycle route.
Or uphill (bridge in my case), at speeds that would hardly be attainable even if it were a flat segment.

Now I encounter something strange at the KOM time of 1 segment;
The average speed of this KOM is 63.1 km/h, but the maximum speed is 23.2 km/h??? Something that can't be right at all.

Many KOMs have therefore been achieved in a wrong/unfair way.

Daredevils dare to cycle down a col at speeds of more than 100 km/h, but then the percentage of the slope/segment must be <0% anyway. Maybe even -10%, but that depends on the length of the descent.

Now I live in the flat Netherlands. Speeds of >100km/h are nowhere attainable by bicycle here, unless you have to cycle over the Afsluitdijk with wind force 8 at your back.

Wouldn't it be an idea that Strava would immediately apply a filter to remove these inaccurate segments from the person's activity?
- If the speed of a segment is above an x level, it must be a segment with a negative percentage anyway. Or if a segment has an average percentage of eg >10%, then the speed cannot be higher than x km/h.
For example, if I turn on my GPS and I drive at an average speed of 50 km/h, by car, to the top of the Alpe d'Huez. Then I am a lot faster at the top than the professional cyclists who cycle upwards at an average of around 20 km/h.
But I have the KOM until someone reports about it.
- Segments that are incorrect because the average speed is higher than the maximum speed.
- Or even does not register activities because the route partly took place over highways or the average speed of an entire route of >x kilometers is higher than x km/h. For example, an activity of 50km where the average speed is higher than 60km/h.

It could remove the necessary "segment fouling".

[anchskier]

Strava claims they have systems in place to automatically detect unrealistic efforts, but it is terrible.  It misses an amazing amount of very obvious situations.  I'm pretty sure a 3rd grader could have built a better filter.  Hopefully they will find a way to improve the system at some point.  

[TrueDisciple]

@anchskier I agree. It seems a little optimistic to say that this functionality already exists, when it's so poor that it barely seems to do anything. I guess we don't see all the really really crazy activities that it sorts out. 🤣

But there are thousands of obvious activities that should be removed from various leaderboards, segments and otherwise, and are not.

[DannyDindia]

@Sorenhow difficult is it to implement an automatic mechanism to flag activity like this one:

https://www.strava.com/activities/8898231382/overview
(if the activity is already flagged it's because I did it)
I flag about 10 such activities every day, I could as well flag many more but you set a limit on daily flagging.

Why also not automatically flag existing activities?

Seriously, it's ridiculous.

[KyleW]

@Soren 

I think it would help massively if you could tell the community what incentives exist to not implement auto-flagging for every world record? There are reliable records for any distance you care to reference, and Strava has been around for many years with many users complaining about this exact problem, so it makes me wonder what back-door dealings must be going on to not implement them. Does Strava make some form of income or are managers within Strava pointing towards user interaction data (regardless of how negative the experience is, such as flagging feels for most of us) as an indicator of success?

Let us take one small example. The world record on the track in the 5km race for the men's category is 12:35. Even if we assume a 5% improvement, which is inhuman, but even assuming so, the algorithm could be programmed so that "any run beating the world record by more than 5% is auto-flagged". This would mean any run of 5km or longer completed at a 5km race time of 11:57 or faster would not make the leaderboards. This would solve 99% of the problems, and even if not implemented retroactively would at least give users a HOPE of cleaning up the leaderboard. This 5% allowance is generous and should an athlete every legitimately attain a record that exceeds a world record by more than this, a moderator could easily verify it and reinstate the activity for leaderboard purposes.