Twitter algorithm catches "90%" of bullies ...

[rslade]

Twitter has a new algorithm which, it says, will catch over 90% of bullies.

Now, I really wish them well on the catching bullies part.  I really do.

It's the 90% part that worries me.

Let's say that 4% of people are bullies online.  (4% seems to be a fairly reliable measure for extremes of behaviour, both good and bad.  4% of people go to jail, 4% of people regularly give more than the standard deduction to charity, 4% of people donate blood, etc.)  So for every thousand people, 40 are online bullies.  The new Twitter algorithm will, because of false negatives, catch 36 of them.  That's good.

Thing is, there are not only false negatives in error rates.  There are also false positives.  And so, out of every thousand people, a hundred are going to be kicked off Twitter for being bullies, even though they aren't.  That could be bad ...

[CISOScott]

And what are they using to define "bullying" behavior?

Will they make it public knowledge so people know what is appropriate and what they are being "caught" for?

Or will Twitter just ban people at their whim? I understand it is their platform and they have the right to do with it what they want, but they could make themselves irrelevant by being biased in thoughts and actions.

[CraginS]

---

@CISOScott wrote:

And what are they using to define "bullying" behavior?

...

---

Ah, the only way to validate the claim of a percentage "caught" is to run tests with a known number of bullies in the test ample. That means an artificial sample, seeded with bullying behavior based on their algorithm.  We have a circular logic problem there. The more accurate statement should be, "Our algorithm identifies 90% of the behaviors we have designated as bullying."

---

@CISOScott wrote:

...Or will Twitter just ban people at their whim? I understand it is their platform and they have the right to do with it what they want, but they could make themselves irrelevant by being biased in thoughts and actions.

---

I think Twitter has already been shown to do exactly that, ban and suppress based on a particular bias. And, yet, Twitter seems to remain relevant for at least part of the population