Constructive criticism of TwitBlock seems to have quite rapidly turned into some quite aggressive complaints including several demands for immediate closure. This is not intended to be a malicious project, but with a sudden rush of large numbers, I am experiencing a minefield of issues that I did not predict. I am doing my best to fight these fires, but please bear a few things in mind a few things when feeding back to me:
First and foremost, this is alpha software.
I did not expect 7,000 visits a day after just three weeks. (I can thank Mashable for that overhead). I am constantly thinking of ways I can please everyone and still keep the application doing what it needs to do, which is help people remove illegitimate followers.
– or – “I told you it was in Alpha”
I’ve rolled out an experimental TwitBlock feature designed to reduce “false positives” for legitimate accounts that are being blocked. Whitelist entries are now subtracted from blocks. i.e. accounts marked as “not spam” will have their blocks counteracted on a 1:1 basis. If this feature is abused, it will be removed. It survives on the premise that the spam bots are not capable of whitelisting each other.
Here’s the full story:
As we approach 3,000 TwitBlock users, we know of over 100,000 blocks and have stored 20,000 profile pic checksums. I figured it was time to start crunching some numbers. Continue reading…
The list of Twitter accounts below all have something in common – They all have an identical profile image, which looks like this:
At the time of writing none of these accounts have been suspended. Whether they are breaking any laws or not I don’t know, but it is clearly a syndicate whichever way you look at it. The profiles all point to a Korean-registered “Cash generator” website, which [I would hazard a guess] is a con.
TwitBlock unearthed this statistic from a list of
only 18,000 100,000 blocked accounts provided by under 400 3,000 TwitBlock users . When you consider the size and growth of Twitter, you can well imagine that there are far more than 120 288 profiles in this syndicate. You also have to wonder how much of Twitter’s growth figures can be attributed to this junk.
[ UPDATE: 18 Aug ]
Many of these accounts have been suspended, but TwitBlock is discovering new ones each day – currently 248 accounts known with this image.
[ UPDATE 19 Aug ]
I’ve produced a report of the top 20 most duplicated profile pics identified by TwitBlock
A detailed explanation of the scoring mechanism used by TwitBlock.
Some people have complained that they get a high spam score and point out that they are not spammers. There are a number of important things to note about this.
- This software is in alpha – these indicators and the scoring mechanisms attached to them will change.
- As the system gathers data it will rely less on heuristics and more on cross-referencing (e.g. how many people have blocked an account)
- Some of these tests are only indicators of automation, not specifically of malicious behaviour.
- The spam rating has no limit – Scoring 40 may be high for a “legimate” account, but in a list with real spammers scoring 300+ you’ll be way down the bottom.
- If you display characteristics of a spammer then perhaps this amounts to the same thing as being a spammer. Most normal users score zero.
Roughly in order of accuracy, here are the 8 tests currently performed in the standard TwitBlock scan.
The day a thousand apps stool still
I noticed some weeks ago that Twitter’s OAuth implementation didn’t appear to be verifying signatures. I knew this because I purposefully set an invalid access token which was accepted unconditionally. I thought this was odd, but as a newbie to OAuth I was just happy that my app was working, so I filed the problem at the back of my mind under “deal with it if it becomes a problem”. Today (the week I release by beloved TwitBlock app) it very suddenly became a problem.
A bulk blocking and spam filter tool for Twitter
I’ve finally got round to building the Twitter app I’ve been thinking about for months. While everyone else is preoccupied with making fun, or cool apps, I’ve been thinking about the increasing problem of spam and junk followers on Twitter. I won’t go into why I think this is such a problem right now, plenty of time for that later.
This is just a quick announcement to say that I’ve released an early alpha version of a tool that I hope to develop into something genuinely useful. Currently it’s a simple scanner that analyses your followers for signs of “spammy” behaviour. I’ll post more details about these indicators soon, and I’ll also share some of the interesting discoveries I’ve been making about Twitter spam as I go on my mission.
UPDATE: I have posted about these indicators
If you know about the Qwitter service, then you may also know what people say about it – that it plain doesn’t work. So for my first Twitter app, I decided to make one that does.
I have been made aware since then that there is also Twitdiff, although I haven’t tried it at time of writing.
If you don’t know about Qwitter, it’s a service that monitors your Twitter followers and emails you if someone unfollows you. My app currently tweets the notification instead, so everyone will know you’ve been qwit.
I’m not offering my app as a public service [yet] I knocked it up in 2 hours and if you know what you’re doing with a LAMP set-up you can download it and run it yourself.
» Download qwitter 0.1.2
Requires PHP >= 5.2.x + json extension, MySQL >= 5.0.45
I’ve knocked up a really simple command line tool for interacting with the bit.ly API.
It’s simple because:
- The output is currently pretty raw
- The bit.ly API doesn’t actually do very much
» Download version 0.1.1
Requires PHP >= 5.2.x + json extension
I’m not one to make predictions, but this one has been at the back of my mind for a while and the evidence is starting to rack up. In my usual style, I shall make my point through an unnecessary number of dodgy anecdotes and opinions rather than through citations and serious research. I hope you enjoy them.