Better stats, please Twitter!

Recently I’ve been asking “is Twitter ‘mainstream’ yet?

I appreciate the subjective nature of this word, and also how much this comes down to perception as much as metrics. However, nobody would argue that Facebook aren’t mainstream, so how far behind are Twitter really?

I asked attendees of DevNest to do a quick poll to gauge opinion, and was surprised to see that the majority (at time of writing) thought Twitter is already mainstream.

My own perception is that Twitter isn’t mainstream, at least not within my wider demographic (i.e. my ‘normal’ friends). But regardless of whether there’s any real way to quantify this, certain things are clear to me and need addressing

  • Twitter’s publishing of user statistics is insufficient;
  • Third party research is largely flawed;
  • There is a bias that only users that send lots of tweets are important;
  • We need to better understand the dollar value of all twitter users.

‘Registered’ users vs ‘active’ users

If Twitter’s recent claim of 165 million registered users is taken at face value, you might draw the conclusion that Twitter are only about two years behind Facebook, and about a third the size. My perception is that this isn’t anything like the case, and of course the contentious word here is ‘registered’.

Facebook’s 500 million members are ‘active’ (i.e. returning within 30 days) and a whopping 50% of these users sign in each day. Perhaps some of these logins are automated (browser toolbars and such) but regardless, can Twitter claim this? We don’t know, because Twitter don’t seem to want to tell us.

How many of Twitter’s 165 million ‘users’ are news feeds, bots, or even spammers? Are suspended and dormant accounts included? In short, who’s really using Twitter? How many people log in, and how often? what do they do? How many don’t tweet, but spend all day reading? How many new users are never seen again?

Third party reports are flawed

The absence of official statistics has resulted in a series of reports from third parties who have attempted to analyse Twitter’s user base externally. These studies are invariably for the purpose of getting press and often to prove a point that somehow gives credence to whatever they’re selling. The insights are often interesting, but they are usually flawed, particularly in their implied conclusions.

The main flaw I see is that the Twitter API does not provide information about users logging in. Nor does it provide data about the browsing activity of non-members, or applications that display tweets to their own audience. Without this data many studies simply claim that Twitter users aren’t active, and therefore worthless. The notion of  ‘active’ often being a flawed definition in the first place, and worthless being a [possibly wrong] assumption about user value – even if this is only implied or interpreted.

Barracuda Networks claimed in their study last March that only 21% of users are ‘active’. Their spurious definition of ‘active’ was having at least 10 followers, 10 friends and having tweeted at least 10 times. By this definition, 99.9% of my own followers are active, although 22% of them haven’t tweeted for over a month.

RJMetrics claimed in October that 38% of users have never sent a single tweet. This may be numerically accurate, and may be telling about barriers to contribution, but what it doesn’t say is what these users are doing. Are they simply lurking, or have they never come back?

A Harvard study in 2009 observed that 10% of Twitter users contribute 90% of the content. Again, I don’t doubt the statistical fact, but what does it really say about user value? Are only ‘contributors’ important users?

User value and the participation inequality

Jakob Nielsen’s 90-9-1 rule says that it’s normal for any ‘online community’ to have a small amount of people contributing (read: tweeting) which is hugely disproportionate to a silent majority who merely lurk. Nielsen claims this has been, and always will be, the case. The Harvard study seems to roughly align with this.

The main bias seems to be that tweeting is the only way to be of value to developers and advertisers – I don’t believe this is the case. A user who just wants to know what Justin Bieber had for lunch may be just as likely (or even more likely) to click on sponsored links, and therefore has a dollar value. Does it matter that this user may not be tweeting, or even logged in to Twitter at all?

What we need to understand in the case of Twitter is whether the Lurkers are still valid consumers, how many of them are dead weight, how many are gone for good, and how many aren’t even members.

Only Twitter seem to be in a position to provide a clear picture of this, and I question whether the reason for their reticence is that the picture doesn’t look too pretty. Perhaps advertisers are privy to data that the rest of us are not. But even if that is the case, it needs to change.

‘Fess up please Twitter! If developers and advertisers are to take the platform seriously, we need better stats. Please take a leaf out of Facebook’s .. erm.. book.