There is indeed a new subsection: 1.5, but it does not explicitly forbid the development of client software. It does however indicate that they will be watching you closely if you choose to do so.

1.5.a: “Your Client must use the Twitter API as the sole source for features that are substantially similar to functionality offered by Twitter”

Somewhat ambiguous, but the important point is they can add clauses to this section whenever they like. Get it?

Ryan’s announcement is more telling, of course -

“Twitter will provide the primary mainstream consumer client experience on phones, computers, and other devices”

(told you)

“If you are an existing developer of client apps, you can continue to serve
your user base, but we will be holding you to high standards [...]“

You’ d be crazy to enter the client market now anyway. I feel for the companies that already have, but Bill Gross seems less concerned. I wonder what he has to say about this – I doubt very much this development is anything like a surprise.

The fact is that this isn’t new. It’s been obvious for some time that Twitter are reclaiming their brand, and that’s only going to continue. Why would you expect this not to be the case?

The only thing that’s new here is Twitter’s verbalising of what we already knew. At least that’s some welcome transparency. The announcement isn’t a press release, mind you. This is a developer writing on a developers’ mailing list. It’s pretty candid, but it’s actually good advice. If you didn’t already know the state of the nation – now you do. I’m taking a copy in case it disappears.

People will complain that the developer community helped Twitter succeed, and now these same people are being told to get lost. The goal posts are Twitter’s to move. Today it’s client software – what’s next?

This is all fair comment, but we need to stop being so surprised by such things. They are a corporation.

It’s no secret that the advertising industry is under pressure to invent new ways of reaching an increasingly savvy and cynical audience. It may be one step ahead of the average consumer – but not us cool Internet kids – we own this place. We trained our eyes not to look at banners, we laughed in the face of FarmVille, and now we’re hiding Groupon from our News Feeds (so last year). We win at Internet.

Sometimes though, I realise I’ve not been quite cynical enough. Like when I found out Cassette Boy was getting paid by ITV. These moments are a little reminder of how the world really works.

Oh, Charlie

What Charlie Sheen is doing would normally (in traditional media)  be seen as a massive sell-out. I feel sad every time I see Iggy Pop making a fool of himself selling  car insurance. Whether it’s a young, cash hungry star, or an old has-been with no options left – it’s just a bit embarrassing.

And yet the Internet (particularly the Twitterverse) seems oblivious to the marketing machine that is the manufactured Charlie Sheen meme. Whether his meltdown was genuine or not (he’s an actor, remember) he’s now publicly cashing in on it. At first glance I just accepted it as another crazy internet meme.

Meet Ad.ly

Ad.ly is a firm that “runs celebrity endorsements in social media”. Short version: companies pay Ad.ly to pay celebrities to mention their brands on social media platforms like Twitter. It’s modern product placement. According to some copy on their site, an Ad.ly employee may even send the Tweets.

Why is this not an embarrassing sell-out, because it’s on the Web? If he was doing a Sugar Puffs TV commercial we’d all be tweeting that Charlie Sheen’s career is over. But we’re all falling in line. Whether we’re lapping up the pseudo-motivational rhetoric, or just finding it all rather funny. We’re just a bunch of eye-balls poised to consume advertising.

Nice try Advertising, but I win because I have Tiger Blood – what else have you got?

Spoiler alert: The project I demoed is a Web of Trust for community-verified Twitter accounts. The site is called cert.me.uk – it’s invite-only at this very early stage.

You can see my less-than-beautiful slide deck here. As it has no bullet points, there are notes for each slide below.

Who are you?

Back in Web 1.0 I used to sign up to ‘community sites’ under a pseudonym and would generally use a silly avatar that in no way represented my ‘real world’ identity. This wasn’t some attempt at being enigmatic, it was just what people did all those many years ago.

Some people still choose to do this on occasion, but for the most part the social web has done a very good job of persuading us to represent our true identities online. In the case of Twitter, this identity is public. Many of us see benefit in asserting our identity through our Twitter account – the most obvious reasons being professional reputation.

Are you who you say you are?

This is trickier.

If you don’t see the problem, consider the little-known case of Australian politician Bronwyn Bishop. A Twitter account was fraudulently set up in her name, and accrued a reasonable following before slipping in some outrageous remarks that no doubt endangered her reputation.

Twitter have guidelines on impersonation, but enforcing them is another matter, and as with the case of Bronwyn Bishop, it’s not always as blatant and easily detectable as the infamous BPGlobalPR profile.

Impersonating another person online was criminalised early this year in California, and has been illegal in Texas for a while longer. This is a new problem, so it’s resulting in new crimes, and no doubt new criminals.

Verified Twitter accounts.

Twitter deal with this problem by providing a Verified badge indicating that a Twitter account is held by THE Justin Bieber, not just A Justin Bieber, and more importantly that it’s not a spoof, or unofficial account representing said pop star.

But ..

Twitter’s original Verified accounts beta programme is now officially “closed to the public”. Translation: you’re not going to get one. There is actually no publicly accessible method of requesting one. Bronwyn Bishop didn’t stand a chance of getting verified by Twitter.

I was amused to see how swiftly Charlie Sheen got a verified when he joined Twitter this week. Proof, if proof be needed, that he meant business.. literally. In Twitter’s own words, this makes Charlie Sheen either an advertiser, or a ‘partner’ .. whatever that is.

So, what was wrong with the verified accounts programme?

It didn’t scale

Although Twitter’s processing of verified account applications is rather opaque to me, it seems fairly obvious that it is/was a human-powered system. Flying in Oprah Winfrey for a coffee and a urine sample (or whatever they do) is not going to scale up to the countless people and businesses that can benefit from a verified account.

In fact, the exact words from Matt Harris when I asked him about this over a DevNest Skype session were “it didn’t scale” … clearly.

It’s corporate owned

Whether you consider this is to be a problem is somewhat a matter of personal opinion.

The ‘corporate way’ to verify user accounts tends to be to verify and store your credit card or mobile phone number. This is how Facebook do it. It’s voluntary of course, but the Facebook kind of voluntary; like breathing is voluntary.

Recently Google  introduced mandatory verification of Gmail accounts by mobile phone number. Who knows what system Twitter will implement for verified accounts in future? But if they want my phone number, I’m going to grumble about it.

Regardless of how safe your data may be from commercial third parties, look at what happens when government authorities want it. (disclaimer: not that I’m planning on committing any crimes).

Decide for yourself where this is going, but if there’s a better way to do profile verification that respects your privacy and doesn’t involve storage of clear text data on servers in California, then I’d say it’s worth exploring.

Centralised authority

The way in which you place trust in a verified Twitter account is somewhat like the way you place trust in a web site when you’re online shopping. It’s an almost implicit trust in a single party.

Behind the scenes there’s a lot more to this trust model. Your browser vendor trusts the root certificates issued by a company like Versign (Symantec), and they have gone to sufficient lengths that they trust the owner of the website. As to who verifies Verisign? I have only the vaguest of ideas, and that’s way more than the average consumer.

Web of Trust

There have been some admirable attempts at undermining the seemingly self-regulated certificate authority industry. The most notable is perhaps cacert.org. CACert implements a Gossamer Spider Web of Trust. As opposed to a linear hierarchy of trust (with a certificate authority at its root), the Web of Trust is less centralised in that it has many, independent members at its virtual ‘centre’ . These members have all been assured of one another’s identity sufficiently that they are trusted, but not by a single entity – by each other. Once a member has been assured sufficiently by others, the member may then certify the identities of others.

It’s worth noting that Thawte used to operate a Web of Trust programme for personal email certificates, but they shut it down in 2009 citing some ambiguous references to ‘standards’ and ‘quality’ that I’d imagine the industry has imposed upon itself to ensure their £1,000+ price tags.

Trust is Relative

Although this Web of Trust may be preferable to a hierarchical, corporate model, it still suffers from one problem – that when it comes to people, it doesn’t reflect the way trust works in real life. You may have been certified by hundreds of trusted members, but if I don’t know those members, then their endorsement of you is not worth anything to me. In fact it’s little better than being certified by a faceless corporation.

The Web of Trust is actually a concept that stems from PGP (Pretty Good Privacy) Ironically now owned by Symantec. The Web of Trust used with PGP (and GPG) provides a relative trust model. You, the observer of a user unknown to you, will place trust according to how much other members that you may trust have trusted them. For example: If I completely trust Jack and Jack completely trusts Jill, I can place a certain amount of trust in Jill despite not knowing her myself.

In this way it is much like a social network. It seems like the prefect way to replace verified Twitter accounts would be a distributed trust model that used the network itself as the Web of Trust.

An Experiment

This was a very long way around of explaining why I made cert.me.uk.

It’s a rough prototype. There’s a lot of work to do, and a lot of problems to solve. I will no doubt blog about these things in weeks to come.

[ Update: this design very quickly reverted, and has not been re-instated ]

It makes sense to create something that Twitter users will be familiar with, but consider how much Twitter have tightened control of their brand recently.

In contrast to TwitPic’s redesign, Tweetphoto recently became Plixi. They diversified by adding Facebook support, as well as events and location features. They’re clearly making the service their own, and shaking off the Twitter brand.

TwitPic have diversified too, adding events, location features and also face tagging. The difference being that their inextricable link to Twitter is unchanged, and now galvanised by the new design.

It would be wise for many Twitter-based services to reconsider not only their identities, but also their competition with Twitter itself. As Fred Wilson put it, “… the time for filling the holes in the Twitter service has come and gone“.

I bet Twitter would like to add photo-upload support to Phoenix, and get tagging into the bargain.

Twitgoo and Yfrog don’t seem to be changing tactic at all. TwitPic looks like an obvious contender for acquisition. They clearly don’t seem to be changing direction to avoid being made redundant – quite the opposite.

A blacklist is now in effect

For reasons that I’ll explain below, we have decided to implement a blacklist system as of today. Blacklisted Twitter accounts always show up with a 100% confidence score and will appear at the top of scan results. You can still see the standard spam score indicators, as the screen-grab below shows.

Who decides what accounts are blacklisted?

Currently myself, and selected other people that I deem trustworthy. We take this seriously and won’t just blacklist an account for being annoying, or noisy. We roughly follow Twitter’s own guidelines on spam, and only blacklist accounts we are certain are malicious, fraudulent, or extremely distasteful.

If you would like to be an administrator, let us know by way of a tweet.

If you think you’ve been blacklisted unfairly, let us know by way of a tweet also.

Why implement blacklisting?

The methods we’ve used to date have worked reasonably well, but a distinct problem has been that if you block spammers on other site, it can take us several days to catch up with that data. Spam accounts can collect thousands of followers in a very short space of time, and do a fair bit of damage before they start to show in TwitBlock scans.

Finally prompting the introduction of blacklists was my observation of the recent Facebook scams. One fake Twitter account was offering invites the new messaging system, another offering Facebook credits. I watched these go viral, racking up 2,000-3,000 followers each before they were finally suspended after about 36 hours.

Even if Twitter were investigating these accounts, no warning was issued via their spam and safety accounts. The fact that these accounts were imitating such a well known company should make them a high priority, as they were far more likely to fool people. In the case of Facebook credits, financial fraud may have been involved.

It is clear that Twitter cannot act fast enough in shutting spam accounts down. If they were to show up in TwitBlock scans sooner, they would be reported by more users. Then perhaps Twitter would notice them sooner, and shut them down faster.

To create this badly photoshopped image for my DevNest talk, I took Facebook’s Connect dialogue and spliced it with Twitter’s new design for their Anywhere platform.

Take in its beauty, and then I’ll explain …

This image is a mock-up – it is not Twitter, or TweetDeck official. (just covering my back, ok?)

Extended permissions

Note the icons on the left, particularly where it states you are granting permission for the developer to access your direct messages and tweet from your account. Does the app you’re accessing need to do perform these actions? If it needs to do one thing, should it be able to do all things? I caused a minor storm when I pointed out that any application you authorize can read your DMs. This is why Twitter (if they want to conquer the mainstream) need to follow in Facebook’s footsteps.

Facebook call this approach ‘extended permissions‘. Currently the Twitter API only supports two access levels: read-only, or read+write. For example: Read access would be required to access your direct messages. Write access would be required to send them from your account.

This access level decision is taken by the developer, not the consumer, and it’s currently very badly expressed to the connecting user via the UI. It’s worth noting that extended permissions are not a part of the OAuth spec itself, rather they are an extra layer on top that is specific to the vendor. Perhaps it should be a part of the spec.

Whether anyone questions an application’s need for write access is an issue in itself, but this is compounded by the fact that write access basically means maximum access. My site TwitBlock needs write access to report spam, but I don’t need (or want) to be able to tweet from your account any time I like.

Reporting applications

Note the ‘report this application’ link in the mock-up – Remember Twifficiency? Not the first auto-tweeting app I’ve grumbled about. Regardless of the case specifics, application developers need to be responsible for what their application does.

OAuth means that Twitter can trace any API access back to the application owner, and revoke access. A good start, but it’s not easy [enough] to report a Twitter application. Facebook have a ‘report’ button on connect dialogues and application profile pages. Twitter require you write out a support ticket. How many people know how to get to that page?

It’s a totally understandable point of view, but my response is usually that there are certain entities that have no place in our online communities. Whether this invades your personal space or not, every opportunity should be taken to render it ineffective for the benefit of everyone.

If these junk accounts didn’t experience some level of success they probably wouldn’t bother, so somebody somewhere is falling for it. If you’re smart enough to know how to deal with spam and avoid the pitfalls, then perhaps you should use that knowledge for the benefit of others.

If you see a bikini clad model looking for ‘friends’ and it turns out she is trying to sell you counterfeit watches, ask yourself whether this ‘person’ has a place in your online community. You could ignore them, or you could report them with a single click.

Of course there are boundaries as to what we should consider harmful. A small business employing questionable tactics out of naivete or misguidedness is on the other end of the spectrum from a phishing attack designed to separate you from your credit card number. If you’re in any doubt as to what kind of things to look out for, Twitter’s own definition of spam is  good start.

Below are the results of my twtpoll on the subject

- but where’s Twitter for Mac, and Twitter for Windows?

Tweetie for Mac came as part of the Atebits deal, but why hasn’t it too been relaunched as the official Mac desktop client? There have been rumours of course,  and with most things Twitter-related one has to interpret their actions and statements. Little of the coverage seems to mention Phoenix - aka #NewTwitter, which is a clear power play in the desktop space.

Raffi from the Twitter API team told DevNest over Skype that “Twitter for Mac is definitely a side project ” and implied strongly that the team working on twitter.com is basically viewed as the desktop team.

One has to question whether desktop clients are required at all. Browser based apps are becoming increasingly sophisticated, and if Google have their way the distinction between the desktop and the browser will eventually disappear completely.

Of course, we’re not quite there yet, but we’re not far off. As it stands, TweetDeck and Seesmic are still superior products to the new twitter.com. It has a serious lack of notifications (direct messages/mentions) and doesn’t have anything like the powerful filtering and customisation features of the better desktop clients.

However, these missing features in twitter.com are power user features. The mainstream user base that Twitter need to conquer are not power users. Seasoned Twitter fans that need these extra features will continue to use third party desktop clients quite happily, and I’d dare say that Twitter are quite comfortable with that. I expect to see Twitter using twitter.com as a priority destination for new users, and more casual users. In other words, they can prioritise a feature set for the masses and keep those users safely under the Twitter brand. I expect this mainstream feature set will prioritise tools for search and discovery, as opposed to conversation and contribution – but that’s another blog post.

The @gmp24_7 account (parodying the official Manchester Police accounts @gmp24_1 -through- @gmp24_6) was created at 9:55 this morning and less than 40 tweets later at 10:42 was ordered (by the police, via Twitter) to stop using their copyright

What amuses me about this the most is that copyright infringement is what they got called on. (Apparently falsely according to lawyer @davidallengreen – unless he’s a spoof too). Personally, if I was told I may be impersonating a police officer I’d be a little more petrified.

For the record, this spoof account made no attempt to indicate it was a parody. That was a bit of an error, but they’re still tweeting apparently. I’m curious to see whether Manchester Police take any further action to shut this account up. Most famously @BPGlobalPR remained operational throughout BP’s recent PR crisis.

I look forward to the social media/PR pundits tearing this campaign to pieces. It seems rather brave to enter the public domain with something like this and not expect a backlash.

In my view, this campaign basically says “See, we’re working hard. Don’t cut our budgets“. Does anyone doubt that the police work hard? I don’t. The campaign in no way highlights the actual impact of budget cuts – other than more PR will be done on Twitter, because it’s cheap.

I appreciate the subjective nature of this word, and also how much this comes down to perception as much as metrics. However, nobody would argue that Facebook aren’t mainstream, so how far behind are Twitter really?

I asked attendees of DevNest to do a quick poll to gauge opinion, and was surprised to see that the majority (at time of writing) thought Twitter is already mainstream.

My own perception is that Twitter isn’t mainstream, at least not within my wider demographic (i.e. my ‘normal’ friends). But regardless of whether there’s any real way to quantify this, certain things are clear to me and need addressing

• Twitter’s publishing of user statistics is insufficient;
• Third party research is largely flawed;
• There is a bias that only users that send lots of tweets are important;
• We need to better understand the dollar value of all twitter users.

‘Registered’ users vs ‘active’ users

If Twitter’s recent claim of 165 million registered users is taken at face value, you might draw the conclusion that Twitter are only about two years behind Facebook, and about a third the size. My perception is that this isn’t anything like the case, and of course the contentious word here is ‘registered’.

Facebook’s 500 million members are ‘active’ (i.e. returning within 30 days) and a whopping 50% of these users sign in each day. Perhaps some of these logins are automated (browser toolbars and such) but regardless, can Twitter claim this? We don’t know, because Twitter don’t seem to want to tell us.

How many of Twitter’s 165 million ‘users’ are news feeds, bots, or even spammers? Are suspended and dormant accounts included? In short, who’s really using Twitter? How many people log in, and how often? what do they do? How many don’t tweet, but spend all day reading? How many new users are never seen again?

Third party reports are flawed

The absence of official statistics has resulted in a series of reports from third parties who have attempted to analyse Twitter’s user base externally. These studies are invariably for the purpose of getting press and often to prove a point that somehow gives credence to whatever they’re selling. The insights are often interesting, but they are usually flawed, particularly in their implied conclusions.

The main flaw I see is that the Twitter API does not provide information about users logging in. Nor does it provide data about the browsing activity of non-members, or applications that display tweets to their own audience. Without this data many studies simply claim that Twitter users aren’t active, and therefore worthless. The notion of  ‘active’ often being a flawed definition in the first place, and worthless being a [possibly wrong] assumption about user value – even if this is only implied or interpreted.

Barracuda Networks claimed in their study last March that only 21% of users are ‘active’. Their spurious definition of ‘active’ was having at least 10 followers, 10 friends and having tweeted at least 10 times. By this definition, 99.9% of my own followers are active, although 22% of them haven’t tweeted for over a month.

RJMetrics claimed in October that 38% of users have never sent a single tweet. This may be numerically accurate, and may be telling about barriers to contribution, but what it doesn’t say is what these users are doing. Are they simply lurking, or have they never come back?

A Harvard study in 2009 observed that 10% of Twitter users contribute 90% of the content. Again, I don’t doubt the statistical fact, but what does it really say about user value? Are only ‘contributors’ important users?

User value and the participation inequality

Jakob Nielsen’s 90-9-1 rule says that it’s normal for any ‘online community’ to have a small amount of people contributing (read: tweeting) which is hugely disproportionate to a silent majority who merely lurk. Nielsen claims this has been, and always will be, the case. The Harvard study seems to roughly align with this.

The main bias seems to be that tweeting is the only way to be of value to developers and advertisers – I don’t believe this is the case. A user who just wants to know what Justin Bieber had for lunch may be just as likely (or even more likely) to click on sponsored links, and therefore has a dollar value. Does it matter that this user may not be tweeting, or even logged in to Twitter at all?

What we need to understand in the case of Twitter is whether the Lurkers are still valid consumers, how many of them are dead weight, how many are gone for good, and how many aren’t even members.

Only Twitter seem to be in a position to provide a clear picture of this, and I question whether the reason for their reticence is that the picture doesn’t look too pretty. Perhaps advertisers are privy to data that the rest of us are not. But even if that is the case, it needs to change.

‘Fess up please Twitter! If developers and advertisers are to take the platform seriously, we need better stats. Please take a leaf out of Facebook’s .. erm.. book.