I wasn’t looking forward to trying them all out to decide which library to use, but as it turns out I didn’t have to. When I discovered wkhtmltopdf, my decision was made. (I know that sounds like a cheesy marketing testimonial, bare with me)

I only needed to convert HTML documents to PDF. i.e. I didn’t need to programmatically draw vectors, or anything like that. That’s a pretty important prerequisite. When you consider how good HTML and CSS are for laying out a document, it makes you wonder why you’d do it any other way. The catch is that your PDF generator needs to render your HTML as well as a browser does.

wkhtmltopdf is built on WebKit which means it does exactly that – it renders the PDF as well as a browser, because it basically is a browser. It even runs the JavaScript. The fact that other libraries have their own [possibly bespoke] rendering engines with huge limitations seems pretty crazy once you’ve seen wkhtmltopdf in action.

Anyway, I thought I’d share a few things that I encountered while getting this integrated to my PHP application.

PHP Bindings

Despite my initial excitement upon discovering  php-wkhtmltox (PHP extension for wkhtmltopdf and wkhtmltoimage), I ended up not using this extension. My main reasons were as follows:

• Rather opaque configuration options
• Clumsy when using with stdin and stdout

I decided to execute the wkhtmltopdf binary via the shell and use PHPs procopen and related functions to pipe in my HTML and grab my PDF without faffing around with temporary files. I’m unsure of the comparative overheads in using the shell versus using a PHP extension, but it works very well regardless.

Compiling from source

Don’t.

I’m not an expert sysadmin but I’ve compiled plenty of software on the Linux and Mac command ines and this did not go well. It requires first compiling the open source version of Quicktime, which is almost 1GB of source code and was building for over an hour before I hit Ctrl-C on my Mac. It failed first time on my Linux machines… basically I gave up.

Static binaries are available for both Linux and Mac, and they worked fine for me. I recommend saving yourself the pain and just using them, even if they’re not available for the latest version of wkhtmltopdf. (The Mac binary is a couple of versions old).

Fonts

On my Linux servers, after installing a whole bunch of X11 stuff I probably didn’t need and definitely didn’t understand, I discovered via the comments here that I needed to install the urw-fonts package. If you don’t have the fonts installed correctly you get black squares instead of glyphs.

JavaScript errors

wkhtmltopdf is WebKit, which means it executes JavaScript too. I was using the same HTML template to render PDFs as our ‘print’ page. This page calls window.print() to invoke the printer dialogue. As it turns out (and it took me some head-scratching) wkhtmlpdf dies when you call this function, and it does so without any decent explanation of the error.

Rather than muck around with my PHP template, I just passed --disable-javascript to wkhtmltopdf, and everything was fine.

I had a quick go with the W3 Total Cache WordPress plug-in, but it seems rather biased to running Apache (which I’m not) and I experienced some strange errors that I failed to immediately fix. Rather than wrestle with it, or try other WordPress caches, I decided to get to grips with Varnish. This is something I’d been meaning to do for ages, and of course it isn’t limited to WordPress – Varnish is a fabulous caching solution for whatever site you’re building.

The challenge with deploying Varnish to any site is to ensure you cache as much as you can while ensuring dynamic content is fresh when required. With WordPress as an example, the cases are fairly obvious. While logged in to the admin screens you don’t want to cache anything, but the outside world reading your posts should get cached pages whenever possible. In their simplest forms, these cases almost work out of the box with Varnish, but there are a few gotchas. There is also scope to cache more aggressively if you choose to, and most importantly you’ll want to ensure that when content is altered, the changes are visible to everyone immediately.

Cookie gotcha no.1

By default, if you send a cookie to Varnish it assumes quite sensibly that you are in some way expecting dynamic content from the back end. e.g. you may be an admin user, logged in to the site. Of course Varnish can’t actually know what the cookie is for, so when your browser sends a completely redundant Google Analytics UTM cookie, you’ll miss the cache. On this site, that means Varnish would only serve you a maximum of one cached page per visit.

Anonymous session cookies

If a visitor comments on a WordPress site, they are pseudo-logged-in by way of a simple cookie exchange. This means the site can do niceties like pre-populate comment forms for commenting again. These niceties are exactly that, so if you overlook this functionality and block those cookies you can serve more cached pages. In my case, I am happy to make this trade.

The following snippet from my configuration shows how I’ve maximised anonymous user caching while ensuring admin users always get dynamic content

sub vcl_recv {
  # ...
  # admin users always miss the cache
  if( req.url ~ "^/wp-(login|admin)" || req.http.Cookie ~ "wordpress_logged_in_" ){
    return (pass);
  }
  # ignore any other cookies
  unset req.http.Cookie;
  return (lookup);
}

Purging when content changes

HTTP defines a wealth of cache control headers that allow applications and clients to specify what is to be cached and for how long, etc.. but there is no standard way to tell a cache that certain content is to be purged. This is the main challenge of deploying Varnish to a site like WordPress.  A post may be updated, or a user may comment. You’ll want to purge any affected pages from the cache immediately, regardless of their original expiry. Putting that business logic into your VCL configuration would be hugely impractical – it’s really your application’s jurisdiction.

I looked at various purging techniques, and by far the best approach in my opinion is for your application to talk directly to the Varnish administration CLI which listens on the network on a different port to the cache itself. As long as your backend can access your front end by a secure, internal address, you can execute commands on all your Varnish front ends any time you want from your application.

To achieve this, I first had a look at the wp-varnish plug-in which uses the HTTP PURGE method. I don’t like this approach at all; it’s very limited, too closely coupled to the VCL config, and also this plug-in doesn’t purge all pages relating to changes, such as tag and archive indexes.

So, I rolled my own

I started by writing a VarnishAdminSocket class, which talks to Varnish from PHP. This was pretty easy to do, because the varnishadm program uses a very basic text-based protocol. This isn’t designed for WordPress, this is designed for use in any PHP application.

My php-varnish toolkit is on Github. It includes a WordPress plug-in which I am using on this site and you can see the VCL configuration I am using too.

I have a like/hate relationship with WordPress. That is to say that it does a lot, it has a great admin area and there’s a large community producing themes and plug-ins. However, I am a PHP developer of many years, and every time I come into direct contact with the core WordPress code-base I end up being sick in my mouth.

What’s to hate? Well it kind of reminds me of that old BMW advert (or was it Audi?) the one with the swan – On the surface it glides effortlessly, but under the water it’s an ugly son of a bitch, churning away to get the job done.  I had the misfortune of working on a nasty WPMU project last year and found myself looking under the bonnet more than was good for me. Not only is WordPress a horribly written piece of PHP, (and possibly responsible for many a bad word said about PHP itself) it has a legacy of incremental improvements alongside a legacy of community-contributed code that strives to remain as reverse compatible as possible.

If you don’t want to read my rantings, skip to the bit at the end, otherwise … WordPress, let me count the ways a hate thee.

1. Base URL hell

I wanted to get as much of my static assets onto a CDN as I could – a topic I’ll cover in full later. This means serving static files (such as images and css) from a different base URL. It should be simple to upload your theme and plug-in directories to another server and ensure that WordPress points to the correct assets in your HTML. It is not simple. Just look at the hideousness of determining base URLs in WordPress. This simple task should not require a complex plug-in to cover all possible legacy issues. Furthermore, WordPress allows all plug-ins to mess around with base URLs, which means one plug-in might completely undo the work of another.

Oh, by the way – WordPress defines the constant WP_CONTENT_URL before it loads any plug-ins, so the only way to override this is to define it in wp-config.php.

2. Canonical domain ridiculousness

If you access a WordPress site from a domain that is not hard-wired into the site’s database, it will redirect you to the ‘correct’ URL. This is the default behaviour. So, if you want to host the same exact site from a different domain, or different port, (e.g. for debugging a distributed system) you will have to write a WordPress plug-in to override this behaviour. Why does this annoy me?

1. It is not PHP’s job to decide what domain a site lives on – that is the job of the web server. A domain redirect like this should be done at server config level.
2. No system should hard-wire its domain into database-saved preferences. This is nuts, and WordPress does it in multiple places.

3. Magic quotes are the Devil\\\’s work

Magic quotes are so utterly evil that they have been removed from PHP 5.3. They are testament to the amateur reputation of PHP. However, even in earlier versions of PHP, magic quotes could be easily disabled, and anyone who knows what they’re doing with PHP will have been doing so  for years. Furthermore any system written properly would insist on them being disabled. Not so with WordPress. In fact, being able to disable magic quotes poses a problem for WordPress. How can it guarantee to work if you can forcibly disable magic quotes?

Answer: By forcing them even more forcibly. WordPress 3 forces all script input through its own wp_magic_quotes function. Funnily enough this isn’t documented, but it looks like this.

4. mysqli support – or not.

The core WordPress code-base uses the old mysql database driver. When I say old, consider that the mysqli (improved) extension was introduced for versions of MySQL > 4.1.3  in PHP 5, which came out of beta in 2004. WordPress does have a drop-in replacement system though, which allows you to place your own db.php file in wp-content. Here’s the third-party-developed mysqli adapter for WordPress that I am using on this site.

So that’s okay then, right? Well, judging by the required location for the db.php file, this adapter system looks rather like an after-thought. Using the alternative database adapter means calling require_wp_db() rather than just including the original wp-includes/wp-db.php file. Great – except is every plug-in out there doing this? Even WordPress itself isn’t fully patched to support this. At the time of writing, the “famous 5-minute installer” uses the mysql adapter only – I had to patch wp-admin/install.php to be able to install WordPress with only mysqli on my system.

5. Depreciated code handling

Watch out for depreciated function names. For example: WordPress changed wp_specialchars to esc_html. This will result in notices being raised in older themes, which you’ll only see if you enable WP_DEBUG. It’s a good idea to find-and-replace these if you’re upgrading an old theme to WP3. I don’t really hate WordPress for this, but it seems a bit of a pointless change, and isn’t it a bit late in the day for being pedantic about naming conventions? A whim like this just contributes to the amount of legacy in the system and the amount of code devoted to handling depreciation. There’s a 2,500 line file devoted to depreciating functions that are to be removed “later”. Making a clean break with WordPress 3 would have prevented many people from upgrading, so I understand why they do this; I just don’t like the effect it has on the code-base.

I could go on, but I’m hungry. If you want more, see what this guy has to say on the topic.

The bit at the end

All in all, I’m disappointed that WordPress 3 wasn’t taken as an opportunity to improve upon its own legacy. It is fairly clear that popularity with amateurs, ‘ease of use’ and backward compatibility is more important to the team than creating good, robust software that developers will appreciate. Drupal sets a much better example. I’m not saying I don’t have any gripes with Drupal, but they are much happier to take reduced backward compatibility on the chin when they release new versions. This means more incompatible modules, but such is the price of progress.

PHP 5.3 with FastCGI Process Manager

Running PHP as a FastCGI (or even a regular CGI) means that it’s decoupled from the web server, and if you’ve historically run mod_php, then this is something you’ll have to get used to. For example, restarting your web server won’t refresh your PHP settings. The CGI listens on a port much like the web server itself does, and that’s how the web server passes requests though to PHP. There’s plenty of information on why this is good news, like here.

The nginx wiki has plenty of information and examples of running PHP as a FastCGI, but I had some bad experiences using these examples. Quite possibly this was due to my tuning it badly, but regardless, discovering PHP-FPM was a major result. Furthermore FPM is now bundled with PHP 5.3.3 – I thoroughly recommend, if you can upgrade to PHP 5.3.x that you do so. If you can’t, and you still want to use PHP-FPM, it’s a lot more effort to install and you may have to roll back your version of PHP 5.2.x – I won’t be covering that here, check the various patches here instead.

There doesn’t seem to be a huge amount of information about configuring PHP-FPM in PHP 5.3.3, so I’ll share my configs here. To start with building php-fpm from the downloaded PHP sources is as simple enabling the option. The rest of the build process is as normal, but you’ll have a php-fpm binary installed at /usr/local/sbin/php-fpm

$ ./configure --enable-fpm

The following is a single nginx server block, which defines a virtual host capable of executing PHP via FastCGI.

server {
  listen       80;
  server_name  *.timwhitlock.info timwhitlock.info;
  root         /home/vhosts/timwhitlock.info/httpdocs;
  access_log   /home/vhosts/timwhitlock.info/log/access.log main;
  error_log    /home/vhosts/timwhitlock.info/log/error.log  warn;
  # Process PHP files with fastcgi
  location ~ \.php$ {
    if ( !-f $request_filename ) {
      return 404;
    }
    include /etc/nginx/fastcgi_params;
    fastcgi_pass   127.0.0.1:9000;
    fastcgi_index  index.php;
  }
  location / {
    index  index.php;
  }
}

Notice the inclusion of /etc/nginx/fastcgi_params. This file may be bundled with your nginx installation already. If not, it looks like this:

fastcgi_param  QUERY_STRING       $query_string;
fastcgi_param  REQUEST_METHOD     $request_method;
fastcgi_param  CONTENT_TYPE       $content_type;
fastcgi_param  CONTENT_LENGTH     $content_length;

fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
fastcgi_param  SCRIPT_FILENAME    $document_root$fastcgi_script_name;
fastcgi_param  REQUEST_URI        $request_uri;
fastcgi_param  DOCUMENT_URI       $document_uri;
fastcgi_param  DOCUMENT_ROOT      $document_root;
fastcgi_param  SERVER_PROTOCOL    $server_protocol;

fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
fastcgi_param  SERVER_SOFTWARE    nginx/$nginx_version;

fastcgi_param  REMOTE_ADDR        $remote_addr;
fastcgi_param  REMOTE_PORT        $remote_port;
fastcgi_param  SERVER_ADDR        $server_addr;
fastcgi_param  SERVER_PORT        $server_port;
fastcgi_param  SERVER_NAME        $server_name;

# PHP only, required if PHP was built with --enable-force-cgi-redirect
#fastcgi_param  REDIRECT_STATUS    200;

I also added the following lines, on the recommendation I found here (in fact this is probably what went wrong with my earlier experiments with FPM under 5.2.x). I recommend adding these lines.

fastcgi_connect_timeout 60;
fastcgi_send_timeout 180;
fastcgi_read_timeout 180;
fastcgi_buffer_size 128k;
fastcgi_buffers 4 256k;
fastcgi_busy_buffers_size 256k;
fastcgi_temp_file_write_size 256k;
fastcgi_intercept_errors on;

That’s got nginx ready to pass PHP execution onto port 9000, so now we have to get PHP-FPM up and running on that port. PHP-FPM has it’s own configuration, which as of this new release has shifted from an XML format to the familiar ini style. This is a great move, firstly because it looks nicer, and secondly because it also allows us to add additional PHP ini settings.

My main php.ini and php-fpm.conf files are under /usr/local/etc which is the default location. The php-fpm.conf file contains global settings and settings for individual resource pools. Each pool listens on its own port, and can have its own PHP settings.

If you need to support multiple hosts as I do, then you’ll probably want each nginx server to pass PHP requests on to a different pool, specified by its port. To achieve this, I removed all resource pools from the main config and added a wild-card inclusion much like I did with the nginx config. My main php-fpm.conf file looks like this:

[global]

pid = /usr/local/var/run/php-fpm.pid
error_log = /usr/local/var/log/php-fpm.log
log_level = notice
emergency_restart_threshold = 0
emergency_restart_interval = 0
process_control_timeout = 0
daemonize = yes

;  pools defined in virtual hosts
include=/home/vhosts/*/conf/php-fpm.include

Then each host has its own config in php-fpm.include as follows.

[mypool]

listen = 127.0.0.1:9000
listen.backlog = -1
listen.allowed_clients = 127.0.0.1

; Unix user/group of processes
user = nginx
group = nginx

; Choose how the process manager will control the number of child processes.
pm = dynamic
pm.max_children = 50
pm.start_servers = 10
pm.min_spare_servers = 5
pm.max_spare_servers = 10
pm.max_requests = 100

; Pass environment variables
env[HOSTNAME] = $HOSTNAME
env[PATH] = /usr/local/bin:/usr/bin:/bin
env[TMP] = /tmp
env[TMPDIR] = /tmp
env[TEMP] = /tmp

; host-specific php ini settings here
php_admin_value[open_basedir] = /home/vhosts/timwhitlock.info/httpdocs:/tmp

Any directives missing here were just left as default. Note that I set the user to nginx, although I don’t know whether this is a good idea, or not. This is possibly just a throw-back to my mod_php days where PHP would always run as the apache user.

Process manager tuning

There’s not much documentation on the pm.* directives, namely max_children, start_servers, min_spare_servers, max_spare_servers, and max_requests. However, they are analogous to other settings you may be familiar with. The well documented Apache directives,  MaxClients, StartServers, MinSpareServers, MaxSpareServers and MaxRequestsPerChild are worth a look to make sense of these options.

Once a process has been used it still occupies memory, even when idle, so you don’t want more of these processes running than your server can handle. I kept my spares low, and my max children under a number I was confident the server would have enough memory to cope with. This is not a particular area of expertise for me, so I suggest doing your own research.

To start up PHP, run the program telling it where your configs are:

# php-fpm -c /usr/local/etc  -y /usr/local/etc/php-fpm.conf

If you need to reload any configurations, you can do a graceful reload will the kill command, as follows taking the process id from disk:

# cat /usr/local/var/run/php-fpm.pid | xargs kill -s USR2

Update

Since writing this I’ve improved my set up a bit

1. Using this init.d script to start, stop and reload PHP configs
2. Upgrading to PHP 5.3.6: the processes now show the pool names, which is handy
3. I’m also using a Unix socket instead of a listening port

Performance issues aren’t just for high traffic sites. I’m lucky if I get 50 visitors a day to this site, but by using scaling techniques popular with the big boys, I figured I could increase page load speeds, (good for visitors and good for SEO). If I could achieve this and use less resources, perhaps I could even save some money on my hosting bills. I currently run a 512MB VPS on Slicehost, and I’d rather not increase this right now.

With a few days off work, I decided to take the plunge and swap out some of the server tech powering this blog. From the bottom up, so to speak, this was as follows -

1. Replace Apache with Nginx (below)
2. Upgrade to PHP 5.3.3 and run as a FastCGI (next post)
3. Upgrade to WordPress 3
4. Deploy a CDN
5. Add a Varnish cache for extra speed

I’ll go through my experience across a number of posts, starting with Nginx. I shan’t replicate any existing documentation; I’m just going to go through what I did and point you at the resources you’ll need.

Nginx (Engine X)

Apache has long been my comfort zone, as I imagine it is for most PHP developers, but I’ve been listening to people over the past couple of years saying the old work horse is bloated, and needs shooting. It has certainly been falling out of favour with high-traffic sites, while Nginx seems to be gaining popularity. Before this upgrade I’d already tried Nginx out on a few other sites, and it’s great. Read about Nginx here. Here’s an Nginx-Apache comparison if you want some stats – I have none to offer you.

Installing nginx

I found I could install a fairly recent version on Fedora using yum, so # yum install nginx was all I needed to do. I found it was totally unavailable in other repos, including CentOS, so you may have to build from source. I won’t replicate any such tutorials here, except to say that you will most likely have to update libevent. You’ll need a recent version of that for the rest of this stack anyway, so may as well build that upfront.

Virtual hosts

As I have a couple of other small sites on this sever, I needed to set up virtual hosting. I did this with an include directive in the main /etc/nginx.conf, as follows:

http {
  # ....
  include /home/vhosts/*/conf/nginx.include;
}

So each of my host directories has a server {…} block defined in nginx.include. I actually removed the default server block from the main config, although that is entirely up to you. All nginx core directives listed here.

Starting nginx is as simple as running # nginx, and a graceful reload of altered configs can be done with # nginx -s reload. If you install via yum, you may also have a service script at /etc/init.d/nginx. To ensure nginx is up on reboot, I just chucked a start command into /etc/rc.local although, I’m sure there’s a better way to do that.

PHP

So, nginx is running, what to do about PHP? Waving goodbye to Apache also means waving goodbye to mod_php. If you’ve previously run PHP as a FastCGI, then you’re ahead of the game on this one. If not, then you’re about to leave another comfort zone. I’ll cover running PHP-FPM next »»

JASPA is/was a cross-compiler written in PHP which converts an ActionScript3-like language to native JavaScript prototypes. Essentially this is a language abstraction; an approach to JavaScript which is has its critics. I felt however that with ActionScript being a derivative of ECMAScript that some such criticism was not quite applicable. It worked pretty well, and I used it some of my own work, but it has received little public interest, and I was unable to maintain it well enough for those who were interested. Thanks to all the people who did email me about the project – I may pick up the project again sometime, but I can’t make any promises.

My ultimate reason for physically bringing the site down was that was that it was powered by Mediawiki, which doesn’t appear to be compatible with PHP 5.3.x, and I wanted to upgrade so I could use the new version of PHP-FPM (post about that to follow). I could have debugged the wiki to see if 5.3.x was really the problem, but I decided instead to cut my losses. R.I.P.

• Download jParser 1.0.0 (recommended)
  
• Download jParser devel package (Full source and build scripts)
• See the library examples running at apps.timwhitlock.info/jparser

The library has been split in two:

1. jTokenizer – A JavaScript tokenizer designed to mimic the PHP tokenizer.
2. jParser - The fully blown JavaScript syntactical parser which generates a parse tree.

The reason for the split is that for most purposes where you think you need a parser, you in fact just need a tokenizer. The tokenizer library is about 15KB, whereas the parser is over 700KB (minified), so you can see why you might not want to include it unnecessarily.

The library files jparser.php and jtokenizer.php are self-contained, minified files for production use. If you wish to inspect or modify the code you will need to download the devel package. This package provides a build script which collates the libraries into their distributable files.

jTokenizer

Possible uses for the tokenizer include code highlighting and simple manipulation of JavaScript source code.

The main function you will want to use is j_token_get_all which behaves the same as the PHP token_get_all function with the addition of a column number as well as a line number. Additionally there is the j_token_name as per the PHP token_name function.

jParser

This is a full, syntactical parser. On its own it simply generates a parse tree which can be traversed and manipulated. There is no proper documentation on this yet, but take a look at the node classes in the devel package if you are serious about doing something useful with this parser.

Some other notes in no particular order

The full parser uses a lot of juice. I recommend giving PHP loads of memory, and be careful what you throw at it if you’re going to run it on a production server.

A parser is not an interpreter or a JavaScript engine. If you want to develop such a thing in PHP you might be insane, but it could be done with this parser as a base.

The JParser parse tree is purposefully not a full tree, it collapses redundant nodes to save memory. If you want to see a full tree then take a look at the JParserRaw class. (devel package required)

Splitting the parsing process into two parts (tokenize/parse) is probably not the most efficient and probably uses more memory than it would another way. However, I figured it would be neat to mimic the PHP tokenizer functionality so that parsers could be built that take a stream of PHP tokens.

I noticed some weeks ago that Twitter’s OAuth implementation didn’t appear to be verifying signatures. I knew this because I purposefully set an invalid access token which was accepted unconditionally. I thought this was odd, but as a newbie to OAuth I was just happy that my app was working, so I filed the problem at the back of my mind under “deal with it if it becomes a problem”. Today (the week I release by beloved TwitBlock app) it very suddenly became a problem.

It seems that eventually they decided to fix the security hole, except they did it without telling anyone. In their defence this needed to be done quickly, and they were only going to be implementing what we all believed was already in place, namely the well-documented OAuth standard. The problem was that many people, including myself have publicly released apps that could never have been fully tested.

The damage was already done, i.e. broken apps in the public domain; but the clean up operation didn’t go well either. At the time of writing, no warning has been posted on status.twitter.com. The first we all knew about the change was broken apps, and disappointed users. A few threads popped up on this Google Group where devs banged their heads together to work out what was wrong with their code libraries. In my case I eventually tracked down the bug: The main cause of my broken library was that I was signing the full request URL complete with query string. This is incorrect, as section.9.1.2 demonstrates. A bug that, if I’d been able to test properly in development, would not have made it into production.

Twitter’s response

Although I’ve not seen an official response, I did see a post on the afore-mentioned Google Group by one of the Twitter API developers. He admitted that they did not approach the situation as they should, but this post may have been deleted because I can no longer find it. I shall post it if I find it again.

URL encoding bug

Another bug (which was not actually the main problem) is that PHP’s rawurlencode function doesn’t quite fit the bill. OAuth parameter encoding implements RFC3986 whereas PHP’s rawurlencode function implements RFC1738… the difference is that PHP will encode the “~” character. A simple fix is to retro decode the offending character, as follows:

function oauth_urlencode( $str ){
return str_replace('%7E', '~', rawurlencode($str) );
}

I have been made aware since then that there is also Twitdiff, although I haven’t tried it at time of writing.

If you don’t know about Qwitter, it’s a service that monitors your Twitter followers and emails you if someone unfollows you. My app currently tweets the notification instead, so everyone will know you’ve been qwit.

I’m not offering my app as a public service [yet] I knocked it up in 2 hours and if you know what you’re doing with a LAMP set-up you can download it and run it yourself.

» Download qwitter 0.1.2
Requires PHP >= 5.2.x  + json extension, MySQL >= 5.0.45

INSTALL

Ensure you have a PHP binary at /usr/bin/php and check the version is suitable

$ which php
/usr/bin/php
$ /usr/bin/php --version
PHP 5.2.x (cli) (built: ..........)
Copyright (c) 1997-2007 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2007 Zend Technologies

Unzip the download, and ensure it’s executable:

$ tar -xvzf qwitter-0-1.tar.gz
$ cd qwitter-0-1
$ chmod u+x qwitter

Configure it for your Twitter account with these two constants:

define( 'TWITTER_USER', '' );
define( 'TWITTER_PASSWORD', '' );

Configure it for your MySQL server. These constants are the default:

define('PLUG_DB_NAME', 'qwitter');
define('PLUG_DB_USER', 'qwitter');
define('PLUG_DB_PASS', '');
define('PLUG_DB_HOST', 'localhost');
define('PLUG_DB_PORT', '3306');

Create the database table.

CREATE TABLE `qwitter_followers` (
`id` varchar(11) collate utf8_unicode_ci NOT NULL,
`modified` timestamp NULL default NULL,
`created` timestamp NOT NULL default '0000-00-00 00:00:00',
`unfollowed` timestamp NULL default NULL,
PRIMARY KEY  (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;

Note that you can configure the table name prefix, so you can run multiple qwitters on the same data source. See QWITTER_TABLE_PREFIX constant

USAGE

You should test it works by simply running it, but the only way the app can do its job for real is to be run frequently – the more frequent the better. To set up a cron job and run the app every 5 minutes, create a crontab entry like this:

*/5   *   *   *   *   /path/to/qwitter >/dev/null 2>&1

If you don’t know how to set up a cronjob, you’d better Google it.

Note that if someone follows and unfollows you within this time window, you won’t be informed. The app needs to see the follower at least once before it will notice that you’ve been since unfollowed.

CHANGELOG

1.0.1
Added QWITTER_TABLE_PREFIX config
Fixed loophole where blocked users were considered qwitters
1.0.2
Added QWITTER_NOTIFY to support emailing of notification

TODO

1. Check missing followers against your block list. Currently blocking a user raises a notification. todone
2. Implement OAuth, so this can be offered as a service

It’s simple because:

1. The output is currently pretty raw
2. The bit.ly API doesn’t actually do very much

» Download version 0.1.1

Requires PHP >= 5.2.x + json extension

INSTALL

Ensure you have a PHP binary at /usr/bin/php and check the version is suitable

$ which php
/usr/bin/php
$ /usr/bin/php --version
PHP 5.2.x (cli) (built: ..........)
Copyright (c) 1997-2007 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2007 Zend Technologies

Unzip the download, and ensure it’s executable:

$ tar -xvzf bitly-0-1-1.tar.gz
bitly-0-1-1/
bitly-0-1-1/bitly
bitly-0-1-1/README
$ cd bitly-0-1-1
$ chmod u+x bitly

Configure it for your bit.ly account, by redefining two constants at the top of the file. They look like this:

define( 'BITLY_USER', '' );
define( 'BITLY_PASSWORD', '' );

Optionally move the executable somewhere useful:

$ mkdir -p ~/bin
$ mv -v bitly ~/bin
bitly ->/home/xxxx/bin/bitly

USAGE

Run the help command to see available commands

$ bitly -h

Example, to shorten a URL:

$ bitly shorten http://web.2point1.com

CHANGELOG

Version 0.1
First release

Version 0.1.1
Added –version switch
Added history=1 into shorten method (ensures it actually goes into your bit.ly history)
Added optional keyword for custom URLs to shorten method

See also

bit.ly API documentation