Status codes are not for humans.

Last week I finally managed to provoke our tech lead into using the 'F-word' in conversation. The trigger for this outburst was none other than HTTP status codes, and my desire to invent a new one. This was clearly beyond the pale.

I have since retreated from my original stance (a custom status code), but have committed a change to our dev branch to use an existing, but rarely seen, status code - 422 "Unprocessable Entity" (more on the situation in which I return this later).

This code is an extension to the base set of codes, proposed as part of the WebDAV extensions to HTTP 1.1 in June 2007 (citation):

The 422 (Unprocessable Entity) status code means the server understands the content type of the request entity (hence a 415(Unsupported Media Type) status code is inappropriate), and the syntax of the request entity is correct (thus a 400 (Bad Request) status code is inappropriate) but was unable to process the contained instructions. For example, this error condition may occur if an XML request body contains well-formed (i.e., syntactically correct), but semantically erroneous, XML instructions.

This status code has seen increasing adoption within the HATEOS / REST community, however it's not without its detractors - as the comment thread with this post illustrates.

My particular crime is not just to use this code, but to use it in a classic vanilla HTML scenario - in the event of form POST validation errors.

It has always seemed to me a peculiar anomaly that whilst the PRG pattern and the use of 3xx status codes (albeit typically the 'wrong' code, 302, instead of the code specifically designed for this situation - 303) has become ubiquitous, it is perfectly acceptable to return a 200 OK status when the form POST is rejected (because some validation has failed).

The common argument against this is that form validation is an application specific issue, and therefore not the responsibility of the protocol used to the transmit the data, but I think that's a thin argument at best. Additional, accepted, status codes exist for "Payment Required (402)", and "Forbidden (403)", both of which relate to the underlying application, and not HTTP per se.

The 2xx code block specifically states:

This class of status code indicates that the client's request was successfully received, understood, and accepted.

But what if the request was successfully received, understood, and not accepted. Surely this is a valid use case - in fact I would suggest that it's one of the canonical HTML/HTTP use cases. And whilst the specific rules around acceptance are application specific, the generic concept of 'Request Declined' is ubiquitous - in 15 years of web development I've never seen an application that does not support this scenario. And the rules around this are no more application-specific than the 'Payment Required' example.

It is (IMO) unfortunate that people who are looking to support this as a valid 4xx exception are having to shoe-horn this into the 422 WebDAV extension, but my suggestion for a new 421 code (421 on the basis that it's between 422, which is the closest formal code, and 420, which Twitter appropriated and therefore also made-up) seems to have really upset people, so I'm happy to stand down on that. (And in fact, re-reading 422 now, it is basically what I'm looking for - just would rather it wasn't within the context of a WebDAV submission. My ideal outcome would be to rewrite the description of 422 in a more generic sense, and have it adopted as the de facto response in the event of a request validation error.)

All of which brings me to the real purpose of this post. In researching 422, I have come across endless discussion around the rights and wrongs of HTTP code assignments, but not one post or comment on what, to me, is the main reason for adopting status codes at all (beyond the functioning of the web, of course) - which is to facilitate testing and analytics, i.e. making it easier for computers to understand what is going on without having to read HTML.

Status codes are transparent and invisible to end users, but stick out like a sore thumb in logs, which makes them invaluable for analysis. A log file that includes nothing but 200s or the occasional 404 provides very little insight in to what is really going on.

Similarly, having to interrogate the HTML body of a response to understand whether the HTTP request was "received, understood and accepted" or not is painful at best, and often misleading.
HTTP status codes aren't there for humans, they are there for non-humans, whether that be the routing infrastructure (understanding what to cache), or HTTP clients (understanding what to do next).

And remember, browsers are not the only clients.

(PS - to everyone who disagrees with me, bear in mind that I do understand your objections, I just don't agree with them. I'm trying to make the web more useful, not studying for my Masters.)

[UPDATE]

I found this in the original HTTP 1.1 specification:

HTTP status codes are extensible. HTTP applications are not required to understand the meaning of all registered status codes, though such understanding is obviously desirable. However, applications MUST understand the class of any status code, as indicated by the first digit, and treat any unrecognized response as being equivalent to the x00 status code of that class, with the exception that an unrecognized response MUST NOT be cached. For example, if an unrecognized status code of 431 is received by the client, it can safely assume that there was something wrong with its request and treat the response as if it had received a 400 status code. In such cases, user agents SHOULD present to the user the entity returned with the response, since that entity is likely to include human- readable information which will explain the unusual status.

You can make up your own mind from this (I have).

Programmers are not interchangeable

[UPDATE: just found this in my drafts folder from a few months ago - not sure why I never published it at the time, but since YJ is out-and-proud, I thought I should push the button.]

This essay by Paul Graham (YC founder) from back in 2007 has been doing the rounds recently (trending on HN), and it struck a chord with me because of the following extract and the new project with which I am involved - YunoJuno - www.yunojuno.com.

YJ is founded on precisely this belief - that certain 'craft skills' (designers, developers, ux etc.) are not interchangeable, and that the success of a project is very largely dependent on the specific individuals involved.

To that end, we want to create exactly the kind of relationship that Paul outlines below - "Maybe we could define a new kind of organization that combined the efforts of individuals without requiring them to be interchangeable." 

It's a vision we share, and one that I hope YunoJuno will be a part of (specifically the part that looks after the individuals). If you have a skill, and you want to treated as an individual, but to feel part of something greater, then head on over and 'Join the Family'.

(We're in beta at the moment, launching in earnest in the new year. Not any more - we are open for business.)

One of the defining qualities of organizations since there have been such a thing is to treat individuals as interchangeable parts. This works well for more parallelizable tasks, like fighting wars. For most of history a well-drilled army of professional soldiers could be counted on to beat an army of individual warriors, no matter how valorous. But having ideas is not very parallelizable. 

And that's what programs are: ideas. 

It's not merely true that organizations dislike the idea of depending on individual genius, it's a tautology. It's part of the definition of an organization not to. Of our current concept of an organization, at least. 

Maybe we could define a new kind of organization that combined the efforts of individuals without requiring them to be interchangeable. Arguably a market is such a form of organization, though it may be more accurate to describe a market as a degenerate case—as what you get by default when organization isn't possible. 

Probably the best we'll do is some kind of hack, like making the programming parts of an organization work differently from the rest. Perhaps the optimal solution is for big companies not even to try to develop ideas in house, but simply to buy them. But regardless of what the solution turns out to be, the first step is to realize there's a problem. There is a contradiction in the very phrase "software company." The two words are pulling in opposite directions. Any good programmer in a large organization is going to be at odds with it, because organizations are designed to prevent what programmers strive for.

Desktop ambivalence (in a post-OS world)

There was a time when your choice of desktop OS said something about you - you were a Windows nerd or an Apple hipster - hell, Apple even based an entire advertising campaign on it. That was then. This is now.

A couple of years ago I was exclusively a Windows person. I wrote software for the .NET platform, which made Visual Studio my tool of choice, and the software I wrote ran on Windows servers. Even if I was working with VMs, I was running Windows Server VMs inside my Windows desktop OS.

I was always aware of Apple, but as I wasn't a hipster, and had never opened Photoshop, I left them to one side. When I was forced to use OSX I felt quite strongly that it was inferior to Windows.

Then I went to work at a creative agency, where I was swimming in an ocean of Mac-luvin. I started using one on the odd occasions when I need to borrow an office laptop. I still didn't like OSX, but I could get along with it.

At the same time I started playing around with Python, and I decided to scratch an itch I had around my complete lack of understanding of Linux as an OS. So I installed Ubuntu on a VM, and started playing with that.

Then I was given an iPad by work. And I bought myself a Windows 7 phone. So now I had Windows 7, Windows Phone 7, iOS, OSX, and Ubuntu in my life. I even installed Windows 8 when it came out (and paid for the upgrade!), but uninstalled it a week later (you can read about that here).

I finally realised that I had become ambivalent to the OS - and started concentrating on the tools I was using instead. I dropped Visual Studio in favour of Sublime Text, and dropped Word in favour of anything that worked with Markdown (which is anything).

I still don't like OSX - the fact that maximising a window doesn't, you know, maximise it, and the woeful Finder being my pet hates - but I bought myself a MacBook Air as my primary computer simply because I like the hardware, and frankly, I don't really care about the OS.

A beautifully designed piece of hardware running beautifully designed, and simple, software (stand up iA Writer), running on an OS that just gets out of my way, is all I need these days. Are we in the post-OS-as-brand world?

Postscript: the logical conclusion of this is no (discernible) OS at all. Over Christmas I thought I'd be a good son and back up the laptop I bought for my mother a couple of years ago. She uses it infrequently, but enough, and I figured she probably had something of value on it worth saving. In fact there was nothing - at all - on it. Every single thing she does on her computer is web-based. She is the Chrome OS poster child (grandmother).

Testing HTML emails

Someone asked me yesterday if I could send the contents of an HTML file as an HTML-formatted email to a client, as they wanted to see what it looked like. Sending HTML emails is usually something that's either done through an online service (MailChimp et al.) or from the internals of a platform build where you (as a dev) have access to templates and SMTP servers and all that that entails.

Just sending a file as an HTML-formatted email is surprisingly hard to do (i.e. you can't type raw HTML into your email client and have it render in the recipient's inbox.)

So I put a script together to do it - https://gist.github.com/4176850

It requires Python (it's a Python script), but runs from the command line, thus:




$ python send_html_email.py --filename='test_file.html' --recipient=hugo@example.com

It runs through Gmail's SMTP service, so you will need a Gmail account - and you'll be prompted for your email and password when you run the script.

Feel free to use, feedback, fork, etc.

[UPDATE]

I have recently come across the service mailtrap (mailtrap.io), which is a brilliant companion to this sort of solution. Mailtrap operates an SMTP service that doesn't go anywhere. Sound crazy, but it's great - you set up an account, get a set of SMTP credentials, and then everything you send to that server gets parked in the Mailtrap inbox - where you can see the emails received as HTML, plain text and even source. Try it out.

Windows 8 - first thoughts

tl;dr I've been through the pain so you don't have to - please do not install
Windows 8 on a desktop, non-touch, computer - you can thank me later.

Last weekend I installed (and paid for) a copy of Windows 8 on my home desktop.
I complained about it at the time (display driver issues), but persevered, so I
thought I'd share my experience.

Before starting, and to give context to my story, here's my history with MSFT:
for many years I made my living building application with Microsoft technologies.
I've created Excel spreadsheets, Access databases, VB6 applications, eMbedded
Visual Basic mobile applications, .NET applications (Windows and Web, from 1.0
through to the current period), and large enterprise implementations of their
core server products; I worked for a year with Redmond on the 'Jupiter' project,
which became BizTalk 2004, and earned a living as a BizTalk consultant for some
time after that. I've owned mobile Windows products since the early days of
WinCE - I had black and white Cassioepia PDA, and all the terrible phones in
between. I'm not a fanboy, but it's not far off to say that Microsoft and their
platform have paid for my house, my holidays and most of my belongings. I owe
them.

Recently I've branched out (not entirely of my own volition): I was given an
iPad (1) as a present a couple of years ago, which is great; I bought myself a
Nexus 7, which is better (for me, as reading is my main activity on a tablet).
I have an Android phone (which I hate), but only because I lost my Windows Phone
7 (which I loved). At work I switch between a MacBook Pro laptop (pretty good),
and a desktop which runs both Windows 7 and Ubuntu, which I use for development.

So I think it's fair to say that I have a pretty good grasp of the main desktop,
tablet and phone operating systems.

Which makes my experience with the Windows 8 all the more extraordinary. Quite
simply it's the worst operating system I've yet encountered. And one week after
installing it I have returned to Windows 7, which is the best desktop operating
system to date (IMO - I know Mac-lovers think OSX is great, but really, it
isn't. Finder, anyone?)

Windows Phone 7 is fantastic, a real break from the competition, and a bold,
imaginative statement from Redmond. It was clearly created by a team with great
talent, given the freedom to do what they are good at. WP7 is what happens when
you get the right people in the room, and let them do their best work. The problem
is that on the back its critical (if not commercial) acclaim, something incredible
appears to have happened.

It feels (and I have no inside knowledge on this) as if the WP7 team were fired,
and then their work was handed to a group with less talent, no background on the
project, and simply told to "make Windows look like this."

I cannot imagine any circumstances under which the new start screen would seem
appropriate on a desktop, operated by a mouse. Everything about it screams
"touch me", which is quite annoying on a 22" non-touch screen. On which subject,
ENORMOUS touch-friendly buttons looking f*king ridiculous on a large screen.
The installed apps (Photos, People etc.) look absurd.

All of which I could forgive, if they hadn't also messed with the 'old-style'
desktop. There is no start button, for which the person responsible should be
fired, and the Metro-style translated to the rich windowed multi-tasking envir-
onment that is Windows, looks like they just forgot to finish the product.

Windows 8 is a catastrophic downgrade in User Experience, which is a shame as I
believe it covers a bunch of really quite important internal changes. Most
companies use pointless eye-candy to distract from the lack of real features,
but it appears as if MSFT have pulled off a first - drawing attention away from
the good stuff to concentrate on the terrible UI/UX.

Do not upgrade.