Saturday, December 01, 2012

Testing HTML emails

Someone asked me yesterday if I could send the contents of an HTML file as an HTML-formatted email to a client, as they wanted to see what it looked like. Sending HTML emails is usually something that's either done through an online service (MailChimp et al.) or from the internals of a platform build where you (as a dev) have access to templates and SMTP servers and all that that entails.

Just sending a file as an HTML-formatted email is surprisingly hard to do (i.e. you can't type raw HTML into your email client and have it render in the recipient's inbox.)

So I put a script together to do it -

It requires Python (it's a Python script), but runs from the command line, thus:

$ python --filename='test_file.html'

It runs through Gmail's SMTP service, so you will need a Gmail account - and you'll be prompted for your email and password when you run the script.

Feel free to use, feedback, fork, etc.


I have recently come across the service mailtrap (, which is a brilliant companion to this sort of solution. Mailtrap operates an SMTP service that doesn't go anywhere. Sound crazy, but it's great - you set up an account, get a set of SMTP credentials, and then everything you send to that server gets parked in the Mailtrap inbox - where you can see the emails received as HTML, plain text and even source. Try it out.

Saturday, November 03, 2012

Windows 8 - first thoughts

tl;dr I've been through the pain so you don't have to - please do not install
Windows 8 on a desktop, non-touch, computer - you can thank me later.

Last weekend I installed (and paid for) a copy of Windows 8 on my home desktop.
I complained about it at the time (display driver issues), but persevered, so I
thought I'd share my experience.

Before starting, and to give context to my story, here's my history with MSFT:
for many years I made my living building application with Microsoft technologies.
I've created Excel spreadsheets, Access databases, VB6 applications, eMbedded
Visual Basic mobile applications, .NET applications (Windows and Web, from 1.0
through to the current period), and large enterprise implementations of their
core server products; I worked for a year with Redmond on the 'Jupiter' project,
which became BizTalk 2004, and earned a living as a BizTalk consultant for some
time after that. I've owned mobile Windows products since the early days of
WinCE - I had black and white Cassioepia PDA, and all the terrible phones in
between. I'm not a fanboy, but it's not far off to say that Microsoft and their
platform have paid for my house, my holidays and most of my belongings. I owe

Recently I've branched out (not entirely of my own volition): I was given an
iPad (1) as a present a couple of years ago, which is great; I bought myself a
Nexus 7, which is better (for me, as reading is my main activity on a tablet).
I have an Android phone (which I hate), but only because I lost my Windows Phone
7 (which I loved). At work I switch between a MacBook Pro laptop (pretty good),
and a desktop which runs both Windows 7 and Ubuntu, which I use for development.

So I think it's fair to say that I have a pretty good grasp of the main desktop,
tablet and phone operating systems.

Which makes my experience with the Windows 8 all the more extraordinary. Quite
simply it's the worst operating system I've yet encountered. And one week after
installing it I have returned to Windows 7, which is the best desktop operating
system to date (IMO - I know Mac-lovers think OSX is great, but really, it
isn't. Finder, anyone?)

Windows Phone 7 is fantastic, a real break from the competition, and a bold,
imaginative statement from Redmond. It was clearly created by a team with great
talent, given the freedom to do what they are good at. WP7 is what happens when
you get the right people in the room, and let them do their best work. The problem
is that on the back its critical (if not commercial) acclaim, something incredible
appears to have happened.

It feels (and I have no inside knowledge on this) as if the WP7 team were fired,
and then their work was handed to a group with less talent, no background on the
project, and simply told to "make Windows look like this."

I cannot imagine any circumstances under which the new start screen would seem
appropriate on a desktop, operated by a mouse. Everything about it screams
"touch me", which is quite annoying on a 22" non-touch screen. On which subject,
ENORMOUS touch-friendly buttons looking f*king ridiculous on a large screen.
The installed apps (Photos, People etc.) look absurd.

All of which I could forgive, if they hadn't also messed with the 'old-style'
desktop. There is no start button, for which the person responsible should be
fired, and the Metro-style translated to the rich windowed multi-tasking envir-
onment that is Windows, looks like they just forgot to finish the product.

Windows 8 is a catastrophic downgrade in User Experience, which is a shame as I
believe it covers a bunch of really quite important internal changes. Most
companies use pointless eye-candy to distract from the lack of real features,
but it appears as if MSFT have pulled off a first - drawing attention away from
the good stuff to concentrate on the terrible UI/UX.

Do not upgrade.

Wednesday, October 31, 2012

Why Google should get behind Markdown

In the war against complexity, Markdown has emerged as the winner in the
simple-text-formatting battle. There are others - notably Textile (supported
by Basecamp prior to its refresh) and reStructuredText (heavily supported by the
Python community) - but Markdown is the clear leader in its field.

In recent days there's been a bit of a hoo-hah about the lack of a formal spec.
for it (@gruber, the BDFL of Markdown doesn't seem that bothered), and the pro-
liferation of different variants / flavours. For instance, StackOverflow and
GitHub both have subtle extensions that favour their own specific use case.

I don't want to get involved in that debate, but now that it has appeared in
the 'public' domain, I thought I'd add one idea that occurred to me this
morning (whilst writing and sharing some notes).

I use iA's writer for taking notes in meetings, which supports .md, and I use
Sublime as my general purpose writing / coding tool, which also has a great
'preview in browser' function for .md files. What I'm missing however is a good
online tool for a.) making notes online, and b.) sharing those notes as HTML
pages (as per the preview in browser function) with other people.

At the same time, I use Google Docs for sharing docs online; one of the
ongoing complaints about gdocs is that it's such a poor relation to Office. So
here's what I suggest (to Google) - instead of chasing Office and adding
features, how about removing features, and simply becoming the de facto Markdown
editor / browser / publishing platform of choice. Put your weight behind the
specification effort (i.e. write a bunch of formal compliance tests for .md),
then allow people to write in .md in gdocs, and to publish them as HTML for

They already have partial support for *bold* and _italic_ markup on Google+
posts, so they clearly have thought about this.

(And yes, I know that people who understand Markdown are a tiny niche audience,
but then I'm not expecting anyone from Google to actually read this, so I don't
need to be practical.)

Monday, October 22, 2012

[draft] Estimation is guesswork.

Estimation (in software development circles) is neither an art nor a science - it's guesswork. And yet we continue to perpetuate the myth that it's possible to predict how long something will take to build.

I have a post in me about the 'immutablility of developers' and their inability to change the rate at which they work, a fact of life that very few project managers seems to have grasped.

This is just a placeholder (note to self to write it!)

Thursday, September 13, 2012

Experiments in rebase.

Thought I'd post this as a visual for anyone else getting their head around the difference between rebase and merge.

I created an project, and then created a series of commits to that project (by 'touch'ing a text file), across two branches, 'master' and 'branch':

- [master] touch readme1 'first commit to master'
- [master] touch readme2 'second commit to master'
- git checkout -b branch
- [branch] touch readme3 'first commit to rebase'
- [branch] touch readme4 'second commit to rebase'
- git checkout 'master'
- [master] touch readme5 'third commit to master'

At this point I had two diverged branches.

Running $ git log --oneline on [master] i have:

518f971 third commit to master
4a6d74b second commit to master
0f788ce first commit to master

on [branch] I have:

10e8078 second commit to branch
0122b8b first commit to branch
4a6d74b second commit to master
0f788ce first commit to master

I now copied the entire directory so that I have two versions of the project in exactly the same state, one of which I will rebase, the other I will merge.


$ git checkout branch
$ git rebase master

7b9e782 second commit to branch
58f65f3 first commit to branch
518f971 third commit to master
4a6d74b second commit to master
0f788ce first commit to master

At this point, master is behind branch, but can be 'fast-forwarded' by merging:

$ git checkout master
$ git merge branch


$ git checkout master
$ git merge branch

a06b559 Merge branch 'branch' 

518f971 third commit to master 
10e8078 second commit to branch 
0122b8b first commit to branch 
4a6d74b second commit to master 
0f788ce first commit to master

This time you get an extra commit - which is the merge commit itself. This method preserves the exact history - i.e. the fact that development branched, and was then merged back in successfully. If this is important (and I'm definitely not in the 'always rebase' camp, then go with merge. Otherwise, rebase will give you a cleaner history. (NB like a supermodel, the better looking the history, the more deceptive the reality. You may prefer to preserve the history of branching.)

Tuesday, September 11, 2012

GitHub goes down, and everyone just carries on...

GitHub has been on & off all day today, and at first one would imagine that that would cause a gigantic PITA for developers, and that productivity in all places 'silicon' (valley, fen, glen, roundabout, savannah, desert, glacier,...) would plummet.

However, since GitHub uses git, and git is a DVCS, whilst users will have lost all the shiny unicorns that GH provides, they could, if their work really mattered, have simply pushed their repository to another hosted git service - like Codebase, BitBucket, Assembla, ..., and then carried on as normal. A little bit of coordination required to get teammates the new repo URL, and they'd be back online in minutes.

Or simply continued working locally, and committing, until GH gets back online. I've often done the reverse - I have private repos in BitBucket (free!) which I have periodically pushed to GH in order to take advantage of some of the branch visualisations.

Try doing that with TFS or SVN.

Thursday, August 09, 2012

Django's extended user profile...

Django, as Python's (arguably) most popular web framework, follows the 'batteries included' philosophy of the language with a large number of built-in web features. One of these is the 'auth' system, which provides simple user and group authentication and permissions management.

The core User class is 'skeletal' in nature - containing as it does just email, username, first name, last name and password profile fields (as well as some other utility attributes such as 'is_staff' etc). This is unlikely to ever satisfy a real-world scenario, and so Django includes a documented mechanism for extending this base user record with your own application-specific extended profile. It's simple to set up and use - you define a profile model, and then add that model as the AUTH_PROFILE_MODULE setting. Once this is set up, you can simply call the get_profile() method of any user object, and it will return the profile object.

Unfortunately, there is one fundamental limitation to this pattern - it is assumed that every user registering on your site has the same profile. So if, for instance, you wanted teachers and pupils, or perhaps suppliers and customers; or buyers and sellers, each with their own profile type, you are out of luck.

Google this particular issue and you will find a plethora of workarounds and hacks, all of which seem to start with the same underlying assumption - that the core django framework does some kind of magic that cannot be subverted; get_profile() is sacrosanct.

One of the joys of using open source software, of course, is that you can open up frameworks and just take a look at how they work. So that's what I've done, and this is what I've found...

When using extended user profiles within django, there are really only two key requirements - first, that when a new user signs up, the user profile is created; and second, that when you call get_profile() on a user, you get the correct profile object back again. Let's dig into both of those in turn, and consider how we might adapt them to use multiple profile types.

1. Creating the user profile.

When creating the extended user profile, the django framework provides precisely no assistance. It is entirely up to the application developer to manage this process. To quote from the docs:
The method get_profile() does not create a profile if one does not exist. You need to register a handler for the User model'sdjango.db.models.signals.post_save signal and, in the handler, if created is True, create the associated user profile:
So, in this case, it makes no difference whether you have one or more user profile classes to use - it's entirely up to you. You would probably have something similar to this in your views:

    if kwargs['type']=='teacher':
        profile = TeacherProfile(user=request.user, subject=kwargs['subject'])
    elif kwargs['type']=='student':
        profile = StudentProfile(user=request.user, year=kwargs['year'])

2. Retrieving the user profile.

This is where django, in theory, starts to add value. The framework includes the user.get_profile() method, which will return the relevant profile object for the user. The problem being that it will only support one, fixed, profile class for all users in the system. This is where the source code helps us:

    def get_profile(self):
        Returns site-specific profile for this user. Raises
        SiteProfileNotAvailable if this site does not allow profiles.
        if not hasattr(self, '_profile_cache'):
            from django.conf import settings
            if not getattr(settings, 'AUTH_PROFILE_MODULE', False):
                raise SiteProfileNotAvailable(
                    'You need to set AUTH_PROFILE_MODULE in your project '
                app_label, model_name = settings.AUTH_PROFILE_MODULE.split('.')
            except ValueError:
                raise SiteProfileNotAvailable(
                    'app_label and model_name should be separated by a dot in '
                    'the AUTH_PROFILE_MODULE setting')
                model = models.get_model(app_label, model_name)
                if model is None:
                    raise SiteProfileNotAvailable(
                        'Unable to load the profile model, check '
                        'AUTH_PROFILE_MODULE in your project settings')
                self._profile_cache = model._default_manager.using(
                self._profile_cache.user = self
            except (ImportError, ImproperlyConfigured):
                raise SiteProfileNotAvailable
        return self._profile_cache

Observation #1 is that it's not a very complicated method; we should be able to work out what this does. Let's remove some of the error checking and reduce it to what we really want:

    def get_profile(self):
        from django.conf import settings
        app_label, model_name = settings.AUTH_PROFILE_MODULE.split('.')
        model = models.get_model(app_label, model_name)
        return model._default_manager.using(

Now let's reduce that to pseudocode:

        # pseudocode - do not attempt to run!
   def get_profile(self):


Observation #2 is that this is trivial - it does exactly what you would think, and no more. There is nothing to be scared of here. The only problem is that is only expects one value to exist in the settings file, and is fixed on that assumption.

So, how do we take this knowledge and use it to our advantage. My own personal view is that, having read through the code, you should feel perfectly happy and comfortable to ignore it, and roll your own. It really isn't doing anything magic. Nothing will break.

Using our teacher/student app above, you can simply create your own get_profile() method, and hard-code the 'get' code (you can stick the method anywhere you feel comfortable using it - I have a profiles app that defines the models, and have the get_profiles() method in file):

        # pseudocode - do not attempt to run!
   def get_profile2(user):

        if user is teacher:
           return get_teacher_profile(user_id =
        elif user is student:
           return get_student_profile(user_id =

The obvious problem in the pseudocode above is how to check if the user is a teacher or student, given that there is nothing on the user object that we can use to determine what type the user is. The simplest answer to this, and it is a hack, is to use Groups. By assigning users to relevant groups when they register you can do the following:

    def get_profile2(user):
        if user.groups.filter(name='teachers').exists():
           return get_teacher_profile(user_id =
        elif user.groups.filter(name='students').exists():
           return get_student_profile(user_id =

This is a workable solution, but it does require hard-coding the groups, which makes it more of a copy-and-paste solution than a pluggable django app or extension.

The ideal answer is for the core django project to support multiple profile types within a project, by changing the way it currently works. The fix is trivial:
  1. Change the AUTH_PROFILE_MODULE to use a dictionary instead of a string to store a mapping of user types and their profile models
  2. Add an additional field to the core user model, called something like 'user_type' (or, heaven forbid, 'profile')
  3. Update auth.models.User.get_profile() to look up the model based on the user_type / profile attribute and use that instead
I have hacked my local copy of django to do just this, and this method works as expected. A fully-integrated, multi-profile, django installation. Of course, this isn't supportable by any hosting company, so is not a viable short-term solution. In fact it's only supportable if it gets pulled into the next version of django. I'd love to see this happen, so if you think this is a good idea, please add any feedback below; if enough people want it to happen...