A/B testing is all the rage in certain web development circles. Naturally, when something becomes popular the criticism starts. I’ve read some unconvincing attacks on A/B testing recently, as well as some good ones, so I want to lay down my thoughts on what A/B testing is and what it isn’t. (more…)
Author Archive
Epistemology and A/B Testing
Thursday, July 22nd, 2010Birmingham Events
Tuesday, July 20th, 2010Birmingham doesn’t have great visibility at the intersection of software development, design, and entrepreneurship in which Untyped operates, but in the last few months there has been a surge of events that suggest this is changing.
Covering the Election
Thursday, May 6th, 2010Today is the closest and most interesting general election in the UK that I can remember. This blog isn’t the place to talk politics, but while reading the manifestos of the three major parties I was struck by their design, and particularly the design of their covers, and I’m going to share some thoughts today on this topic. I think it’s interesting to look at the message the each party is trying to convey with their design and in particular how they all, for me, got it wrong. In alphabetical order, here they are:
The Conservatives
Gold lettering on a blue cloth binding. I knew I’d seen this before but it took a while before I remembered where: my parent’s old textbooks, which I used to leaf through as a child, had this kind of binding beneath their dust jackets. I looked through all my and my wife’s textbooks and didn’t find any the same. Add in the stuffy “Invitation” and to me this says old. Very traditional, very establishment, and very much at odds with the image of David Cameron, with no tie and top button undone, presented in the Tory advertising.
Labour
This is not a subtle cover. The illustration benefits from the fact the UK is a small country and most places look more or the less the same, so the patchwork fields will look like “home” to almost everyone. I’m a bit surprised the couple looks so white; I’d expect Labour to embrace diversity a bit more. However the whole feel of the cover is quite retro. The style of illustration and the rural setting (the UK is very urbanised) both seem to looking backward to me. I like the alliteration in the text. That blazing hot sun disturbs me; it looks more like a nuke going off than the gentle British sun I’m used to.
Liberal Democrats
It’s hard to say much about this cover, as it doesn’t say much to me. The repetition of “fair” is effective, and this is continued inside. The colours are washed out. This cover doesn’t really inspire any emotion in me; it looks more like an annual report than a manifesto!
Of the three I like the Labour cover the most, but as you’ve seen none of them really worked for me. This isn’t too surprising; major political parties must paint with such a broad brush that they cannot really address any small demographic. Now enough about the manifestos; go out and vote!
Formalising Bonds with the Informal
Tuesday, April 20th, 2010There is an interesting move underway by the US Securities and Exchange Commission (SEC) to more precisely define the meaning of certain asset backed securities (like the now infamous mortgage-backed securities that were central to the recent crash). The NY times has covered the story from a high level, but what of particular interest to me is the proposal to specify the meaning of the bonds in Python. This is a step is the right direction but Python is not the answer.
The core problem here is to give a clear and unambiguous meaning to a bond. This requires the language in which it is written is precisely defined. Python is not precisely defined. There is only a prose definition of the language, which is inadequate in the same way that the prose definitions of bonds are inadequate, and of course there are differences between various versions and implementation of Python. Since Python is not precisely defined the only meaning one can give to a program in Python is whatever the particular implementation one uses does with that program.
In contrast there are languages that are formally defined, such Standard ML and Scheme. These would make a sound basis for the formal definition of bonds. In turns out that functional languages also make a good (meaning expressive and convenient) basis for the formal definition of bonds. There is a great paper on expressing contracts in Haskell and at least one company has implemented this idea in a commercial system (in O’Caml, I believe). So my advice to the SEC: use an appropriate subset of Scheme or Standard ML, or hire someone to create a formally defined DSL, but don’t use a language without a formal definition if precision is your goal.
Hacking Motivation
Friday, March 12th, 2010Like most people there are parts of my job I don’t enjoy. Writing quotes, for example, is not my favourite activity. Recently I’ve been thinking about a way to “hack” my motivation, to make these parts of my job more interesting and enjoyable. Here are my ideas.
Self-determination theory posits that motivation derives from autonomy, relatedness, and competency. The first two are easy to come by in a small business. I’m my own boss and what I do is critical to the success of the business, and hence to my continued mortgage payments. So it seems the later is the limiting factor. This matches my experience; I can happily program for hours (days? years?) and I consider my pretty damn good at it. Writing quotes is painful and it takes me a long time to finish one. So perhaps if I can gain competency I’ll enjoy writing quotes more and thus become more motivated to complete them.
How does one gain competency? Deliberate practice is how. It may take 10000 hours to become an expert but I don’t need to be an expert quote writer, just a better one than I am today. Luckily the learning curve suggests that I can get to a good level of competency with relatively little effort. So this is my hack for improving motivation: pick something I do often that I’m bad at and practice it in a fun way. For quote writing that might mean trying to write a good quote for a preposterous imagined product. If you enjoy the ridiculous like I do you’d probably find that fun.
I’m not actually going to practice quote writing right now as I have other big projects occupying me, but I intend to try this technique out in the future. I’ll let you all know how this turns out for me. Finally, if you try my technique I’d love to hear how it works out for you.
Is the iPad the beginning of the end for Intel?
Tuesday, February 2nd, 2010Much has been written about the iPad since its launch. I’m sympathetic to the concerns about the closed nature of the iPad, and I think the iBookStore (along with the Kindle) is going to have a big effect on the book market, but I want to focus on something I haven’t seen much discussed: the A4 chip powering the iPad.
What you need to know about the A4 is this: at its core is an ARM Cortex-A9 MP CPU. ARM cores also power the iPhone and about every other smartphone out there. Intel just can’t compete in this market as their chips require too much power. This weakness is, paradoxically, a result of their greatest strength: the Intel instruction set. Even the most modern Intel chip still retains the ability to execute code for the ancient 8086. This ensures you can run just about any program ever written for an Intel machine on the latest CPU, giving Intel an enormous software base to leverage. However supporting this instruction set comes at a cost. The 8086 instruction set is not a good fit for modern CPU designs, and the instruction set has accreted decades of cruft to try and wedge modern features into it. To get acceptable performance all modern Intel chips have vast amounts of silicon devoted to instruction decoding; that is, the process of turning instructions into so-called micro-ops, which are what the CPU actually executes. All this silicon takes power, which is why ARM handily beats Intel on performance-per-Watt.
Now this wasn’t an issue for Intel even a few years ago. But we’re seeing three things that ought to make them worried. The first is the iPad, showing that little devices can grow bigger and perhaps move into the PC market, in much the same way PCs took over from the scientific workstations of Apollo, DEC, Sun and others. The second is the increasing concern for performance-per-Watt from people like Google and Amazon whose huge server farms power the major Internet services. The third is open source software, and particularly GCC’s support for just about every CPU on the market. This means the software can be easily recompiled for a new architecture. Suddenly Intel’s dominance doesn’t seem so assured.
So perhaps in a few years ARM will become the dominant architecture, rather than Intel. Apple have already shown that switching architecture (twice!) isn’t so painful. And as someone who has been writing an Intel assembler for fun I can’t say I see this as a bad thing.
Biased Guide to DEFUN 2009
Friday, July 10th, 2009The DEFUN 2009 schedule has recently gone up. I haven’t seen many posts about it, so here’s my take on what looks good. DEFUN is looking pretty good for Scheme, which is a pleasant surprise; it is well known that Scheme is not mainstream at ICFP. There are three awesome tutorials on the schedule, covering DSLs, web programming, and embedded programming respectively. I’m particularly excited by the DSL tutorial, given by Matthew Flatt and Eli Barzilay. It is my belief that PLT Scheme is the best platform available for developing DSLs, but not many people are aware of its full abilities. I hope this tutorial will go a way to getting the information out there.
Session A5(c) most intrigues me of the remaining tutorials. Ur/Web throws a whole pile of research into type systems at the problem of developing web applications. Through the power of dependent types Ur/Web claims to eliminate invalid HTML, invalid SQL, code-injection attacks, and bunch of other issues. I certainly want to better understand how it works, and perhaps see if we can apply some of the metaprogramming ideas to our own libraries.
If you’re interested in DEFUN you’ll probably also be interested in the CUFP 2009 schedule. I should be at both, as well as most of ICFP. If you’d like to chat, drop me an email.
Flapjax: Second Batch
Friday, April 17th, 2009Flapjax is the awesome functional reactive Javascript library from Brown PLT. We had a good experience with Flapjax some time ago, but in the interim it seemed that the project died. Turns out it was just hibernating. In the last few days Flapjax 2.0 has been released, along with a tech. report describing the system in more detail than the somewhat brief documentation.
To celebrate I coded up a small animation library for Flapjax. It’s hosted on Github, not our usual Subversion server as I wanted to gain a bit more experience with Git.
More State on the Web
Thursday, March 26th, 2009As a followup to The State of State on the Web I want to mention stateless servlets, a relatively new feature of the PLT web server that make continuations (even) more usable. Stateless servlets are essentially a kind of servlet with serializable continuations. A serialized continuation can then be stored on the hard disk, in the URL, in a cookie, or using any other mechanism you desire. This gets around the issue of memory consumption that is a concern with normal continuations. I don’t have a lot of experience with this kind of servlet, but Jay’s experience is that they are faster than normal servlets and the continuations are typically less than 100 bytes (and so can easily be encoded in a URL). Very nice!
The State of State on the Web
Friday, March 20th, 2009There seems to be a miscomprehension that continuation based and RESTful web apps are mutually exclusive. Witness Nagare proudly proclaiming “no explicit URL routing / mapping … no global session object … no REST” as if continuation based frameworks were violently in opposition to these features. This is not the case. Fundamentally the issue is about managing state, and continuations, cookies, and friends are all approaches to solving the problem of encoding state over a stateless protocol. At Untyped we develop web apps that use a combination of continuations, RESTful URLs, and cookies for managing state and I believe this is the correct way to approach the problem. I hope this post will convince you of the merits of our approach.
Before looking at the tradeoffs of the different approaches I want to summarise continuations and their use in web applications. Simply put, the continuation of a program is what happens next. In the program (+ 5 (+ 2 1)) the continuation of (+ 2 1) is to evaluate (+ 5 []), where I’ve written [] to indicate the place where the value of (+ 2 1) goes. Now in Scheme we can capture a continuation, store it in a variable, and generally pass it around like any other value. This means we can effectively suspend a computation (by capturing a continuation) and then resume it at some time in the future (by invoking the continuation, which in Scheme appears as any a function application).
Now let’s look at what continuations do for web applications. A continuation-based framework associates a specific server state with a URL, which it does by capturing a continuation when a response is sent to a user. Everytime the user visits that URL they visit the same server state, invoking the captured continuation. As the user navigates around the site they build a history of server states that can be revisited using the back and forward buttons. This has several advantages. Firstly, if you don’t use mutation the back button will just work, because the user is just back to the same program state. Pretty neat. Furthermore, continuations give you procedure call semantics in your web app. Because a continuation is resumed when a URL is visited, to your program it appears as if the user’s request is the returned value of the function that sends your response. It’s as if you were using display and read on the web. This makes programming a lot simpler. For example, if you want to forward the user to a login page you just call the login page function, and it will return to the right place. No need to pass that page a URL to redirect the user to. This can be incredibly productive.
Now we’ve seen some of the advantages of continuations, we must consider the cases where the model falls down. There are two main issues: server load, and scope. Server load is simple. Every time you store a continuation on the server you use up some memory (RAM or disk space). At some point you have to reclaim that resource, so people may see “continuation expired” pages if they leave a long time between visits (though this is no worse that session expiry, which is quite common). Often a website has pages that are just displaying the results of simple queries to a database. These pages have no interesting state and using continuations in this case is wasteful of resources. Here RESTful approaches are appropriate, and we use them with, for example, the web server’s dispatchers.
Scope is another issue with continuation-based apps. Recall that continuation-based frameworks associate a particular URL, meaning a particular browser window (or tab), with a particular server state. There are some kinds of state that should be shared across all browser windows. Login information is a prevalent example. If I login to a site via one browser window, and then visit that site in another browser window I expect to already be logged in. This isn’t possible with continuations, as they are per window. Cookies, on the other hand, are per browser. So storing my login status in a cookie is the right thing to do.
In summary, RESTful approaches (URL routing, for example), cookies, and continuations are complementary and all have a place in web applications. Don’t think, for example, that is you use continuations you automatically reject everything RESTful! Finally, the Anton of Straaten addressed this issue from a different direction in his LL4 talk. Check it out for a different take on the problem.
Equivalently we could say the continuation of (+ 2 1) is (lambda (x) (+ 5 x)). This realisation is the key to continuation passing style, a program transformation useful in compilers and, perhaps surprisingly, AJAX web applications.