untyping: Web Development Archives

April 21, 2008

The Biographicon

The Biographicon is a very pretty web site of user contributed biographies. It is also written in Scheme, and here's one of the developers discussing how it was done.

Posted by Noel at 01:23 PM | Comments (2) | TrackBack

February 29, 2008

Announcing: Instaservlet

We've just released a package called Instaservlet, which enables you to get a servlet running in two lines of code, plus the servlet code. Here's a simple example:

  (require (planet "instaservlet.ss" ("untyped" "instaservlet.plt" 1)))

  (define (servlet request)
    '(html (head (title "It's working!"))
           (body (h1 "Instaservlet is in the house!"))))

  (go! servlet)

Try it in PLT Scheme 3.99 and see!

This package arose out of the development work I did on Smut Shorts, which made me I realise we needed to support a quick start for web development. Instaservlet is the first step in doing this. Not only does it setup the servlet, it does it in a robust manner. Continuations are managed using the LRU manager, which can be a bit difficult to setup but it much more robust than the time-out manager. Instaservlet uses some sensible defaults which should work for most small to medium size sites — they're the setting used on Smut Shorts, so we know they can handle a reasonable load. It also sets up a nice default page to handle continuation expiry. Instaservlet is good enough to get you started with, and future versions will enable more customisations so it can scale to any website build using PLT Scheme.

Posted by Noel at 02:43 PM | Comments (0) | TrackBack

February 14, 2008

Happy Valentine's from Untyped

For Valentine's Day we have created a new website, Smut Shorts. If you have something to say about love or lust, and can do so in 500 characters or less, then please add it to the growing number of “shorts”. It's anonymous and fun. Just, no porn thanks.

If you're reading this site, you're probably interested in the technical details behind Smut Shorts. It is a collaboration between a number of people, most of whom have chosen to be anonymous. The majority of the coding was done by yours truly, and therefore in Scheme. It is running on the PLT Scheme webserver (version 3.99) and uses PostgreSQL as the back-end. I coded it up in about two days. It was a side-project, so it was a bit of rush job and there is lots still to do. If you break the site let me know and I'll try to fix it.

A few interesting lessons were learned from doing this site. It all comes down to scalability, which is something that has recently been on my mind a lot. In this case we want to scale down to the low end — the guy who is just hacking up something in his spare time and wants to get it done in a hurry. Our frameworks, Snooze and Lylux, are pretty good but they don't support a fast start. You have to create a whole bunch of files before you've even got your first page up. Furthermore, we've always avoided creating a templating mechanism, as we've said that we'd rather use smart people who can balance parens than create this unnecessary divide between designers and programmers. I now recognise this is a mistake. Had we a templating mechanism I could have pushed more design work to my collaborators. It's not that they're stupid (far far from it) but they're busy and don't have time to learn even Scheme basics just so they change a few lines of text. If we're gonna grow the Scheme web-hacking community it has to start with dudes messing around in their spare time, so we need to address the low-end of scale. The high end of scale can wait till the IPO ;-)

Posted by Noel at 01:46 PM | Comments (3) | TrackBack

December 07, 2007

Announcing: Delirium

Delirium is a web-browser automation toolkit, which means it's a Scheme library that provides a bunch of functions that you can use to control a web browser. We expect the primary use will be for web testing, and Delirium can be used inside SchemeUnit like any other Scheme library.

For the Schemers Delirium isn't anything special, but we believe the use of continuations make Delirium a major advance over similar web testing tools like Selenium. If you write your server code in Scheme you can directly test how your server side responds to web tests with tests running on the same server. That is to say a test can interleave calls to the web browser and to the server side code, which is impractical without continuations. This features makes it much easier to write reliable and comprehensive tests.

Delirium is on PLaneT. Note the documentation was translated by hand from Scribble source. Some errors may have been introduced during the translation.

Posted by Noel at 02:08 PM | Comments (0) | TrackBack

November 23, 2007

Custom Dispatchers in the PLT Scheme Web Server

We've just released Instaweb 2.0. Instaweb is our utility that takes care of setting up the PLT web-server and running servlets. If you have a servlet in a file called servlet.ss with Instaweb you just need to write the following lines to get it running:

  (require (planet "instaweb.ss" ("schematics" "instaweb.plt" 2)))
  (instaweb)

The new version of Instaweb includes many new options and works in a slightly different way to the 1.0 branch. To my mind the best new feature is that Instaweb now configures the web-server to pass to the servlet all requests that don't match a file in the htdocs directory. This means your servlet no longer has to live under a URL starting with /servlets. You can read the documentation to get the full details of what's new. What I want to talk about here is how we implelemented this, as it illustrates some very nice features of the web-server that aren't well known.

In the web-server's terminology a dispatcher is a function that may generate a response given a request. Examples includes the filesystem dispatcher, which responds to requests with the contents of a file, and the servlet dispatcher, which invokes a servlet. Dispatchers are arranged in a list. The first dispatcher in the list inspects the request and, if it decides the request is relevant, generates a response. Otherwise control is passed to the next dispatcher in the list. For some time now the web-server has had a configurable dispatcher pipeline, which can be set by simply passing a value with the #:dispatch keyword to the serve function.

The web-server provides a number of dispatchers, all in the dispatchers subdirectory of the web-server collection. They all provide a make function that does most of the work. Here's how to use the file, servlet, and sequence dispatchers, the most generally useful ones:

The file dispatcher, in dispatch-files.ss, takes a single parameter, a function that converts a URL to a path (and another value that the dispatcher ignores). The path can name a file, which the dispatcher will serve if such a file actually exists, or it can name a directory, in which case the dispatcher will look for a file within that directory called index.html or index.htm.

To use the file dispatcher you will probably want the handy make-url->path function in filesystem-map.ss. Pass this function a base path (the directory where your files live), and it will return a function suitable to pass to the file dispatcher.

Here's an example of use:
```
  (require 
    (prefix file: (lib "dispatch-files.ss" "web-server" "dispatchers"))
    (lib "filesystem-map.ss" "web-server" "dispatchers"))

  (define base-path (string->path "/my/directory/of/files"))

  ;; htdocs-url->path : path -> (url -> path (list-of path-element))
  (define (htdocs-url->path path)
    (make-url->path (path->complete-path path)))
  
  ;; dispatch-htdocs : (connection request -> response)
  (define dispatch-htdocs
    (file:make #:url->path (htdocs-url->path base-path)))
```
The servlet dispatcher, in dispatch-servlets.ss is a bit more difficult to use as you need a function from the private subcollection of the web-server, suggesting the code reorganisation isn't quite finished. The make function takes two arguments, the first being a cache-table, and the second being a function that, like for the file dispatcher, maps URLs to paths. To construct a cache-table use the following lines of code:
```
  (require
    (lib "cache-table.ss" "web-server" "private"))

  (define cache-table (box (make-cache-table)))
```
If you want all URLs to go a particular servlet, as in Instaweb, the URL to path function just needs to return the path of the servlet. The function used in Instaweb is this:
```
  ;; serlvet-url->path : url -> path (list-of path-element)
  (define (servlet-url->path url)
    (let ([complete-servlet-path (path->complete-path servlet-path)])
      (values complete-servlet-path (explode-path* complete-servlet-path))))
```
Now we can create a dispatcher as follows:
```
  ;; clear-servlet-cache! : -> void
  ;; dispatch-servlets:    connection request -> response
  (define-values (clear-servlet-cache! dispatch-servlets)
    (servlet:make (box (make-cache-table))
                  #:url->path servlet-url->path))
```
The sequencer dispatcher couldn't be easier to use. It just takes any numbe of dispatchers and creates new dispatcher that tries them in sequence. For example:
```
  ;; dispatch-all : connection request -> response
  (define dispatch-all
    (sequencer:make dispatch-htdocs
                    dispatch-servlets))
```

With the above you should be able to create your own custom dispatchers. If you have problems just read the (very short) Instaweb code!

Posted by Noel at 04:12 PM | Comments (0) | TrackBack

August 02, 2007

A Scheme Case Study

If you've looked at the ICFP 2007 preliminary program you'll have noticed we're presenting “Experience Report: Scheme in Commercial Web Application Development”. We submitted the final version of the paper a couple of weeks ago, and I've finally got around to putting it online for your reading pleasure. The contents shouldn't come as a surprise: a summary of our experiences developing commercial web applications in PLT Scheme over the last year. We've tried to be honest, including the good and bad. Hopefully the points you'll take away are that we've been able to overcome initial problems with stability, and in a fairly short time we've developed a framework that compares well to popular alternatives such as Ruby on Rails.

The four page limit on experience reports is very tight, and unfortunately our experiences with Flapjax were cut from the final version. So let me say here that if you write Javascript code you need to check out Flapjax! Our Flapjax code is about half the size of the equivalent Javascript, and this is without using the Flapjax compiler. The only problem with Flapjax is performance in large networks. This is more a property of the poor quality of Javascript interpreters: Wolfenstein 3D on my 286 back in 1990-something was smoother than Javascript raycaster running today on my Powerbook. Luckily the new developments taking place at Mozilla will alleviate this problem in the next few years.

Posted by Noel at 09:47 PM | Comments (6) | TrackBack

June 20, 2007

Selenium Code Released

The Selenium code I talked about in an earlier post has been released to PLaneT

Posted by Noel at 01:21 PM | Comments (0)

January 22, 2007

Large Scale Web Site Development

Two interesting case studies on large web sites: MySpace and EBay. They are remarkably similar. Or perhaps it should be unremarkable. You don't get many chances to get that sort of system wrong, so it is natural that different groups should converge to tried and tested solutions. Ezra Kilty offers some thoughts on EBay and the great Goog, who of course have a system way different and way cooler than anyone else's.

Posted by Noel at 05:30 PM | Comments (0)

January 19, 2007

Flapjax in Action

In a few days we'll be releasing our first code using Flapjax to drive the user interface. The core of the system is a grid view that runs in the browser. The user can edit data in place, and when they move to a new row changes are sent to the server. Invalid data is flagged in the grid, and mousing over shows the reason for the error. Given we're probably the first people in the world to release commercial code using Flapjax a few notes on our experience are worthwhile.

The summary is: Flapjax worked very well for us. Flapjax code is much smaller and faster to write than the equivalent plain Javasscript, and Flapjax insulates you from most of the cross-browser issues. If your application revolves around events, and most Javscript applications do, Flapjax will probably be a good fit. However current Javascript implementations are so slow that they can limit how much you can take advantage of Flapjax.

Now, the details. We went through two design. Our first prototype rendered the TABLE that contained the grid data as one enormous Behaviour. Every field was an “edit in place” field, using more Flapjax code. (See here if you don't know what an edit in place field is, though note that this isn't the code we used.) This plays very nicely with the Flapjax model, as changes to any part grid automatically get updated in the browser. It worked fine for small tables, but scaled really badly; for large tables it was so slow to load and redraw that the browser would kill the script.

The main problem with our first prototype is that browsers are really slow at rendering elements inserted using the DOM. Our second design made two major changes to overcome these problems. First we dropped edit in place fields in favour of plain old INPUT elements, and then got the server to render the HTML instead of building the grid dynamically on the client side. So essentially we used Flapjax to handle events from the grid, but not to render the grid. This design is much faster; for example a grid with a thousand cells renders in about a second.

As with any new technology, the community still has to develop best practices and design patterns to make it easier to adopt Flapjax. The best design for error handling is still an open issue. Architectures for mixing OO and FRP (Flapjax) code are not well defined, though there is prior work. However we feel the benefit we got from Flapjax, in terms of shorter code and a faster development cycle, outweighed the cost.

Posted by Noel at 12:49 PM | Comments (1)

October 25, 2006

Firefox 2.0 Review

I'm editing this post using the Performancing extension for the new Firefox 2.0 browser. Being a web whore I simply had to download the latest version of Firefox as soon as it was out.

If anything, Firefox 2.0 is proof that the browser is now a stable product. The changes from 1.5 are incremental. There is nothing that particularly excites me on the user end.

From the point of view of a developer, it's a different story. Javascript 1.7 continues the evolution of Javascript from a hairball to a language you wouldn't be embarassed to take to school to meet your Professors. Array comprehensions, proper lexical scoping, and generators are the main features. Support for the WhatWG client-side session and persistent store adds more possibilities for storing continuations on the client-side. I'm interested to see that their motivating example is exactly the same as the one Shriram uses when proselytizing continuations (the “Orbitz bug”).

Of course, to use most of this you're stuck with Firefox. However, as Firefox evolves further from the functionality offered by IE it becomes more compelling as a platform is its own right. Some browser stats show encouraging numbers of people are using Firefox. Give it a few years and IE support may legitimately be optional.

Posted by Noel at 10:57 AM | Comments (0)

October 11, 2006

Debugging File Handle Exhaustion

Dave has been working like a maniac to switch our database code over from SQLite to PostgreSQL. PostgreSQL has two main advantages: it is much faster, and we can open up ODBC connections to the database for other uses that don't require a web interface. The change is now complete, however it hasn't been without some difficulties. One problem that bit us was running out of file handles. If you ever have a similar problem, here is how to debug it.

On Linux the /proc filesystem reflects a great many kernel resources. The particularly interesting directories for our purposes are:

The files file-nr and file-max in /proc/sys/fs.
The per process directories keyed by process ID

The first thing to check is the value of /proc/sys/fs/file-max, which is the maximum number of file handles allowed on your system. This shouldn't be a problem, but just ensure it isn't something ridiculously small. On our system we get:

$ cat /proc/sys/fs/file-max 89367

That should be plenty under any reasonable usage, but we can check how many file handles are open by reading the value of /proc/sys/fs/file-nr. On our system this is:

$ cat /proc/sys/fs/file-nr 920 0 89367

This first number is the number of file handles in use. Definitely no problem there. It must be that a process is exceeding the per-process limit on file handles. In our setup this could be either PostgreSQL or MzScheme. We need the process IDs to find out how many handles each is using.

$ ps -A | grep postmaster 12936 ? 00:00:00 postmaster 12937 ? 00:00:00 postmaster 12939 ? 00:00:00 postmaster 12940 ? 00:00:00 postmaster 12941 ? 00:00:00 postmaster $ ps -A | grep mzscheme 20382 ? 00:00:26 mzscheme

We can see how many handles are in use by looking in the directory for each process ID. For example, for the first PostgeSQL process:

$ sudo ls -l /proc/12936/fd/ | wc -l 4

So that PostgreSQL process is using 4 handles. The other processes are using similar numbers. So it must be our MzScheme process that is using up all the handles. We check that in a similar way, and the result is:

$ sudo ls -l /proc/20382/fd/ | grep socket | wc -l 193

Looks like we've found our culprit.

Posted by Noel at 04:06 PM | Comments (0)

July 14, 2006

Unaccustomed as I am to Public Speaking

If you happen to be in Birmingham on the 18th I'm presenting our current ideas on web development as part of the School of Computer Science's Cake Talk series. The abstract is below. If you intend to attend follow the link for location and time. My slides will go up after the talk.

Functional Programming and the Web

Continuations, functional reactive programming, and bidirectional programming. A random walk down Lambda the Ultimate or the next Big Thing in web development? In the long and glorious tradition of Cake Talks I will present some half-baked ideas that argue for the later interpretation. Turn up and decide for yourself.

Posted by Noel at 03:06 PM | Comments (0)

July 12, 2006

Unlib unchained

We're pleased to announce the release of Unlib, a library of utility functions. Like most PLT Scheme libraries it is available from PLaneT. You can also track development via our Subversion server. For now the URL is https://ssl.untyped.com/svn/repos/untyped.com/unlib/ so you can checkout the code like this:

svn checkout https://ssl.untyped.com/svn/repos/untyped.com/unlib/trunk unlib

It's mostly Dave G's work, so congratulations to Dave! (And extra congratulation to Dave G who graduated yesterday with a PhD in Computer Science!!)

Posted by Noel at 02:19 PM | Comments (0)

July 04, 2006

mod_authz_svn versus htpasswd

mod_authz_svn files have commas separating members of a group, but htpasswd files don't. Don't spend an hour wondering why your Subversion configuration isn't working because of this difference.

Posted by Noel at 02:46 PM | Comments (0)

July 03, 2006

More on the Pre-registration Project

Time for a bit more detail on the SBCS pre-registration project.

A bit of background: at QMUL, student information is held by the central Registry, but each School is responsible for collecting that information and verifying it. The old system collected all the information on paper. This information has to be checked for correctness (is the student's name spelled correctly; are they on the right degree programme; have they registred for the correct units?) before being typed up and sent on. With a thousand students you can imagine how much work this is during the short registration period at the beginning of the semester.

We've created an online replacement for the old system. Using a variety of information sources we've uploaded information on students, staff, programmes, and courses. Students are assigned to members of staff who act as their advisors, and are responsible for editing their registration information. Certain members of staff are capable of editing the other information we store, including programme and course descriptions. In effect we've created a complete custom content management system for all information related to registration. The School is very happy with the result, and so are we.

As we mentioned before, the site runs on PLT Scheme and SQLite, with Apache forwarding to the PLT web server. We use the 3m version of PLT Scheme. Memory consumption seems stable at around 200MB. We don't have any impressive uptime stats to report as we've been updating our PLT Scheme installation quite regularly which neccesitates bringing down the web server (current uptime is about a day). Peak load is modest; there are about 40 advisors who will be using the site at any one time. So far our highest daily load has been about a thousand hits. Last I checked we had about 20K lines of code. It has probably increased since.

Development has been interesting. We've run into two bugs in the PLT web server (both fixed), and found a way to reliably make SQLite crash (also now fixed). Oh yeah, and we've had our server flooded. We will be releasing some of our libraries as open-source as we get time to clean them up. Look for Unlib, our utility library, Snooze, our persistence manager, and Lylux, our web framework, at a PLaneT near you.

Finally, we made the claim “This is, to our knowledge, first large site to run the PLT Scheme web server continuously for any length of time”. There are at least two other prior contenders for the crown: Jacob Matthews, who puts the science back into speed-dating, and Shriram Krishnamurthi et al with Continue.

Posted by Noel at 04:09 PM | Comments (0)

May 31, 2006

Compile Your Code!

Here at Untyped Central Dave and I are hacking away like lumberjacks, which is a good thing as the project we're working on is due soon. We're constantly running tests and loading code into the web server, and we've noticed that these processes have been getting slower and slower. It turns out the bottleneck is the time to parse and byte compile our Scheme code. Simply byte compiling the code beforehand has made an incredible difference. Tests that used to take minutes now run in seconds. Two features of PLT Scheme make it really easy to integrate byte compilation into our development process. Firstly, mzc will follow dependencies when given the -k flag. So we just run mzc -k main.ss and all our code is compiled. Also useful is that PLT Scheme does the Right Thing and loads source code if it's newer than byte compiled code, so we don't have to constantly recompile our code. So we can just code away as normal, except every time we take a break we run mzc. Eventually we might write some code to recompile at regular intervals (say, every 10 minutes) but for now it isn't worth the effort.

Posted by Noel at 05:43 PM | Comments (1)

February 24, 2006

Nancy Typing

Dave's on a roll. A few days ago we heard he's on the ECMAScript committee. Now he follows up with a great post on Nancy typing (read the post to get the joke), and a PLT Scheme language that implements Javascript (available from PLaneT, of course).

Posted by Noel at 09:32 AM | Comments (0)

February 15, 2006

Design Patterns for Web Applications

The Yahoo! Design Pattern Library is a collection of design patterns for web applications, along with links to the also-newly-released Yahoo! User Interface Code Library. Most of the patterns should be familiar to web application developers but it is good to have them all collected in one place. The code should be good to, though I haven't had a chance to look at it yet.

Posted by Noel at 04:50 PM | Comments (0)

Lessons Learned from Big Web Apps

There's a good summary of lessons learned from building del.icio.us. It's a bit telepathic at times but there's a lot of good stuff in there.

Posted by Noel at 04:45 PM | Comments (0)

December 22, 2005

printf in AJAX? Sorry, that's not debugging.

This post is one in a continuing series where we internally debate the merits of the over-hyped promise of AJAX...

Debugging Ajax Requests in Prototype:

How do we debug our Ajax applications? The Rails Weenie has taken the Ajax Responder feature in Prototype...

I'd like to remind Noel that the distance of the Atlantic and business of the Christmas holidays are not going to keep me from pointing out that AJAX is an immature and dangerous platform to build a business on. Yes, I'm glad that GMail is there... and I suppose the Yahoo! Mail beta. However, these are fragile technologies to build upon.

How do we debug things written using the Prototype framework? printf. I mean, I'm glad that with AJAX and Rails I can whip something up quickly that "just works." However, it doesn't "just have test cases," or "just get internationalized", or "just stand up under load." They're rapid-prototyping tools, certainly nothing more. AJAX breaks usability standards, pushes data and computation to an unreliable substrate (the client's web browser), and as the post above provides some evidence for, there are no good debugging or tracing tools available for developers working in heterogeneous browser environments in Javascript.

So, Noel, riddle me this: why should developers be willing to take eight steps backwards and be shafted with printf as their primary debugging tool when working with AJAX?

Posted by jadudm at 08:06 PM | Comments (0)

November 19, 2005

Rich Web Clients: The Revenge

Earlier this month Matt posted a provocative article on rich internet applications. Here's a taster:

Ajax is a dangerous and immature technology. It is, in fact, a hack—a kludget—to provide rich-client functionality in the browser.

And here's my equally blistering riposte.

Hear that sound? It's the CEO of Oddpost, which was one of the first Ajax apps, and which sold for a reported US$30 million to Yahoo. He's crying himself to sleep 'cause Matt said his technology sucked. My point is not (just) to lampoon Matt, but to point that Ajax does work, and is worth a lot of cabbages. Clearly a lot of people are using it so it can't be that hard.

Matt says “Ajax (as currently exemplified by Javascript and XML over HTTP) is certainly only version 0.5”. Now I don't think that is entirely fair. There a many toolkits and libraries that make developing Ajax applications easier and more robust. Debugging support isn't so bad either. As for type checking...well I'm giving that one to Matt. At least for now.

So in conclusion I argue that's Matt case is not proved. In your face Matt! Tune in next time when we kiss and make up. Or Matt gives me a black eye. :-)

Posted by Noel at 09:10 AM | Comments (0)

October 27, 2005

SISCWeb Petstore

Ben Simon implements the Java Petstore in Scheme using the excellent SISCWeb framework, which runs in the equally excellent SISC Scheme implementation. Well worth a look, particularly the writeup which explains some of the advantages Scheme brings to web development.

Posted by Noel at 11:44 AM | Comments (0)

September 15, 2005

The Birmingham Course: OpenLaszlo to the Rescue!

We needed to deliver a prototype on-line exam quickly. Despite their simple appearance, on-line exams can have some complex interaction modes that are annoying to deal with in a client/server model. OpenLaszlo provided an excellent framework for quickly going from idea to solution.

The Birmingham Course approached us with the idea of placing online a mock exam for their MRCPsych course. While the time constraints were tight, we thought there was an interesting opportunity in the project, and enjoyed the enthusiasm that The Birmingham Course brought to the table.

I would have liked to do some paper prototyping, and work through the issues that might arise before starting. However, we also wanted students currently enrolled in the course to trial an on-line version of the mock exam. This left us in a bit of a pickle: can you, in one-and-one-half weeks, go from concept to implementation for an on-line examination, and get it right? (I live far enough away from Birmingham that I didn't have the ability to take a day or two and go up there to meet with the client.)

The mock exam had no data gathering requirements, nor were there any concerns with authentication and identifying users. Therefore, some of the trickier aspects of web-based application development were not concerns. However, we still needed to deliver 133 True/False questions, and 30 multiple-choice questions in a stateful way, so students could easily "go back" and change their answers, as well as get their end-of-exam score.

While there may be AJAX gurus out there who think this would be a piece of cake, I do almost all of this development work in Scheme, and have no particular expertise in HTML/CSS/Javascript interaction on twelve different browser platforms (that's why I run with a team). I certainly don't have a lot of patience for developing code in an unstructured and unsupportive environment. Put another way, I think the browser makes a lousy run-time environment; I want my compiler to check for obvious problems up front, and I want my runtime environment to be rich enough that I can go about debugging a running application. Your typical AJAX app fails on these counts far too readily for me to think that any AJAX code I develop in a hurry will be reliable. (I'd like to point out that I've never even given AJAX enough of my time to decide if my biases and opinions are right; I've only read code and stories from others, and I don't like what I see.)

Enter OpenLaszlo. It has a compiler, that tells me when I make silly mistakes. It has a rich run-time environment, which includes an interactive debugging REPL. I can create a window as easily as saying

<canvas><window>Hi there!</window></canvas>

It uses JavaScript 1.5 as it's scripting language, and has a really sweet little constraint engine built in. The upcoming 3.1 release has some very swish enhancements on the way as well.

I was able to quickly prototype ideas for the customer, push them to the WWW, and let them interact with the application. While it is possible to do all kinds of RPC from OpenLaszlo, I chose to employ another really nice feature of the environment: SOLO application deployment. If you're willing to give up some (not all) connectivity with remote hosts, OpenLaszlo apps can be compiled to a single, static Flash file. This means that any user with Flash 6 or better can interact with a very rich net-based app, and get a very desktop-like experience.

The end-product of a one-week mad dash can be seen online; I was quite pleased with it. I know I made some choices that were less than optimal, but the code can be refactored and improved in a future iteration of the project. I was also happy working with OpenLaszlo as a development tool---putting together GUIs was no more complex than writing a webpage, the event model and constraint system are both well-behaved and useful, and being able to deliver a static Flash document as an application means I don't have to worry about a timeouts on client/server interactions, nor do I have to wonder if IE4/IE5/IE6/Opera/Firefox/Mozilla/Safari is going to choke on my particular choice of HTML, CSS, and Javascript idioms. If you're some kind of "purist" who abhors Flash, then... I can't help you. But it's widely deployed, and much better behaved than a gaggle of browsers.

Food for thought. Point is, I liked it. And the customer loved it; the app behaved just the way they wanted. I don't know what else I can say.

Posted by jadudm at 09:14 AM | Comments (0)

September 12, 2005

Firefox 1.5 Beta 1

Firefox 1.5 beta 1 is out. I've been tracking Deer Park (the pre-beta releases of Firefox) for a while now so I'm excited to see this new release. I'm particularly enthusiastic about the canvas support as I'm having fun playing around with vector graphics. The Firefox canvas is much more complete, though slower, than Apple's buggy implementation in Safari. The other new features don't excite me as much, as the web applications I develop run on the general Internet. If you have a closed environment they will be more useful. In such an environment, XULRunner becomes very interesting.

Posted by Noel at 02:33 PM | Comments (0)

August 18, 2005

A little OpenLaszlo lovin'

No doubt I'll be playing with this technology more in the future, so I thought I'd introduce it now.

OpenLaszlo, released unto the world by Laszlo Systems, is an interesting combination of languages and ideas. It is an XML-based language for specifying the layout and behavior of rich internet applications. For example, I wrote a tabbed slideshow for photographs that you can find on my personal weblog, here....

No doubt I'll be playing with this technology more in the future, so I thought I'd introduce it now.

When you open a slide, audio begins playing.
When you switch slides, the current audio stops, and new audio is started.
I can layer objects (and remove them) over the photo content quickly and easily, when and where I want them.

I imagine this is all possible in JavaScript/DHTML, but... I don't know how. And, I don't feel I need to; the OpenLaszlo crew has provided a powerful platform for me to work from---a compiler, GUI toolkit, RPC mechanisms, and more. It seems to me that the whole JavaScript/CSS/AJAX thing is nothing more but a poor reinvention of the technology that OpenLaszlo provides. Granted, for full RPC-functionality, OpenLaszlo currently requires a servlet container, whereas AJAX-apps just need a browser with a JavaScript engine. However, you don't get much support from the compiler or run-time for debugging AJAX applications, whereas OpenLaszlo apps have a compiler and run-time debugger. That's worth installing a server to me. (It was a double-click operation on my Mac.)

If I forgo the OpenLaszlo server, I can statically compile my applications; although RPC is no longer available, I can still do HTTP POSTs, and that's enough to do something RESTful. But, instead of a webpage, I can create a very rich, interactive GUI-based environment that can be delivered to any browser that is Flash-capable. I hear Macromedia has pretty good penetration, and generally things Just Work within that environment, regardless of whether you are on Windows or Mac, IE or Firefox, etc.

So, I think it's a cool technology. I've been poking at the edges of it on-and-off for a few weeks now, and will follow up with some more about what I've been doing with it. My tutorial builder/photo slideshow demo just met a need I had; I think OpenLaszlo is capable of much, much more.

Posted by jadudm at 10:14 AM | Comments (0)

August 02, 2005

Two Databases Worth A Look

If MonetDB is half as good as claimed it will be a big improvement over current databases. Certainly worth a look.

Also worth a look is Sedna, a XML database with a Scheme API.

Posted by Noel at 12:14 PM | Comments (0)

Bug Fixes for IE 7

I see via Ditchnet.org that IE 7 Beta 1 is out, and support for CSS 2.1 is in the works. For comparison, see what Deer Park Alpha 2 (the next version of Firefox) includes. It looks like IE 7 is going to be a good browser for 2004, and an ok browser for today. This is still good news for web developers, because IE 6 is a terrible browser for today.

Posted by Noel at 12:09 PM | Comments (0)

July 07, 2005

Tags and the Post-Google World of Implicit Feedback

Go to Yahoo and you'll see a riot of news stories, links to services such as Yahoo Messenger, and right at the bottom of the page… a link to the service that launched the company, the Yahoo Directory. It is laughable now to think that a human maintained directory could keep pace with an powerful search engines and exponentially growing web, but for a long time Yahoo's directory was the web's best way to find information.

Early search engines considered only the text of a web page when searching. They could find all the pages that contained the words you were looking for, but they could not tell you which ones were important, leaving you to sift through the hundreds or thousands of results. In this situation going to Yahoo's directory was a short-cut to finding the most important pages on any topic. It might take you six or seven clicks to drill down through the hierarchy but you'd get there in the end.

Then along came Google. Enter some keywords and Google will give you the all pages that match those keywords, ordered by importance. In most cases, Google is sufficient. The key innovation in Google is that recognises that people indicate the pages they consider important by linking to them. Google uses that information to rank pages via the PageRank algorithm. With millions of web pages individual variations don't count for much, and what you end up with is the pages generally considered best.

There is a big idea behind the PageRank algorithm, which is that people implicitly tell you what is important to them. The Yahoo model was to explicitly ask people what is important to them. This doesn't scale. Implicit measures scale better, and in fact perform better as size grows, as noise in the measurements cancels out.

Fast-forward to today. One of the hottest things on the web at the moment are services like del.icio.us that asks people to tag pages or assign ratings to them. This is another example of explicit feedback. You don't have to look long at the most popular tags on del.icio.us to see that the demographics are skewed heavily towards the geek end. Now explicit feedback is great if you can get it, but while you can expect obsessive-compulsive geeks like myself to meticulously organise their links, but you can't expect the man-on-the-street to do the same. What is needed is implicit feedback.

For services like del.icio.us that aggregating pages on a common theme, there are a number of algorithms that will apply PageRank in a topic-sensitive manner (for example, the aptly named topic-sensitive PageRank). For services that maintain rating of pages, there has been quite a lot of work on implicit ratings, collated in this excellent recent survey. Several studies show that reading time is correlated with interest. If you run a web site, this is one statistic you can easily gather, and one that the Attention.XML spec includes. If you write web browsers there are bunch of other measurements you can gather, such as scrolling and bookmarking. Of course for most of us these measurements are out of reach!

Explicit measures have one advantage: they are much easier to get started with. If I rate a blog post as "good", the meaning is unambiguous. If I spend 30 seconds reading it you need some sort of model to convert that into a rating, and to build that model you need a fair amount of data, knowledge of machine learning techniques (we can help), and probably some beefy hardware to handle large numbers of users. For small services this may be too much, but for large services it is the way forward.

Posted by Noel at 01:08 PM | Comments (0)

June 17, 2005

The Canvas: Why It's Important

In Safari 1.3 Apple introduced a new canvas tag along with Javascript extensions that enable 2D vector graphics to be drawn directly onto a web page. This extension has since been incorporated in a WhatWG specification and Mozilla have followed up with an implementation that's available in the latest Firefox 1.1 preview release. We have two demos up here and here. This essentially enables a lot of the power of Flash from within a normal web page.

Apple's motivation to was to enable Dashboard, but as we discussed earlier the potential for this technology extends much further. Existing web technologies such as CSS, the DOM, and Javascript allow highly interactive applications, as GMail and Google Maps have shown. However there are still some limitations: the developer cannot create custom widgets or draw complex graphics in the web-browser. The canvas solves this problem. Now that Firefox is implementing the canvas it makes it viable to release cross-platform graphic rich applications. This is a whole class of applications that so far the web hasn't been able to touch. Think about Adobe Illustrator running within your web browser. This is one to watch!

Posted by Noel at 12:01 PM | Comments (0)

May 25, 2005

How Did They Do That?

Xyle looks like a nifty tool for working out how a CSS design was created. It only works on Safari, so for most people the Web Developer plugin for Firefox will be more useful. Xyle does have a few features that WD doesn't, so if you have Safari and the $15 I'd say it's worth it.

Posted by Noel at 03:38 PM | Comments (0)

May 10, 2005

Dashboard: A Great Move, Even Though Apple Got It Wrong

One of the highly talked about features in Tiger is the Dashboard. Basically it is a platform for little widgets coded in Javascript and CSS; things like calculators and email notifiers. At least that's how Apple present Dashboard, and they've got it wrong.

To see the true importance of Dashboard we need to look at what makes Microsoft the dominant player in the computer industry. Microsoft makes most of its money from the sale of Windows and Office. Now the only reason to buy Windows is for the applications that run on it. So long as Microsoft controls the largest pool of developers Windows remain the number one OS.

It seemed that the web threatened Window's dominance, by making the platform irrelevant. Microsoft realised this a long time ago, which is why they killed Netscape and then promptly stopped development on IE. Now browsers have continued to get better, thanks largely to the efforts of Mozilla, but yet Microsoft continues to prosper. The reason: web-based interfaces, compared to their native counterparts, suck. Right now we're seeing some exciting developments. Starting with Oddpost and continuing with GMail and other so-called AJAX applications, web-based applications have been delivering interfaces that approach their desktop counterparts, but they aren't there yet.

This is where Dashboard fits in. It lets you deliver rich interfaces using Javascript, HTML and CSS, principally via the canvas extension, which is independent of Dashboard per se (it works in Safari and any other WebKit application). This is a huge win for Apple. There are thousands of people who don't know any Objective-C but do know web technologies, and now they are all potential OS X developers. Web applications are only going to grow in number, and enhancing an existing web application with a Dashboard interface isn't that much work, so Dashboard has the potential to make OS X the preferred platform for delivering rich interfaces. Additionally, using skills learned from web development, people can start developing Dashboard applications that have nothing to do with the web. Unfortunately Apple has marketed Dashboard as a platform for little toys, not serious applications, but I think its potential will become apparent as people explore it further. And when they do Apple might finally have the momentum to seriously tackle Microsoft's dominance.

Posted by Noel at 10:14 PM | Comments (0)