Developing a REST API And Client Using Node

The web is full of APIs. These days mashups are getting more and more popular. People aggregate information from various sources to produce new, sometimes interesting, services. REST (Representational State Transfer) provides one nice way to come up with an API that is fairly easy to consume.

In this post I'll discuss REST briefly and introduce you to a few tools I developed in order to make it easier to implement and consume REST APIs using Node. As an API is somewhat useless without some kind of a database behind it, I'll discuss MongoDB (or rather Mongoose) very briefly as well.

Introduction to REST

So what is REST? You can probably find a lot of definitions out there. For me it's all about resources and HTTP verbs POST, GET, PUT and DELETE. These verbs map to CRUD (Create, Read, Update, Delete) familiar from persistent storage. You can think it as a web specific way of doing the same thing.

This mapping means we can model our backend behind a REST API. This decoupling is quite powerful as it allows us to use whatever technology we like on the backend and frontend side. In fact we could write multiple different frontends hooking into our backend. The frontends could even mash data from multiple sources. Compared to the traditional "lets lump it all together and write some APIs later" approach this seems like a more novel way to go.

REST doesn't specify in any manner how to deal with auth. There are multiple possible solutions for this particular problem. In this post I will use a combination of API key over SSL since it's quite simple to set up and fits my purposes for now. OAuth and Amazon style HMAC are popular alternatives too.

Anatomy of a REST API

I really like the RESTful API Design slides. I think they summarize quite well what REST is about and how you might go and and design your API. Consider we're writing a TODO application. We might design a Todo model with fields such as these:

• Name - String, required
• Priority - Integer within [0, 5], defaults to zero
• Description - String, required
• Status - String enum (either "Waiting", "Doing" or "Done"), defaults to "Waiting"

Our REST API for this very simple schema can be modeled simply as a route /api/v1/todo. The following list describes the operations we can provide through the API:

• POST /todo - Creates a new Todo resource and returns it
• GET /todo - Returns existing Todo resources
• PUT /todo - Not allowed (403 error)
• DELETE /todo - Not allowed (403 error)

Those operations function on collection level so to speak. The following operate on an individual item.

• POST /todo/ - Not allowed (403 error)
• GET /todo/ - Returns resource and returns it if found, else 404 error
• PUT /todo/ - Updates resource and returns it if found, else 404 error 
• DELETE /todo/ - Deletes resource and returns empty if found, else 404 error

"RESTful API Design" slides describe some additional functionality. For instance it might make sense to constrain fields returned (ie. fields=name,priority), match by some property (ie. name=some+todo), count (ie. /count) or perhaps use pagination (ie. limit=25&offset=50).

rest-sugar, mongoose-sugar, rest-sugar-client

In fact a utility of mine, rest-sugar, does what I just described and then some. It has been designed to work with mongoose-sugar, another package of mine. There is a third package, rest-sugar-client, that makes it easy to consume the API constructed by rest-sugar.

So how this all works? What's up with all the sugar? Not so sure about sugar. Just popped into mind, really. The whole point of these tools is to make it easy to generate a REST API based on the given database schemas and provide a high level API for the client.

In this case the database schema could be defined using mongoose-sugar. That works as a light wrapper to Mongoose that in turn wraps MongoDB. rest-sugar hooks into this definition and queries provided by mongoose-sugar and constructs a REST API based on that. Finally rest-sugar-client creates the client side API based on the metadata available on the REST API.

mongoose-sugar

mongoose-sugar wraps Mongoose and makes schemas behave in a way more appropriate for me. For instance I made sure all my schemas are set as "strict" by default. This means they will get validated on database level. I also made sure certain extra fields (ie. deleted flag and created date) get attached to each schema. I added that date information there just for convenience. As I prefer to deal with deletions in a soft manner I decided to use a flag instead of the native delete.

mongoose-sugar also attaches the schema definition to the schema itself so that it is easy for me to access that later. This makes it possible for me to serve the description of the database schemas via my REST API.

There is also a specific "refs" utility that makes it easy to make enforced references between models. This is something that takes quite a bit of syntax to do so I decided to add it into this package.

The schema I gave above maps directly to mongoose-sugar. Examine the demo provided with the package to get a better idea of how to do this. Mongoose documentation likely helps as well.

The syntax is somewhat light and quite comfortable to use even. Pretty much the only thing I'm still a bit puzzled about is migrations. I know the situation is a bit different with document databases due to the way they have been designed. Still, this is something I will need to look into.

rest-sugar

As mentioned earlier rest-sugar is an utility that construct a REST API based on a given schema. In order to do this it uses a specific type of query API (with getAll, update etc.) to access the actual models. So far I've defined this only for mongoose-sugar and a memory based storage (a demo). It should be easy to implement that for other drivers too.

The following snippet shows how you might construct a server providing a REST API based on your schemas:

#!/usr/bin/env node
var express = require('express');
var mongoose = require('mongoose');
var sugar = require('mongoose-sugar');
var rest = require('rest-sugar');
var models = require('./models');

main();

function main() {
    mongoose.connect('mongodb://localhost/mongoosedemo');

    var app = express.createServer();

    app.configure(function() {
        app.use(express.methodOverride()); // handles PUT
        app.use(express.bodyParser()); // handles POST
        app.use(app.router);
    });

    rest.init(app, '/api/v1/', {
        'libraries': models.Library,
        'tags': models.Tag
    }, sugar);

    console.log('Surf to localhost:8000');

    app.listen(8000);
}

As you can see the definition is somewhat light. The actual magic lies within the implementation of rest-sugar. If you for some reason wanted to change your schema considerably, you might want to define another init for /api/v2 and use the updated schemas there. Still, it would remain somewhat nice and simple.

Note that you can easily set up some authentication scheme just by passing an extra parameter to rest.init. If you wish to use key based authentication (you should use SSL with this!), pass rest.auth.key('apikey', APIKEY) (where APIKEY is your key) to it. After that the API expects that "apikey" is passed as a part of your query string. It might be interesting to support various other schemes as well. Ideas are definitely welcome!

There are a couple of demos at the repository you might want to examine if this sounded interesting. You will find an example of authentication there too.

rest-sugar-client

So we have got a nice REST API running. We're just missing the frontend bit. This is where rest-sugar-client kicks in. The way it works is quite simple really. All it does is to introspect the metadata provided by the REST API and use that to construct a simple API of its own.

The advantage of this is that we don't need to maintain another set of APIs on the client side. It will definitely stay always in sync. The next step would be to hook this up with some nice frontend library and start coding the user interface.

Even though that sounds simple I find it somewhat cool. In fact I just wrote a simple command line based utility based on this idea. I know curl can do the same thing. This is something more domain specific, though. A web based interface might be interesting to have too.

Conclusion

I hope this post gave you some idea of how I might implement a REST interface and client for it. In my mind introspection is one of the coolest things out there. Just having some metadata around makes it possible to implement nice, generic solutions.

I will likely have to develop a couple of services to see how well this kind of setup works out in practice. There are some aspects I'm a bit doubtful about still (migrations and possible offline usage come to mind). Still, it feels like I managed to develop something that is perhaps useful for some other people too.