Tutorial: Node.js and MongoDB JSON REST API server with Mongoskin and Express.js

Tutorial: building a JSON REST API server with Node.js and MongoDB using Mongoskin and Express.js utilizing Mocha and Super Agent for BDD/TDD.

Update3: Expess 4 version of this tutorial is available at Express.js 4, Node.js and MongoDB REST API Tutorial, and github.com/azat-co/rest-api-express (master branch). This tutorial will work with Express 3.x.

Update2: “Mongoskin removed ‘db.collection.id’ and added some actionById methods” from this pull request with this code changes. To use the code in this post, just install an older version of Mongoskin (0.5.0?). The code in the GitHub will work with Mongoskin 1.3.20.

Update2: “Mongoskin removed ‘db.collection.id’ and added some actionById methods” from this pull request with this code changes. To use the code in this post, just install an older version of Mongoskin (0.5.0?)

Update: use the enhanced code from this repository github.com/azat-co/rest-api-express (express3 branch).

Note: This text is a part of Express.js Guide: The Comprehensive Book on Express.js.

This tutorial will walk you through writing test using the Mocha and Super Agent libraries and then use them in a test-driven development manner to build a Node.js free JSON REST API server utilizing Express.js framework and Mongoskin library for MongoDB. In this REST API server, we’ll perform create, read, update and delete (CRUD) operations and harness Express.js middleware concept with app.param() and app.use() methods.

Continue reading “Tutorial: Node.js and MongoDB JSON REST API server with Mongoskin and Express.js”

Intro to Express.js: Parameters, Error Handling and Other Middleware

Express.js is a node.js framework that among other things provides a way to organize routes. Each route is defined via a method call on an application object with a URL patter as a first parameter (RegExp is also supported). The best practice is to keep router lean and thin by moving all the logic into corresponding external modules/files. This way important server configuration parameters will be neatly in one place, right there when you need them! :-)

Note: This text is a part of Express.js Guide: The Comprehensive Book on Express.js.

Express.js is one of the most popular and mature Node.js frameworks. You can read more about in Intro to Express.js series on webapplog.com:

To learn how to create an application from scratch please refer to the earlier post.

Request Handlers

Express.js is a node.js framework that among other things provides a way to organize routes. Each route is defined via a method call on an application object with a URL patter as a first parameter (RegExp is also supported), for example:

app.get('api/v1/stories/', function(res, req){
  ...
})

or, for a POST method:

app.post('/api/v1/stories'function(req,res){
  ...
})

It’s needless to say that DELETE and PUT methods are supported as well.
The callbacks that we pass to get() or post() methods are called request handlers, because they take requests (req), process them and write to response (res) objects. For example:

app.get('/about', function(req,res){
  res.send('About Us: ...');
});

We can have multiple request handlers, hence the name middleware. They accept a third parameter next calling which (next()) will switch the execution flow to the next handler:

app.get('/api/v1/stories/:id', function(req,res, next) {
  //do authorization
  //if not authorized or there is an error 
  // return next(error);
  //if authorized and no errors
  return next();
}), function(req,res, next) {
  //extract id and fetch the object from the database
  //assuming no errors, save story in the request object
  req.story = story;
  return next();
}), function(req,res) {
  //output the result of the database search
  res.send(res.story);
});

ID of a story in URL patter is a query string parameter which we need for finding a matching items in the database.

Parameters Middleware

Parameters are values passed in a query string of a URL of the request. If we didn’t have Express.js or similar library, and had to use just the core Node.js modules, we’d had to extract parameters from HTTP.request object via some require('querystring').parse(url) or require('url').parse(url, true) functions trickery.

Thanks to Connect framework and people at VisionMedia, Express.js already has support for parameters, error handling and many other important features in the form of middlewares. This is how we can plug param middleware in our app:

app.param('id', function(req,res, next, id){
  //do something with id
  //store id or other info in req object
  //call next when done
  next();
});

app.get('/api/v1/stories/:id',function(req,res){
  //param middleware will be execute before and
  //we expect req object already have needed info
  //output something
  res.send(data);
});

For example:

app.param('id', function(req,res, next, id){
  req.db.get('stories').findOne({_id:id}, function (e, story){
    if (e) return next(e);
    if (!story) return next(new Error('Nothing is found'));
    req.story = story;
    next();
  });
});

app.get('/api/v1/stories/:id',function(req,res){
  res.send(req.story);
});

Or we can use multiple request handlers but the concept remains the same: we can expect to have req.story object or an error thrown prior to the execution of this code so we abstract common code/logic of getting parameters and their respective objects:

app.get('/api/v1/stories/:id', function(req,res, next) {
  //do authorization
  }),
  //we have an object in req.story so no work is needed here
  function(req,res) {
  //output the result of the database search
  res.send(story);
});

Authorization and input sanitation are also good candidates for residing in the middlewares.

Function param() is especially cool because we can combine different keys, e.g.:

app.get('/api/v1/stories/:storyId/elements/:elementId',function(req,res){
  res.send(req.element);
});

Error Handling

Error handling is typically used across the whole application, therefore it’s best to implement it as a middleware. It has the same parameters plus one more, error:

app.use(function(err, req, res, next) {
  //do logging and user-friendly error message display
  res.send(500);
})

In fact, the response can be anything:

JSON string

app.use(function(err, req, res, next) {
  //do logging and user-friendly error message display
  res.send(500, {status:500, message: 'internal error', type:'internal'});
})

Text message

app.use(function(err, req, res, next) {
  //do logging and user-friendly error message display
  res.send(500, 'internal server error');
})

Error page

app.use(function(err, req, res, next) {
  //do logging and user-friendly error message display
  //assuming that template engine is plugged in
  res.render('500');
})

Redirect to error page

app.use(function(err, req, res, next) {
  //do logging and user-friendly error message display
  res.redirect('/public/500.html');
})

Error HTTP response status (401, 400, 500, etc.)

app.use(function(err, req, res, next) {
  //do logging and user-friendly error message display
  res.end(500);
})

By the way, logging is also should be abstracted in a middleware!

To trigger an error from within your request handlers and middleware you can just call:

next(error);

or

next(new Error('Something went wrong :-(');

You can also have multiple error handlers, and use named instead of anonymous functions as its shows in Express.js Error handling guide.

Other Middleware

In addition to extracting parameters, it can be used for many things, like authorization, error handling, sessions, output, and others.

res.json() is one of them. It conveniently outputs JavaScript/Node.js object as a JSON. For example:

app.get('/api/v1/stories/:id', function(req,res){
  res.json(req.story);
});

is equivalent to (if req.story is an Array and Object):

app.get('/api/v1/stories/:id', function(req,res){
  res.send(req.story);
});

or

app.get('api/v1/stories/:id',function(req,res){
  res.set({
    'Content-Type': 'application/json'
  });
  res.send(req.story);
});

Abstraction

Middleware is flexible. You can use anonymous or named functions, but the best thing is to abstract request handlers into external modules based on the functionality:

var stories = require.('./routes/stories');
var elements = require.('./routes/elements');
var users = require.('./routes/users');
...
app.get('/stories/,stories.find);
app.get('/stories/:storyId/elements/:elementId', elements.find);
app.put('/users/:userId',users.update);

routes/stories.js:

module.exports.find = function(req,res, next) {
};

routes/elements.js:

module.exports.find = function(req,res,next){
};

routes/users.js:

module.exports.update = function(req,res,next){
};

You can use some functional programming tricks, like this:

function requiredParamHandler(param){
  //do something with a param, e.g., check that it's present in a query string
  return function (req,res, next) {
    //use param, e.g., if token is valid proceed with next();
    next();
  });
}

app.get('/api/v1/stories/:id', requiredParamHandler('token'), story.show);

var story  = {
  show: function (req, res, next) {
    //do some logic, e.g., restrict fields to output
    return res.send();
  }
}   

As you can see middleware is a powerful concept for keeping code organized. The best practice is to keep router lean and thin by moving all the logic into corresponding external modules/files. This way important server configuration parameters will be neatly in one place, right there when you need them! :-)