Express.js 4, Node.js and MongoDB REST API Tutorial

Usually good things don’t stay the same, so our tutorial on building a JSON REST API server with Node.js and MongoDB using Mongoskin and Express.js, and testing it with Mocha and Superagent, has became a bit outdated with the new Express.js 4 version release. Here’s a brand new, revisited tutorial for Express.js 4, Node.js and MongoDB (Mongoskin) free-JSON RESTful API server.

The code for this new tutorial is available at github.com/azat-co/rest-api-express (master branch). The old tutorial’s code for Express 3.x, is still working and in the express3 branch.

Express.js 4 and MongoDB REST API Tutorial consists of these parts:

1. Node.js and MongoDB REST API Overview
2. REST API Tests with Mocha and Superagent
3. NPM-ing Node.js Server Dependencies
4. Express.js 4.x Middleware Caveat
5. Express.js and MongoDB (Mongoskin) Implementation
6. Running The Express.js 4 App and Testing MongoDB Data with Mocha
7. Conclusion and Further Express.js and Node.js Reading

Instead of TL;DR:

If you’re only interested in a working code from the repository and know what to do, here are brief instructions on how to download and run the REST API server:

$ git clone git@github.com:azat-co/rest-api-express.git
$ npm install
$ node express.js

Start MongoDB with $ mongod. Then, in a new terminal window run the Mocha tests:

$ mocha express.test.js

Or, if you don’t have mocha installed globally:

$ ./node_modules/mocha/bin/mocha express.test.js

Node.js and MongoDB REST API Overview

This Node.js, Express.js and MongoDB (Mongoskin) tutorial will walk you through writing the test using the Mocha and Super Agent libraries. This is needed for a test-driven development building of a Node.js free JSON REST API server.

The server application itself will utilize Express.js 4.x framework and Mongoskin library for MongoDB. In this REST API server, we’ll perform create, read, update and delete (CRUD) operations and harness Express.js middleware concept with app.param() and app.use() methods.

First of all, make sure you have MongoDB installed. You can follow the steps on the official website.

We’ll be using the following versions of libraries:

• express: ~4.1.1
• body-parser: ~1.0.2
• mongoskin: ~1.4.1
• expect.js: ~0.3.1
• mocha: ~1.18.2
• superagent: ~0.17.0

If you try to attempt to use later or older versions the code might not work. :-(

REST API Tests with Mocha and Superagent

Before anything else, let’s write functional tests that make HTTP requests to our soon-to-be-created REST API server. If you know how to use Mocha or just want to jump straight to the Express.js app implementation, feel free to do so. You can use CURL terminal commands for testing too.

Assuming we already have Node.js, NPM and MongoDB installed, let’s create a new folder (or if you wrote the tests use that folder):

$ mkdir rest-api
$ cd rest-api

We’ll use Mocha, Expect.js and Super Agent libraries. To install them, run these commands from the project folder:

$ npm install mocha@1.18.2 --save-dev
$ npm install expect.js@0.3.1 --save-dev 
$ npm install superagent@0.17.0 --save-dev

Note: You can also install Mocha globally with the -g flag.

Now, let’s create the express.test.js file in the same folder which will have six suites:

• Creating a new object
• Retrieving an object by its ID
• Retrieving the whole collection
• Updating an object by its ID
• Checking an updated object by its ID
• Removing an object by its ID

HTTP requests are just a breeze with Super Agent’s chained functions which we’ll put inside of each test suite.
To keep this tutorial focused on the REST API with Express.js 4 and MongoDB, and not on Mocha, we won’t go into the details of test suits. Feel free to copy and paste the code!

Here is the full source code for the express.test.js file:

var superagent = require('superagent')
var expect = require('expect.js')

describe('express rest api server', function(){
  var id

  it('post object', function(done){
    superagent.post('http://localhost:3000/collections/test')
      .send({ name: 'John'
        , email: 'john@rpjs.co'
      })
      .end(function(e,res){
        // console.log(res.body)
        expect(e).to.eql(null)
        expect(res.body.length).to.eql(1)
        expect(res.body[0]._id.length).to.eql(24)
        id = res.body[0]._id
        done()
      })    
  })

  it('retrieves an object', function(done){
    superagent.get('http://localhost:3000/collections/test/'+id)
      .end(function(e, res){
        // console.log(res.body)
        expect(e).to.eql(null)
        expect(typeof res.body).to.eql('object')
        expect(res.body._id.length).to.eql(24)        
        expect(res.body._id).to.eql(id)        
        done()
      })
  })

  it('retrieves a collection', function(done){
    superagent.get('http://localhost:3000/collections/test')
      .end(function(e, res){
        // console.log(res.body)
        expect(e).to.eql(null)
        expect(res.body.length).to.be.above(0)
        expect(res.body.map(function (item){return item._id})).to.contain(id)        
        done()
      })
  })

  it('updates an object', function(done){
    superagent.put('http://localhost:3000/collections/test/'+id)
      .send({name: 'Peter'
        , email: 'peter@yahoo.com'})
      .end(function(e, res){
        // console.log(res.body)
        expect(e).to.eql(null)
        expect(typeof res.body).to.eql('object')
        expect(res.body.msg).to.eql('success')        
        done()
      })
  })
  it('checks an updated object', function(done){
    superagent.get('http://localhost:3000/collections/test/'+id)
      .end(function(e, res){
        // console.log(res.body)
        expect(e).to.eql(null)
        expect(typeof res.body).to.eql('object')
        expect(res.body._id.length).to.eql(24)        
        expect(res.body._id).to.eql(id)        
        expect(res.body.name).to.eql('Peter')        
        done()
      })
  })    
  
  it('removes an object', function(done){
    superagent.del('http://localhost:3000/collections/test/'+id)
      .end(function(e, res){
        // console.log(res.body)
        expect(e).to.eql(null)
        expect(typeof res.body).to.eql('object')
        expect(res.body.msg).to.eql('success')    
        done()
      })
  })      
})

To run the tests, we can use the $ mocha express.test.js command (if you have Mocha globally) or $ ./node_modules/mocha/bin/mocha express.test.js.

NPM-ing Node.js Server Dependencies

In this tutorial, we’ll utilize Mongoskin, a MongoDB library which is a better alternative to the plain, good old native MongoDB driver for Node.js. In addition, Mongoskin is more lightweight than Mongoose and schema-less. For more insight, please check out Mongoskin comparison blurb.

Express.js is a wrapper for the core Node.js HTTP module objects. The Express.js framework is built on top of Connect middleware and provides tons of convenience. Some people compare the framework to Ruby’s Sinatra in terms of how it’s non-opinionated and configurable.

If you’ve create a rest-api folder in the previous section Test Coverage, simply run these commands to install modules for the application:

$ npm install express@4.1.1 --save
$ npm install mongoskin@1.4.1 --save

Express.js 4.x Middleware Caveat

Sadly, just NPM-ing express is not enough anymore for building minimal REST API servers with Express.js, because in version 4.x the middlewares are not bundled with the framework! Developers have to install separate modules, except for express.static, which was left in the Express.js 4.x. So to parse incoming information, we add body-parser:

$ npm install body-parser@1.0.2 --save

Express.js 4 and MongoDB (Mongoskin) Implementation

First thing’s first, so let’s define our dependencies in express.js:

var express = require('express'),
  mongoskin = require('mongoskin'),
  bodyParser = require('body-parser')  

After the version 3.x (this of course includes v4), Express.js streamlines the instantiation of its app instance, this line will give us a server object:

var app = express()

To extract params from the body of the requests, we’ll use bodyParser() middleware which looks more like a configuration statement:

app.use(bodyParser())

Middleware (in this and other forms) is a powerful and convenient pattern in Express.js and Connect to organize and re-use code.

As with the bodyParser() method that saves us from the hurdles of parsing a body object of HTTP request, Mongoskin makes it possible to connect to the MongoDB database in one effortless line of code:

var db = mongoskin.db('mongodb://@localhost:27017/test', {safe:true})

Note: If you wish to connect to a remote database, e.g., MongoHQ instance, substitute the string with your username, password, host and port values. Here is the format of the URI string: mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]].

The app.param() method is another Express.js middleware. It basically says “do something every time there is this value in the URL pattern of the request handler”. In our case, we select a particular collection when request pattern contains a sting collectionName prefixed with a colon (you’ll see it later in the routes). Then, we save that collection as a property (collection but could be anything) of the request object (widespread req), which will be available in the next request handlers:

app.param('collectionName', function(req, res, next, collectionName){
  req.collection = db.collection(collectionName)
  return next()
})

Merely to be user-friendly, let’s put a root route with a message:

app.get('/', function(req, res) {
  res.send('please select a collection, e.g., /collections/messages')
})

Now the real work begins, here is how we retrieve a list of any items (first parameter is an empty object {} which means any). The results will be capped at a limit of 10 and sorted by _id (second parameter). The find() method returns a cursor, so we call toArray() to get the JavaScript/Node.js array:

app.get('/collections/:collectionName', function(req, res, next) {
  req.collection.find({} ,{limit:10, sort: [['_id',-1]]}).toArray(function(e, results){
    if (e) return next(e)
    res.send(results)
  })
})

Have you noticed a :collectionName string in the URL pattern parameter? This and the previous app.param() middleware is what gives us the req.collection object, which points to a specified collection in our database.

The object creating endpoint is slightly easier to grasp since we just pass the whole payload to the MongoDB. This method often called free JSON REST API because server and then the database accept any data structure. Parse.com and other Back-end as a Service providers pioneered the free JSON approach. In our Express.js app, we use req.body for this:

app.post('/collections/:collectionName', function(req, res, next) {
  req.collection.insert(req.body, {}, function(e, results){
    if (e) return next(e)
    res.send(results)
  })
})

Single object retrieval functions like findById and findOne are faster than find(), but they use a different interface (they return the object directly instead of a cursor). So please be aware of that. In addition, we’re extracting the ID from :id part of the path with req.params.id Express.js magic:

app.get('/collections/:collectionName/:id', function(req, res, next) {
  req.collection.findById(req.params.id, function(e, result){
    if (e) return next(e)
    res.send(result)
  })
})

PUT request handler gets more interesting because update() doesn’t return the augmented object. Instead it returns us a count of affected objects.

Also {$set:req.body} is a special MongoDB operator (operators tend to start with a dollar sign) that sets values.

The second {safe:true, multi:false} parameter is an object with options that tell MongoDB to wait for the execution before running the callback function and to process only one (first) item.

app.put('/collections/:collectionName/:id', function(req, res, next) {
  req.collection.updateById(req.params.id, {$set:req.body}, {safe:true, multi:false}, function(e, result){
    if (e) return next(e)
    res.send((result===1)?{msg:'success'}:{msg:'error'})
  })
})

Finally, the DELETE HTTP method is processed by app.del(). In the request handler, we utilize removeById() which does exactly what it sounds like it should do, and takes an ID and a callback. Then, we output a custom JSON message success on the deletion:

app.del('/collections/:collectionName/:id', function(req, res, next) {
  req.collection.remove({_id: req.collection.id(req.params.id)}, function(e, result){
    if (e) return next(e)
    res.send((result===1)?{msg:'success'}:{msg:'error'})
  })
})

Note: The delete is an operator in JavaScript, so Express.js uses app.del instead.

The last line that actually starts the server on port 3000 in this case:

app.listen(3000)

Just in case something is not working quite well, here is the full code of the express.js file (also check with the GitHub which is sometimes more up-to-date and workable):

var express = require('express')
  , mongoskin = require('mongoskin')
  , bodyParser = require('body-parser')

var app = express()
app.use(bodyParser())

var db = mongoskin.db('mongodb://@localhost:27017/test', {safe:true})

app.param('collectionName', function(req, res, next, collectionName){
  req.collection = db.collection(collectionName)
  return next()
})

app.get('/', function(req, res, next) {
  res.send('please select a collection, e.g., /collections/messages')
})

app.get('/collections/:collectionName', function(req, res, next) {
  req.collection.find({} ,{limit:10, sort: [['_id',-1]]}).toArray(function(e, results){
    if (e) return next(e)
    res.send(results)
  })
})

app.post('/collections/:collectionName', function(req, res, next) {
  req.collection.insert(req.body, {}, function(e, results){
    if (e) return next(e)
    res.send(results)
  })
})

app.get('/collections/:collectionName/:id', function(req, res, next) {
  req.collection.findById(req.params.id, function(e, result){
    if (e) return next(e)
    res.send(result)
  })
})

app.put('/collections/:collectionName/:id', function(req, res, next) {
  req.collection.updateById(req.params.id, {$set:req.body}, {safe:true, multi:false}, function(e, result){
    if (e) return next(e)
    res.send((result===1)?{msg:'success'}:{msg:'error'})
  })
})

app.del('/collections/:collectionName/:id', function(req, res, next) {
  req.collection.removeById(req.params.id, function(e, result){
    if (e) return next(e)
    res.send((result===1)?{msg:'success'}:{msg:'error'})
  })
})

app.listen(3000)

Save the code and exit your editor, because we’re done with our small Express.js REST API server.

Running The Express.js 4 App and Testing MongoDB Data with Mocha

Now, assuming you have MongoDB installed and running ($ mongod), we should be able to run this in your terminal (separate window from mongod):

$ node express.js

And in a different window (without closing the first one):

$ mocha express.test.js

Or, if you don’t have mocha installed globally:

$ ./node_modules/mocha/bin/mocha express.test.js

If you really don’t like Mocha and/or BDD, CURL is always there for you. :-)

For example, CURL data to make a POST request:

$ curl -X POST -d "name=azat" http://localhost:3000/collections/test13

And the result should look something like this:

{"name":"azat","_id":"535e180dad6d2d2e797830a5"}]%

We can easily check this object either by using our REST API server:

$ curl http://localhost:3000/collections/test13

GET requests also work in the browser. For example, open this link while your local server is running on port 3000 http://localhost:3000/collections/test13.

Or if we don’t trust server results (why wouldn’t we? but let’s pretend we don’t), we can open MongoDB shell ($ mongo) and type:

> db.test13.find()

Note: If you changed the database name and it’s not test, then precede the above command with > use your_database_name.

In this tutorial, our tests are longer than the app code itself. For some it might be tempting to abandon the test-driven development, but believe me the good habits of TDD will save you hours and hours during any serious development when the complexity of the applications you work on is big.

Conclusion and Further Express.js and Node.js Reading

The Express.js 4 and MongoDB/Mongoskin libraries are great when you need to build a simple REST API server in a few lines of code. Later, if you need to expand the libraries they also provide a way to configure and organize your code.

NoSQL databases like MongoDB are good at free-REST APIs where we don’t have to define schemas and can throw any data and it’ll be saved.

The full source code for express.test.js, express.js and package.json is available at github.com/azat-co/rest-api-express.

If you would like to learn more about Express.js and other JavaScript libraries, take a look at the following books by Azat:

• Practical Node.js: Building Real-world Scalable Web Apps
• Express.js Guide: The Comprehensive Book on Express.js

In addition, checkout the free series of posts on webapplog.com: Intro to Express.js tutorials.

Note: In this example I’m using semi-colon less style. Semi-colons in JavaScript are absolutely optional except in two cases: in the for loop and before expression/statement that starts with parenthesis (e.g., Immediately-Invoked Function Expression).