Express.js 4, Node.js and MongoDB REST API Tutorial

Express.js 4, Node.js and MongoDB REST API Tutorial

Usually good things don’t stay the same, so our tutorial on building a JSON REST API server with Node.js and MongoDB using Mongoskin and Express.js, and testing it with Mocha and Superagent, has became a bit outdated with the new Express.js 4 version release. Here’s a brand new, revisited tutorial for Express.js 4, Node.js and MongoDB (Mongoskin) free-JSON RESTful API server.

The code for this new tutorial is available at github.com/azat-co/rest-api-express (master branch). The old tutorial’s code for Express 3.x, is still working and in the express3 branch.

Express.js 4 and MongoDB REST API Tutorial consists of these parts:

  1. Node.js and MongoDB REST API Overview
  2. REST API Tests with Mocha and Superagent
  3. NPM-ing Node.js Server Dependencies
  4. Express.js 4.x Middleware Caveat
  5. Express.js and MongoDB (Mongoskin) Implementation
  6. Running The Express.js 4 App and Testing MongoDB Data with Mocha
  7. Conclusion and Further Express.js and Node.js Reading

Instead of TL;DR:

If you’re only interested in a working code from the repository and know what to do, here are brief instructions on how to download and run the REST API server:

$ git clone git@github.com:azat-co/rest-api-express.git
$ npm install
$ node express.js

Start MongoDB with $ mongod. Then, in a new terminal window run the Mocha tests:

$ mocha express.test.js

Or, if you don’t have mocha installed globally:

$ ./node_modules/mocha/bin/mocha express.test.js

Node.js and MongoDB REST API Overview

This Node.js, Express.js and MongoDB (Mongoskin) tutorial will walk you through writing the test using the Mocha and Super Agent libraries. This is needed for a test-driven development building of a Node.js free JSON REST API server.

The server application itself will utilize Express.js 4.x framework and Mongoskin library for MongoDB. In this REST API server, we’ll perform create, read, update and delete (CRUD) operations and harness Express.js middleware concept with app.param() and app.use() methods.

First of all, make sure you have MongoDB installed. You can follow the steps on the official website.

We’ll be using the following versions of libraries:

  • express: ~4.1.1
  • body-parser: ~1.0.2
  • mongoskin: ~1.4.1
  • expect.js: ~0.3.1
  • mocha: ~1.18.2
  • superagent: ~0.17.0

If you try to attempt to use later or older versions the code might not work. :-(

REST API Tests with Mocha and Superagent

Before anything else, let’s write functional tests that make HTTP requests to our soon-to-be-created REST API server. If you know how to use Mocha or just want to jump straight to the Express.js app implementation, feel free to do so. You can use CURL terminal commands for testing too.

Assuming we already have Node.js, NPM and MongoDB installed, let’s create a new folder (or if you wrote the tests use that folder):

$ mkdir rest-api
$ cd rest-api

We’ll use Mocha, Expect.js and Super Agent libraries. To install them, run these commands from the project folder:

$ npm install mocha@1.18.2 --save-dev
$ npm install expect.js@0.3.1 --save-dev 
$ npm install superagent@0.17.0 --save-dev

Note: You can also install Mocha globally with the -g flag.

Now, let’s create the express.test.js file in the same folder which will have six suites:

  • Creating a new object
  • Retrieving an object by its ID
  • Retrieving the whole collection
  • Updating an object by its ID
  • Checking an updated object by its ID
  • Removing an object by its ID

HTTP requests are just a breeze with Super Agent’s chained functions which we’ll put inside of each test suite.
To keep this tutorial focused on the REST API with Express.js 4 and MongoDB, and not on Mocha, we won’t go into the details of test suits. Feel free to copy and paste the code!

Here is the full source code for the express.test.js file:

var superagent = require('superagent')
var expect = require('expect.js')

describe('express rest api server', function(){
  var id

  it('post object', function(done){
    superagent.post('http://localhost:3000/collections/test')
      .send({ name: 'John'
        , email: 'john@rpjs.co'
      })
      .end(function(e,res){
        // console.log(res.body)
        expect(e).to.eql(null)
        expect(res.body.length).to.eql(1)
        expect(res.body[0]._id.length).to.eql(24)
        id = res.body[0]._id
        done()
      })    
  })

  it('retrieves an object', function(done){
    superagent.get('http://localhost:3000/collections/test/'+id)
      .end(function(e, res){
        // console.log(res.body)
        expect(e).to.eql(null)
        expect(typeof res.body).to.eql('object')
        expect(res.body._id.length).to.eql(24)        
        expect(res.body._id).to.eql(id)        
        done()
      })
  })

  it('retrieves a collection', function(done){
    superagent.get('http://localhost:3000/collections/test')
      .end(function(e, res){
        // console.log(res.body)
        expect(e).to.eql(null)
        expect(res.body.length).to.be.above(0)
        expect(res.body.map(function (item){return item._id})).to.contain(id)        
        done()
      })
  })

  it('updates an object', function(done){
    superagent.put('http://localhost:3000/collections/test/'+id)
      .send({name: 'Peter'
        , email: 'peter@yahoo.com'})
      .end(function(e, res){
        // console.log(res.body)
        expect(e).to.eql(null)
        expect(typeof res.body).to.eql('object')
        expect(res.body.msg).to.eql('success')        
        done()
      })
  })
  it('checks an updated object', function(done){
    superagent.get('http://localhost:3000/collections/test/'+id)
      .end(function(e, res){
        // console.log(res.body)
        expect(e).to.eql(null)
        expect(typeof res.body).to.eql('object')
        expect(res.body._id.length).to.eql(24)        
        expect(res.body._id).to.eql(id)        
        expect(res.body.name).to.eql('Peter')        
        done()
      })
  })    
  
  it('removes an object', function(done){
    superagent.del('http://localhost:3000/collections/test/'+id)
      .end(function(e, res){
        // console.log(res.body)
        expect(e).to.eql(null)
        expect(typeof res.body).to.eql('object')
        expect(res.body.msg).to.eql('success')    
        done()
      })
  })      
})

To run the tests, we can use the $ mocha express.test.js command (if you have Mocha globally) or $ ./node_modules/mocha/bin/mocha express.test.js.

NPM-ing Node.js Server Dependencies

In this tutorial, we’ll utilize Mongoskin, a MongoDB library which is a better alternative to the plain, good old native MongoDB driver for Node.js. In addition, Mongoskin is more lightweight than Mongoose and schema-less. For more insight, please check out Mongoskin comparison blurb.

Express.js is a wrapper for the core Node.js HTTP module objects. The Express.js framework is built on top of Connect middleware and provides tons of convenience. Some people compare the framework to Ruby’s Sinatra in terms of how it’s non-opinionated and configurable.

If you’ve create a rest-api folder in the previous section Test Coverage, simply run these commands to install modules for the application:

$ npm install express@4.1.1 --save
$ npm install mongoskin@1.4.1 --save

Express.js 4.x Middleware Caveat

Sadly, just NPM-ing express is not enough anymore for building minimal REST API servers with Express.js, because in version 4.x the middlewares are not bundled with the framework! Developers have to install separate modules, except for express.static, which was left in the Express.js 4.x. So to parse incoming information, we add body-parser:

$ npm install body-parser@1.0.2 --save

Express.js 4 and MongoDB (Mongoskin) Implementation

First thing’s first, so let’s define our dependencies in express.js:

var express = require('express'),
  mongoskin = require('mongoskin'),
  bodyParser = require('body-parser')  

After the version 3.x (this of course includes v4), Express.js streamlines the instantiation of its app instance, this line will give us a server object:

var app = express()

To extract params from the body of the requests, we’ll use bodyParser() middleware which looks more like a configuration statement:

app.use(bodyParser())

Middleware (in this and other forms) is a powerful and convenient pattern in Express.js and Connect to organize and re-use code.

As with the bodyParser() method that saves us from the hurdles of parsing a body object of HTTP request, Mongoskin makes it possible to connect to the MongoDB database in one effortless line of code:

var db = mongoskin.db('mongodb://@localhost:27017/test', {safe:true})

Note: If you wish to connect to a remote database, e.g., MongoHQ instance, substitute the string with your username, password, host and port values. Here is the format of the URI string: mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]].

The app.param() method is another Express.js middleware. It basically says “do something every time there is this value in the URL pattern of the request handler”. In our case, we select a particular collection when request pattern contains a sting collectionName prefixed with a colon (you’ll see it later in the routes). Then, we save that collection as a property (collection but could be anything) of the request object (widespread req), which will be available in the next request handlers:

app.param('collectionName', function(req, res, next, collectionName){
  req.collection = db.collection(collectionName)
  return next()
})

Merely to be user-friendly, let’s put a root route with a message:

app.get('/', function(req, res) {
  res.send('please select a collection, e.g., /collections/messages')
})

Now the real work begins, here is how we retrieve a list of any items (first parameter is an empty object {} which means any). The results will be capped at a limit of 10 and sorted by _id (second parameter). The find() method returns a cursor, so we call toArray() to get the JavaScript/Node.js array:

app.get('/collections/:collectionName', function(req, res, next) {
  req.collection.find({} ,{limit:10, sort: [['_id',-1]]}).toArray(function(e, results){
    if (e) return next(e)
    res.send(results)
  })
})

Have you noticed a :collectionName string in the URL pattern parameter? This and the previous app.param() middleware is what gives us the req.collection object, which points to a specified collection in our database.

The object creating endpoint is slightly easier to grasp since we just pass the whole payload to the MongoDB. This method often called free JSON REST API because server and then the database accept any data structure. Parse.com and other Back-end as a Service providers pioneered the free JSON approach. In our Express.js app, we use req.body for this:

app.post('/collections/:collectionName', function(req, res, next) {
  req.collection.insert(req.body, {}, function(e, results){
    if (e) return next(e)
    res.send(results)
  })
})

Single object retrieval functions like findById and findOne are faster than find(), but they use a different interface (they return the object directly instead of a cursor). So please be aware of that. In addition, we’re extracting the ID from :id part of the path with req.params.id Express.js magic:

app.get('/collections/:collectionName/:id', function(req, res, next) {
  req.collection.findById(req.params.id, function(e, result){
    if (e) return next(e)
    res.send(result)
  })
})

PUT request handler gets more interesting because update() doesn’t return the augmented object. Instead it returns us a count of affected objects.

Also {$set:req.body} is a special MongoDB operator (operators tend to start with a dollar sign) that sets values.

The second {safe:true, multi:false} parameter is an object with options that tell MongoDB to wait for the execution before running the callback function and to process only one (first) item.

app.put('/collections/:collectionName/:id', function(req, res, next) {
  req.collection.updateById(req.params.id, {$set:req.body}, {safe:true, multi:false}, function(e, result){
    if (e) return next(e)
    res.send((result===1)?{msg:'success'}:{msg:'error'})
  })
})

Finally, the DELETE HTTP method is processed by app.del(). In the request handler, we utilize removeById() which does exactly what it sounds like it should do, and takes an ID and a callback. Then, we output a custom JSON message success on the deletion:

app.del('/collections/:collectionName/:id', function(req, res, next) {
  req.collection.remove({_id: req.collection.id(req.params.id)}, function(e, result){
    if (e) return next(e)
    res.send((result===1)?{msg:'success'}:{msg:'error'})
  })
})

Note: The delete is an operator in JavaScript, so Express.js uses app.del instead.

The last line that actually starts the server on port 3000 in this case:

app.listen(3000)

Just in case something is not working quite well, here is the full code of the express.js file (also check with the GitHub which is sometimes more up-to-date and workable):

var express = require('express')
  , mongoskin = require('mongoskin')
  , bodyParser = require('body-parser')

var app = express()
app.use(bodyParser())

var db = mongoskin.db('mongodb://@localhost:27017/test', {safe:true})

app.param('collectionName', function(req, res, next, collectionName){
  req.collection = db.collection(collectionName)
  return next()
})

app.get('/', function(req, res, next) {
  res.send('please select a collection, e.g., /collections/messages')
})

app.get('/collections/:collectionName', function(req, res, next) {
  req.collection.find({} ,{limit:10, sort: [['_id',-1]]}).toArray(function(e, results){
    if (e) return next(e)
    res.send(results)
  })
})

app.post('/collections/:collectionName', function(req, res, next) {
  req.collection.insert(req.body, {}, function(e, results){
    if (e) return next(e)
    res.send(results)
  })
})

app.get('/collections/:collectionName/:id', function(req, res, next) {
  req.collection.findById(req.params.id, function(e, result){
    if (e) return next(e)
    res.send(result)
  })
})

app.put('/collections/:collectionName/:id', function(req, res, next) {
  req.collection.updateById(req.params.id, {$set:req.body}, {safe:true, multi:false}, function(e, result){
    if (e) return next(e)
    res.send((result===1)?{msg:'success'}:{msg:'error'})
  })
})

app.del('/collections/:collectionName/:id', function(req, res, next) {
  req.collection.removeById(req.params.id, function(e, result){
    if (e) return next(e)
    res.send((result===1)?{msg:'success'}:{msg:'error'})
  })
})

app.listen(3000)

Save the code and exit your editor, because we’re done with our small Express.js REST API server.

Running The Express.js 4 App and Testing MongoDB Data with Mocha

Now, assuming you have MongoDB installed and running ($ mongod), we should be able to run this in your terminal (separate window from mongod):

$ node express.js

And in a different window (without closing the first one):

$ mocha express.test.js

Or, if you don’t have mocha installed globally:

$ ./node_modules/mocha/bin/mocha express.test.js

If you really don’t like Mocha and/or BDD, CURL is always there for you. :-)

For example, CURL data to make a POST request:

$ curl -X POST -d "name=azat" http://localhost:3000/collections/test13

And the result should look something like this:

{"name":"azat","_id":"535e180dad6d2d2e797830a5"}]%

We can easily check this object either by using our REST API server:

$ curl http://localhost:3000/collections/test13
Using CURL with Express 4 and MongoDB REST API
Using CURL with Express 4 and MongoDB REST API

GET requests also work in the browser. For example, open this link while your local server is running on port 3000 http://localhost:3000/collections/test13.

Or if we don’t trust server results (why wouldn’t we? but let’s pretend we don’t), we can open MongoDB shell ($ mongo) and type:

> db.test13.find()

Note: If you changed the database name and it’s not test, then precede the above command with > use your_database_name.

In this tutorial, our tests are longer than the app code itself. For some it might be tempting to abandon the test-driven development, but believe me the good habits of TDD will save you hours and hours during any serious development when the complexity of the applications you work on is big.

Conclusion and Further Express.js and Node.js Reading

The Express.js 4 and MongoDB/Mongoskin libraries are great when you need to build a simple REST API server in a few lines of code. Later, if you need to expand the libraries they also provide a way to configure and organize your code.

NoSQL databases like MongoDB are good at free-REST APIs where we don’t have to define schemas and can throw any data and it’ll be saved.

The full source code for express.test.js, express.js and package.json is available at github.com/azat-co/rest-api-express.

If you would like to learn more about Express.js and other JavaScript libraries, take a look at the following books by Azat:

In addition, checkout the free series of posts on webapplog.com: Intro to Express.js tutorials.

Note: In this example I’m using semi-colon less style. Semi-colons in JavaScript are absolutely optional except in two cases: in the for loop and before expression/statement that starts with parenthesis (e.g., Immediately-Invoked Function Expression).

Author: Azat

Techies, entrepreneur, 20+ years in tech/IT/software/web development expert: NodeJS, JavaScript, MongoDB, Ruby on Rails, PHP, SQL, HTML, CSS. 500 Startups (batch Fall 2011) alumnus. http://azat.co http://github.com/azat-co

41 thoughts on “Express.js 4, Node.js and MongoDB REST API Tutorial”

  1. Hi, thanks for your good tutorial. It is a good basis for designing a RESTful api, but I want to criticise the RESTful approach used in this tutorial (https://github.com/azat-co/rest-api-express). As you know, the REST approach is based on the HTTP protocol and HTTP protocol is proven to be used in web-scale. So I think we should use HTTP protocol properly when designing RESTful apis. The github code doesn’t use HTTP status codes. For example, for successful POST, you should return 201, if no resource exist, use HTTP 404 and etc.
    Here, i don’t actually criticise you completely. Everyone knows there is no standard for RESTful api, but at least a RESTful api should use HTTP protocol properly.

  2. Thank you! Node is amazing and fun!

    I’m not sure I understand your question. CURL is a command-line tool. You can get it for Windows too. Just Google it. There was link in the comment on this blog on some post as well (Googling is faster to find)… So why would you use it within you app? Within the app I wound use http.request or superagent libraries.

  3. Hey,

    1st off – your collection of tutorials literally helped me build a website from scratch in a week – though I’m using EJS instead of Jade – I prefer being able to write pseudo JavaScript in a more html format – that and i wasn’t a big fan of the fussy nature of Jade and it’s space/tab rigidity.

    2nd: I come from an RDMS background, and tossing everything in as a string (for the most part) sure saves a lot of time.

    I did make a few mods. I didn’t create a separate route page for the collections, preferring to load them from app.js feeling an extra page superfluous, especially as the only insert is contact form requests.

    Which brings me to my question. I’ve never used curl in nodeJS/ExpressJS and am running on a windows system (client requirements), so I’m not 100% on how that is to be accomplished. The contact form itself does – using the following (within app.js):

    app.post('/submit', function(req, res) {
    if(!req.body){
    res.status(500).send({status: 500, message: 'empty submission', type:'internal'});
    }else{
    res.status(200).send({status: 200, message: JSON.stringify(req.body) });
    }
    });

    submit properly allowing me to leave my form itself with the “action=”/submit” rather than exposing the collection/:collectionName path in the source (seems bad to me if anyone wanted to look around).

    So I’m curious if you’re aware of how to go about using curl within expressJS/nodeJS on a windows 2008 server, calling from within the application itself?

    I realize I could simply take the req.body which I’ve already received and route it to:


    app.post('/collections/:collectionName', function(req, res, next) {
    req.collection.insert(req.body, {}, function(e, results){
    if (e) return next(e)
    res.send(results)
    })
    })

    with

    app.route('/collection/contact_form', function(req, res, next){
    req.collection.insert( req.body, {}, function(e, results){
    if (e) return next(e)
    res.status(200).send({message: 'success' });
    });
    });

    (i haven’t tried this yet – I just wrote it here for the question, so I don’t know if or what errors exist by doing this which I’ll attempt waiting for your reply).

    Any suggestions? and/or methods you recommend? and of course, the response answer to the curl on windows from within the app itself… that would be a sexy method to load the pages more efficiently, from both expressJS and jquery/client side if I need anything a little more robust in future…

    And of course, thanks so much for your online tutorials! I’m almost completely disinterested in traditional programming web methods with the ease of accomplishment this is wrought!

    cheers!

  4. I got the following problem when deleting the collection:

    TypeError: Object # has no method ‘id’

    I guess it’s because MongoSkin doesn’t supper this. It works when I changed the code to:

    req.collection.removeById(req.params.id,function(e,result)

  5. nice tutorial.
    i was developing a REST API from php as a part of my final year college research. but after considering performance I had to find another way. This article leads me to the right direction.
    i tried out this tutorial on Ubuntu. Currently I’m working on windows. Is there any way to run this application on windows with out curl (on Ubuntu also i successfully run it with out curl) ??

    fyi : im getting an reference error when running express.test.js and i couldnt get any results with express.js

  6. In the express.js file using your code editor like Sublime Text 2 or TextMate. If you use VIM or Emacs, you’ll be typing in your terminal.

  7. For the code that starts after the section “Express.js 4 and MongoDB (Mongoskin) Implementation”, where are we typing this in? Does this go into a new file? Are we typing directly into terminal?

  8. The full source code for express.text.js, express.js and package.json is available at github.com/azat-co/rest-api-express.

    s/text/test

    :)

  9. Great tutorial! Thanks for this!
    But I have a question,
    POST, GET, PUT is working properly. But delete isn’t.
    Here’s the error.

    TypeError: Object #<SkinClass> has no method 'id'

    I’m using Postman .

  10. The command you put npm install of body-parser has the correct version but the product is wrong …its showing mongoskin

    $ npm install mongoskin@1.0.2 –save

  11. Thanks for this tutorial!
    But I think there’s a typo in the “Express.js 4.x Middleware Caveat” section.
    Didn’t you mean
    $ npm install body-parser@1.0.2 --save
    instead of
    $ npm install mongoskin@1.0.2 --save

  12. Yes, Koa.js is coming soon, if you would like to be notified about the post, you can sign up using the newsletter opt-in form.

    Not sure about C#/Java, maybe some driver?

  13. Can you please show how to use Node.js with Kao.js. And also how to connect Kao.js with C#/Java code.

  14. Great post…

    Just find this:
    Start MongoDB with $ mognod.

    Don’t should be “mongod”?

    []’s

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.