URL Parameters and Routing in Express.js

TL;DR: This post is about URL parameters and routing in Express.js, and it’s an excerpt (Chapter 6) from my new book Pro Express.js: Master Express.js—The Node.js Framework For Your Web Development. The book was released this week (~December 24, 2014), but we have a great limited-time offer for you which will be announced on Sunday, December 28, 2014 on Webapplog.com. This post is the last post in the series of excerpts from Pro Express.js with other posts as follow: Error Handling and Running an Express.js App, Express.js Security Tips, LoopBack 101: Express.js on Steroids, Sails.js 101 and Secret Express.js Settings.

To review, the typical structure of an Express.js app fig(which is usually a server.js or app.js file) roughly consists of these parts, in the order shown:

1. Dependencies : A set of statements to import dependencies

2. Instantiations : A set of statements to create objects

3. Configurations : A set of statements to configure system and custom settings

4. Middleware : A set of statements that is executed for every incoming request

5. Routes : A set of statements that defines server routes, endpoints, and pages

6. Bootup : A set of statements that starts the server and makes it listen on a specific port for incoming requests

This chapter covers the fifth category, routes and the URL parameters that we define in routes. These parameters, along with the app.param() middleware, are essential because they allow the application to access information passed from the client in the URLs (e.g., books/proexpressjs). This is the most common convention for REST APIs. For example, the http://hackhall.com/api/posts/521eb002d00c970200000003 route will use the value of 521eb002d00c970200000003 as the post ID.

Parameters are values passed in a query string of a URL of the request. If we didn’t have Express.js or a similar library, and had to use just the core Node.js modules, we’d have to extract parameters from an HTTP.request object via some require('querystring').parse(url) or require('url').parse(url, true) function “trickery.”

Let’s look closer at how to define a certain rule or logic for a particular URL parameter.

URL Parameters in Express.js

The first approach to extracting parameters from the URLs is to write some code in the request handler (route). In case you need to repeat this snippet in other routes, you can abstract the code and manually apply the same logic to many routes. (To abstract code means to refactor the code so that it can be reused in other places and/or be organized better. This improves maintainability and readability of the code.)

For example, imagine that we need user information on a user profile page ( /v1/users/azat defined as /v1/users/:username) and on an admin page (/v1/admin/azat defined as /v1/admin/:username). One way to do this is to define a function that looks up the user information (findUserByUsername) and call this function twice inside of each of the routes. This is how we can implement it (example ch6/app.js is in GitHub):

var users = {
  'azat': {
    email: 'hi@azat.co',
    website: ' http://azat.co ',
    blog: ' http://webapplog.com '
  }
};

var findUserByUsername = function (username, callback) {
  // Perform database query that calls callback when it's done
  // This is our fake database
  if (!users[username])
    return callback(new Error(
      'No user matching '
       + username
      )
    );
  return callback(null, users[username]);
};

app.get('/v1/users/:username', function(request, response, next) {
  var username = request.params.username;
  findUserByUsername(username, function(error, user) {
    if (error) return next(error);
    return response.render('user', user);
  });
});

app.get('/v1/admin/:username', function(request, response, next) {
  var username = request.params.username;
  findUserByUsername(username, function(error, user) {
    if (error) return next(error);
    return response.render('admin', user);
  });
}); 

You can run the app from the ch6 folder with $ node app command. Then, open a new terminal tab/window, and CURL a GET request with:

$ curl http://localhost:3000/v1/users/azat 

To see this:

user profile</h2><p> http://azat.co</p><p>http://webapplog.com</p >

And with:

$ curl http://localhost:3000/v1/admin/azat 

To see this:

admin: user profile</h2><p>hi@azat.co</p><p> http://azat.co</p><p>http://webapplog.com</p><div><Practical>Node.js is your step-by-step guide to learning how to build scalable real-world web applications, taking you from installing Express.js to writing full-stack web applications with powerful libraries such as Mongoskin, Everyauth, Mongoose, Socket.IO, Handlebars, and everything in between.</Practical></div> 

Note: Windows users can download CURL from http://curl.haxx.se/download.html. Alternatively, you can use the Postman Chrome extension at http://bit.ly/JGSQwr. Or, for GET requests only, you can use your browser—just go to the URL. The browser won’t make PUT or DELETE requests, and it will make POST requests only if you submit a form.

The last approach is to use jQuery to make AJAX/XHR requests, but be mindful about the cross-origin limitations, which means using the same domain, or CORS headers on the server. Or you could simply go to http://localhost:3000/v1/users/azat (see Figure 6–1 ) and http://localhost:3000/v1/admin/azat (see Figure 6–2 ) in your browser.

The admin.jade template (Figure 6–2 ) has slightly different content from user.jade (Figure 6–1 ) to help you differentiate between the two pages/routes so you can be assured that both of them parse and use the parameters correctly.

Even after abstracting the bulk of the code into the findUserByUsername() function , we still ended up with ineloquent code. If we use the middleware approach, the code becomes a little bit better. The idea is to write a custom middleware findUserByUsernameMiddleware and use it with each route that needs the user information. Here’s how you can refactor the same two routes and use the /v2 prefix (prefixes are usually used to differentiate REST API versions):

var findUserByUsername = function (username, callback) {
  // Perform database query that calls callback when it's done
  // This is our fake database!
  if (!users[username])
    return callback(new Error(
      'No user matching '
       + username
      )
    );
  return callback(null, users[username]);
};

var findUserByUsernameMiddleware = function(request, response, next){
  if (request.params.username) {
    console.log('Username param was detected: ', request.params.username)
    findUserByUsername(request.params.username, function(error, user){
      if (error) return next(error);
      request.user = user;
      return next();
    })
 } else {
   return next();
 }
}
// The v2 routes that use the custom middleware
app.get('/v2/users/:username',
  findUserByUsernameMiddleware,
  function(request, response, next){
  return response.render('user', request.user);
});
app.get('/v2/admin/:username',
  findUserByUsernameMiddleware,
  function(request, response, next){
  return response.render('admin', request.user);
});

The middleware findUserByUsernameMiddleware checks for the presence of the parameter (request.params.username) and then, if it’s present, proceeds to fetch the information. This is a better pattern because it keeps routes lean and abstracts logic. However, Express.js has an even better solution. It’s similar to the middleware method , but it makes our lives a bit easier by automatically performing the parameter presence checks (i.e., a check to see if the
parameter is in the request). Meet the app.param() method!

app.param()

Anytime the given string (e.g., username ) is present in the URL pattern of the route, and server receives a request that matches that route, the callback to the app.param() will be triggered. For example, with app.param('username', function(req, res, next, username){...}) and app.get('/users/:username', findUser) every time we have a request /username/azat or /username/tjholowaychuk , the closure in app.param() will be executed (before findUser).

The app.param() method is very similar to app.use() but it provides the value ( username in our example) as the fourth, last parameter, to the function. In this snippet, the username will have the value from the URL (e.g., ‘azat’ for /users/azat):

app.param('username', function (request, response, next, username) {
  // ... Perform database query and
  // ... Store the user object from the database in the req object
  req.user = user;
  return next();
});

No need of extra lines of code since we have req.user object populated by the app.param() :

app.get('/users/:username', function(request, response, next) {
  //... Do something with req.user
  return res.render(req.user);
}); 

No need for extra code in this route either. We get req.user for free because of the app.param() defined earlier:

app.get('/admin/:username', function(request, response, next) {
  //... Same thing, req.user is available!
  return res.render(user);
}); 

Here is another example of how we can plug param middleware into our app:

app.param('id', function(request, response, next, id){
  // Do something with id
  // Store id or other info in req object
  // Call next when done
  next();
});  

app.get('/api/v1/stories/:id', function(request, response){
  // Param middleware will be executed before and
  // We expect req objects to already have needed info
  // Output something
  res.send(data);
}); 

Tip: If you have a large application with many versions of API and routes (v1, v2, etc.), then it’s better to use the Router class/object to organize the code of these routes. You create a Router object and mount it on a path, such as /api or /api/v1 . Router is just a stripped-down version of the var app = express() object. More details about the Router class are provided later in the chapter.

The following is an example of plugging param middleware into an app that has a Mongoskin/Monk-like database connection in req.db:

app.param('id', function(request, response, next, id){
  req.db.get('stories').findOne({_id: id}, function (error, story){
    if (error) return next(error);
    if (!story) return next(new Error('Nothing is found'));
    req.story = story;
    next();
  });
});  

app.get('/api/v1/stories/:id', function(request, response){
  res.send(req.story);
}); 

Or we can use multiple request handlers, but the concept remains the same: we can expect to have a req.story object or an error thrown prior to the execution of this code, so we abstract the common code/logic of getting parameters and their respective objects. Here is an example:

app.get('/api/v1/stories/:id', function(request, response, next) {
  //do authorization
  },
  //we have an object in req.story so no work is needed here
  function(request, response) {
    //output the result of the database search
    res.send(story);
}); 

Note Authorization and input sanitation are good candidates for residing in the middleware. For extensive examples of OAuth and Express.js, refer to Practical Node.js (Apress, 2014).

The param() function is especially cool, because we can combine different variables in the routes; for example:

app.param('storyId', function(request, response, next, storyId) {
  // Fetch the story by its ID (storyId) from a database
  // Save the found story object into request object
  request.story = story;

});
app.param('elementId', function(request, response, next, elementId) {
  // Fetch the element by its ID (elementId) from a database
  // Narrow down the search when request.story is provided
  // Save the found element object into request object
  request.element = element;
});
app.get('/api/v1/stories/:storyId/elements/:elementId', function(request, response){
  // Now we automatically get the story and element in the request object
  res.send({ story: request.story, element: request.element});
});
app.post('/api/v1/stories/:storyId/elements', function(request, response){
  // Now we automatically get the story in the request object
  // We use story ID to create a new element for that story
  res.send({ story: request.story, element: newElement});
}); 

To summarize, by defining app.param once, its logic will be triggered for every route that has the matching URL parameter name. You might be wondering, “How is it different from writing your own function and calling it, or from writing your own custom middleware?” They will both execute the code properly, but param is a more elegant approach. We can refactor our earlier example to show the difference.

Let’s go back to the ch6 project. If we refactor our previous example from ch6/app.js and use v3 as a new route prefix, we might end up with elegant code like this:

app.param('v3Username', function(request, response, next, username){
  console.log(
    'Username param was is detected: ',
    username
  )
  findUserByUsername(
    username,
    function(error, user){
      if (error) return next(error);
      request.user = user;
      return next();
    }
  );
});

app.get('/v3/users/:v3Username',
  function(request, response, next){
    return response.render('user', request.user);
  }
);
app.get('/v3/admin/:v3Username',
  function(request, response, next){
    return response.render('admin', request.user);
  }
); 

So, extracting parameters is important, but defining routes is more important. Defining routes is also an alternative to using app.param() to extract values from URL parameters—this method is recommended when a parameter is used only once. If it’s used more than once, param is a better pattern.

A lot of routes have already been defined in the five previous chapters. In the next section, we’ll explore in more detail how to define various HTTP methods , chain middleware, abstract middleware code, and define all-method routes.

Routing in Express.js

Express.js is a Node.js framework that, among other things, provides a way to organize routes into smaller subsections (Routers—instances of Router class/object). In Express.js 3.x and earlier, the only way to define routes is to use the app.VERB() pattern, which we’ll cover next. However, starting with Express.js v4.x, using the new Router class is the recommended way to define routes, via router.route(path) . We’ll cover the traditional approach first.

app.VERB()

Each route is defined via a method call on an application object with a URL pattern as the first parameter (Regular Expressions are also supported); that is, app.METHOD(path, [callback...], callback).

For example, to define a GET /api/v1/stories endpoint:

app.get('/api/v1/stories/', function(request, response){
  // ...
})

Or, to define an endpoint for the POST HTTP method and the same route:

app.post('/api/v1/stories', function(request, response){
  // ...
}) 

DELETE, PUT, and other methods are supported as well. For more information, see http://expressjs.com/api.html#app.VERB.

The callbacks that we pass to get() or post() methods are called request handlers (covered in detail in Chapter 7), because they take requests (req), process them, and write to the response (res) objects. For example:

app.get('/about', function(request, response){
  res.send('About Us: ...');
}); 

We can have multiple request handlers in one route. All of them except the first and the last will be in the middle of the flow (order in which they are executed), hence the name middleware . They accept a third parameter/function,
next , which when called (next()), switches the execution flow to the next handler. For example, we have three functions that perform authorization, database search and output:

app.get('/api/v1/stories/:id', function(request, response, next) {
  // Do authorization
  // If not authorized or there is an error
  // Return next(error);
  // If authorized and no errors
  return next(); 
}), function(request, response, next) {
  // Extract id and fetch the object from the database
  // Assuming no errors, save story in the request object
  request.story = story;
  return next(); 

}), function(request, response) {
  // Output the result of the database search
  res.send(response.story);
});

The name next() is an arbitrary convention, which means you can use anything else you like instead of next(). The Express.js uses the order of the arguments in the function to determine their meaning. The ID of a story is the URL parameter, which we need for finding matching items in the database.

Now, what if we have another route /admin. We can define multiple request handlers, which perform authentication, validation, and loading of resources:

app.get('/admin',
  function(request, response, next) {
    // Check active session, i.e.,
    // Make sure the request has cookies associated with a valid user session
    // Check if the user has administrator privileges
    return next();
  }, function(request, response, next){
    // Load the information required for admin dashboard
    // Such as user list, preferences, sensitive info
    return next();
  }, function(request, response) {
    // Render the information with proper templates
    // Finish response with a proper status
    res.end();
 }) 

But what if some of the code for /admin , such as authorization/authentication, is duplicated from the /stories ? The following accomplishes the same thing, but is much cleaner with the use of named functions:

var auth = function (request, response, next) {
  // ... Authorization and authentication
  return next();
}
var getStory = function (request, response, next) {
  // ... Database request for story
  return next();
}
var getUsers = function (request, response, next) {
  // ... Database request for users
  return next();
}
var renderPage = function (request, response) {
 if (req.story) res.render('story', story);
 else if (req.users) res.render('users', users);
 else res.end();
}
app.get('/api/v1/stories/:id', auth, getStory, renderPage);
app.get('/admin', auth, getUsers, renderPage);

Another useful technique is to pass callbacks as items of an array, made possible thanks to the inner workings of the arguments JavaScript mechanism:

var authAdmin = function (request, response, next) {
  // ...
  return next();
}
var getUsers = function (request, response, next) {
  // ...
  return next();
}
var renderUsers = function (request, response) {
  // ...
  res.end();
}
var admin = [authAdmin, getUsers, renderUsers];
app.get('/admin', admin); 

One distinct difference between request handlers in routes and middleware is that we can bypass the rest of the callbacks in the chain by calling next('route'); . This might come in handy if, in the previous example with the
/admin route, a request fails authentication in the first callback, in which case there’s no need to proceed. You can also use next() to jump to the next route if you have multiple routes matching the same URL.

Please note that, if the first parameter we pass to app.VERB() contains query strings (e.g., /?debug=true), that information is disregarded by Express.js. For example, app.get('/?debug=true', routes.index); will be treated exactly as app.get('/', routes.index);.

The following are the most commonly used Representational State Transfer (REST) server architecture HTTP methods and their counterpart methods in Express.js along with the brief meaning:

• GET: app.get() —Retrieves an entity or a list of entities
• HEAD: app.head() —Same as GET, only without the body
• POST: app.post() —Submits a new entity
• PUT: app.put() —Updates an entity by complete replacement
• PATCH: app.patch() —Updates an entity partially
• DELETE: app.delete() and app.del() —Deletes an existing entity
• OPTIONS: app.options() —Retrieves the capabilities of the server

Tip: An HTTP method is a special property of every HTTP(S) request, similar to its headers or body. Opening a URL in your browser is a GET request, and submitting a form is a POST request. Other types of requests, such as PUT, DELETE, PATCH, and OPTIONS, are only available via special clients such as CURL, Postman, or custom-built applications (both front-end and back-end).

For more information on HTTP methods, please refer to RFC 2616 and its “Method Definitions” section (section 9).

app.all()

The app.all() method allows the execution of specified request handlers on a particular path regardless of what the HTTP method of the request is. This procedure might be a lifesaver when defining global or namespace logic, as in
this example:

app.all('*', userAuth);
...
app.all('/api/*', apiAuth); 

Trailing Slashes

Paths with trailing slashes at the end are treated the same as their normal counterparts by default. To turn off this feature, use app.enable('strict routing'); or app.set('strict routing', true); . You can learn more about
setting options in Chapter 3.

Router Class

The Router class is a mini Express.js application that has only middleware and routes. This is useful for abstracting certain modules based on the business logic that they perform. For example, all /users/* routes can be defined in
one router, and all /posts/* routes can be defined in another. The benefit is that, after we define a portion of the URL in the router with router.path() (see the next section), we don’t need to repeat it over and over again, such as is the case with using the app.VERB() approach.

The following is an example of creating a router instance:

var express = require('express');
var router = express.Router(options);
// ... Define routes
app.use('/blog', router);

where options is an object that can have following properties:

• caseSensitive : Boolean indicating whether to treat routes with the same name but different letter case as different, false by default; e.g., if it’s set to false, then /Users is the same as
  /users.

• strict: Boolean indicating whether to treat routes with the same name but with or without a trailing slash as different, false by default; e.g., if it’s set to false, then /users is the same as
  /users/.

router.route(path)

The router.route(path) method is used to chain HTTP verb methods. For example, in a create, read, update, and delete (CRUD) server that has POST, GET, PUT, and DELETE endpoints for the /posts/:id URL (e.g., /posts/53fb401dc96c1caa7b78bbdb ), we can use the Router class as follows:

var express = require('express');
var router = express.Router();
// ... Importations and configurations
router.param('postId', function(request, response, next) {
  // Find post by ID
  // Save post to request
  request.post = {
    name: 'PHP vs. Node.js',
    url: ' http://webapplog.com/php-vs-node-js '
  };
  return next();
});

router
  .route('/posts/:postId')
  .all(function(request, response, next){
    // This will be called for request with any HTTP method
    })
  .post(function(request, response, next){
  })
  .get(function(request, response, next){
    response.json(request.post);
  })
  .put(function(request, response, next){
    // ... Update the post
    response.json(request.post);
  })
  .delete(function(request, response, next){
    // ... Delete the post
    response.json({'message': 'ok'});
  }) 

The Router.route(path) method provides the convenience of chaining methods, which is a more appealing way to structure, your code than re-typing router for each route.

Alternatively, we can use router.VERB(path, [callback...], callback) to define the routes just as we would use app.VERB(). Similarly, the router.use() and router.param() methods work the same as app.use() and app.param().

Going back to our example project (in the ch6 folder), we can implement v4/users/:username and v4/admin/:username with Router:

router.param('username', function(request, response, next, username){
  console.log(
    'Username param was detected: ',
    username
  )
  findUserByUsername(
    username,
    function(error, user){
      if (error) return next(error);
      request.user = user;
      return next();
    }
  );
})
router.get('/users/:username',
  function(request, response, next){
    return response.render('user', request.user);
  }
);
router.get('/admin/:username',
  function(request, response, next){
    return response.render('admin', request.user);
  }
);
app.use('/v4', router); 

As you can see, router.get() methods include no mention of v4 . Typically, the router.get() and router.param() methods are abstracted into a separate file. This way, the main file (app.js in our example) stays lean and easy to read and maintain—a nice principle to follow!

Request Handlers

Request handlers in Express.js are strikingly similar to callbacks in the core Node.js http.createServer() method , because they’re just functions (anonymous, named, or methods) with req and res parameters:

var ping = function(req, res) {
  console.log('ping');
  res.end(200);
};
app.get('/', ping); 

In addition, we can utilize the third parameter, next(), for control flow. It’s closely related to the topic of error handling, which is covered in Chapter 9. Here is a simple example of two request handlers, ping and pong where the
former just skips to the latter after printing a word ping:

var ping = function(req, res, next) {
console.log(‘ping’);
return next();
};
var pong = function(req, res) {
console.log(‘pong’);
res.end(200);
};
app.get(‘/’, ping, pong);

When a request comes on the / route, Express.js calls ping() , which acts as middleware in this case (because it’s in the middle!). Ping, in turn, when it’s done, calls pong with that finished response with res.end().

The return keyword is also very important. For example, we don’t want to continue processing the request if the authentication has failed in the first middleware:

// Instantiate app and configure error handling

// Authentication middleware
var checkUserIsAdmin = function (req, res, next) {
  if (req.session && req.session._admin !== true) {
    return next (401);
  }
  return next();
};

// Admin route that fetches users and calls render function
var admin = {
  main: function (req, res, next) {
    req.db.get('users').find({}, function(e, users) {
      if (e) return next(e);
      if (!users) return next( new Error('No users to display.'));
      res.render('admin/index.html', users);
   });
  }
};

// Display list of users for admin dashboard
app.get('/admin', checkUserIsAdmin, admin.main); 

The return keyword is essential, because if we don’t use it for the next(e) call, the application will try to render (res.render()) even when there is an error and/or we don’t have any users. For example, the following is probably a
bad idea because after we call next(), which will trigger the appropriate error in the error handler, the flow goes on and tries to render the page:

var admin = {
  main: function (req, res, next) {
    req.db.get('users').find({}, function(e, users) {
      if (e) next(e);
      if (!users) next( new Error('No users to display.'));
      res.render('admin/index.html', users);
   });
  }
 };

We should be using something like this:

if (!users) return next(new Error('No users to display.'));
res.render('admin/index.html', users);

or something like this:

if (!users)
  return next(new Error('No users to display.'));
else
  res.render('admin/index.html', users); 

Summary

In this chapter we covered two major aspects of the typical structure of an Express.js app: defining routes and extracting URL parameters. We explored three different ways of how to get them out of the URL and use them in request handlers ( req.params , custom middleware and app.param()). You learned how to define routes for various HTTP methods. Finally, we delved deep into the Router class, which acts as a mini Express.js application, and implemented yet another set of routes for the example project using the Router class.

Every time we defined a router (or a middleware), we used either an anonymous function definition or a named function in callbacks to define the request handler. The request handler usually has three parameters: request (or
req ), response (or res ), and next . In the next chapter, you’ll learn more about these objects, and how, in Express.js, they are different from the core Node.js http module’s request and response . Knowing these differences will give you more features and functionality!

PS: Make sure to check sign up for the Webapplog.com and check your inbox on Sunday December 28, 2014 for the special time-sensitive offer on Pro Express.js.