Understanding Fat Arrows (=>) in CoffeeScript

The fat arrows (=>) are special function expressions that confuse even the smartest of developers. Most JavaScript professionals know that closure changes the meaning of this (@ in CoffeeScript).

There’s no reason to be afraid of fat arrows in CoffeeScript. They are time savers (and clarity achievers) when it comes to the nested functions and jQuery event handlers. When they are applied to class methods, the good rule of thumb is: use => when we need @ to be the object in which method is written; use-> when we need @ to be the object in which method is executed.

The fat arrows (=>) are special function expressions that confuse even the smartest of developers. Most JavaScript professionals know that closure changes the meaning of this (@ in CoffeeScript).

Continue reading “Understanding Fat Arrows (=>) in CoffeeScript”

Q&A: Thank You Web App, Express.js and Emails

I recently received a question from Joanne Daudier. She is a up and coming web developer, middle term student at a JS/Node.js bootcamp called RefactorU and a reader of Express.js Guide. This is what she wrote:

Hi Azat:

I skimmed over your book today. I feel like it is a bit advanced for me since I’m just learning Express. I do have a few questions for you though.

I recently received a question from Joanne Daudier. She is an up and coming web developer, middle term student at a JS/Node.js bootcamp called RefactorU and a reader of Express.js Guide. This is what she wrote:

Continue reading “Q&A: Thank You Web App, Express.js and Emails”

Solving Programming Problems

Programming problems are not that much different from mathematics or physics problems. There are usually an input and an output to which someone needs to arrive by providing an algorithm. This algorithm is typically a function or series of functions.

Programming puzzles and toy problems are good exercises to sharpen skills and prepare for technical interviews. No wonder that more and more online coding schools (e.g., CodeAcademy) make those metal workouts main staple of their courses.

Beginner programmers might benefit by applying these steps to their process of solving a programming problem:

Programming problems are not that much different from mathematics or physics problems. There are usually an input and an output to which someone needs to arrive by providing an algorithm. This algorithm is typically a function or series of functions.

Programming puzzles and toy problems are good exercises to sharpen skills and prepare for technical interviews. No wonder that more and more online coding schools (e.g., CodeAcademy) make those metal workouts main staple of their courses.

Beginner programmers might benefit by applying these steps to their process of solving a programming problem:

Continue reading “Solving Programming Problems”

The Release of Express.js Guide: The Comprehensive Book on Express.js

After weeks of writing and editing, Azat and his team are happy to announce the release of Express.js Guide: The Most Popular Node.js Framework Manual! The book is very approachable and suitable for beginners. If someone wants to save time searching the web and learn the best practices from the trenches, Express.js Guide is the book that has everything: Express.js API reference, quick start guides, 20+ meticulously explained examples and tutorials on over 270 pages with more than 60 illustrations.

Express.js is a de facto standard of Node.js development and the most popular NPM library as of today! However, as with any framework, sometimes the learning curve is steep. At HackReactor, I often asked the same questions about code organization, authentication, database connections and deployment.

Continue reading “The Release of Express.js Guide: The Comprehensive Book on Express.js”

Todo App with Express.js/Node.js and MongoDB

Todo apps are considered to be quintessential in showcasing frameworks akin to famous Todomvc.com for front-end JavaScript frameworks. In this example, we’ll use Jade, forms, LESS, AJAX/XHR and CSRF.

Note: This tutorial is a part of Express.js Guide: The Comprehensive Book on Express.js.

Todo apps are considered to be quintessential in showcasing frameworks akin to famous Todomvc.com for front-end JavaScript frameworks. In this example, we’ll use Jade, forms, LESS, AJAX/XHR and CSRF.

In our Todo app, we’ll intentionally not use Backbone.js or Angular to demonstrate how to build traditional websites with the use of forms and redirects. In addition to that, we’ll explain how to plug-in CSRF and LESS.

Example: All the source code is in the github.com/azat-co/todo-express for your convenience.

Continue reading “Todo App with Express.js/Node.js and MongoDB”

JS FUNdamentals: An Essential Overview of JavaScript

Programming languages like BASIC, Python, C has boring machine-like nature which requires developers to write extra code that’s not directly related to the solution itself. Think about line numbers in BASIC or interfaces, classes and patterns in Java.

On the other hand JavaScript inherits the best traits of pure mathematics, LISP, C# which lead to a great deal of expressiveness (and fun!).

More about Expressive Power in this post: What does “expressive” mean when referring to programming languages?

If it’s not fun, it’s not JavaScript.

Note: This text is a part of upcoming ebook JavaScript and Node FUNdamentals: A Collection of Essential Basics.

Expressiveness

Programming languages like BASIC, Python, C has boring machine-like nature which requires developers to write extra code that’s not directly related to the solution itself. Think about line numbers in BASIC or interfaces, classes and patterns in Java.

Continue reading “JS FUNdamentals: An Essential Overview of JavaScript”

Tutorial: Node.js and MongoDB JSON REST API server with Mongoskin and Express.js

Tutorial: building a JSON REST API server with Node.js and MongoDB using Mongoskin and Express.js utilizing Mocha and Super Agent for BDD/TDD.

Update3: Expess 4 version of this tutorial is available at Express.js 4, Node.js and MongoDB REST API Tutorial, and github.com/azat-co/rest-api-express (master branch). This tutorial will work with Express 3.x.

Update2: “Mongoskin removed ‘db.collection.id’ and added some actionById methods” from this pull request with this code changes. To use the code in this post, just install an older version of Mongoskin (0.5.0?). The code in the GitHub will work with Mongoskin 1.3.20.

Update2: “Mongoskin removed ‘db.collection.id’ and added some actionById methods” from this pull request with this code changes. To use the code in this post, just install an older version of Mongoskin (0.5.0?)

Update: use the enhanced code from this repository github.com/azat-co/rest-api-express (express3 branch).

Note: This text is a part of Express.js Guide: The Comprehensive Book on Express.js.

This tutorial will walk you through writing test using the Mocha and Super Agent libraries and then use them in a test-driven development manner to build a Node.js free JSON REST API server utilizing Express.js framework and Mongoskin library for MongoDB. In this REST API server, we’ll perform create, read, update and delete (CRUD) operations and harness Express.js middleware concept with app.param() and app.use() methods.

Continue reading “Tutorial: Node.js and MongoDB JSON REST API server with Mongoskin and Express.js”

Cisco, Node.js and other musings

The hands-on Introduction to Node.js training consisted of two days and started badly: I was late driving from Oakland to San Jose through traffic, and security personnel in the lobby took extra 15 minutes to clear and escort me to the classroom. Nevertheless, attendees quickly plunged into installing Node.js, Node Package Manager and MongoDB using hard copies of Rapid Prototyping with JS: Agile JavaScript Development that I brought with me.

Continue reading “Cisco, Node.js and other musings”

Not Everything has to be done Server-side

Here is a little JavaScript trick I did to randomly include LeanPub embed (e.g., like the one in the right sidebar

Here is a little JavaScript trick I did to randomly include LeanPub embed (e.g., like the one in the right sidebar).

Continue reading “Not Everything has to be done Server-side”

Express.js Tutorial: Instagram Gallery Example App with Storify API

Storify runs on Node.js and Express.js, therefore why not use these technologies to write an application that demonstrates how to build apps that rely on third-party APIs and HTTP requests to them?

Note: This text is a part of Express.js Guide: The Comprehensive Book on Express.js.

An example of an Express.js app using Storify API as a data source is a continuation of introduction to Express.js tutorials.

Continue reading “Express.js Tutorial: Instagram Gallery Example App with Storify API”

LeanPub Compilation and Sanitizing Tool Written in Node.js

LeanPub Tool — Node.js script for sanitizing and compiling a book’s manuscript. LeanPub uses multi-file book format and a special markdown markup (that confuses Marked app). I found that to convert to MS Word (for editing and other needs) this workflow is the best: 1. run leanpub-tool.js with node.js (node leanpub-tool.js) 2. open file in Marked app and convert to HTML 3. open HTML in MS Word and enjoy.

LeanPub is an awesome publishing platform, but often we need to communicate with other  team members involved in the book production, e.g., content and copy editors. In such times, I’ve learned that Guy Kawasaki was right — when we suggested in his book APE: Author, Publisher, Entrepreneur — that author must use MS Word!

The problem is that LeanPub uses multi-file book format and a special markdown markup (that confuses Marked app). I was wrapping up a new revision of the Rapid Prototyping with JS book and previous experience of handing a txt file to an editor was poor. Luckily, Node.js came to help! Through some trail and error attempts, I found this workflow to be the best for converting LeanPub manuscript to a one MS Word file (for editing and other needs):

  1. Run leanpub-tool.js with Node.js (node leanpub-tool.js)
  2. Open file in Marked app and convert to HTML
  3. Open HTML in MS Word and enjoy.

The full code of leanpub-tool.js which is also available at https://gist.github.com/azat-co/5674684:


var fs = require('fs');
//change these to YOUR filenames
//probably we can read it from Book.txt but I was too lazy to implement it :-) and somebody might want to compile only sertain parts of the book
var book =[
"frontmatter.txt",
"mainmatter.txt",
"part1.txt",
"chapter1.txt",
"part2.txt",
"chapter2.txt",
"backmatter.txt",
"acknowledgment.txt"
];
 
var sanitizeText = [
  '{frontmatter}',
  '{backmatter}',
  '{mainmatter}',
  "I>## Note",
  "T>## Tip",
  "W>## Warning",
  '{lang="javascript"}',
  '{:lang="javascript"}',
  '{lang="css"}',
  '{:lang="css"}',
  '{lang="json"}',
  '{lang="ruby"}',
  '{lang="php"}',
  '{lang="text"}',
  '{lang="bash"}',
  '{lang="html"}',
  "I>",
  "T>",
  "W>"
]
 
 
var str = '';
//read files
book.forEach(function(chapter){
  str +=  fs.readFileSync(chapter,'utf8');
})
//sanitize LeanPub specific Markdown tags
sanitizeText.forEach(function(text){
//  console.log(text)
  //this loops through while there is not matches
  while (new RegExp (text).test(str)) {
    str = str.replace(text,'','gm')
  };
})
 
//write output to a file
fs.writeFileSync('leanpub-tool.txt',str);

Rapid Prototyping with JS: Agile JavaScript Development is done with v0.5 (1.0RC)!

This is a release candidate for 1.0 version which is going to be available to public in print. Page count has increase from 151 to 211 pages (PDF), which include the following updates:

More digestible (smaller) code examples with better comments
Express.js middleware section with an example
Express.js + MongoDB REST API server section with an example
Derby section with an example
Grammar and typo fixes
Illustrations
Summaries in the beginning of each chapter
Code examples formatting fixes

Rapid Prototyping with JS: Agile JavaScript Development
Rapid Prototyping with JS: Agile JavaScript Development

This is a release candidate for 1.0 version which is going to be available to public in print. Page count has increase from 151 to 211 pages (PDF), which include the following updates:

  • More digestible (smaller) code examples with better comments
  • Express.js middleware section with an example
  • Express.js + MongoDB REST API server section with an example
  • Derby section with an example
  • Grammar and typo fixes
  • Illustrations
  • Summaries in the beginning of each chapter
  • Code examples formatting fixes

Download a free chapter Intro to Backbone.js or full PDF, ePub or Mobi copy at LeanPub.

Rapid Prototyping with JS touched topics such as:

  • jQuery
  • AJAX
  • CSS and LESS
  • JSON and BSON
  • Twitter Bootstrap
  • Node.js
  • MongoDB
  • Parse.com
  • Agile methodologies
  • Git
  • Heroku, MongoHQ and Windows Azure
  • REST API
  • Backbone.js
  • AMD and Require.js
  • Express.js
  • Monk
  • Derby

If you need in-depth knowledge or references, they are usually one click or one Google search away.

Practical aspect included building multiple versions of the Message Board app:

  • jQuery + Parse.com JS REST API
  • Backbone and Parse.com JS SDK
  • Backbone and Node.js
  • Backbone and Node.js + MongoDB

The Message Board application has all the foundation of a typical web/mobile application: fetching data, displaying it, submitting new data. Other examples include:

  • jQuery + Twitter RESP API “Tweet Analyzer”
  • Parse.com “Save John”
  • Node.js “Hello World”
  • MongoDB “Print Collections”
  • Derby + Express “Hello World”
  • Backbone.js “Hello World”
  • Backbone.js “Apple Database”
  • Monk + Expres.js “REST API Server”

Oh My JS: The Best JavaScript Articles (https://leanpub.com/ohmyjs/) is in the works!

Nested Objects in Mongoose

There is a certain magic in ORMs like Mongoose. I learned it the hard way (as usual!), when I was trying to iterate over nested object’s properties…

There is a certain magic in ORMs like Mongoose. I learned it the hard way (as usual!), when I was trying to iterate over nested object’s properties. For example, here is a schema with a nested object features defines like this:

var User = module.exports = new Schema({
  features: { 
    realtime_updates: {
      type: Boolean
    },
    storylock: {
      type: Boolean
    },
    custom_embed_style: {
      type: Boolean
    },
    private_stories: {
      type: Boolean
    },
    headerless_embed:{
      type: Boolean
    }
};

Let’s say I want to overwrite object features_enabled with these properties:

if (this.features) { 
  for (var k in this.features) {
    features_enabled[k] = this.features[k];
  }
}
console.log(features_enabled)
return features_enabled;

Not so fast, I was getting a lot of system properties specific to Mongoose. Instead we need to use toObject(), e.g.:

if (this.features.toObject()) { 
  for (var k in this.features.toObject()) {
    console.log('!',k)
    features_enabled[k] = this.features.toObject()[k];
  }
}

Remember rule number one, computer is always right. If we think that it’s wrong — look up the rule number one. :-)

Intro to Express.js: Parameters, Error Handling and Other Middleware

Express.js is a node.js framework that among other things provides a way to organize routes. Each route is defined via a method call on an application object with a URL patter as a first parameter (RegExp is also supported). The best practice is to keep router lean and thin by moving all the logic into corresponding external modules/files. This way important server configuration parameters will be neatly in one place, right there when you need them! :-)

Note: This text is a part of Express.js Guide: The Comprehensive Book on Express.js.

Express.js is one of the most popular and mature Node.js frameworks. You can read more about in Intro to Express.js series on webapplog.com:

To learn how to create an application from scratch please refer to the earlier post.

Request Handlers

Express.js is a node.js framework that among other things provides a way to organize routes. Each route is defined via a method call on an application object with a URL patter as a first parameter (RegExp is also supported), for example:

app.get('api/v1/stories/', function(res, req){
  ...
})

or, for a POST method:

app.post('/api/v1/stories'function(req,res){
  ...
})

It’s needless to say that DELETE and PUT methods are supported as well.
The callbacks that we pass to get() or post() methods are called request handlers, because they take requests (req), process them and write to response (res) objects. For example:

app.get('/about', function(req,res){
  res.send('About Us: ...');
});

We can have multiple request handlers, hence the name middleware. They accept a third parameter next calling which (next()) will switch the execution flow to the next handler:

app.get('/api/v1/stories/:id', function(req,res, next) {
  //do authorization
  //if not authorized or there is an error 
  // return next(error);
  //if authorized and no errors
  return next();
}), function(req,res, next) {
  //extract id and fetch the object from the database
  //assuming no errors, save story in the request object
  req.story = story;
  return next();
}), function(req,res) {
  //output the result of the database search
  res.send(res.story);
});

ID of a story in URL patter is a query string parameter which we need for finding a matching items in the database.

Parameters Middleware

Parameters are values passed in a query string of a URL of the request. If we didn’t have Express.js or similar library, and had to use just the core Node.js modules, we’d had to extract parameters from HTTP.request object via some require('querystring').parse(url) or require('url').parse(url, true) functions trickery.

Thanks to Connect framework and people at VisionMedia, Express.js already has support for parameters, error handling and many other important features in the form of middlewares. This is how we can plug param middleware in our app:

app.param('id', function(req,res, next, id){
  //do something with id
  //store id or other info in req object
  //call next when done
  next();
});

app.get('/api/v1/stories/:id',function(req,res){
  //param middleware will be execute before and
  //we expect req object already have needed info
  //output something
  res.send(data);
});

For example:

app.param('id', function(req,res, next, id){
  req.db.get('stories').findOne({_id:id}, function (e, story){
    if (e) return next(e);
    if (!story) return next(new Error('Nothing is found'));
    req.story = story;
    next();
  });
});

app.get('/api/v1/stories/:id',function(req,res){
  res.send(req.story);
});

Or we can use multiple request handlers but the concept remains the same: we can expect to have req.story object or an error thrown prior to the execution of this code so we abstract common code/logic of getting parameters and their respective objects:

app.get('/api/v1/stories/:id', function(req,res, next) {
  //do authorization
  }),
  //we have an object in req.story so no work is needed here
  function(req,res) {
  //output the result of the database search
  res.send(story);
});

Authorization and input sanitation are also good candidates for residing in the middlewares.

Function param() is especially cool because we can combine different keys, e.g.:

app.get('/api/v1/stories/:storyId/elements/:elementId',function(req,res){
  res.send(req.element);
});

Error Handling

Error handling is typically used across the whole application, therefore it’s best to implement it as a middleware. It has the same parameters plus one more, error:

app.use(function(err, req, res, next) {
  //do logging and user-friendly error message display
  res.send(500);
})

In fact, the response can be anything:

JSON string

app.use(function(err, req, res, next) {
  //do logging and user-friendly error message display
  res.send(500, {status:500, message: 'internal error', type:'internal'});
})

Text message

app.use(function(err, req, res, next) {
  //do logging and user-friendly error message display
  res.send(500, 'internal server error');
})

Error page

app.use(function(err, req, res, next) {
  //do logging and user-friendly error message display
  //assuming that template engine is plugged in
  res.render('500');
})

Redirect to error page

app.use(function(err, req, res, next) {
  //do logging and user-friendly error message display
  res.redirect('/public/500.html');
})

Error HTTP response status (401, 400, 500, etc.)

app.use(function(err, req, res, next) {
  //do logging and user-friendly error message display
  res.end(500);
})

By the way, logging is also should be abstracted in a middleware!

To trigger an error from within your request handlers and middleware you can just call:

next(error);

or

next(new Error('Something went wrong :-(');

You can also have multiple error handlers, and use named instead of anonymous functions as its shows in Express.js Error handling guide.

Other Middleware

In addition to extracting parameters, it can be used for many things, like authorization, error handling, sessions, output, and others.

res.json() is one of them. It conveniently outputs JavaScript/Node.js object as a JSON. For example:

app.get('/api/v1/stories/:id', function(req,res){
  res.json(req.story);
});

is equivalent to (if req.story is an Array and Object):

app.get('/api/v1/stories/:id', function(req,res){
  res.send(req.story);
});

or

app.get('api/v1/stories/:id',function(req,res){
  res.set({
    'Content-Type': 'application/json'
  });
  res.send(req.story);
});

Abstraction

Middleware is flexible. You can use anonymous or named functions, but the best thing is to abstract request handlers into external modules based on the functionality:

var stories = require.('./routes/stories');
var elements = require.('./routes/elements');
var users = require.('./routes/users');
...
app.get('/stories/,stories.find);
app.get('/stories/:storyId/elements/:elementId', elements.find);
app.put('/users/:userId',users.update);

routes/stories.js:

module.exports.find = function(req,res, next) {
};

routes/elements.js:

module.exports.find = function(req,res,next){
};

routes/users.js:

module.exports.update = function(req,res,next){
};

You can use some functional programming tricks, like this:

function requiredParamHandler(param){
  //do something with a param, e.g., check that it's present in a query string
  return function (req,res, next) {
    //use param, e.g., if token is valid proceed with next();
    next();
  });
}

app.get('/api/v1/stories/:id', requiredParamHandler('token'), story.show);

var story  = {
  show: function (req, res, next) {
    //do some logic, e.g., restrict fields to output
    return res.send();
  }
}   

As you can see middleware is a powerful concept for keeping code organized. The best practice is to keep router lean and thin by moving all the logic into corresponding external modules/files. This way important server configuration parameters will be neatly in one place, right there when you need them! :-)

Node.js MVC: Express.js + Derby.js Hello World Tutorial

Express.js is a popular node frameworks which uses middleware concept to enhance functionality of applications. Derby is a new sophisticated Model View Controller (MVC) framework which is designed to be used with Express as it’s middleware. Derby also comes with the support of Racer, data synchronization engine, and Handlebars-like template engine among many other features.

DerbyJS — Node.js MVC Framework

Express.js is a popular node frameworks which uses middleware concept to enhance functionality of applications. Derby is a new sophisticated Model View Controller (MVC) framework which is designed to be used with Express as it’s middleware. Derby also comes with the support of Racer, data synchronization engine, and Handlebars-like template engine among many other features.

Derby.js Installation

Let’s set up a basic Derby application architecture without the use of scaffolding. Usually project generators are confusing when people just start to learn a new comprehensive framework. This is a bare minimum “Hello World” application tutorial that still illustrates Derby skeleton and demonstrates live-templates with websockets.

Of course we’ll need Node.js and NPM which can be obtained at nodejs.org. To install derby globally run:

$ npm install -g derby

To check the installation:

$ derby -V

My version as of April 2013 is 0.3.15. We should be good to go to creating our first app!

File Structure in a Derby.js App

This is the project folder structure:

project/
  -package.json
  -index.js
  -derby-app.js
  views/
    derby-app.html
  styles/
    derby-app.less

Dependencies for The Derby.js Project

Let’s include dependencies and other basic information in package.json file:

 {
  "name": "DerbyTutorial",
  "description": "",
  "version": "0.0.0",
  "main": "./server.js",
  "dependencies": {
    "derby": "*",
    "express": "3.x"
  },
  "private": true
}

Now we can run npm install which will download our dependencies into node_modules folder.

Views in Derby.js

Views must be in views folder and they must be either in index.html under a folder which has the same name as your derby app JavaScript file, i.e., views/derby-app/index.html, or be inside of a file which has the same name as your derby app JS file, i.e., derby-app.html.

In this example “Hello World” app we’ll use <Body:> template and {message} variable. Derby uses mustach-handlebars-like syntax for reactive binding. index.html looks like this:

<Body:>
  <input value="{message}"><h1>{message}</h1>

Same thing with Stylus/LESS files, in our example index.css has just one line:

h1 {
  color: blue;
}

To find out more about those wonderful CSS preprocessors check out documentation at Stylus and LESS.

Building The Main Derby.js Server

index.js is our main server file, and we begin it with an inclusion of dependencies with require() function:

var http = require('http'),
  express = require('express'),
  derby = require('derby'),
  derbyApp = require('./derby-app');

Last line is our derby application file derby-app.js.

Now we’re creating Express.js application (v3.x has significant differences between 2.x) and an HTTP server:

var expressApp = new express(),
  server = http.createServer(expressApp);

Derby uses Racer data synchronization library which we create like this:

var store = derby.createStore({
  listen: server
});

To fetch some data from back-end to the front-end we instantiate model object:

var model = store.createModel();

Most importantly we need to pass model and routes as middlewares to Express.js app. We need to expose public folder for socket.io to work properly.

expressApp.
  use(store.modelMiddleware()).
  use(express.static(__dirname + '/public')).
  use(derbyApp.router()).
  use(expressApp.router);

Now we can start the server on port 3001 (or any other):

server.listen(3001, function(){
  model.set('message', 'Hello World!');
});

Full code of index.js file:

var http = require('http'),
  express = require('express'),
  derby = require('derby'),
  derbyApp = require('./derby-app');

var expressApp = new express(),
  server = http.createServer(expressApp);

var store = derby.createStore({
  listen: server
});

var model = store.createModel();

expressApp.
  use(store.modelMiddleware()).
  use(express.static(__dirname + '/public')).
  use(derbyApp.router()).
  use(expressApp.router);

server.listen(3001, function(){
  model.set('message', 'Hello World!');
});

Derby.js Application

Finally, Derby app file which contains code for both a front-end and a back-end. Front-end only code is inside of app.ready() callback. To start, let’s require and create an app. Derby uses unusual construction (not the same familiar good old module.exports = app):

var derby = require('derby'),
  app = derby.createApp(module);

To make socket.io magic work we need to subscribe model attribute to its visual representation, in other words bind data and view. We can do it in the root route, and this is how we define it (patter is /, a.k.a. root):

app.get('/', function(page, model, params) {
  model.subscribe('message', function() {
    page.render();  
  })  
});

Full code of derby-app.js file:

var derby = require('derby'),
  app = derby.createApp(module);

app.get('/', function(page, model, params) {
  model.subscribe('message', function() {
    page.render();  
  })  
});  

Launching Hello World App

Now everything should be ready to boot our server. Execute node . or node index.js and open a browser at http://localhost:3001. You should be able to see something like this: http://cl.ly/image/3J1O0I3n1T46.

Derby + Express.js Hello World App

Passing Values to Back-End in Derby.js

Of course static data is not much, so we can slightly modify our app to make back-end and front-end pieces talks with each other.

In the server file index.js add store.afterDb to listen to set events on message attribute:

server.listen(3001, function(){
  model.set('message', 'Hello World!');
  store.afterDb('set','message', function(txn, doc, prevDoc, done){
    console.log(txn)
    done();
  }) 
});

Full code of index.js after modifications:

var http = require('http'),
  express = require('express'),
  derby = require('derby'),
  derbyApp = require('./derby-app');

var expressApp = new express(),
  server = http.createServer(expressApp);

var store = derby.createStore({
  listen: server
});

var model = store.createModel();

expressApp.
  use(store.modelMiddleware()).
  use(express.static(__dirname + '/public')).
  use(derbyApp.router()).
  use(expressApp.router);

server.listen(3001, function(){
  model.set('message', 'Hello World!');
  store.afterDb('set','message', function(txn, doc, prevDoc, done){
    console.log(txn)
    done();
  })   
});

In Derby application file derby-app.js add model.on() to app.ready():

  app.ready(function(model){
	    model.on('set', 'message',function(path, object){
	    console.log('message has been changed: '+ object);
	  })
  });

Full derby-app.js file after modifications:

var derby = require('derby'),
  app = derby.createApp(module);

app.get('/', function(page, model, params) {
  model.subscribe('message', function() {
    page.render();
  })
});

app.ready(function(model) {
  model.on('set', 'message', function(path, object) {
    console.log('message has been changed: ' + object);
  })
});

Now we’ll see logs both in the terminal window and in the browser Developer Tools console. The end result should look like this in the browser: http://cl.ly/image/0p3z1G3M1E2c, and like this in the terminal: http://cl.ly/image/322I1u002n38.

Hello World App: Browser Console Logs

Hello World App: Terminal Console Logs

For more magic in the persistence area, check out Racer’s db property. With it you can set up an automatic synch between views and database!

Let me know if you’re interested in any specific topic for future blog post and don’t forget to checkout my JavaScript books:

The full code of all the files in this Express.js + Derby Hello World app is available as a gist at https://gist.github.com/azat-co/5530311.

Node.js OAuth1.0 and OAuth2.0: Twitter API v1.1 Examples

Recently we had to work on modification to accommodate Twitter API v1.1. The main difference between Twitter API v1.1 and, soon to be deprecated, Twitter API v1.0 is that most of the REST API endpoints now require user or application context. In other words, each call needs to be performed via OAuth 1.0A or OAuth 2.0 authentication.

Recently we had to work on modification to accommodate Twitter API v1.1. The main difference between Twitter API v1.1 and, soon to be deprecated, Twitter API v1.0 is that most of the REST API endpoints now require user or application context. In other words, each call needs to be performed via OAuth 1.0A or OAuth 2.0 authentication.

Node.js OAuth
Node.js OAuth

At Storify we run everything on Node.js so it was natural that we used oauth module by Ciaran Jessup: NPM and GitHub. It’s mature and supports all the needed functionality but lacks any kind of examples and/or interface documentation.

Here are the examples of calling Twitter API v1.1, and a list of methods. I hope that nobody will have to dig through the oauth module source code anymore!

OAuth 1.0

Let start with a good old OAuth 1.0A. You’ll need four values to make this type of a request to Twitter API v1.1 (or any other service):

  1. Your Twitter application key, a.k.a., consumer key
  2. Your Twitter secret key
  3. User token for your app
  4. User secret for your app

All four of them can be obtained for your own apps at dev.twitter.com. In case that the user is not youself, you’ll need to perform 3-legged OAuth, or Sign in with Twitter, or something else.

Next we create oauth object with parameters, and call get() function to fetch a secured resource. Behind the scene get() function constructs unique values for the request header — Authorization header. The method encrypts URL, timestamp, application and other information in a signature. So the same header won’t work for another URL or after a specific time window.

var OAuth = require('OAuth');
var oauth = new OAuth.OAuth(
      'https://api.twitter.com/oauth/request_token',
      'https://api.twitter.com/oauth/access_token',
      'your Twitter application consumer key',
      'your Twitter application secret',
      '1.0A',
      null,
      'HMAC-SHA1'
    );
    oauth.get(
      'https://api.twitter.com/1.1/trends/place.json?id=23424977',
      'your user token for this app', 
      //you can get it at dev.twitter.com for your own apps
      'your user secret for this app', 
      //you can get it at dev.twitter.com for your own apps
      function (e, data, res){
        if (e) console.error(e);        
        console.log(require('util').inspect(data));
        done();      
      });    
});

OAuth Echo

OAuth Echo is similar to OAuth 1.0. If you’re a Delegator (service to which requests to Service Provider are delegated by Consumer) all you need to do is just pass the value of x-verify-credentials-authorization header to the Service Provider in Authorization header. Twitter has a good graphics on OAuth Echo.

There is OAuthEcho object which inherits must of its methods from normal OAuth class. In case if you want to write Consumer code (or for functional tests, in our case Storify is the delegator) and you need x-verify-credentials-authorization/Authorization header values, there is a authHeader method. If we look at it, we can easily reconstruct the headers with internal methods of oauth module such as _prepareParameters() and _buildAuthorizationHeaders(). Here is a function that will give us required values based on URL (remember that URL is a part of Authorization header):

  function getEchoAuth(url) { 
  //helper to construct echo/oauth headers from URL
    var oauth = new OAuth('https://api.twitter.com/oauth/request_token',
      'https://api.twitter.com/oauth/access_token',
      "AAAAAAAAAAAAAAAAAAAA",
      //test app token
      "BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB", 
      //test app secret
    '1.0A',
    null,
      'HMAC-SHA1');
    var orderedParams = oauth._prepareParameters(
      "1111111111-AAAAAA", //test user token
    "AAAAAAAAAAAAAAAAAAAAAAA", //test user secret
    "GET",
    url
    );
    return oauth._buildAuthorizationHeaders(orderedParams);
  }

From your consumer code you can maker request with superagent or other http client library (e.g., node.js core http module’s http.request):

var request = require('super agent');

request.post('your delegator api url')
  .send({...}) 	
  //your json data
  .set(
    'x-auth-service-provider',
    'https://api.twitter.com/1.1/account/verify_credentials.json')
  .set(
    'x-verify-credentials-authorization',
    getEchoAuth("https://api.twitter.com/1.1/account/verify_credentials.json"))
  .end(function(res){console.log(res.body)});

OAuth2

OAuth 2.0 is a breeze to use comparing to the other authentication methods. Some argue that it’s not as secure, so make sure that you use SSL and HTTPS for all requests.

 var OAuth2 = OAuth.OAuth2;    
 var twitterConsumerKey = 'your key';
 var twitterConsumerSecret = 'your secret';
 var oauth2 = new OAuth2(
   twitterconsumerKey,
   twitterConsumerSecret, 
   'https://api.twitter.com/', 
   null,
   'oauth2/token', 
   null);
 oauth2.getOAuthAccessToken(
   '',
   {'grant_type':'client_credentials'},
   function (e, access_token, refresh_token, results){
     console.log('bearer: ',access_token);
     oauth2.get('protected url', 
       access_token, function(e,data,res) {
         if (e) return callback(e, null);
         if (res.statusCode!=200) 
           return callback(new Error(
             'OAuth2 request failed: '+
             res.statusCode),null);
         try {
           data = JSON.parse(data);        
         }
         catch (e){
           return callback(e, null);
         }
         return callback(e, data);
      });
   });

Please note the JSON.parse() function, oauth module returns string, not a JavaScript object.

Consumers of OAuth2 don’t need to fetch the bearer/access token for every request. It’s okay to do it once and save value in the database. Therefore, we can make requests to protected resources (i.e. Twitter API v.1.1) with only one secret password. For more information check out Twitter application only auth.

Node.js oauth API

Node.js oauth OAuth

oauth.OAuth()

Parameters:

  • requestUrl
  • accessUrl
  • consumerKey
  • consumerSecret
  • version
  • authorize_callback
  • signatureMethod
  • nonceSize
  • customHeaders

Node.js oauth OAuthEcho

oauth.OAuthEcho()

Parameters:

  • realm
  • verify_credentials
  • consumerKey
  • consumerSecret
  • version
  • signatureMethod
  • nonceSize
  • customHeaders

OAuthEcho sharers the same methods as OAuth

Node.js oauth Methods

Secure HTTP request methods for OAuth and OAuthEcho classes:

OAuth.get()

Parameters:

  • url
  • oauth_token
  • oauth_token_secret
  • callback

OAuth.delete()

Parameters:

  • url
  • oauth_token
  • oauth_token_secret
  • callback

OAuth.put()

Parameters:

  • url
  • oauth_token
  • oauth_token_secret
  • post_body
  • post_content_type
  • callback

OAuth.post()

Parameters:

  • url
  • oauth_token
  • oauth_token_secret
  • post_body
  • post_content_type
  • callback

https://github.com/ciaranj/node-oauth/blob/master/lib/oauth.js

Node.js oauth OAuth2

OAuth2 Class

OAuth2()

Parameters:

  • clientId
  • clientSecret
  • baseSite
  • authorizePath
  • accessTokenPath
  • customHeaders

OAuth2.getOAuthAccessToken()

Parameters:

  • code
  • params
  • callback

OAuth2.get()

Parameters:

  • url
  • access_token
  • callback

https://github.com/ciaranj/node-oauth/blob/master/lib/oauth2.js

The authors of node.js oauth did a great job but currently there are 32 open pull requests (mine is one of them) and it makes me sad. Please let them know that we care about improving Node.js ecosystem of modules and developers community!

UPDATE: Pull request was successfully merged!

Useful Twitter API v1.1 Resources

Just because they are vast and not always easy to find.

Tools

Intro to Express.js: Simple REST API app with Monk and MongoDB

After looking at Google Analytics stats I’ve realized that there is a demand for short Node.js tutorial and quick start guides. This is an introduction to probably the most popular (as of April 2013) Node.js framework Express.js.

Why?

After looking at Google Analytics stats I’ve realized that there is a demand for short Node.js tutorial and quick start guides. This is an introduction to probably the most popular (as of April 2013) Node.js framework Express.js.

Express.js — Node.js framework
Express.js — Node.js framework

mongoui

This app is a start of mongoui project. A phpMyAdmin counterpart for MongoDB written in Node.js. The goal is to provide a module with a nice web admin user interface. It will be something like Parse.com, Firebase.com, MongoHQ or MongoLab has but without trying it to any particular service. Why do we have to type db.users.findOne({'_id':ObjectId('...')}) any time we want to look up the user information? The alternative of MongoHub mac app is nice (and free) but clunky to use and not web based.

REST API app with Express.js and Monk

Ruby enthusiasts like to compare Express to Sinatra framework. It’s similarly flexible in the way how developers can build there apps. Application routes are set up in a similar manner, i.e., app.get('/products/:id', showProduct);. Currently Express.js is at version number 3.1. In addition to Express we’ll use Monk module.

We’ll use Node Package Manager which is usually come with a Node.js installation. If you don’t have it already you can get it at npmjs.org.

Create a new folder and NPM configuration file, package.json, in it with the following content:

{
  "name": "mongoui",
  "version": "0.0.1",
  "engines": {
    "node": ">= v0.6"
  },
  "dependencies": {
    "mongodb":"1.2.14",
    "monk": "0.7.1",
    "express": "3.1.0"
  }
}

Now run npm install to download and install modules into node_module folder. If everything went okay you’ll see bunch of folders in node_modules folders. All the code for our application will be in one file, index.js, to keep it simple stupid:

var mongo = require('mongodb');
var express = require('express');
var monk = require('monk');
var db =  monk('localhost:27017/test');
var app = new express();

app.use(express.static(__dirname + '/public'));
app.get('/',function(req,res){
  db.driver.admin.listDatabases(function(e,dbs){
      res.json(dbs);
  });
});
app.get('/collections',function(req,res){
  db.driver.collectionNames(function(e,names){
    res.json(names);
  })
});
app.get('/collections/:name',function(req,res){
  var collection = db.get(req.params.name);
  collection.find({},{limit:20},function(e,docs){
    res.json(docs);
  })
});
app.listen(3000)

Let break down the code piece by piece. Module declaration:

var mongo = require('mongodb');
var express = require('express');
var monk = require('monk');

Database and Express application instantiation:

var db =  monk('localhost:27017/test');
var app = new express();

Tell Express application to load and server static files (if there any) from public folder:

app.use(express.static(__dirname + '/public'));

Home page, a.k.a. root route, set up:

app.get('/',function(req,res){
  db.driver.admin.listDatabases(function(e,dbs){
      res.json(dbs);
  });
});

get() function just takes two parameters: string and function. The string can have slashes and colons, for example product/:id. The function must have two parapemets request and response. Request has all the information like query string parameters, session, headers and response is an object to with we output the results. In this case we do it by calling res.json() function. db.driver.admin.listDatabases() as you might guess give us a list of databases in async manner.

Two other routes are set up in a similar manner with get() function:

app.get('/collections',function(req,res){
  db.driver.collectionNames(function(e,names){
    res.json(names);
  })
});
app.get('/collections/:name',function(req,res){
  var collection = db.get(req.params.name);
  collection.find({},{limit:20},function(e,docs){
    res.json(docs);
  })
});

Express conveniently supports other HTTP verbs like post and update. In the case of setting up a post route we write this:

app.post('product/:id',function(req,res) {...});

Express also has support for middeware. Middleware is just a request function handler with three parameters: request, response, and next. For example:

app.post('product/:id', authenticateUser, validateProduct, addProduct);

function authenticateUser(req,res, next) {
  //check req.session for authentication
  next();
}

function validateProduct (req, res, next) {
   //validate submitted data
   next();
}

function addProduct (req, res) {
  //save data to database
}

validateProduct and authenticateProduct are middleware. They are usually put into separate file (or files) in a big projects.

Another way to set up middle ware in Express application is to use use() function. For example earlier we did this for static assets:

app.use(express.static(__dirname + '/public'));

We can also do it for error handlers:

app.use(errorHandler);

Assuming you have mongoDB installed this app will connect to it (localhost:27017) and display collection name and items in collections. To start mongo server:

$ mongod

to run app (keep the mongod terminal window open):

$ node .

or

$ node index.js

To see the app working, open http://localhost:3000 in Chrome with JSONViewer extension (to render JSON nicely).

Tom Hanks' The Polar Express
Tom Hanks’ The Polar Express

Rapid Prototyping with JS Update v0.4

Why Backbone.js? Because a lot of people expressed desire to use it but being a framework Backbone has a learning curve. Not a steep one like Rails but still it takes time to master and learn Backbone. The new chapter “Intro to Backbone.js” will show readers how to:

  • Create Backbone architecture from scratch
  • Construct proper architecture with Routes
  • Use Collections
  • Apply Templates in Subviews
  • Split code into multiple JS files
  • Organize code into modules with AMD and Require.js
  • Optimize code for production with r.js library

Link to the new copy of Rapid Prototyping with JS at https://leanpub.com/rapid-prototyping-with-js.

Rapid Prototyping with JS: Agile JavaScript Developement
Rapid Prototyping with JS: Agile JavaScript Developement

TL;DR

Here is a list of the update for Rapid Prototyping with JS v0.4:

  • Brand new chapter: Intro to Backbone.js
  • Re-structured table of contents (chapters and parts)
  • Extended list of resources for further reading
  • Fixed code formatting and highlighting
  • Fixed grammar, style and typos mistakes
  • Brand new book cover optimized for ebook format
  • Reduced size of ebook (faster download)
  • Inspirational quotes in each chapter
  • Updated “Basics” chapter
  • New headline “Agile JavaScript Development”
  • Extended list of ways to reach us (Twitter, Facebook, Storify, etc.)

LeanPub Updates

LeanPub changed their purchasing. Now it’s even better for readers. Anybody can “return” the book they didn’t like within 45 days and get a full refund. Word return is in double quotes because all content is Digital Right Management (DRM) free. Does it mean that somebody can buy and download a book, get their money back, but keep the copy? Yes, but they will get bad karma for that! And not like a bad karma on Hacker News but a real bad karma. Everybody else deservers authors and publishers trust and respect.

Backbone.js

Why Backbone.js? Because a lot of people expressed desire to use it but being a framework Backbone has a learning curve. Not a steep one like Rails but still it takes time to master and learn Backbone. The new chapter “Intro to Backbone.js” will show readers how to:

  • Create Backbone architecture from scratch
  • Construct proper architecture with Routes
  • Use Collections
  • Apply Templates in Subviews
  • Split code into multiple JS files
  • Organize code into modules with AMD and Require.js
  • Optimize code for production with r.js library

Write and follow @RPJSbook.

Test-Driven Development in Node.js With Mocha

Don’t waste time writing tests for throwaway scripts, but please adapt the habit of Test-Driven Development for the main code base. With a little time spent in the beginning, you and your team will save time later and have confidence when rolling out new releases. Test Driven Development is a really really really good thing.

Who needs Test-Driven Development?

Imagine that you need to implement a complex feature on top of an existing interface, e.g., a ‘like’ button on a comment. Without tests you’ll have to manually create a user, log in, create a post, create a different user, log in with a different user and like the post. Tiresome? What if you’ll need to do it 10 or 20 times to find and fix some nasty bug? What if your feature breaks existing functionality, but you notice it 6 months after the release because there was no test!

Mocha: simple, flexible, fun
Mocha: simple, flexible, fun

Don’t waste time writing tests for throwaway scripts, but please adapt the habit of Test-Driven Development for the main code base. With a little time spent in the beginning, you and your team will save time later and have confidence when rolling out new releases. Test Driven Development is a really really really good thing.

Quick Start Guide

Follow this quick guide to set up your Test-Driven Development process in Node.js with Mocha.

Install Mocha globally by executing this command:

$ sudo npm install -g mocha

We’ll also use two libraries, Superagent and expect.js by LeanBoost. To install them fire up npm commands in your project folder like this:

$ npm install superagent
$ npm install expect.js   

Open a new file with .js extension and type:

var request = require('superagent');
var expect = require('expect.js');

So far we’ve included two libraries. The structure of the test suite going to look like this:

describe('Suite one', function(){
  it(function(done){
  ...
  });
  it(function(done){
  ...
  });
});
describe('Suite two', function(){
  it(function(done){
  ...
  });
});

Inside of this closure we can write request to our server which should be running at localhost:8080:

...
it (function(done){
  request.post('localhost:8080').end(function(res){
    //TODO check that response is okay
  });
});
...

Expect will give us handy functions to check any condition we can think of:

...
expect(res).to.exist;
expect(res.status).to.equal(200);
expect(res.body).to.contain('world');
...

Lastly, we need to add done() call to notify Mocha that asynchronous test has finished its work. And the full code of our first test looks like this:

var request = require('superagent');
var expect = require('expect.js');
  
describe('Suite one', function(){
 it (function(done){
   request.post('localhost:8080').end(function(res){
    expect(res).to.exist;
    expect(res.status).to.equal(200);
    expect(res.body).to.contain('world');
    done();
   });
  });
});

If we want to get fancy, we can add before and beforeEach hooks which will, according to their names, execute once before the test (or suite) or each time before the test (or suite):

before(function(){
  //TODO seed the database
});
describe('suite one ',function(){
  beforeEach(function(){
    //todo log in test user
  });
  it('test one', function(done){
  ...
  });
});

Note that before and beforeEach can be placed inside or outside of describe construction.

To run our test simply execute:

$ mocha test.js

To use different report type:

$ mocha test.js -R list
$ mocha test.js -R spec