Tag: node.js

Express.js Tutorial: Instagram Gallery Example App with Storify API

Storify runs on Node.js and Express.js, therefore why not use these technologies to write an application that demonstrates how to build apps that rely on third-party APIs and HTTP requests to them?

Note: This text is a part of Express.js Guide: The Comprehensive Book on Express.js.

An example of an Express.js app using Storify API as a data source is a continuation of introduction to Express.js tutorials.

Continue reading “Express.js Tutorial: Instagram Gallery Example App with Storify API”

LeanPub Compilation and Sanitizing Tool Written in Node.js

LeanPub Tool — Node.js script for sanitizing and compiling a book’s manuscript. LeanPub uses multi-file book format and a special markdown markup (that confuses Marked app). I found that to convert to MS Word (for editing and other needs) this workflow is the best: 1. run leanpub-tool.js with node.js (node leanpub-tool.js) 2. open file in Marked app and convert to HTML 3. open HTML in MS Word and enjoy.

LeanPub is an awesome publishing platform, but often we need to communicate with other  team members involved in the book production, e.g., content and copy editors. In such times, I’ve learned that Guy Kawasaki was right — when we suggested in his book APE: Author, Publisher, Entrepreneur — that author must use MS Word!

The problem is that LeanPub uses multi-file book format and a special markdown markup (that confuses Marked app). I was wrapping up a new revision of the Rapid Prototyping with JS book and previous experience of handing a txt file to an editor was poor. Luckily, Node.js came to help! Through some trail and error attempts, I found this workflow to be the best for converting LeanPub manuscript to a one MS Word file (for editing and other needs):

  1. Run leanpub-tool.js with Node.js (node leanpub-tool.js)
  2. Open file in Marked app and convert to HTML
  3. Open HTML in MS Word and enjoy.

The full code of leanpub-tool.js which is also available at https://gist.github.com/azat-co/5674684:


var fs = require('fs');
//change these to YOUR filenames
//probably we can read it from Book.txt but I was too lazy to implement it :-) and somebody might want to compile only sertain parts of the book
var book =[
"frontmatter.txt",
"mainmatter.txt",
"part1.txt",
"chapter1.txt",
"part2.txt",
"chapter2.txt",
"backmatter.txt",
"acknowledgment.txt"
];
 
var sanitizeText = [
  '{frontmatter}',
  '{backmatter}',
  '{mainmatter}',
  "I>## Note",
  "T>## Tip",
  "W>## Warning",
  '{lang="javascript"}',
  '{:lang="javascript"}',
  '{lang="css"}',
  '{:lang="css"}',
  '{lang="json"}',
  '{lang="ruby"}',
  '{lang="php"}',
  '{lang="text"}',
  '{lang="bash"}',
  '{lang="html"}',
  "I>",
  "T>",
  "W>"
]
 
 
var str = '';
//read files
book.forEach(function(chapter){
  str +=  fs.readFileSync(chapter,'utf8');
})
//sanitize LeanPub specific Markdown tags
sanitizeText.forEach(function(text){
//  console.log(text)
  //this loops through while there is not matches
  while (new RegExp (text).test(str)) {
    str = str.replace(text,'','gm')
  };
})
 
//write output to a file
fs.writeFileSync('leanpub-tool.txt',str);

Rapid Prototyping with JS: Agile JavaScript Development is done with v0.5 (1.0RC)!

This is a release candidate for 1.0 version which is going to be available to public in print. Page count has increase from 151 to 211 pages (PDF), which include the following updates:

More digestible (smaller) code examples with better comments
Express.js middleware section with an example
Express.js + MongoDB REST API server section with an example
Derby section with an example
Grammar and typo fixes
Illustrations
Summaries in the beginning of each chapter
Code examples formatting fixes

Rapid Prototyping with JS: Agile JavaScript Development
Rapid Prototyping with JS: Agile JavaScript Development

This is a release candidate for 1.0 version which is going to be available to public in print. Page count has increase from 151 to 211 pages (PDF), which include the following updates:

  • More digestible (smaller) code examples with better comments
  • Express.js middleware section with an example
  • Express.js + MongoDB REST API server section with an example
  • Derby section with an example
  • Grammar and typo fixes
  • Illustrations
  • Summaries in the beginning of each chapter
  • Code examples formatting fixes

Download a free chapter Intro to Backbone.js or full PDF, ePub or Mobi copy at LeanPub.

Rapid Prototyping with JS touched topics such as:

  • jQuery
  • AJAX
  • CSS and LESS
  • JSON and BSON
  • Twitter Bootstrap
  • Node.js
  • MongoDB
  • Parse.com
  • Agile methodologies
  • Git
  • Heroku, MongoHQ and Windows Azure
  • REST API
  • Backbone.js
  • AMD and Require.js
  • Express.js
  • Monk
  • Derby

If you need in-depth knowledge or references, they are usually one click or one Google search away.

Practical aspect included building multiple versions of the Message Board app:

  • jQuery + Parse.com JS REST API
  • Backbone and Parse.com JS SDK
  • Backbone and Node.js
  • Backbone and Node.js + MongoDB

The Message Board application has all the foundation of a typical web/mobile application: fetching data, displaying it, submitting new data. Other examples include:

  • jQuery + Twitter RESP API “Tweet Analyzer”
  • Parse.com “Save John”
  • Node.js “Hello World”
  • MongoDB “Print Collections”
  • Derby + Express “Hello World”
  • Backbone.js “Hello World”
  • Backbone.js “Apple Database”
  • Monk + Expres.js “REST API Server”

Oh My JS: The Best JavaScript Articles (https://leanpub.com/ohmyjs/) is in the works!

Nested Objects in Mongoose

There is a certain magic in ORMs like Mongoose. I learned it the hard way (as usual!), when I was trying to iterate over nested object’s properties…

There is a certain magic in ORMs like Mongoose. I learned it the hard way (as usual!), when I was trying to iterate over nested object’s properties. For example, here is a schema with a nested object features defines like this:

var User = module.exports = new Schema({
  features: { 
    realtime_updates: {
      type: Boolean
    },
    storylock: {
      type: Boolean
    },
    custom_embed_style: {
      type: Boolean
    },
    private_stories: {
      type: Boolean
    },
    headerless_embed:{
      type: Boolean
    }
};

Let’s say I want to overwrite object features_enabled with these properties:

if (this.features) { 
  for (var k in this.features) {
    features_enabled[k] = this.features[k];
  }
}
console.log(features_enabled)
return features_enabled;

Not so fast, I was getting a lot of system properties specific to Mongoose. Instead we need to use toObject(), e.g.:

if (this.features.toObject()) { 
  for (var k in this.features.toObject()) {
    console.log('!',k)
    features_enabled[k] = this.features.toObject()[k];
  }
}

Remember rule number one, computer is always right. If we think that it’s wrong — look up the rule number one. :-)

Intro to Express.js: Parameters, Error Handling and Other Middleware

Express.js is a node.js framework that among other things provides a way to organize routes. Each route is defined via a method call on an application object with a URL patter as a first parameter (RegExp is also supported). The best practice is to keep router lean and thin by moving all the logic into corresponding external modules/files. This way important server configuration parameters will be neatly in one place, right there when you need them! :-)

Note: This text is a part of Express.js Guide: The Comprehensive Book on Express.js.

Express.js is one of the most popular and mature Node.js frameworks. You can read more about in Intro to Express.js series on webapplog.com:

To learn how to create an application from scratch please refer to the earlier post.

Request Handlers

Express.js is a node.js framework that among other things provides a way to organize routes. Each route is defined via a method call on an application object with a URL patter as a first parameter (RegExp is also supported), for example:

app.get('api/v1/stories/', function(res, req){
  ...
})

or, for a POST method:

app.post('/api/v1/stories'function(req,res){
  ...
})

It’s needless to say that DELETE and PUT methods are supported as well.
The callbacks that we pass to get() or post() methods are called request handlers, because they take requests (req), process them and write to response (res) objects. For example:

app.get('/about', function(req,res){
  res.send('About Us: ...');
});

We can have multiple request handlers, hence the name middleware. They accept a third parameter next calling which (next()) will switch the execution flow to the next handler: 

app.get('/api/v1/stories/:id', function(req,res, next) {
  //do authorization
  //if not authorized or there is an error 
  // return next(error);
  //if authorized and no errors
  return next();
}), function(req,res, next) {
  //extract id and fetch the object from the database
  //assuming no errors, save story in the request object
  req.story = story;
  return next();
}), function(req,res) {
  //output the result of the database search
  res.send(res.story);
});

ID of a story in URL patter is a query string parameter which we need for finding a matching items in the database.

Parameters Middleware

Parameters are values passed in a query string of a URL of the request. If we didn’t have Express.js or similar library, and had to use just the core Node.js modules, we’d had to extract parameters from HTTP.request object via some require('querystring').parse(url) or require('url').parse(url, true) functions trickery.

Thanks to Connect framework and people at VisionMedia, Express.js already has support for parameters, error handling and many other important features in the form of middlewares. This is how we can plug param middleware in our app:

app.param('id', function(req,res, next, id){
  //do something with id
  //store id or other info in req object
  //call next when done
  next();
});

app.get('/api/v1/stories/:id',function(req,res){
  //param middleware will be execute before and
  //we expect req object already have needed info
  //output something
  res.send(data);
});

For example: 

app.param('id', function(req,res, next, id){
  req.db.get('stories').findOne({_id:id}, function (e, story){
    if (e) return next(e);
    if (!story) return next(new Error('Nothing is found'));
    req.story = story;
    next();
  });
});

app.get('/api/v1/stories/:id',function(req,res){
  res.send(req.story);
});

Or we can use multiple request handlers but the concept remains the same: we can expect to have req.story object or an error thrown prior to the execution of this code so we abstract common code/logic of getting parameters and their respective objects:

app.get('/api/v1/stories/:id', function(req,res, next) {
  //do authorization
  }),
  //we have an object in req.story so no work is needed here
  function(req,res) {
  //output the result of the database search
  res.send(story);
});

Authorization and input sanitation are also good candidates for residing in the middlewares.

Function param() is especially cool because we can combine different keys, e.g.:

app.get('/api/v1/stories/:storyId/elements/:elementId',function(req,res){
  res.send(req.element);
});

Error Handling

Error handling is typically used across the whole application, therefore it’s best to implement it as a middleware. It has the same parameters plus one more, error:

app.use(function(err, req, res, next) {
  //do logging and user-friendly error message display
  res.send(500);
})

In fact, the response can be anything:

JSON string

app.use(function(err, req, res, next) {
  //do logging and user-friendly error message display
  res.send(500, {status:500, message: 'internal error', type:'internal'});
})

Text message

app.use(function(err, req, res, next) {
  //do logging and user-friendly error message display
  res.send(500, 'internal server error');
})

Error page

app.use(function(err, req, res, next) {
  //do logging and user-friendly error message display
  //assuming that template engine is plugged in
  res.render('500');
})

Redirect to error page

app.use(function(err, req, res, next) {
  //do logging and user-friendly error message display
  res.redirect('/public/500.html');
})

Error HTTP response status (401, 400, 500, etc.)

app.use(function(err, req, res, next) {
  //do logging and user-friendly error message display
  res.end(500);
})

By the way, logging is also should be abstracted in a middleware!

To trigger an error from within your request handlers and middleware you can just call:

next(error);

or 

next(new Error('Something went wrong :-(');

You can also have multiple error handlers, and use named instead of anonymous functions as its shows in Express.js Error handling guide.

Other Middleware

In addition to extracting parameters, it can be used for many things, like authorization, error handling, sessions, output, and others.

res.json() is one of them. It conveniently outputs JavaScript/Node.js object as a JSON. For example:

app.get('/api/v1/stories/:id', function(req,res){
  res.json(req.story);
});

is equivalent to (if req.story is an Array and Object):

app.get('/api/v1/stories/:id', function(req,res){
  res.send(req.story);
});

or

app.get('api/v1/stories/:id',function(req,res){
  res.set({
    'Content-Type': 'application/json'
  });
  res.send(req.story);
});

Abstraction

Middleware is flexible. You can use anonymous or named functions, but the best thing is to abstract request handlers into external modules based on the functionality:

var stories = require.('./routes/stories');
var elements = require.('./routes/elements');
var users = require.('./routes/users');
...
app.get('/stories/,stories.find);
app.get('/stories/:storyId/elements/:elementId', elements.find);
app.put('/users/:userId',users.update);

routes/stories.js:

module.exports.find = function(req,res, next) {
};

routes/elements.js:

module.exports.find = function(req,res,next){
};

routes/users.js:

module.exports.update = function(req,res,next){
};

You can use some functional programming tricks, like this:

function requiredParamHandler(param){
  //do something with a param, e.g., check that it's present in a query string
  return function (req,res, next) {
    //use param, e.g., if token is valid proceed with next();
    next();
  });
}

app.get('/api/v1/stories/:id', requiredParamHandler('token'), story.show);

var story  = {
  show: function (req, res, next) {
    //do some logic, e.g., restrict fields to output
    return res.send();
  }
}

As you can see middleware is a powerful concept for keeping code organized. The best practice is to keep router lean and thin by moving all the logic into corresponding external modules/files. This way important server configuration parameters will be neatly in one place, right there when you need them! :-)

Node.js MVC: Express.js + Derby.js Hello World Tutorial

Express.js is a popular node frameworks which uses middleware concept to enhance functionality of applications. Derby is a new sophisticated Model View Controller (MVC) framework which is designed to be used with Express as it’s middleware. Derby also comes with the support of Racer, data synchronization engine, and Handlebars-like template engine among many other features.

DerbyJS — Node.js MVC Framework

Express.js is a popular node frameworks which uses middleware concept to enhance functionality of applications. Derby is a new sophisticated Model View Controller (MVC) framework which is designed to be used with Express as it’s middleware. Derby also comes with the support of Racer, data synchronization engine, and Handlebars-like template engine among many other features.

Derby.js Installation

Let’s set up a basic Derby application architecture without the use of scaffolding. Usually project generators are confusing when people just start to learn a new comprehensive framework. This is a bare minimum “Hello World” application tutorial that still illustrates Derby skeleton and demonstrates live-templates with websockets.

Of course we’ll need Node.js and NPM which can be obtained at nodejs.org. To install derby globally run:

$ npm install -g derby

To check the installation:

$ derby -V

My version as of April 2013 is 0.3.15. We should be good to go to creating our first app!

File Structure in a Derby.js App

This is the project folder structure:

project/
  -package.json
  -index.js
  -derby-app.js
  views/
    derby-app.html
  styles/
    derby-app.less

Dependencies for The Derby.js Project

Let’s include dependencies and other basic information in package.json file:

 {
  "name": "DerbyTutorial",
  "description": "",
  "version": "0.0.0",
  "main": "./server.js",
  "dependencies": {
    "derby": "*",
    "express": "3.x"
  },
  "private": true
}

Now we can run npm install which will download our dependencies into node_modules folder.

Views in Derby.js

Views must be in views folder and they must be either in index.html under a folder which has the same name as your derby app JavaScript file, i.e., views/derby-app/index.html, or be inside of a file which has the same name as your derby app JS file, i.e., derby-app.html.

In this example “Hello World” app we’ll use <Body:> template and {message} variable. Derby uses mustach-handlebars-like syntax for reactive binding. index.html looks like this:

<Body:>
  <input value="{message}"><h1>{message}</h1>

Same thing with Stylus/LESS files, in our example index.css has just one line:

h1 {
  color: blue;
}

To find out more about those wonderful CSS preprocessors check out documentation at Stylus and LESS.

Building The Main Derby.js Server

index.js is our main server file, and we begin it with an inclusion of dependencies with require() function:

var http = require('http'),
  express = require('express'),
  derby = require('derby'),
  derbyApp = require('./derby-app');

Last line is our derby application file derby-app.js.

Now we’re creating Express.js application (v3.x has significant differences between 2.x) and an HTTP server:

var expressApp = new express(),
  server = http.createServer(expressApp);

Derby uses Racer data synchronization library which we create like this:

var store = derby.createStore({
  listen: server
});

To fetch some data from back-end to the front-end we instantiate model object:

var model = store.createModel();

Most importantly we need to pass model and routes as middlewares to Express.js app. We need to expose public folder for socket.io to work properly.

expressApp.
  use(store.modelMiddleware()).
  use(express.static(__dirname + '/public')).
  use(derbyApp.router()).
  use(expressApp.router);

Now we can start the server on port 3001 (or any other):

server.listen(3001, function(){
  model.set('message', 'Hello World!');
});

Full code of index.js file:

var http = require('http'),
  express = require('express'),
  derby = require('derby'),
  derbyApp = require('./derby-app');

var expressApp = new express(),
  server = http.createServer(expressApp);

var store = derby.createStore({
  listen: server
});

var model = store.createModel();

expressApp.
  use(store.modelMiddleware()).
  use(express.static(__dirname + '/public')).
  use(derbyApp.router()).
  use(expressApp.router);

server.listen(3001, function(){
  model.set('message', 'Hello World!');
});

Derby.js Application

Finally, Derby app file which contains code for both a front-end and a back-end. Front-end only code is inside of app.ready() callback. To start, let’s require and create an app. Derby uses unusual construction (not the same familiar good old module.exports = app):

var derby = require('derby'),
  app = derby.createApp(module);

To make socket.io magic work we need to subscribe model attribute to its visual representation, in other words bind data and view. We can do it in the root route, and this is how we define it (patter is /, a.k.a. root):

app.get('/', function(page, model, params) {
  model.subscribe('message', function() {
    page.render();  
  })  
});

Full code of derby-app.js file:

var derby = require('derby'),
  app = derby.createApp(module);

app.get('/', function(page, model, params) {
  model.subscribe('message', function() {
    page.render();  
  })  
});

Launching Hello World App

Now everything should be ready to boot our server. Execute node . or node index.js and open a browser at http://localhost:3001. You should be able to see something like this: http://cl.ly/image/3J1O0I3n1T46.

Derby + Express.js Hello World App

Passing Values to Back-End in Derby.js

Of course static data is not much, so we can slightly modify our app to make back-end and front-end pieces talks with each other.

In the server file index.js add store.afterDb to listen to set events on message attribute:

server.listen(3001, function(){
  model.set('message', 'Hello World!');
  store.afterDb('set','message', function(txn, doc, prevDoc, done){
    console.log(txn)
    done();
  }) 
});

Full code of index.js after modifications:

var http = require('http'),
  express = require('express'),
  derby = require('derby'),
  derbyApp = require('./derby-app');

var expressApp = new express(),
  server = http.createServer(expressApp);

var store = derby.createStore({
  listen: server
});

var model = store.createModel();

expressApp.
  use(store.modelMiddleware()).
  use(express.static(__dirname + '/public')).
  use(derbyApp.router()).
  use(expressApp.router);

server.listen(3001, function(){
  model.set('message', 'Hello World!');
  store.afterDb('set','message', function(txn, doc, prevDoc, done){
    console.log(txn)
    done();
  })   
});

In Derby application file derby-app.js add model.on() to app.ready():

  app.ready(function(model){
	    model.on('set', 'message',function(path, object){
	    console.log('message has been changed: '+ object);
	  })
  });

Full derby-app.js file after modifications:

var derby = require('derby'),
  app = derby.createApp(module);

app.get('/', function(page, model, params) {
  model.subscribe('message', function() {
    page.render();
  })
});

app.ready(function(model) {
  model.on('set', 'message', function(path, object) {
    console.log('message has been changed: ' + object);
  })
});

Now we’ll see logs both in the terminal window and in the browser Developer Tools console. The end result should look like this in the browser: http://cl.ly/image/0p3z1G3M1E2c, and like this in the terminal: http://cl.ly/image/322I1u002n38.

Hello World App: Browser Console Logs

Hello World App: Terminal Console Logs

For more magic in the persistence area, check out Racer’s db property. With it you can set up an automatic synch between views and database!

Let me know if you’re interested in any specific topic for future blog post and don’t forget to checkout my JavaScript books:

The full code of all the files in this Express.js + Derby Hello World app is available as a gist at https://gist.github.com/azat-co/5530311.

Node.js OAuth1.0 and OAuth2.0: Twitter API v1.1 Examples

Recently we had to work on modification to accommodate Twitter API v1.1. The main difference between Twitter API v1.1 and, soon to be deprecated, Twitter API v1.0 is that most of the REST API endpoints now require user or application context. In other words, each call needs to be performed via OAuth 1.0A or OAuth 2.0 authentication.

Recently we had to work on modification to accommodate Twitter API v1.1. The main difference between Twitter API v1.1 and, soon to be deprecated, Twitter API v1.0 is that most of the REST API endpoints now require user or application context. In other words, each call needs to be performed via OAuth 1.0A or OAuth 2.0 authentication.

Node.js OAuth
Node.js OAuth

At Storify we run everything on Node.js so it was natural that we used oauth module by Ciaran Jessup: NPM and GitHub. It’s mature and supports all the needed functionality but lacks any kind of examples and/or interface documentation.

Here are the examples of calling Twitter API v1.1, and a list of methods. I hope that nobody will have to dig through the oauth module source code anymore!

OAuth 1.0

Let start with a good old OAuth 1.0A. You’ll need four values to make this type of a request to Twitter API v1.1 (or any other service):

  1. Your Twitter application key, a.k.a., consumer key
  2. Your Twitter secret key
  3. User token for your app
  4. User secret for your app

All four of them can be obtained for your own apps at dev.twitter.com. In case that the user is not youself, you’ll need to perform 3-legged OAuth, or Sign in with Twitter, or something else.

Next we create oauth object with parameters, and call get() function to fetch a secured resource. Behind the scene get() function constructs unique values for the request header — Authorization header. The method encrypts URL, timestamp, application and other information in a signature. So the same header won’t work for another URL or after a specific time window.

var OAuth = require('OAuth');
var oauth = new OAuth.OAuth(
      'https://api.twitter.com/oauth/request_token',
      'https://api.twitter.com/oauth/access_token',
      'your Twitter application consumer key',
      'your Twitter application secret',
      '1.0A',
      null,
      'HMAC-SHA1'
    );
    oauth.get(
      'https://api.twitter.com/1.1/trends/place.json?id=23424977',
      'your user token for this app', 
      //you can get it at dev.twitter.com for your own apps
      'your user secret for this app', 
      //you can get it at dev.twitter.com for your own apps
      function (e, data, res){
        if (e) console.error(e);        
        console.log(require('util').inspect(data));
        done();      
      });    
});

OAuth Echo

OAuth Echo is similar to OAuth 1.0. If you’re a Delegator (service to which requests to Service Provider are delegated by Consumer) all you need to do is just pass the value of x-verify-credentials-authorization header to the Service Provider in Authorization header. Twitter has a good graphics on OAuth Echo.

There is OAuthEcho object which inherits must of its methods from normal OAuth class. In case if you want to write Consumer code (or for functional tests, in our case Storify is the delegator) and you need x-verify-credentials-authorization/Authorization header values, there is a authHeader method. If we look at it, we can easily reconstruct the headers with internal methods of oauth module such as _prepareParameters() and _buildAuthorizationHeaders(). Here is a function that will give us required values based on URL (remember that URL is a part of Authorization header):

  function getEchoAuth(url) { 
  //helper to construct echo/oauth headers from URL
    var oauth = new OAuth('https://api.twitter.com/oauth/request_token',
      'https://api.twitter.com/oauth/access_token',
      "AAAAAAAAAAAAAAAAAAAA",
      //test app token
      "BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB", 
      //test app secret
    '1.0A',
    null,
      'HMAC-SHA1');
    var orderedParams = oauth._prepareParameters(
      "1111111111-AAAAAA", //test user token
    "AAAAAAAAAAAAAAAAAAAAAAA", //test user secret
    "GET",
    url
    );
    return oauth._buildAuthorizationHeaders(orderedParams);
  }

From your consumer code you can maker request with superagent or other http client library (e.g., node.js core http module’s http.request):

var request = require('super agent');

request.post('your delegator api url')
  .send({...}) 	
  //your json data
  .set(
    'x-auth-service-provider',
    'https://api.twitter.com/1.1/account/verify_credentials.json')
  .set(
    'x-verify-credentials-authorization',
    getEchoAuth("https://api.twitter.com/1.1/account/verify_credentials.json"))
  .end(function(res){console.log(res.body)});

OAuth2

OAuth 2.0 is a breeze to use comparing to the other authentication methods. Some argue that it’s not as secure, so make sure that you use SSL and HTTPS for all requests.

 var OAuth2 = OAuth.OAuth2;    
 var twitterConsumerKey = 'your key';
 var twitterConsumerSecret = 'your secret';
 var oauth2 = new OAuth2(
   twitterconsumerKey,
   twitterConsumerSecret, 
   'https://api.twitter.com/', 
   null,
   'oauth2/token', 
   null);
 oauth2.getOAuthAccessToken(
   '',
   {'grant_type':'client_credentials'},
   function (e, access_token, refresh_token, results){
     console.log('bearer: ',access_token);
     oauth2.get('protected url', 
       access_token, function(e,data,res) {
         if (e) return callback(e, null);
         if (res.statusCode!=200) 
           return callback(new Error(
             'OAuth2 request failed: '+
             res.statusCode),null);
         try {
           data = JSON.parse(data);        
         }
         catch (e){
           return callback(e, null);
         }
         return callback(e, data);
      });
   });

Please note the JSON.parse() function, oauth module returns string, not a JavaScript object.

Consumers of OAuth2 don’t need to fetch the bearer/access token for every request. It’s okay to do it once and save value in the database. Therefore, we can make requests to protected resources (i.e. Twitter API v.1.1) with only one secret password. For more information check out Twitter application only auth.

Node.js oauth API

Node.js oauth OAuth

oauth.OAuth()

Parameters:

  • requestUrl
  • accessUrl
  • consumerKey
  • consumerSecret
  • version
  • authorize_callback
  • signatureMethod
  • nonceSize
  • customHeaders

Node.js oauth OAuthEcho

oauth.OAuthEcho()

Parameters:

  • realm
  • verify_credentials
  • consumerKey
  • consumerSecret
  • version
  • signatureMethod
  • nonceSize
  • customHeaders

OAuthEcho sharers the same methods as OAuth

Node.js oauth Methods

Secure HTTP request methods for OAuth and OAuthEcho classes:

OAuth.get()

Parameters:

  • url
  • oauth_token
  • oauth_token_secret
  • callback

OAuth.delete()

Parameters:

  • url
  • oauth_token
  • oauth_token_secret
  • callback

OAuth.put()

Parameters:

  • url
  • oauth_token
  • oauth_token_secret
  • post_body
  • post_content_type
  • callback

OAuth.post()

Parameters:

  • url
  • oauth_token
  • oauth_token_secret
  • post_body
  • post_content_type
  • callback

https://github.com/ciaranj/node-oauth/blob/master/lib/oauth.js

Node.js oauth OAuth2

OAuth2 Class

OAuth2()

Parameters:

  • clientId
  • clientSecret
  • baseSite
  • authorizePath
  • accessTokenPath
  • customHeaders

OAuth2.getOAuthAccessToken()

Parameters:

  • code
  • params
  • callback

OAuth2.get()

Parameters:

  • url
  • access_token
  • callback

https://github.com/ciaranj/node-oauth/blob/master/lib/oauth2.js

The authors of node.js oauth did a great job but currently there are 32 open pull requests (mine is one of them) and it makes me sad. Please let them know that we care about improving Node.js ecosystem of modules and developers community!

UPDATE: Pull request was successfully merged!

Useful Twitter API v1.1 Resources

Just because they are vast and not always easy to find.

Tools

Intro to Express.js: Simple REST API app with Monk and MongoDB

After looking at Google Analytics stats I’ve realized that there is a demand for short Node.js tutorial and quick start guides. This is an introduction to probably the most popular (as of April 2013) Node.js framework Express.js.

Why?

After looking at Google Analytics stats I’ve realized that there is a demand for short Node.js tutorial and quick start guides. This is an introduction to probably the most popular (as of April 2013) Node.js framework Express.js.

Express.js — Node.js framework
Express.js — Node.js framework

mongoui

This app is a start of mongoui project. A phpMyAdmin counterpart for MongoDB written in Node.js. The goal is to provide a module with a nice web admin user interface. It will be something like Parse.com, Firebase.com, MongoHQ or MongoLab has but without trying it to any particular service. Why do we have to type db.users.findOne({'_id':ObjectId('...')}) any time we want to look up the user information? The alternative of MongoHub mac app is nice (and free) but clunky to use and not web based.

REST API app with Express.js and Monk

Ruby enthusiasts like to compare Express to Sinatra framework. It’s similarly flexible in the way how developers can build there apps. Application routes are set up in a similar manner, i.e., app.get('/products/:id', showProduct);. Currently Express.js is at version number 3.1. In addition to Express we’ll use Monk module.

We’ll use Node Package Manager which is usually come with a Node.js installation. If you don’t have it already you can get it at npmjs.org.

Create a new folder and NPM configuration file, package.json, in it with the following content:

{
  "name": "mongoui",
  "version": "0.0.1",
  "engines": {
    "node": ">= v0.6"
  },
  "dependencies": {
    "mongodb":"1.2.14",
    "monk": "0.7.1",
    "express": "3.1.0"
  }
}

Now run npm install to download and install modules into node_module folder. If everything went okay you’ll see bunch of folders in node_modules folders. All the code for our application will be in one file, index.js, to keep it simple stupid: 

var mongo = require('mongodb');
var express = require('express');
var monk = require('monk');
var db =  monk('localhost:27017/test');
var app = new express();

app.use(express.static(__dirname + '/public'));
app.get('/',function(req,res){
  db.driver.admin.listDatabases(function(e,dbs){
      res.json(dbs);
  });
});
app.get('/collections',function(req,res){
  db.driver.collectionNames(function(e,names){
    res.json(names);
  })
});
app.get('/collections/:name',function(req,res){
  var collection = db.get(req.params.name);
  collection.find({},{limit:20},function(e,docs){
    res.json(docs);
  })
});
app.listen(3000)

Let break down the code piece by piece. Module declaration:

var mongo = require('mongodb');
var express = require('express');
var monk = require('monk');

Database and Express application instantiation:

var db =  monk('localhost:27017/test');
var app = new express();

Tell Express application to load and server static files (if there any) from public folder:

app.use(express.static(__dirname + '/public'));

Home page, a.k.a. root route, set up:

app.get('/',function(req,res){
  db.driver.admin.listDatabases(function(e,dbs){
      res.json(dbs);
  });
});

get() function just takes two parameters: string and function. The string can have slashes and colons, for example product/:id. The function must have two parapemets request and response. Request has all the information like query string parameters, session, headers and response is an object to with we output the results. In this case we do it by calling res.json() function. db.driver.admin.listDatabases() as you might guess give us a list of databases in async manner.

Two other routes are set up in a similar manner with get() function:

app.get('/collections',function(req,res){
  db.driver.collectionNames(function(e,names){
    res.json(names);
  })
});
app.get('/collections/:name',function(req,res){
  var collection = db.get(req.params.name);
  collection.find({},{limit:20},function(e,docs){
    res.json(docs);
  })
});

Express conveniently supports other HTTP verbs like post and update. In the case of setting up a post route we write this:

app.post('product/:id',function(req,res) {...});

Express also has support for middeware. Middleware is just a request function handler with three parameters: request, response, and next. For example:

app.post('product/:id', authenticateUser, validateProduct, addProduct);

function authenticateUser(req,res, next) {
  //check req.session for authentication
  next();
}

function validateProduct (req, res, next) {
   //validate submitted data
   next();
}

function addProduct (req, res) {
  //save data to database
}

validateProduct and authenticateProduct are middleware. They are usually put into separate file (or files) in a big projects.

Another way to set up middle ware in Express application is to use use() function. For example earlier we did this for static assets:

app.use(express.static(__dirname + '/public'));

We can also do it for error handlers:

app.use(errorHandler);

Assuming you have mongoDB installed this app will connect to it (localhost:27017) and display collection name and items in collections. To start mongo server: 

$ mongod

to run app (keep the mongod terminal window open):

$ node .

or 

$ node index.js

To see the app working, open http://localhost:3000 in Chrome with JSONViewer extension (to render JSON nicely).

Tom Hanks' The Polar Express
Tom Hanks’ The Polar Express

Rapid Prototyping with JS Update v0.4

Why Backbone.js? Because a lot of people expressed desire to use it but being a framework Backbone has a learning curve. Not a steep one like Rails but still it takes time to master and learn Backbone. The new chapter “Intro to Backbone.js” will show readers how to:

  • Create Backbone architecture from scratch
  • Construct proper architecture with Routes
  • Use Collections
  • Apply Templates in Subviews
  • Split code into multiple JS files
  • Organize code into modules with AMD and Require.js
  • Optimize code for production with r.js library

Link to the new copy of Rapid Prototyping with JS at https://leanpub.com/rapid-prototyping-with-js.

Rapid Prototyping with JS: Agile JavaScript Developement
Rapid Prototyping with JS: Agile JavaScript Developement

TL;DR

Here is a list of the update for Rapid Prototyping with JS v0.4:

  • Brand new chapter: Intro to Backbone.js
  • Re-structured table of contents (chapters and parts)
  • Extended list of resources for further reading
  • Fixed code formatting and highlighting
  • Fixed grammar, style and typos mistakes
  • Brand new book cover optimized for ebook format
  • Reduced size of ebook (faster download)
  • Inspirational quotes in each chapter
  • Updated “Basics” chapter
  • New headline “Agile JavaScript Development”
  • Extended list of ways to reach us (Twitter, Facebook, Storify, etc.)

LeanPub Updates

LeanPub changed their purchasing. Now it’s even better for readers. Anybody can “return” the book they didn’t like within 45 days and get a full refund. Word return is in double quotes because all content is Digital Right Management (DRM) free. Does it mean that somebody can buy and download a book, get their money back, but keep the copy? Yes, but they will get bad karma for that! And not like a bad karma on Hacker News but a real bad karma. Everybody else deservers authors and publishers trust and respect.

Backbone.js

Why Backbone.js? Because a lot of people expressed desire to use it but being a framework Backbone has a learning curve. Not a steep one like Rails but still it takes time to master and learn Backbone. The new chapter “Intro to Backbone.js” will show readers how to:

  • Create Backbone architecture from scratch
  • Construct proper architecture with Routes
  • Use Collections
  • Apply Templates in Subviews
  • Split code into multiple JS files
  • Organize code into modules with AMD and Require.js
  • Optimize code for production with r.js library

Write and follow @RPJSbook.

Test-Driven Development in Node.js With Mocha

Don’t waste time writing tests for throwaway scripts, but please adapt the habit of Test-Driven Development for the main code base. With a little time spent in the beginning, you and your team will save time later and have confidence when rolling out new releases. Test Driven Development is a really really really good thing.

Who needs Test-Driven Development?

Imagine that you need to implement a complex feature on top of an existing interface, e.g., a ‘like’ button on a comment. Without tests you’ll have to manually create a user, log in, create a post, create a different user, log in with a different user and like the post. Tiresome? What if you’ll need to do it 10 or 20 times to find and fix some nasty bug? What if your feature breaks existing functionality, but you notice it 6 months after the release because there was no test!

Mocha: simple, flexible, fun
Mocha: simple, flexible, fun

Don’t waste time writing tests for throwaway scripts, but please adapt the habit of Test-Driven Development for the main code base. With a little time spent in the beginning, you and your team will save time later and have confidence when rolling out new releases. Test Driven Development is a really really really good thing.

Quick Start Guide

Follow this quick guide to set up your Test-Driven Development process in Node.js with Mocha.

Install Mocha globally by executing this command:

$ sudo npm install -g mocha

We’ll also use two libraries, Superagent and expect.js by LeanBoost. To install them fire up npm commands in your project folder like this:

$ npm install superagent
$ npm install expect.js

Open a new file with .js extension and type:

var request = require('superagent');
var expect = require('expect.js');

So far we’ve included two libraries. The structure of the test suite going to look like this:

describe('Suite one', function(){
  it(function(done){
  ...
  });
  it(function(done){
  ...
  });
});
describe('Suite two', function(){
  it(function(done){
  ...
  });
});

Inside of this closure we can write request to our server which should be running at localhost:8080:

...
it (function(done){
  request.post('localhost:8080').end(function(res){
    //TODO check that response is okay
  });
});
...

Expect will give us handy functions to check any condition we can think of:

...
expect(res).to.exist;
expect(res.status).to.equal(200);
expect(res.body).to.contain('world');
...

Lastly, we need to add done() call to notify Mocha that asynchronous test has finished its work. And the full code of our first test looks like this:

var request = require('superagent');
var expect = require('expect.js');
  
describe('Suite one', function(){
 it (function(done){
   request.post('localhost:8080').end(function(res){
    expect(res).to.exist;
    expect(res.status).to.equal(200);
    expect(res.body).to.contain('world');
    done();
   });
  });
});

If we want to get fancy, we can add before and beforeEach hooks which will, according to their names, execute once before the test (or suite) or each time before the test (or suite):

before(function(){
  //TODO seed the database
});
describe('suite one ',function(){
  beforeEach(function(){
    //todo log in test user
  });
  it('test one', function(done){
  ...
  });
});

Note that before and beforeEach can be placed inside or outside of describe construction.

To run our test simply execute:

$ mocha test.js

To use different report type:

$ mocha test.js -R list
$ mocha test.js -R spec

Asynchronicity in Node.js

One of the biggest advantages of using Node.js over Python or Ruby is that Node has a non-blocking I/O mechanism. To illustrate this let me use an example of a line in a Starbucks coffeeshop. Let’s pretend that each person standing in line for a drink is a task, and everything behind the counter — cashier, register, barista — is a server or server application. When we order a cup of regular drip coffee, like Pike, or hot tea, like Earl Grey, the barista makes it. While the whole line waits while that drink is made, and the person is charged the appropriate amount…

Non-Blocking I/O

One of the biggest advantages of using Node.js over Python or Ruby is that Node has a non-blocking I/O mechanism. To illustrate this, let me use an example of a line in a Starbucks coffee shop. Let’s pretend that each person standing in line for a drink is a task, and everything behind the counter — cashier, register, barista — is a server or server application. When we order a cup of regular drip coffee, like Pike, or hot tea, like Earl Grey, the barista makes it. The whole line waits while that drink is made, and the person is charged the appropriate amount.

Asynchronicity in Node.js
Asynchronicity in Node.js

Of course, we know that these kinds of drinks are easy to make; just pour the liquid and it’s done. But what about those fancy choco-mocha-frappe-latte-soy-decafs? What if everybody in line decides to order these time-consuming drinks? The line will be held up by each order, and it will grow longer and longer. The manager of the coffee shop will have to add more registers and put more baristas to work (or even stand behind the register him/herself). This is not good, right? But this is how virtually all server-side technologies work, except Node. Node is like a real Starbucks. When you order something, the barista yells the order to the other employee, and you leave the register. Another person gives their order while you wait for your state-of-the-art eye-opener in a paper cup. The line moves, the processes are executed asynchronously and without blocking the queue by waiting.

This is why Node.js blows everything else away (except maybe low-level C/C++) in terms of performance and scalability. With Node, you just don’t need that many CPUs and servers to handle the load.

Asynchronous Way of Coding

Asynchronicity requires a different way of thinking for programmers familiar with Python, PHP, C or Ruby. It’s easy to introduce a bug unintentionally by forgetting to end the execution of the code with a proper return expression.

Here is a simple example illustrating this scenario:

var test = function (callback) {
  return callback();  
  console.log('test') //shouldn't be printed
}

var test2 = function(callback){
  callback();
  console.log('test2') //printed 3rd
}

test(function(){
  console.log('callback1') //printed first
  test2(function(){
  console.log('callback2') //printed 2nd
  })
});

If we don’t use return callback() and just use callback() our string test2 will be printed (test is not printed).

callback1
callback2
tes2

For fun I’ve added a setTimeout() delay for the callback2 string, and now the order has changed:

var test = function (callback) {
  return callback();  
  console.log('test') //shouldn't be printed
}

var test2 = function(callback){
  callback();
  console.log('test2') //printed 2nd
}

test(function(){
  console.log('callback1') //printed first
  test2(function(){
    setTimeout(function(){
      console.log('callback2') //printed 3rd
    },100)
  })
});

Prints:

callback1
tes2
callback2

The last example illustrates that the two functions are independent of each other and run in parallel. The faster function will finish sooner than the slower one. Going back to our Starbucks examples, you might get your drink faster than the other person who was in front of you in the line. Better for people, and better for programs! :-)

Cheat Sheets for Web Development

Web development usually involves a large number of languages each with its own syntax, keywords, special sauce and magic tricks. Here is a collection of web development cheat sheets, in no particular order, which I’ve amassed by browsing the Internet over many years of web development

Cheat sheets are great ways to organize frequently used information and keep it handy. I used cheat sheets for learning and memorizing during my crams at school, and use them now for reference.

Cheat sheet for Web Development
Cheat sheet for Web Development

Web development usually involves a large number of languages each with its own syntax, keywords, special sauce and magic tricks.
Here is a collection of web development cheat sheets, in no particular order, which I’ve amassed by browsing the Internet over many years of web development. They cover the following topics:

  • jQuery
  • CSS3
  • Git
  • Heroku
  • HTML5
  • Linux Command Line
  • Mod reWrite
  • CoffeeScript
  • JavaScript
  • CSS2
  • JavaScript DOM
  • Mac Glyphs
  • Node.js
  • PHP
  • RGB Hex
  • Sublime Text 2
  • SEO
  • WordPress

Get zip archive at http://www.webapplog.com/wp-content/uploads/web-dev-cheat-sheets.zip.

Full list of the files:


jquery12_colorcharge.png
cdiehl_firefox.pdf
CSS Help Sheet outlined.pdf
css-cheat-sheet-v2.pdf
css-cheat-sheet.pdf
CSS3 Help Sheet outlined.pdf
css3-cheat-sheet.pdf
dan-schmidt_jquery-utility-functions-type-testing.pdf
danielschmitz_jquery-mobile.pdf
davechild_css2.pdf
davechild_html-character-entities.pdf
davechild_javascript.pdf
davechild_linux-command-line.pdf
davechild_mod-rewrite.pdf
davechild_regular-expressions.pdf
dimitrios_coffeescript-cheat-sheet.pdf
gelicia_sublime-text-2-shortcuts-verbose-mac.pdf
Git_Cheat_Sheet_grey.pdf
heroku_cheatsheet.pdf
HTML Help Sheet 02.pdf
html5-cheat-sheet.pdf
i3quest_jquery.pdf
javascript_refererence.pdf
javascript-cheat-sheet-v1.pdf
JavaScript-DOM-Cheatsheet.pdf
javascriptcheatsheet.pdf
jQuery-1_3-Visual-Cheat-Sheet.pdf
Mac_Glyphs_All.pdf
Node Help Sheet.pdf
PHP Help Sheet 01.pdf
pyro19d_javascript.pdf
rgb-hex-cheat-sheet-v1.pdf
salesforce_git_developer_cheatsheet.pdf
samcollett_git.pdf
sanoj_web-programming.pdf
SEO_Web_Developer_Cheat_Sheet.pdf
skrobul_sublime-text-2-linux.pdf
WordPress-Help-Sheet.pdf
Help Sheets.zip

MongoDB migration with Node and Monk

Recently one of our top users complained that their Storify account is unaccessible. We’ve checked the production database and it appeared to be that the account might have been compromised and maliciously deleted by somebody using user’s account credentials. Thanks for a great MongoHQ service we had a backup database in less than 15 minutes.

Recently one of our top users complained that their Storify account was unaccessible. We’ve checked the production database and it appeares to be that the account might have been compromised and maliciously deleted by somebody using user’s account credentials. Thanks to a great MongoHQ service, we had a backup database in less than 15 minutes.
There were two options to proceed with the migration:

  1. Mongo shell script
  2. Node.js program

Because Storify user account deletion involves deletion of all related objects — identities, relationships (followers, subscriptions), likes, stories — we’ve decided to proceed with the latter option. It worked perfectly, and here is a simplified version which you can use as a boilerplate for MongoDB migration (also at gist.github.com/4516139).

Restoring MongoDB Records
Restoring MongoDB Records

Let’s load all the modules we need: Monk, Progress, Async, and MongoDB: 

var async = require('async');
var ProgressBar = require('progress');
var monk = require('monk');
var ObjectId=require('mongodb').ObjectID;

By the way, made by LeanBoost, Monk is a tiny layer that provides simple yet substantial usability improvements for MongoDB usage within Node.JS.

Monk takes connection string in the following format:

username:password@dbhost:port/database

So we can create the following objects:

var dest = monk('localhost:27017/storify_localhost');
var backup = monk('localhost:27017/storify_backup');

We need to know the object ID which we want to restore:

var userId = ObjectId(YOUR-OBJECT-ID);

This is a handy restore function which we can reuse to restore objects from related collections by specifying query (for more on MongoDB queries go to post Querying 20M-Record MongoDB Collection. To call it, just pass a name of the collection as a string, e.g., "stories" and a query which associates objects from this collection with your main object, e.g., {userId:user.id}. The progress bar is needed to show us nice visuals in the terminal.

var restore = function(collection, query, callback){
  console.info('restoring from ' + collection);
  var q = query;
  backup.get(collection).count(q, function(e, n) {
    console.log('found '+n+' '+collection);
    if (e) console.error(e);
    var bar = new ProgressBar('[:bar] :current/:total :percent :etas', { total: n-1, width: 40 })
    var tick = function(e) {
      if (e) {
        console.error(e);
        bar.tick();
      }
      else {
        bar.tick();
      }
      if (bar.complete) {
        console.log();
        console.log('restoring '+collection+' is completed');
        callback();                
      }
    };
    if (n>0){
      console.log('adding '+ n+ ' '+collection);
      backup.get(collection).find(q, { stream: true }).each(function(element) {
        dest.get(collection).insert(element, tick);
      });        
    } else {
      callback();
    }
  });
}

Now we can use async to call the restore function mentioned above:

async.series({
  restoreUser: function(callback){   // import user element
    backup.get('users').find({_id:userId}, { stream: true, limit: 1 }).each(function(user) {
      dest.get('users').insert(user, function(e){
        if (e) {
          console.log(e);
        }
        else {
          console.log('resored user: '+ user.username);
        }
        callback();
      });
    });
  },

  restoreIdentity: function(callback){  
    restore('identities',{
      userid:userId
    }, callback);
  },

  restoreStories: function(callback){
    restore('stories', {authorid:userId}, callback);
  }

  }, function(e) {
  console.log();
  console.log('restoring is completed!');
  process.exit(1);
});

The full code is available at gist.github.com/4516139 and here: 

var async = require('async');
var ProgressBar = require('progress');
var monk = require('monk');
var ms = require('ms');
var ObjectId=require('mongodb').ObjectID;

var dest = monk('localhost:27017/storify_localhost');
var backup = monk('localhost:27017/storify_backup');

var userId = ObjectId(YOUR-OBJECT-ID); // monk should have auto casting but we need it for queries

var restore = function(collection, query, callback){
  console.info('restoring from ' + collection);
  var q = query;
  backup.get(collection).count(q, function(e, n) {
    console.log('found '+n+' '+collection);
    if (e) console.error(e);
    var bar = new ProgressBar('[:bar] :current/:total :percent :etas', { total: n-1, width: 40 })
    var tick = function(e) {
      if (e) {
        console.error(e);
        bar.tick();
      }
      else {
        bar.tick();
      }
      if (bar.complete) {
        console.log();
        console.log('restoring '+collection+' is completed');
        callback();                
      }
    };
    if (n>0){
      console.log('adding '+ n+ ' '+collection);
      backup.get(collection).find(q, { stream: true }).each(function(element) {
        dest.get(collection).insert(element, tick);
      });        
    } else {
      callback();
    }
  });
}

async.series({
  restoreUser: function(callback){   // import user element
    backup.get('users').find({_id:userId}, { stream: true, limit: 1 }).each(function(user) {
      dest.get('users').insert(user, function(e){
        if (e) {
          console.log(e);
        }
        else {
          console.log('resored user: '+ user.username);
        }
        callback();
      });
    });
  },

  restoreIdentity: function(callback){  
    restore('identities',{
      userid:userId
    }, callback);
  },

  restoreStories: function(callback){
    restore('stories', {authorid:userId}, callback);
  }

  }, function(e) {
  console.log();
  console.log('restoring is completed!');
  process.exit(1);
});

To launch it, run npm install/update and change hard-coded database values.

Sample of Rapid Prototyping with JS

Free sample chapter of Rapid Prototyping with JS which is a hands-on book which introduces you to rapid software prototyping using the latest cutting-edge web and mobile technologies including NodeJS, MongoDB, BackboneJS, Twitter Bootstrap, LESS, jQuery, Parse.com, Heroku and others.

Rapid Prototyping with JS is a hands-on book which introduces you to rapid software prototyping using the latest cutting-edge web and mobile technologies including NodeJS, MongoDB, BackboneJS, Twitter Bootstrap, LESS, jQuery, Parse.com, Heroku and others.

Rapid Prototyping with JS

Here is a free sample, first chapter — Introduction, of Rapid Prototyping with JS. You can also get a free PDF from LeanPub and explore code examples at github.com/azat-co/rpjs. To buy a full version in PDF, Mobi/Kindle and ePub/iPad formats go to leanpub.com/rapid-prototyping-with-js.

Introduction

Rapid Prototyping with JS is a hands-on book which introduces you to rapid software prototyping using the latest cutting-edge web and mobile technologies including Node.js, MongoDB, Twitter Bootstrap, LESS, jQuery, Parse.com, Heroku and others.

Who This Book is For

The book is designed for advanced-beginner and intermediate level web and mobile developers: somebody who has just started programming and somebody who is an expert in other languages like Ruby on Rails, PHP, and Java and wants to learn JavaScript and Node.js.

Rapid Prototyping with JS, as you can tell from the name, is about taking your idea to a functional prototype in the form of a web or a mobile application as fast as possible. This thinking adheres to the Lean Startup methodology; therefore, this book would be more valuable to startup founders, but big companies’ employees might also find it useful, especially if they plan to add new skills to their resume.

Prerequisite

Mac OS X or UNIX/Linux systems are highly recommended for this book’s examples and for web development in general, although it’s still possible to hack your way on a Windows-based system.

Some cloud services require users’ credit/debit card information even for free accounts.

What to Expect

Expect a lot of coding and not much of a theory. All the theory we cover is directly related to some of the practical aspects and essential for better understanding of technologies and specific approaches in dealing with them, e.g., JSONP and cross-domain calls.

In addition to coding examples, the book covers virtually all setup and deployment step-by-step.

You’ll learn on the example of Message Board web/mobile applications starting with front-end components. There are a few versions of these applications, but by the end we’ll put front-end and back-end together and deploy to production environment. The Message Board application contains all the necessary components typical for a basic web app, and will give you enough confidence to continue developing on your own, apply for a job/promotion or build a startup!

This is a digital version of the book, so most of the links are hidden just like on any other web page, e.g., jQuery instead of http://jquery.com. The content of the book has local hyperlinks which allow you to jump to any section.

All the source code for examples used in this book is available in the book as well as in a public GitHub repository github.com/azat-co/rpjs. You can also download files as a ZIP archive or use Git to pull them. More on how to install and use Git will be covered later in the book. The source code files, folder structure and deployment files are supposed to work locally and/or remotely on PaaS solutions, i.e., Windows Azure and Heroku, with minor or no modifications.

Notation

This is what source code blocks look like:

var object = {};
object.name = "Bob";

Terminal commands have a similar look but start with dollar sign, $:

$ git push origin heroku
$ cd /etc/
$ ls

Inline filenames, path/folder names, quotes and special words/names are italicized while command names, e.g., mongod, and emphasized words, e.g., Note, are bold.

Web Basics

Overview

The bigger picture of web and mobile application development consists of the following steps:

  1. User types a URL or follows a link in her browser (aka client);
  2. Browser makes HTTP request to the server;
  3. Server processes the request, and if there’re any parameters in a query string and/or body of the request takes them into account;
  4. Server updates/gets/transforms data in the database;
  5. Server responds with HTTP response containing data in HTML, JSON or other formats;
  6. Browser receives HTTP response;
  7. Browser renders HTTP response to the user in HTML or any other format, e.g., JPEG, XML, JSON.

Mobile applications act in the same manner as regular websites, only instead of a browser there might be a native app. Other minor differences include: data transfer limitation due to carrier bandwidth, smaller screens, and the more efficient use of the local storage.

There are a few approaches to mobile development, each with its own advantages and disadvantages:

  • Native iOS, Android, Blackberry apps build with Objective-C and Java;
  • Native apps build with JavaScript in Appcelerator and then complied into native Objective-C or Java;
  • Mobile websites tailored for smaller screens with responsive design, CSS frameworks like Twitter Bootstrap or Foundation, regular CSS or different templates;
  • HTML5 apps which consists of HTML, CSS and JavaScript, and are usually build with frameworks like Sencha Touch, Trigger.io, JO, and then wrapped into native app with PhoneGap.

Hyper Text Markup Language

Hyper Text Markup Language, or HTML, is not a programming language in itself. It is a set of markup tags which describes the content and presents it in a structured and formatted way. HTML tags consist of a tag name inside of the angle brackets (<>). In most cases tags surround the content with the end tag having forward slash before the tag name.

In this example each line is an HTML element:

<h2>Overview of HTML</h2>
<div>HTML is a ...</div>
<link rel="stylesheet" type="text/css" href="style.css" />

The HTML document itself is an element of html tag and all other elements are children of that html tag:

<!DOCTYPE html>
<html lang="en">
  <head>
    <link rel="stylesheet" type="text/css" href="style.css" />
  </head>
  <body>
    <h2>Overview of HTML</h2>
    <p>HTML is a ...</p>
  </body>
</html>

There are different flavors and versions of HTML, e.g., DHTML, XHTML 1.0, XHTML 1.1, XHTML 2, HTML 4, HTML 5. This article does a good job of explaining the differences — Misunderstanding Markup: XHTML 2/HTML 5 Comic Strip.

More information is available at Wikipedia and w3schools.

Cascading Style Sheets

Cascading Style Sheets, or CSS, is a way to format and present content. An HTML document can have several stylesheets with the tag link as in previous examples or style tag:

<style>
  body {
  padding-top: 60px; /* 60px to make some space */
  }
</style>

Each HTML element can have id and class attribute:

<div id="main" class="large">Lorem ipsum dolor sit amet,  Duis sit amet neque eu.</div>

In CSS we access elements by their id, class, tag name and in some edge cases by parent-child relationship or element attribute value:

p {
  color:#999999;
}
div#main {
  padding-bottom:2em;
  padding-top:3em;
}
.large {
  font-size:14pt;
}
body > div {
  display:none;         
}
input[name="email"] {
  width:150px;
}

More information for further reading is available at Wikipedia and w3schools.

CSS3 is an upgrade to CSS which includes new ways of doing things such as rounded corners, borders and gradients, which were possible in regular CSS only with the help of PNG/GIF images and by using other tricks.

For more information refer to CSS3.info, w3school
and CSS3 vs CSS comparison article on Smashing.

JavaScript

JavaScript was started in 1995 at Netscape as LiveScript. It has the same relationship with Java as a hamster and a ham :)
It is used for both client and server side development as well as in desktop applications.

There is a script tag to use JavaScript in the HTML document:

<script type="text/javascript" language="javascript>
  alert("Hello world!");
  //simple alert dialog window
</script>

Usually it a good idea to separate JavaScript code from HTML; in this example we include app.js file:

<script src="js/app.js" type="text/javascript" language="javascript" />

Here are the main types of JavaScript objects/classes:

  • Array object, e.g., var arr = ["apple", "orange", 'kiwi"];
  • Boolean primitive object, e.g., var bool = true;
  • Date object, e.g., var d = new Date();
  • Math object, e.g., var x = Math.floor(3.4890);
  • Number primitive object, e.g., var num = 1;
  • String primitive object, e.g., var str = "some string";
  • RegExp object, e.g., var pattern = /[A-Z]+/;
  • Global properties and functions, e.g., NaN
  • Browser objects, e.g., window.location = 'http://google.com';
  • DOM objects, e.g., var table = document.createElement('table');

Full JavaScript and DOM objects and classes reference with examples are available at w3school.

Typical syntax for function declaration:

function Sum(a,b) {
  var sum = a+b;
  return sum;
}
console.log(Sum(1,2));

Functions in JavaScript are first-class citizens due to functional programming nature of the language. Therefore functions can be used as other variables/objects; for example, functions can be passed to other functions as arguments:

var f = function (str1){
  return function(str2){
  return str1+' '+str2;
  };
};
var a = f('hello');
var b = f('goodbye');
console.log((a('Catty'));
console.log((b('Doggy'));

JavaScript has a loose/weak typing, as opposed to strong typing in languages like C and Java, which makes JavaScript a better programming language for prototyping.

More information about browser-run JavaScript is available at Wikipedia and w3schools.

Agile Methodologies

Agile software development methodology evolved due to the fact that traditional methods, like Waterfall, weren’t good enough in situations of high unpredictability, i.e., when the solution is unknown. Agile methodology includes Scrum/Sprint, Test-Driven Development, Continuous Deployment, Paired Programming and other practical techniques many of which were borrowed from Extreme Programming.

Scrum

In regard to the management, Agile methodology uses Scrum approach. More about Scrum can be read at:

Scrum methodology is a sequence of short cycles, and each cycle is called sprint. One sprint usually lasts from one to two weeks. Sprint starts and ends with sprint planning meeting where new tasks can be assigned to team members. New tasks cannot be added to the sprint in progress; they can be added only at the sprint meetings.

An essential part of the Scrum methodology is the daily scrum meeting, hence the name. Each scrum is a 5–15 minutes long meeting which is often conducted in the hallways. On scrum meetings each team member answers three questions:

  1. What have you done since yesterday?
  2. What are you going to do today?
  3. Do you need anything from other team members?

Flexibility makes Agile an improvement over Waterfall methodology, especially in situations of high uncertainty, i.e., startups.

Advantage of Scrum methodology: effective where it is hard to plan ahead of the time, and also in situations where a feedback loop is used as a main decision-making authority.

Test-Driven Development

Test-Driven Development, or TDD, consists of following steps:

  1. Write failing automated test cases for new feature/task or enhancement by using assertions that are either true or false.
  2. Write code to successfully pass the test cases.
  3. Refactor code if needed, and add functionality while keeping the test cases passed.
  4. Repeat until the task is complete.

Advantages of Test-Driven Development:

  • fewer bugs/defects,
  • more efficient codebase,
  • provides programmers with confidence that code works and doesn’t break old functionality.

Continuous Deployment

Continuous Deployment, or CD, is the set of techniques to rapidly deliver new features, bug fixes, and enhancements to the customers. CD includes automated testing and automated deployment. By utilizing Continuous Deployment the manual overheard is decreased, and the feedback loop time is minimized. Basically, the faster developer can get the feedback from the customers, the sooner the product can pivot, which leads to more advantages over the competition. Many startups deploy multiple times in a single day in comparison to the 6–12 month release cycle which is still typical for corporations and big companies.

One of the most popular solutions for CD is Continuous Integration server Jenkins.

Advantages of Continuous Deployment approach: decreases feedback loop time and manual labor overhead.

Pair Programming

Pair Programming is a technique when two developers work together on one machine. One of the developers is a driver and the other is observer. The driver writes the code and the observer watches it, assists, and makes suggestions. Then they switch the roles. The driver has a more tactical role of focusing on the current task. In contrast, the observer has a more strategic role overseeing “the bigger picture,” and the ways to improve the codebase and to make it more efficient.

Advantages of Paired Programming:

  • Pair attributes to shorter and more efficient codebase, and introduces fewer bugs and defects.
  • As an added bonus, knowledge is passed along programmers as they work together. However, situations of conflicts between developers are possible.

Node.js

Node.js is an event-driven asynchronous I/O server-side technology for building scalable and efficient web servers. Node.js consists of Google’s V8 JavaScript engine.

The purpose and use of Node.js is similar to Twisted for Python and EventMachine for Ruby. The JavaScript implementation of Node was the third one after attempts at using the Ruby and C++ programming languages.

Node.js is not in itself a framework like Ruby on Rails; it’s more comparable to the pair PHP+Apache. Here are some of Node.js frameworks: Express, Meteor, Tower.js, Railsway JS, Geddy, Derby.

Advantages of using NodeJS:

  • Developers have high chances of familiarity with JavaScript due to its status as a de facto standard of the application development for web and mobile.
  • One language for front-end and back-end development speeds up coding process. A developer’s brain doesn’t have to switch between different syntaxes. The learning of methods and classes goes faster.
  • With NodeJS, you could prototype quickly and go to market to do your customer development and customer acquisition early. This is an important competitive advantage over the other companies, which use less agile technologies, e.g., PHP and MySQL.
  • NodeJS is build to support real-time applications by utilizing web-sockets.

For more information go to Wikipedia, Nodejs.org, and articles on ReadWrite and O’Reilly.

NoSQL and MongoDB

MongoDB, from huMONGOus, is a high-performance no-relationship database for huge data. NoSQL concept came out when traditional Relational Database Management Systems, or RDBMS, were unable to meet the challenges of huge amounts of data.

Advantages of using MongoDB:

  • Scalable due to distributed nature: multiple servers and data centers could have redundant data.
  • High-performance: MongoDB is very effective for storing and retrieving data, not the relationship between elements.
  • Key-value store is ideal for prototyping because it doesn’t require one to know the schema and there is no need for a fixed data model.

Cloud Computing

Could computing consists of:

  • Infrastructure as s Service (IaaS), e.g., Rackspace, Amazon Web Services;
  • Platform as a Service (PaaS), e.g., Heroku, Windows Azure;
  • Software as a Service (SaaS), e.g., Google Apps, Salesforce.com.

Cloud application platforms provide:

  • scalability, e.g., spawn new instances in a matter of minutes;
  • easy deployment, e.g., to push to Heroku you can just use $ git push;
  • pay-as-you-go plan: add or remove memory and disk space based on demands;
  • usually there is no need to install and configure databases, app servers, packages, etc.;
  • security and support.

PaaS are ideal for prototyping, building minimal viable products (MVP) and for early stage startups in general.

Here is the list of most popular PaaS solutions:

HTTP Requests and Responses

Each HTTP Request and Response consists of the following components:

  1. Header: information about encoding, length of the body, origin, content type, etc.;
  2. Body: content, usually parameters or data which is passed to the server or sent back to a client;

In addition, HTTP Request contains:

  • Method: There are several methods; the most common are GET, POST, PUT, DELETE.
  • URL: host, port, path;
  • Query string, i.e., everything after a question mark in the URL.

RESTful API

RESTful (REpresentational State Transfer) API became popular due to the demand in distributed systems where each transaction needs to include enough information about the state of the client. In a sense this standard is stateless because no information about the clients’ state is stored on the server, thus making it possible for each request to be served by a different system.

Distinct characteristics of RESTful API:

  • Has better scalability support due to the fact that different components can be independently deployed to different servers;
  • Replaced Simple Object Access Protocol (SOAP) because of the simpler verb and noun structure;
  • Utilizes HTTP methods: GET, POST, DELETE, PUT, OPTIONS etc.

Here is an example of simple Create, Read, Update and Delete (CRUD) REST API for Message Collection:

Method URL Meaning
GET /messages.json Return list of messages in JSON format
PUT /messages.json Update/replace all messages and return status/error in JSON
POST /messages.json Create new message and return its id in JSON format
GET /messages/{id}.json Return message with id {id} in JSON format
PUT /messages/{id}.json Update/replace message with id {id}, if {id} message doesn’t exists create it
DELETE /messages/{id}.json Delete message with id {id}, return status/error in JSON format

REST is not a protocol; it is an architecture in the sense that it’s more flexible than SOAP, which is a protocol. Therefore, REST API URLs could look like /messages/list.html or /messages/list.xml in case we want to support these formats.

PUT and DELETE are idempotent methods, which means that if the server receives two or more similar requests, the end result will be the same.

GET is nullipotent and POST is not idempotent and might affect state and cause side-effects.

Further reading on REST API at Wikipedia and A Brief Introduction to REST article.

Wintersmith — Node.js static site generator

This past weekend was a very productive one for me, because I’ve started to work on and released my book’s one-page website —rapidprototypingwithjs.com. I’ve used Wintersmith to learn something new and to ship fast. Wintersmith is a Node.js static site generator. It greatly impressed me with flexibility and ease of development. In addition I could stick to my favorite tools such as Markdown, Jade and Underscore.

This past weekend was a very productive one for me, because I’ve started to work on and released my book’s one-page website —rapidprototypingwithjs.com. I’ve used Wintersmith to learn something new and to ship fast. Wintersmith is a Node.js static site generator. It greatly impressed me with flexibility and ease of development. In addition I could stick to my favorite tools such as Markdown, Jade and Underscore.

Wintersmith is a Node.js static site generator

Why Static Site Generators

Here is a good article on why using a static site generator is a good idea in general, An Introduction to Static Site Generators. It basically boils down to a few main things:

Templates

You can use template engine such as Jade. Jade uses whitespaces to structure nested elements and its syntax is similar to Ruby on Rail’s Haml markup.

Markdown

I’ve copied markdown text from my book’s Introduction chapter and used it without any modifications. Wintersmith comes with marked parser by default. More on why Markdown is great in my old post, Markdown Goodness.

Simple Deployment

Everything is HTML, CSS and JavaScript so you just upload the files with FTP client, e.g., Transmit by Panic or Cyberduck.

Basic Hosting

Due to the fact that any static web server will work well, there is no need for Heroku or Nodejitsu PaaS solutions, or even PHP/MySQL hosting.

Performance

There are no database calls, no server-side API calls, no CPU/RAM overhead.

Flexibility

Wintersmith allows for different plugins for contents and templates and you can even write you own plugin.

Getting Started with Wintersmith

There is a quick getting started guide on github.com/jnordberg/wintersmith.

To install Wintersmith globally, run NPM with -g and sudo:

$ sudo npm install wintersmith -g

Then run to use default blog template:

$ wintersmith new <path>

or for empty site:

$ wintersmith new <path> -template basic

or use a shortcut:

$ wintersmith new <path> -T basic

Similar to Ruby on Rails scaffolding Wintersmith will generate a basic skeleton with contents and templates folders. To preview a website, run these commands:

$ cd <path>
$ wintersmith preview
$ open http://localhost:8080

Most of the changes will be updates automatically in the preview mode except for the config.json file.

Images, CSS, JavaScript and other files go into contents folder.
Wintersmith generator has the following logic:

  1. looks for *.md files in contents folder,
  2. reads metadata such as template name,
  3. processes *.jade templates per metadate in *.md files.

When you’re done with your static site, just run:

$ wintersmith build

Other Static Site Generators

Here are some of the other Node.js static site generators:

More detailed overview of these static site generators is available in the post, Node Based Static Site Generators.

For other languages and frameworks like Rails and PHP take a look at Static Site Generators by GitHub Watcher Count and the “mother of all site generator lists”.

Querying 20M-Record MongoDB Collection

Storify saves a lot of meta data about social elements: tweets, Facebook status updates, blog posts, news articles, etc. MongoDB is great for storing such unstructured data but last week I had to fix some inconsistency in 20-million-record Elements collection.

Storify saves a lot of meta data about social elements: tweets, Facebook status updates, blog posts, news articles, etc. MongoDB is great for storing such unstructured data but last week I had to fix some inconsistency in 20-million-record Elements collection.

The script was simple: find elements, see if there are no dependencies, delete orphan elements, neveretheless it was timing out or just becoming unresponsive. After a few hours of running different modifications I came up with the working solution.

Here are some of the suggestions when dealing with big collections on Node.js + MongoDB stack:

Befriend Shell

Interactive shell, or mongo, is a good place to start. To launch it, just type mongo in your terminal window:

$ mongo

Assuming you have correct paths set-up during your MongoDB installation, the command will start the shell and present angle brace.

>

Use JS files

To execute JavaScript file in a Mongo shell run:

$ mongo fix.js --shell

Queries look the same: 

db.elements.find({...}).limit(10).forEach(printjson);

To output results use:

print();

or

printjson();

To connect to a database:

db = connect("<host>:<port>/<dbname>")

Break Down

Separate your query into a few scripts with smaller queries. You can output each script to a file (as JSON or CSV) and then look at the output and see if your script is doing what it is actually supposed to do.

To execute JavaScript file (fix.js) and output results into another file (fix.txt) instead of the screen, use:

$ mongo fix.js > fix.txt --shell

or

$ mongo --quiet fix.js > fix.txt --shell

Check count()

Simply run count() to see the number of elements in the collection:

 db.collection.count();

or a cursor:

 db.collection.find({…}).count();

Use limit()

You can apply limit() function to your cursor without modifying anything else in a script to test the output without spending too much time waiting for the whole result.

For example:

 db.find({…}).limit(10).forEach(function() {…});

or

 db.find({…}).limit(1).forEach(function() {…});

is better than using:

 db.findOne({…})

because findOne() returns single document while find() and limit() still returns a cursor.

Hit Index

hint() index will allow you to manually use particular index:

 db.elemetns.find({…}).hint({active:1, status:1, slug:1});

Make sure you have actual indexes with ensureIndex():

 db.collection.ensureIndex({…})

Narrow Down

Use additional criteria such as $ne, $where, $in, e.g.:

db.elements.find({ $and:[{type:'link'}
  ,{"source.href":{$exists:true}}
  ,{'date.created':{$gt: new Date("November 30 2012")}}
  ,{$where: function () {
    if (this.meta&&this.data&&this.data&&this.data.link) {
      return this.meta.title!=this.data.link.title;
    } else {
      return false;
    }}} 
  , {'date.created': {$lt: new Date("December 2 2012")}}]}).forEach(function(e, index, array){
    print(e._id.str);
    });

My First Week At Storify

Last week I joined Storify — a destination for curated social media news. Storify helps you sort through the noise to find the voices online that matter.

Last week I joined Storify — a destination for curated social media news. Storify helps you sort through the noise to find the voices online that matter. To find more about Storify take a look at the guided tour.

Storify co-founder Burt and I met a couple months ago for the first time and I’m glad that we did. There were three main reasons for me to come on board: great team, awesome product and company vision, and cool tech stack that I’m passionate about: Node.js+Express+MongoDB.

Storify on Nodejs.org
Storify on Nodejs.org

The first week at Storify exceeded my expectations! So far there were: 4 team lunches, one birthday party, two (!) break-ins. In addition, I’ve worked on the front-page on my second day and had a chance to SSH to production servers.

A few word about the office, besides free snacks and espresso and being close to everything, there are two other startups, Buffer and HomeLight. The funny thing is that I’ve discovered and fallen in love with Buffer just a few weeks ago and now I’ve met with Leo and sit next to their brilliant team!

By the way, Storify is hiring bright minds: Operations Engineer and Front-End Engineer. If you want to do work on interesting things check out full job description.

Rapid Prototyping with JS is out!

Rapid Prototyping with JS is a hands-on book which introduces you to rapid software prototyping using the latest cutting-edge web and mobile technologies including NodeJS, MongoDB, BackboneJS, Twitter Bootstrap, LESS, jQuery, Parse.com, Heroku and others.

The Book is on LeanPub

Rapid Prototyping with JS is a hands-on book which introduces you to rapid software prototyping using the latest cutting-edge web and mobile technologies including NodeJS, MongoDB, BackboneJS, Twitter Bootstrap, LESS, jQuery, Parse.com, Heroku and others.

The book has 84 pages (in PDF format) or 13,616 words to be precise, step-by-step set-up, best practice advices, web development overview, 11 code examples (also available ready-to-go in GitHub repository azat-co/rpjs), flexible pricing ($9.99–19.99).

Order your copy of Rapid Prototyping with JS at LeanPub: leanpub.com/rapid-prototyping-with-js.

Rapid Prototyping with JS
Rapid Prototyping with JS: Learn how to build web and mobile apps using JavaScript and Node.js

LeanPub platform allows readers to receive infinite future updates (current version of the book is 0.3) and read the book in the most popular digital formats: PDF, ePub/iPad, MOBI/Kindle. The PDF version has footnote links which make it suitable for printing.

Download a free sample at samples.leanpub.com/rapid-prototyping-with-js-sample.pdf.

What Readers Say

Rapid Prototyping with JS is being successfully used at StartupMonthly as a training manual. Here are some of our trainees’ testimonials:

“Thanks a lot to all and special thanks to Azat and Yuri. I enjoyed it a lot and felt motivated to work hard to know these technologies.” — Shelly Arora

“Thanks for putting this workshop together this weekend… what we did with Bootstrap + Parse was really quick & awesome.” — Mariya Yao

“Thanks Yuri and all of you folks. It was a great session – very educative, and it certainly helped me brush up on my Javascript skills. Look forward to seeing/working with you in the future.” — Sam Sur

Who This Book is For

The book is designed for advanced-beginner and intermediate level web and mobile developers: somebody who has just started programming and somebody who is an expert in other languages like Ruby on Rails, PHP, and Java and wants to learn JavaScript and Node.js.

Rapid Prototyping with JS, as you can tell from the name, is about taking your idea to a functional prototype in the form of a web or a mobile application as fast as possible. This thinking adheres to the Lean Startup methodology. Therefore, this book would be more valuable to startup founders, but big companies’ employees might also find it useful, especially if they plan to add new skills to their resume.

Prerequisite

Mac OS X or UNIX/Linux systems are highly recommended for this book’s examples and for web development in general, although it’s still possible to hack your way on a Windows-based system.

Contents

Acknowledgment

Introduction

  1. Who This Book is For
  2. Prerequisite
  3. What to Expect
  4. Notation
  5. Web Basics: Hyper Text Markup Language, Cascading Style Sheets, JavaScript
  6. Agile Methodologies: Scrum, Test-Driven Development, Continuous Deployment, Paired Programming
  7. Node.js
  8. NoSQL and MongoDB
  9. Cloud Computing
  10. HTTP Requests and Responses
  11. RESTful API

Getting Started

  1. Development Folder
  2. Browsers
  3. IDEs and Text Editors
  4. Version Control Systems
  5. Local HTTP Servers
  6. Database: MongoDB
  7. Other Components: NodeJS, jQuery, LESS
  8. SSH Keys
  9. GitHub
  10. Windows Azure
  11. Heroku
  12. Cloud9

Building Front-End Application

  1. JSON
  2. AJAX
  3. Cross-Domain Calls
  4. jQuery
  5. Twitter Bootstrap
  6. LESS
  7. BackboneJS
  8. Example of using Twitter REST API and jQuery
  9. Parse.com
  10. Message Board with Parse.com
  11. Message Board with Parse.com: REST API and jQuery version
  12. Pushing to GitHub
  13. Deployment to Windows Azure
  14. Deployment to Heroku
  15. Message Board with Parse.com: JavaScript SDK and BackboneJS version
  16. Deploying Message Board to PaaS
  17. Enhancing Message Board
  18. Building Back-End Application

Building “Hello World” in NodeJS

  1. NodeJS Core Modules
  2. Node Package Manager
  3. Deploying “Hello World” to PaaS
  4. Deploying to Windows Azure
  5. Deploying to Heroku
  6. Message Board: Run-Time Memory version
  7. Test Case for Message Board
  8. MongoDB Shell
  9. MongoDB Native Driver
  10. MongoDB on Heroku: MongoHQ MongoHQ URL
  11. BSON
  12. Message Board: MongoDB version

Putting it All Together

  1. Different Domain Deployment
  2. Changing Endpoints
  3. Message Board Application
  4. Deployment
  5. Same Domain Deployment

Further Reading

About the Author

Order your copy of Rapid Prototyping with JS at LeanPub: leanpub.com/rapid-prototyping-with-js.

Most Common Technical Questions for Early-Stage Startups

A lot of people like to pick my brain on technical things related to early stage startups, e.g., what framework to use, how long will it take to build an app or a website, should I hire or outsource. I decided to organize my answers to the most common technical questions in this post:

A lot of people like to pick my brain on technical things related to early stage startups, e.g., what framework to use, how long will it take to build an app or a website, should I hire or outsource. I decided to organize my answers to the most common technical questions in this post:

  • Don’t outsource your core product. Outsourcing to a digital agency is the best way to spend your money fast and usually fruitlessly.
  • Use frameworks, don’t write trivial things like URL parsers from scratch.
  • Use available solutions to your none-core components. Things like: CMS, blog, analytics, landing page, forum, version-control, bug-tracking/customer feedback, project management, etc.
  • Use PaaS or IaaS. Don’t use IaaS unless you really need to, for example when PaaS becomes cost prohibitive;
  • Make mobile your first initiative, if and when it makes sense for your business; don’t even build a website if mobile app is your bread and butter!
    • Code snippet: Learn Programming
    Code Snippet: Learn Programming
  • Use social connect APIs for smooth user on-boarding, most of your early adopter probably have at least Facebook or Twitter accounts; don’t make them type their emails and remember passwords.
  • Use the most appropriate programming language don’t rely on your geeky “guru” friend’s advice that programming language is a matter of personal preference, to go with PHP, or that Ruby on Rails and Node.js are just a fads.
  • Don’t over-rely on remote developers. It often leads to miscommunication; your teams will work longer, will have to spend more time and money; also, not everybody is disciplined well enough to work remotely without social interaction and direct supervision.
  • Don’t over-rely on part-time developers. It will take longer to execute, the code will be less efficient due to distractions and as a result harder to maintain and more expensive to run in the future.
  • Learn programming. Business co-founders without any programming knowledge will have a hard time understanding trade-off in features and available resources; expect longer sprint meetings and leap of trust between technical and business people.
  • Don’t have more that one language for your core-product on a prototype stage because code is not an asset, but a liability and maintaining fragmented codebase could lead to a disaster.
  • Start from scratch for your core-product or refactor your code often if you or your team are still learning a language or a framework; the longer you keep old spaghetti code the harder it will be to come back to fix it later.
  • Use Test-Driven Development and Pair-Programming.
  • Use peers and mentors help/feedback from fellow founders, programmer friends on meet-ups and at the office.
  • Involve developers in customer development process. Tech people just love to build things for the sake of building something complicated. It’s so hard to practice Lean Startup methodology, because you need a lot of self-discipline. And it’s just so tempting to go to your basement for 6 months, into your hacker/coding nirvana mode, instead of doing endless customer interview, landing pages, mock-ups, paper prototypes and face website! The problem is that you’ll usually end up with another useless app or website. I had similar a phat startup experience with what later became open-sourced project http://openList.co.

Best of the web goodies for agile web development, startups and Lean Startup startups:

If you like this post and want to lean more about building your web or mobile app using the latest tech and agile practices check out my new book on how to take your idea to a prototype with JavaScript, Node.js and MongoDB — Rapid Prototyping with JS.

What is Accelerator.IO?

It’s a new social network for entrepreneurs with productivity tools based on Lean Startup methodology. Accelerator.IO is a niche social network similar to dribbble, which is for designers, and Forrst, which is for developers. Main focus for us, team of Accelerator.IO, is to provide valuable tools and content for startupers, founders, hackers and hustlers by connecting them with mentors, investors and most importantly fellow entrepreneurs across the globe. It is important to state that Accelerator.IO is not a competitor to AngelList but rather a complimentary tool. In fact, we leverage AngelList API to gather entrepreneurial information about users such as company name and website URL. In a sense Accelerator.IO is a global online co-working space where smart and bright minds collaborate and learn from each others failures and successes.

Accelerator.IO
Accelerator.IO

The key features of Accelerator.IO:

  • curated access and content moderation to keep discussions interesting and engaging;
  • access to global network of like-minded people;
  • leveraging AngelList API for easier adoption;
  • reliance on the over 6,000 people network of StartupMonthly fund and startup accelerator, across Silicon Valley/San Francisco, Middle East and Europe;
  • productivity tools for entrepreneurs and startups founders based on Lean Startup methodology.

Right now Accelerator.IO is in beta testing. We accept early access sign ups and beta testers at http://signup.accelerator.io.

Last but not least, if you are looking for a side-project to pick-up a new skills in web and mobile development, or get a free access to startup social events and get exposure on web — Accelerator.IO is accepting volunteers: JavaScript developers, front-end or back-end (we’ll teach you Node.js if you’re only front-end dev right now).