This post has been written by Scott Hasbrouck. You can find him on Twitter or his website.

It’s been nothing but wonderful to see JavaScript truly catch fire the past few years in web technology, ultimately becoming the most used language in 2016, according to StackOverflow data. My history with JavaScript began about 15 years ago, not all that long after it was first released as part of Netscape Navigator 2, in 1996. My most used learning resource was DynamicDrive, and their tutorials and code snippets of “Dynamic HTML” or DHTML – a term coined by Internet Explorer 4. Really, DHTML was a set of browser features implemented with JavaScript, CSS, and HTML that could get you nifty elements like rollover buttons and stock tickers.

Fasting forward to today, we now live in a world where JavaScript has grown to take over web technology. Not just in the browser, but it is now the most popular backend language according to that same StackOverflow report! Naturally, there are always those who dislike the language citing things like the ease of creating a global variable, or null being an object and undefined being its own datatype. But I’ve found that every language I pick up has quirks that are easily avoidable once you learn to properly use it. And we do want to become experts in our craft and truly learn to master our tools, do we not?

Here are the primary factors (good or not), that I believe are why JavaScript has so quickly taken over the internet:

JavaScript is the only universal client side language.
JavaScript is relatively easy to learn, especially coming from any other C-like language.
With the advent of Node.js, JavaScript can now run on servers (and Node/V8 is extremely resource efficient doing so).
ES6 came along at just the right time, and “fixed” a lot of the problems with JavaScript’s syntax and lacking features.
Mature front end frameworks. Let’s face it, building a front end application in vanilla JavaScript requires a lot of discipline to keep it from turning into spaghetti code. React/Redux/Reflux and Angular provides the framework to keep it organized.
The breadth and quality of open source projects and the ease of installing those modules with npm.

In particular, the advent of Node.js has driven JavaScript adoption to an all-time high. With it, we are only required to learn one language for an entire stack, and are capable of building things like background workers and HTTP servers with it! I’ve even recently finished my first book about charging credit cards with stripe using JavaScript and Node.js – something I never thought I would ever be able to do when I first learned the language over a decade ago. So whether you like it or not, here we are, living in a JavaScript internet world. But here you are. My guess is that you probably like it. Which is great, welcome! Because now I want to share with you how I’ve managed to capitalize on this new expansive JavaScript world to build a truly reactive web app stack – all in one language from top to bottom.

The 3RES Stack

Yeah, I don’t know how to pronounce that either… threes? Sure. Let’s start at the top with React.

Frontend Only Libraries

React

React is a declarative way of building user interfaces, which heavily leans on its XML-like syntax extension, called JSX. Your application is built up from “components” – each of which encapsulate small, often reusable parts of your UI. These components each have their own immutable state, which contains information about how the components should render. The state has a pure setter function (no side effects), and should not be changed directly. This overview of the proposed 3RES stack will only require basic knowledge of React. Of course, you want to become a React master! Be sure to learn more about React at SurviveJS – one of the best comprehensive React books with a free version.

Redux

If React encapsulates all of your UI components, Redux encapsulates all of your data represented as a JavaScript object. This state object is immutable and should not be modified directly, but only by dispatching an action. In this way, React/Redux combined can automatically react to state changes, and update the relevant DOM elements to reflect the new values. Redux has some awesome documentation – probably some of the best for any open source library I’ve used. To top it off, Redux also has 30 free videos on egghead.

Frontend and Backend Libraries

Socket.IO

Most likely, your web apps to date have relied on AJAX to communicate with the server – which is built on a Microsoft introduced JavaScript API called XMLHttpRequest. For many one-time user induced actions, such as logging in, AJAX makes a lot of sense. However, it is extremely wasteful to rely on it for data that updates continuously, and for multiple clients. The only real way to handle this is by regularly polling the backend at short intervals, asking for new data. WebSockets are a relatively new technology that was not even standardized until 2011. A WebSocket opens a continuously pending TCP connection, and allows for frames of data to be sent by either the server or the client. It is initiated with an HTTP “handshake” as an upgrade request. However, similar to how we often don’t use the vanilla XMLHttpRequest API (trust me, I’ve had to do it, you don’t want to implement this yourself and support every browser), we also typically don’t use the JavaScript WebSocket API directly. Socket.io is the most widely accepted library for both client and server-side WebSocket communications, and also implements an XMLHttpRequest/polling fallback for when WebSockets fail. We’ll be using this library in conjunction with RethinkDB changefeeds (described below), and Redux, to continually keep all of our clients’ states up-to-date with our database!

Backend Libraries and Technologies

RethinkDB

RethinkDB is an open-source NoSQL datastore that stores JSON documents. It’s often compared to MongoDB, but vastly superior in many key ways that are relevant to making our 3RES stack work. Primarily, RethinkDB comes out of the box with query changefeeds – the ability to attach an event listener to a query which will receive real-time updates anytime a document selected by that query is added, updated, or removed! As mentioned above, we’ll be emiting Socket.io events from our RethinkDB changefeeds. In addition, RethinkDB is amazingly simple to scale via sharding, and implements redundancy with replication. It has an amazing developer outreach program and crystal clear documentation, and is constantly improving with feedback from engineers like us.

Express

Lastly, our application will still need to accept HTTP requests as routes. Express is the accepted minimalistic Node.js framework for building HTTP routes. We will use this for everything that requires a one-time event that is outside the scope of Socket.io: initial page load, logging in, signing up, logging out, etc.

Building the Server Code

Our sample application will be a simple Todo Checklist with no authentication. One of my common complaints is when the sample app for a simple tutorial has a huge code base – it just makes it way too time consuming to pick out the relevant parts of the app. So this sample app will be very minimal, but will show exactly one example of every required piece of this stack for end-to-end reactivity. The only folder is a /public folder with all of our built JavaScript. One important point that this app leaves out in that spirit is authentication and sessions – anyone on the internet can read and edit Todo’s! If you’re interested in adding authentication to this app with both Socket.io and Express, I’ve got a complete tutorial on how to do this on my website!

Let’s start with the backend. First, you need to grab a copy of RethinkDB, then start it up with:

$ rethinkdb

Once you start RethinkDB, navigate to the super-handy web interface at http://localhost:8080. Click on the ‘Tables’ tab at the top, then add a database named ‘3RES_Todo’, then once that is created, add a table called ‘Todo’.

The complete code for this sample is on Github, so we’ll just walk through the key points here, assuming you’re familiar with Node.js basics. The repo includes all required modules in package.json, but if you’d like to manually install the modules needed for the backend portion of the app, run:

$ npm install --save rethinkdb express socket.io

Now that we have the required packages, let’s setup a basic node app which serves index.html.

// index.js

// Express
var express = require('express');
var app = express();
var server = require('http').Server(app);
var path = require('path');

// Socket.io
var io = require('socket.io')(server);

// Rethinkdb
var r = require('rethinkdb');

// Socket.io changefeed events
var changefeedSocketEvents = require('./socket-events.js');

app.use(express.static('public'));

app.get('*', function(req, res) {
  res.sendFile(path.join(__dirname + '/index.html'));
});

r.connect({ db: '3RES_Todo' })
.then(function(connection) {
    io.on('connection', function (socket) {

        // insert new todos
        socket.on('todo:client:insert', function(todo) {
            r.table('Todo').insert(todo).run(connection);
        });

        // update todo
        socket.on('todo:client:update', function(todo) {
            var id = todo.id;
            delete todo.id;
            r.table('Todo').get(id).update(todo).run(connection);
        });

        // delete todo
        socket.on('todo:client:delete', function(todo) {
            var id = todo.id;
            delete todo.id;
            r.table('Todo').get(id).delete().run(connection);
        });

        // emit events for changes to todos
        r.table('Todo').changes({ includeInitial: true, squash: true }).run(connection)
        .then(changefeedSocketEvents(socket, 'todo'));
    });
    server.listen(9000);
})
.error(function(error) {
    console.log('Error connecting to RethinkDB!');
    console.log(error);
});

Past the few lines boilerplate Express/Node.js that you’ve probably seen a hundred times, the first new thing you’ll notice is the connection to RethinkDB. The connect() method specifies the ‘3RES_Todo’ database we setup earlier. Once a connection is made, we listen for Socket.io connections from clients, and then tell Express to listen to whatever port we’d like. The connection event in turn provides the socket we emit events from.

Now that we have a RethinkDB connection, and a Socket to a client, let’s setup the changefeed query on the RethinkDB ‘Todo’ table! The changes() method accepts an object literal of properties, which we will make use of two: The includeInitial property tells RethinkDB to send the entire table as the first event, then listens for changes. The squash property will ensure that simultaneuos changes are combined into a single event, in case two users change a Todo at the same instant.
Listening for Socket.io events prior to initiating the RehtinkDB changefeed, allows us to modify the query by user. For example, in a real world application, you probably want to broadcast todos for that specific user session, so you would add the userId into your RethinkDB query. As mentioned before, if you’d like some direction on how to use sessions with Socket.io, I have a complete writeup on my blog.

Next, we register three socket event listeners for client induced events: insert, update, and delete. These events in turn make the necessary RethinkDB queries.

Lastly, you’ll see the changefeed invoke a function we are importing. This function accepts two arguments: the socket reference, and a string of what we want to call these individual rows in our sockets (‘todo’ in this case). Here’s the changefeed handler function that emits Socket.io events:

// socket-events.js

module.exports = function(socket, entityName) {
    return function(rows) {
        rows.each(function(err, row) {
            if (err) { return console.log(err); }
            else if (row.new_val && !row.old_val) {
                socket.emit(entityName + ":insert", row.new_val);
            }
            else if (row.new_val && row.old_val) {
                socket.emit(entityName + ":update", row.new_val);
            }
            else if (row.old_val && !row.new_val) {
                socket.emit(entityName + ":delete", { id: row.old_val.id });
            }
        });
    };
};

As you can see, passing in the socket reference and the entityName, returns a function which accepts the rows cursor from RethinkDB. All RethinkDB cursors have an each() method, which can be used to traverse the cursor row by row. This allows us to analyze the new_val and the old_val of each row, and then by some simple logic, we determine whether each change is an insert, update, or delete event. These event types are then appended to the entityName string, to produce events that map to objects of the entity themselves such as:

'todo:new' => { name: "Make Bed", completed: false, id: ''48fcfafa-a2fa-454c-9ab4-8a5540d07ee0'' }

'todo:update' => { name: "Make Bed", completed: true, id: ''48fcfafa-a2fa-454c-9ab4-8a5540d07ee0'' }

'todo:delete' => { id: ''48fcfafa-a2fa-454c-9ab4-8a5540d07ee0'' }

Finally, to try this out, let’s make an index.html file with some simple JavaScript capable of listening for these events:

<html>
    <head>
        <script src="/socket.io/socket.io.js"></script>
        <script>
            var socket = io.connect('/');
            socket.on('todo:insert', function (data) {
                console.log("NEW");
                console.log(data);
            });
            socket.on('todo:update', function (data) {
                console.log("UPDATE");
                console.log(data);
            });
            socket.on('todo:delete', function (data) {
                console.log("DELETE");
                console.log(data);
            });
        </script>
    </head>
    <body>Checkout the Console!</body>
<html>

Let’s give it a spin! Go to your terminal (assuming you still have RethinkDB running in another tab), and run:

$ node index.js

Open two tabs in Chrome: http://localhost:9000 and http://localhost:8080. In the tab with our simple node app, open your JavaScript console, you’ll notice there is nothing there – because we haven’t added any Todo’s yet! Now open the RethinkDB console in the port 8080 tab in Chrome, navigate to the Data Explorer tab, and run this query:

r.db("3RES_Todo").table("Todo").insert({ name: "Make coffee", completed: false })

Now go back to your other Chrome tab with the Node app. Viola! There’s the todo we just added into the database, clearly identified as a new record. Now try updating the todo, using the id that RethinkDB assigned to your todo:

r.db("3RES_Todo").table("Todo").get("YOUR_TODO_ID").update({ completed: true })

Once again, the change event was recognized as an update, and the new todo object was pushed to our client. Finally, let’s delete the todo:

r.db("3RES_Todo").table("Todo").get("YOUR_TODO_ID").delete()

Our changefeed handler recognized this as a delete event, and returned an object with just the id (so that we can remove it from the array of todos in our Redux state!).

This completes everything required on the backend to push todos and changes in real-time to our front end. Let’s move on to the React/Redux code and how to integrate these socket events with Redux dispatchers.

Basic React Todo App

To begin, let’s setup our front end requirements and bundling with WebPack. First, install the required modules (if you’ve pulled down the repo and run npm install you don’t need to do this):

$ npm install --save react react-dom material-ui react-tap-event-plugin redux react-redux
$ npm install --save-dev webpack babel-loader babel-core babel-preset-es2015 babel-preset-react babel-plugin-transform-class-properties

Now let’s get Webpack setup, our webpack.config.js should also include babel, and the babel transform-class-properties plugin:

var path = require('path');
var webpack = require('webpack');

module.exports = {
    entry: './components/index.jsx',
    output: { path: __dirname + '/public', filename: 'bundle.js' },
    module: {
        loaders: [{
            test: /.jsx?$/,
            loader: 'babel-loader',
            exclude: /node_modules/,
            query: {
                presets: ['es2015', 'react'],
                plugins: ['transform-class-properties']
            }
        }]
    }
}

We’re all set to start building the React/Redux front end app! If you need to brush up on React and/or Redux, the resources mentioned in the introduction will help. Let’s strip out the code we had in index.html to demonstrate how Socket.IO works, add a few fonts in, put an id on an empty div we can attach the React app to, and import the webpack bundle:

<html>
    <head>
        <link href='https://fonts.googleapis.com/css?family=Roboto:400,300,500' rel='stylesheet' type='text/css'>
        <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.5.0/css/font-awesome.min.css">
        <script src="/socket.io/socket.io.js"></script>
    </head>
    <body style="margin: 0px;">
        <div id="main"></div>
        <script src="bundle.js"></script>
    </body>
<html>

Let’s put all of our React rendering and some other setup in components/index.js:

import React from 'react';
import ReactDOM from 'react-dom';
import { Provider } from 'react-redux';
import store from '../stores/todos.js';

import App from './app.jsx';

// Setup our socket events to dispatch
import TodoSocketListeners from '../socket-listeners/todos.js';
TodoSocketListeners(store);

// Needed for Material-UI
import injectTapEventPlugin from 'react-tap-event-plugin';
injectTapEventPlugin();

// Render our react app!
ReactDOM.render(<Provider store={store} ><App /></Provider>, document.getElementById('main'));

Notice we have to import an annoying tap event listener for Material-UI (looks like they’re working on removing this requirement). After importing the root App component, we import a socket event listener which dispatches Redux actions, in /socket-listeners/todos.js:

// socket-listeners/todos.js
import io from 'socket.io-client';
const socket = io.connect('/');

export default function(store) {
    socket.on('todo:insert', (todo) => {
        store.dispatch({
            type: 'todo:insert',
            todo: todo
        });
    });

    socket.on('todo:update', function (todo) {
        store.dispatch({
            type: 'todo:update',
            todo: todo
        });
    });

    socket.on('todo:delete', function (todo) {
        store.dispatch({
            type: 'todo:delete',
            todo: todo
        });
    });
}

This function is fairly straightforward. All we’re doing is listening to the socket events emited from the backend socket-events.js. Then dispatching the inserted, updated, or deleted todo, which are in turn triggered by RethinkDB changefeeds. This ties together all the RehtinkDB/Socket magic!

And now let’s build the React components that makeup the app. As imported in components/index.jsx, let’s make components/app.jsx:

import React from 'react';
import AppBar from 'material-ui/lib/app-bar';
import TodoList from './todoList.jsx';
import AddTodo from './addTodo.jsx';

import { connect } from 'react-redux';

class Main extends React.Component {
    render() {
        return (<div>
            <AppBar title="3RES Todo" iconClassNameRight="muidocs-icon-navigation-expand-more" />
            <TodoList todos={this.props.todos} />
            <AddTodo />
        </div>);
    }
}

function mapStateToProps(todos) {
    return { todos };
}

export default connect(mapStateToProps)(Main);

This is all boilerplate React and React-Redux. We import connect from react-redux, and map the state to the props for the TodoList component, which is components/todoList.jsx:

import React from 'react';
import Table from 'material-ui/lib/table/table';
import TableBody from 'material-ui/lib/table/table-body';
import Todo from './todo.jsx';

export default class TodoList extends React.Component {
    render() {
        return (<Table>
            <TableBody>
                {this.props.todos.map(todo => <Todo key={todo.id} todo={todo} /> )}
            </TableBody>
        </Table>);
    }
}

The todo list is made up from a Material-UI table, and we’re simply mapping the todos from the props, to an individual Todo component:

import React from 'react';
import TableRow from 'material-ui/lib/table/table-row';
import TableRowColumn from 'material-ui/lib/table/table-row-column';
import Checkbox from 'material-ui/lib/checkbox';
import IconButton from 'material-ui/lib/icon-button';

// Import socket and connect
import io from 'socket.io-client';
const socket = io.connect('/');

export default class Todo extends React.Component {
    handleCheck(todo) {
        socket.emit('todo:client:update', {
            completed: !todo.completed,
            id: todo.id
        });
    };

    handleDelete(todo) {
        socket.emit('todo:client:delete', todo);
    };

    render() {
        return (<TableRow>
            <TableRowColumn>
                <Checkbox label={this.props.todo.name} checked={this.props.todo.completed} onCheck={this.handleCheck.bind(this, this.props.todo)} />
            </TableRowColumn>
            <TableRowColumn>
                <IconButton iconClassName="fa fa-trash" onFocus={this.handleDelete.bind(this, this.props.todo)} />
            </TableRowColumn>
        </TableRow>)
    }
}

The individual Todo component attaches emitters for the Socket.IO events to the proper UI events for the checkbox and the delete button. This emits the updated or deleted todo to the Socket event listeners in the server.

The last React component we need is a button to add todos! We’ll attach a hovering add button to the bottom right corner of the app:

import React from 'react';
import Popover from 'material-ui/lib/popover/popover';
import FloatingActionButton from 'material-ui/lib/floating-action-button';
import ContentAdd from 'material-ui/lib/svg-icons/content/add';
import RaisedButton from 'material-ui/lib/raised-button';
import TextField from 'material-ui/lib/text-field';

// Import socket and connect
import io from 'socket.io-client';
const socket = io.connect('/');

export default class AddTodo extends React.Component {
    constructor(props) {
        super(props);
        this.state = { open: false };
    };

    handlePopoverTap = (event) => {
        this.setState({
            open: true,
            anchor: event.currentTarget
        });
    };

    handlePopoverClose = () => {
        this.setState({ open: false });
    };

    handleNewTaskInput = (event) => {
        if (event.keyCode === 13) {
            if (event.target.value && event.target.value.length > 0) {

                // Emit socket event for new todo
                socket.emit('todo:client:insert', {
                    completed: false,
                    name: event.target.value
                });

                this.handlePopoverClose();
            }
            else {
                this.setState({ error: 'Tasks must have a name'});
            }
        }
    };

    render() {
        return (<div>
            <Popover
                open = { this.state.open }
                anchorEl = { this.state.anchor }
                anchorOrigin={{ horizontal: 'right', vertical: 'top' }}
                targetOrigin={{ horizontal: 'left', vertical: 'bottom' }}
                onRequestClose={this.handlePopoverClose}>
                <TextField
                    style={{ margin: 20 }}
                    hintText="new task"
                    errorText={ this.state.error }
                    onKeyDown={this.handleNewTaskInput} />
            </Popover>
            <FloatingActionButton onTouchTap={this.handlePopoverTap} style={{ position: 'fixed', bottom: 20, right: 20 }}>
                <ContentAdd />
            </FloatingActionButton>
        </div>)
    };
}

The render method of this component includes the add button, which then shows a popover with an input field. The popover is hidden and shown based on the boolean state.open. With every keypress of the input, we invoke handleNewTaskInput, which listens for keycode 13 (the enter key). If the input field is empty, an error is shown (improvement note: it would be good to validate this on the backend). If the input field isn’t empty, we emit the new todo and close the popover.

Now, we just need a bit more boilerplate Redux to tie all of this together. First, a reducer for the todos, and combine them (planning ahead for when we build out this app and have multiple reducers):

// reducers/todos.js

// todos reducer
const todos = (state = [], action) => {
    // return index of action's todo within state
    const todoIndex = () => {
        return state.findIndex(thisTodo => {
            return thisTodo && thisTodo.id === action.todo.id;
        });
    };

    switch(action.type) {
        case 'todo:insert':
            // append todo at end if not already found in state
            return todoIndex() < 0 ? [...state, action.todo] : state;

        case 'todo:update':
            // Merge props to update todo if matching id
            var index = todoIndex();
            if (index > -1) {
                var updatedTodo = Object.assign({}, state[index], action.todo);
                return [...state.slice(0, index), updatedTodo, ...state.slice(index + 1)]
            }
            else {
                return state;
            }

        case 'todo:delete':
            // remove matching todo
            var index = todoIndex();
            if (index > -1) {
                return [...state.slice(0, index), ...state.slice(index + 1)];
            }
            else {
                return state;
            }

        default:
            return state;
    }
};

export default todos;

And to combine the reducers:

// reducers/index.js

import { combineReducers } from 'redux';
import todos from './todos.js';

const todoApp = combineReducers({ todos });

export default todoApp;

The reducers have a utility function for checking if the todo already exists in the state (you’ll notice that if you leave the browser window open, and restart the server, socket.IO will emit all of the events to the client again). Updating a todo makes use of Object.assign() to return a new object with the updated properties of the todo. Lastly, delete makes use of slice() – which returns a new array, unlike splice().

The actions for these reducers:

// actions/index.js

// Socket triggered actions
// These map to socket-events.js on the server
export const newTodo = (todo) => {
    return {
        type: 'todo:new',
        todo: todo
    }
}

export const updateTodo = (todo) => {
    return {
        type: 'todo:update',
        todo: todo
    }
}

export const deleteTodo = (todo) => {
    return {
        type: 'todo:delete',
        todo: todo
    }
}

Let’s put this all together and build it with webpack!

$ webpack --progress --colors --watch

Our final product is a beautiful, and simple todo app which is reactive to all state changes for all clients. Open two browser windows side-by-side then try adding, checking off, and deleting todos. This is a very simple example of how I’ve tied together RethinkDB changefeeds, Socket.IO, and Redux state, and is really just scratching the surface of what is possible. Authentication and sessions would really make this a really awesome webapp. I could imagine a shareable todo list for user groups such as households, partners, etc. complete with an event feed of who’s completing each task that instantly updates to all users that are subscribed to receive each specific group of todos.

Going forward, I plan to do more work on finding a more general way of tying together any array of objects within a Redux state that requires less boilerplate – a way of connecting a state array to a Socket.IO endpoint similar to React-Redux’s connect(). I’d love to hear feedback from anyone who’s done this, or plans to implement these awesome technologies together in the same stack!

—

Scott Hasbrouck

Bio: Scott is a lifelong software engineer, that loves to share his skills with others through writing and mentorship. As a serial entrepreneur, he is currently the CTO of ConvoyNow, one of three companies he has started as a technical founder, bootstrapping one to over one million users. He’s always seeking the next adventure through hiking remote places, flying small airplanes, and traveling.

Convoy is an in-home tech support solution! We match customers that are having issues how to fix or use their devices, with friendly and knowledgeable tech support professionals.