Decreasing 64-bit Tweet ID in JavaScript

JavaScript is only able to handle integers up to 53-bit in size, here is a script to decrease tweet ID which is a 64-bit number in JavaScript without libraries or recursion, to use with max_id or since_id in Twitter API

As some of you might know, JavaScript is only able to handle integers up to 53-bit in size. This post, Working with large integers in JavaScript (which is a part of Numbers series) does a great job at explaining general concepts on dealing with large numbers in JS.

64-bit Tweet ID is "rounded" in JS
64-bit Tweet ID is “rounded” in JS

I had to do some research on the topic when I was re-writing some JavaScript code responsible for handling Twitter search in Storify editor: we had tweet duplicates in results! In this article, Working with Timelines, Twitter official documentation says:

Environments where a Tweet ID cannot be represented as an integer with 64 bits of precision (such as JavaScript) should skip this step.

So true, because id and id_str fields in a Twitter API response were different. Apparently, JavaScript engine just “rounds” inappropriately large numbers. :-( The task was complicated by the fact that I needed to subtract 1 from the last tweet’s ID to prevent its reappearance in a second search response. After the subtraction I could have easily passed the value to max_id parameter of Twitter API.

I’ve come across different solutions, but decided to write my own function which is simple to understand and not heavy on resources. Here is a script to decrease tweet ID which is a 64-bit number in JavaScript without libraries or recursion, to use with max_id or since_id in Twitter API:

function decStrNum (n) {
    n = n.toString();
    var result=n;
    var i=n.length-1;
    while (i>-1) {
      if (n[i]==="0") {
        result=result.substring(0,i)+"9"+result.substring(i+1);
        i --;
      }
      else {
        result=result.substring(0,i)+(parseInt(n[i],10)-1).toString()+result.substring(i+1);
        return result;
      }
    }
    return result;
}

To check if it works, you can run these logs:

console.log("290904187124985850");
console.log(decStrNum("290904187124985850"));
console.log("290904187124985851");
console.log(decStrNum("290904187124985851"));
console.log("290904187124985800");
console.log(decStrNum("290904187124985800"));
console.log("000000000000000001");
console.log(decStrNum("0000000000000000001"));

Alternative solution which I’ve found in a StackOverflow question was suggested by Bob Lauer, but it involves recursion and IMHO is more complicated:

function decrementHugeNumberBy1(n) {
    // make sure s is a string, as we can't do math on numbers over a certain size
    n = n.toString();
    var allButLast = n.substr(0, n.length - 1);
    var lastNumber = n.substr(n.length - 1);

    if (lastNumber === "0") {
        return decrementHugeNumberBy1(allButLast) + "9";
    }
    else {      
        var finalResult = allButLast + (parseInt(lastNumber, 10) - 1).toString();
        return trimLeft(finalResult, "0");
    }
}

function trimLeft(s, c) {
    var i = 0;
    while (i < s.length && s[i] === c) {
        i++;
    }

    return s.substring(i);
}

Now, if you’re the type of person who likes to shoot sparrows with a howitzer, there are full-blown libraries to handle operations on large numbers in JavaScript; just to name a few: BigInteger, js-numbers and javascript-bignum.

Wintersmith — Node.js static site generator

This past weekend was a very productive one for me, because I’ve started to work on and released my book’s one-page website —rapidprototypingwithjs.com. I’ve used Wintersmith to learn something new and to ship fast. Wintersmith is a Node.js static site generator. It greatly impressed me with flexibility and ease of development. In addition I could stick to my favorite tools such as Markdown, Jade and Underscore.

This past weekend was a very productive one for me, because I’ve started to work on and released my book’s one-page website —rapidprototypingwithjs.com. I’ve used Wintersmith to learn something new and to ship fast. Wintersmith is a Node.js static site generator. It greatly impressed me with flexibility and ease of development. In addition I could stick to my favorite tools such as Markdown, Jade and Underscore.

Wintersmith is a Node.js static site generator

Why Static Site Generators

Here is a good article on why using a static site generator is a good idea in general, An Introduction to Static Site Generators. It basically boils down to a few main things:

Templates

You can use template engine such as Jade. Jade uses whitespaces to structure nested elements and its syntax is similar to Ruby on Rail’s Haml markup.

Markdown

I’ve copied markdown text from my book’s Introduction chapter and used it without any modifications. Wintersmith comes with marked parser by default. More on why Markdown is great in my old post, Markdown Goodness.

Simple Deployment

Everything is HTML, CSS and JavaScript so you just upload the files with FTP client, e.g., Transmit by Panic or Cyberduck.

Basic Hosting

Due to the fact that any static web server will work well, there is no need for Heroku or Nodejitsu PaaS solutions, or even PHP/MySQL hosting.

Performance

There are no database calls, no server-side API calls, no CPU/RAM overhead.

Flexibility

Wintersmith allows for different plugins for contents and templates and you can even write you own plugin.

Getting Started with Wintersmith

There is a quick getting started guide on github.com/jnordberg/wintersmith.

To install Wintersmith globally, run NPM with -g and sudo:

$ sudo npm install wintersmith -g

Then run to use default blog template:

$ wintersmith new <path>

or for empty site:

$ wintersmith new <path> -template basic

or use a shortcut:

$ wintersmith new <path> -T basic

Similar to Ruby on Rails scaffolding Wintersmith will generate a basic skeleton with contents and templates folders. To preview a website, run these commands:

$ cd <path>
$ wintersmith preview
$ open http://localhost:8080

Most of the changes will be updates automatically in the preview mode except for the config.json file.

Images, CSS, JavaScript and other files go into contents folder.
Wintersmith generator has the following logic:

  1. looks for *.md files in contents folder,
  2. reads metadata such as template name,
  3. processes *.jade templates per metadate in *.md files.

When you’re done with your static site, just run:

$ wintersmith build

Other Static Site Generators

Here are some of the other Node.js static site generators:

More detailed overview of these static site generators is available in the post, Node Based Static Site Generators.

For other languages and frameworks like Rails and PHP take a look at Static Site Generators by GitHub Watcher Count and the “mother of all site generator lists”.

Most Common Technical Questions for Early-Stage Startups

A lot of people like to pick my brain on technical things related to early stage startups, e.g., what framework to use, how long will it take to build an app or a website, should I hire or outsource. I decided to organize my answers to the most common technical questions in this post:

A lot of people like to pick my brain on technical things related to early stage startups, e.g., what framework to use, how long will it take to build an app or a website, should I hire or outsource. I decided to organize my answers to the most common technical questions in this post:

  • Don’t outsource your core product. Outsourcing to a digital agency is the best way to spend your money fast and usually fruitlessly.
  • Use frameworks, don’t write trivial things like URL parsers from scratch.
  • Use available solutions to your none-core components. Things like: CMS, blog, analytics, landing page, forum, version-control, bug-tracking/customer feedback, project management, etc.
  • Use PaaS or IaaS. Don’t use IaaS unless you really need to, for example when PaaS becomes cost prohibitive;
  • Make mobile your first initiative, if and when it makes sense for your business; don’t even build a website if mobile app is your bread and butter!
  • Code snippet: Learn Programming
    Code Snippet: Learn Programming
  • Use social connect APIs for smooth user on-boarding, most of your early adopter probably have at least Facebook or Twitter accounts; don’t make them type their emails and remember passwords.
  • Use the most appropriate programming language don’t rely on your geeky “guru” friend’s advice that programming language is a matter of personal preference, to go with PHP, or that Ruby on Rails and Node.js are just a fads.
  • Don’t over-rely on remote developers. It often leads to miscommunication; your teams will work longer, will have to spend more time and money; also, not everybody is disciplined well enough to work remotely without social interaction and direct supervision.
  • Don’t over-rely on part-time developers. It will take longer to execute, the code will be less efficient due to distractions and as a result harder to maintain and more expensive to run in the future.
  • Learn programming. Business co-founders without any programming knowledge will have a hard time understanding trade-off in features and available resources; expect longer sprint meetings and leap of trust between technical and business people.
  • Don’t have more that one language for your core-product on a prototype stage because code is not an asset, but a liability and maintaining fragmented codebase could lead to a disaster.
  • Start from scratch for your core-product or refactor your code often if you or your team are still learning a language or a framework; the longer you keep old spaghetti code the harder it will be to come back to fix it later.
  • Use Test-Driven Development and Pair-Programming.
  • Use peers and mentors help/feedback from fellow founders, programmer friends on meet-ups and at the office.
  • Involve developers in customer development process. Tech people just love to build things for the sake of building something complicated. It’s so hard to practice Lean Startup methodology, because you need a lot of self-discipline. And it’s just so tempting to go to your basement for 6 months, into your hacker/coding nirvana mode, instead of doing endless customer interview, landing pages, mock-ups, paper prototypes and face website! The problem is that you’ll usually end up with another useless app or website. I had similar a phat startup experience with what later became open-sourced project http://openList.co.

Best of the web goodies for agile web development, startups and Lean Startup startups:

If you like this post and want to lean more about building your web or mobile app using the latest tech and agile practices check out my new book on how to take your idea to a prototype with JavaScript, Node.js and MongoDB — Rapid Prototyping with JS.