Unit Testing Strategies in Node.js

In the world of JavaScript development, it’s easy to get lost and do things wrong, especially when it comes to unit testing. JavaScript is a powerful language, but it is also one where it is incredibly easy to do things the wrong way, and the utmost care must be taken to ensure you don’t end up with a situation where you have no hope to correct yourself. In this article, I’ll explain my experiences with unit testing in Node.js, specifically for a web application using express and MongoDB via mongoose, what I’ve learned and what I believe is the “right” philosophy of how to organize your code to make it testable.

When you’re designing a web application like twitter where you have a bunch of users sending public broadcasts and you’re doing this using express/mongoose. You’re probably going to have a structure somewhat like this:

 | 
 +-- index.js (require()'s express, defines routes, maps them to ./controller.js exports)
 | 
 +-- controller.js (require()'s the models in db folder, has the route callbacks of the form function(req,res))
 | 
 +-- db
      |
      +-- user.js (require()'s mongoose and contains the connection + model for your user)
      |
      +-- message.js (require()'s mongoose and contains the connection + model for your messages)

Now here’s the problem; with this kind of set up, if you want to unit test that a route in index.js does what it needs to do; say it’s the route that creates and broadcasts a message, you can do one of two things:

  1. Use something like supertest to spin up a “simulation” (kinda) of your web server then pass it simulated HTTP requests and wait for a response from the server, and compare that response
  2. Grab simply the controller.js ignoring index.js, and pass it a req/res object, maybe even a next() callback, and verify that what we want is happening

In both cases, ultimately the data will be read/written to the actual database via the database models because index.js require()‘s controller.js and controller.js require()‘s the db models. There’s no way in your unit test, no matter how you write it, to stop controller.js from using the real database, because the code to require the database is inside the controller. You could try to make some sort of if(unittest) { ... do this ... } else { ... do that ... } but even if you could pull it off, why would you want to? It’s messy and error-prone.

Let’s back up here a bit, why don’t we want the unit tests to actually connect to the database? I mean what’s the problem – if the database is on a development machine we can easily inject the data we want into it, clear it before and after every test, etc. Well, the problem is that this isn’t really a unit test anymore – it’s an integration test. This may seem like a “pfft, big deal” distinction but it’s not. Unit tests and integration tests are fundamentally different because a unit test should only test one unit of something at a time. When we test controllers, we should be only testing controllers and not any kind of middleware. When we test the database model, we shouldn’t be testing mongoose itself and its connection to the database in the process because this is not following the best practice of separation of concerns – maybe your actual application is following it, but in this instance your tests are not – they are testing many things at once. Don’t get me wrong, integration tests are important, but they’re just not unit tests and frankly I’d place higher importance on unit tests than integration tests since they are more reliable in telling you what is actually broken when changes are made to your code. If I change my models and how they work, I’d expect my model tests to break but my controller and middleware tests to most likely continue passing. In an integration test, all of them would probably break and that doesn’t help me truly narrow down “what is actually broken” so much as just telling me “something is broken.”

So what’s the solution? Well, I have a fairly decent C# background and one of the things I love about C# is how easy and awesome it is to do unit testing of things in C# when you use dependency injection. If you use Microsoft’s Unity for dependency injection and combine that with Moq for mocking out your services, it is just a developer’s dream (at least this one’s). The way the Angular.js guys defined their framework for unit testing is I think definitely a fantastic leap forward for how unit testing should be done in JavaScript but what they’re doing doesn’t quite work for Node.js. Why? Because we don’t actually have ‘dependency injection’ per se in Node.js. There isn’t really an [easy, scalable, non-hairy] way to make a module require() something different depending on what the parent tells it to do. Therefore, to tackle this problem we have to think a little bit outside the box.

Now you’re thinking with dependency injection

Disclaimer: I’m not saying this solution is a magic bullet for all situations, nor that it may even fit your application’s design, but it is what I have found works great for me after making multiple node.js projects.

The way we have to structure our application to make it truly unit-testable is closer to this:

 | 
 +-- index.js (require()'s express, defines routes, maps them to ./controller.js exports)
 | 
 +-- controller.js (require()'s nothing)
 | 
 +-- db.js (require()'s mongoose and exports the models after connecting to the real db)
 | 
 +-- models
      |
      +-- user.js (require()'s mongoose and contains only the model for user)
      |
      +-- message.js (require()'s mongoose and contains only the model for messages)

Some key takeaways here:

  1. controller.js doesn’t directly require() anything at all (doesn’t have to be the case but certainly it does not require anything that we want to mock out in unit tests, like the database)
  2. db.js is broken out separately from the models. The model files do not contain any kind of database connection information, they are just models. This allows us to use them and test with them without actually connecting to a real database
  3. The db.js is almost nothing – it is probably a few lines of code, simply requiring mongoose and the models, connecting to the db, and exporting the models.

How does it work? Your controller needs access to the database, because it needs to pull messages and users out of it, but it doesn’t require() the database file. What does? the index.js file. Why do we allow index.js to require the database but not the controller.js? Because we don’t intend to test index.js in any way – all index.js does is define the routes in express and map them to controller functions; that is, it doesn’t really “do” anything in terms of logic to test. Testing index.js would be like testing express, which already has its own unit tests that its developers made. What we really want to test is the controller.js file. To get the controller access to the database, we pass it from index.js to controller.js by making controller.js not export the callback functions, but export only a single function (like express does) which we call from index.js and it returns an object with the controller callbacks. Let’s take a look:

index.js

'use strict';
var express = require('express')
  , app = express()
  , db = require('./db')
  , controller = require('./controller')(db);

app.get('/someroute', controller.someRouteHandler);
app.post('/foo', controller.fooHandler);
app.post('/bar', controller.barHandler);

app.listen(3000 || env.PORT);

controller.js

'use strict';
module.exports = function(db) {
  var someRouteHandler = function(req, res) {
    // do stuff with the database
    db.message.findOne({id:req.params.id}, function(err, message) {
      res.send(message);
    });
  };
  var fooHandler = function(req, res) {
    // do stuff
    res.send('foo');
  };
  var barHandler = function(req, res) {
    // do stuff
    res.send('bar');
  };
  return {
    someRouteHandler: someRouteHandler,
    fooHandler: fooHandler,
    barHandler: barHandler
  };
});

So how does this help us at all? Well, now we can mock out everything and run a test of only the controller function. Here’s how it would look:

'use strict';
var assert = require('assert')
  , message = require('./models/message')
  , controllerToTest = require('./controller.js');
  
describe('Controller', function() {
  describe('someRouteHandler', function() {
    it('returns the model', function(done) {
      // define the mock DB, which should export the models
      var mockDb = {
        message: message
      };
      // override the findOne of message model to find this doc with no error
      mockDb.message.findOne = function(search, callback) {
        callback(null,{id:'123', body:'Hello, world!'});
      };
      // require our controller, pass it the mock db instead of the real one
      var controller = require('./controller')(mockDb);
      // we provide the ID to look up, as express would
      var req = {
        params: {
          id: '123'
        }
      };
      // we provide the response object which the controller uses
      var res = {
        send: function(data) {
          assert.equal('Hello, world!', data.body, 'Message must match')
          // test is done, call done() so it doesn't fail due to timeout
          done();
        }
      };
      controllerToTest.someRouteHandler(req,res); // call the function to be tested
    });
  });
});

Using this method, we never have to touch the real database, we test exactly the unit we’d like to test (and nothing more), and we can test situations that would be hard to simulate like a database failure. I hope this article helps you get your project’s unit tests off the ground; good luck!

5 Comments

bobjandal says:

Hey did you leave out the actual function call to someRouteHandler in the unit test by accident ?
Very good strategy, thank you.

* Petro says:

Yes it seems I did forget it, sorry! Thanks for catching that, I will make sure to fix it. Glad you enjoyed it.

Richard says:

Hi!

Great article, I come from a similar background(c#, injecting dependencies).

Question – why not override the db require with sinon stub? I am more comfortable with your style of doing this but
I was wondering if you had considered this?

Thanks

* Petro says:

Hi, glad to hear you liked it! I have not heard of sinon stub so that’s why I didn’t mention it 🙂 I’ll look into that, thanks.

Mike says:

Hi!
Really nice article and I like this way of testing server.

But overriding db methods with constant returning results looks a little bit strange to me. Would it be enough just to verify arguments which we provide to db methods? In that case sinon could be better solution.

P.S. Please fix your comment form) When I start typing grey rectangle with ‘comment’ title covers half of input block.

Leave a Reply

Your email address will not be published.