This talk, as delivered at DevReach in Sofia, Bulgaria, is the culmination of years of practicing to write beautiful JavaScript tests that provide just the right feedback to prompt me to take actions which dramatically improve the design of my code. As a result of my singular focus, my approach is very different than what you’d find searching for a tutorial or a README about JavaScript testing, TDD, or unit testing in general.
It would make my day if you’d watch it with an open mind and thoughtfully consider what about the approach I’ve arrived at might prove valuable in your own practice.
Here are the slides, hosted on SpeakerDeck:
Finally, you might have noticed that the slide deck was written in Markdown (as a wink to the fact that the talk was about advice that you wouldn't find in a README). So, naturally, I've decided to post the entire talk's text as a README to Github.
Transcript of the talk
[00:00:00] Hi everybody, I'm here to talk about JavaScript testing tactics. An alternate title might be how my JavaScript tests differ from the readme. It's not stuff that you'd necessarily see if you were just to download a framework or read a simple tutorial. This is just based on several years of, in the wild actually trying to do test driven development in JavaScript.
[00:00:19] My name is Justin. I'd love if you'd follow me or tweet at me. My, my Twitter handle is my last name Searls. And I come from a software agency from Ohio. Thanks for hosting us. And if you're interested in working for a company that cares a lot about, crafted software, please shoot us an email.
[00:00:33] Hello at test double we're hiring. Or if you work for a company that needs help and you'd like us to help accelerate your team on this kind of stuff, please contact us. We're always looking for new clients. A little bit of background. First of all, this is only a 20 minute slot, which makes this, as far as I care, a lightning talk.
[00:00:47] But it means that we don't have time to talk about the purpose of each different type of test, nor do we have time to really discuss anything about integration tests. We're not going to talk about frameworks and how they compete with TDD. But we are going to talk about a handful of situational tactics.
[00:00:59] And I'm going to use Jasmine as the test framework throughout all of these examples. But my hope is that all the lessons are generally applicable ish, but your mileage may vary. Little note on syntax. First of all, I'm going to talk about what I don't do. I don't actually use, if you're familiar with the Jasmine framework, I don't use the domain specific language provided by Jasmine, which is inherited from RSpec.
[00:01:18] Just if you're not familiar, it looks a little like this. I might describe an object called math with a subject and then a result variable defined. In a beforeEach, I might instantiate the thing that I'm putting under test. And then I might describe some method like add. And here I'd take an action in another beforeEach.
[00:01:33] And finally, I'd have an it block that would specify the result I expect. With an expectation statement like that. I, I don't do that, and I also, I don't write my specs in Javascript. And I know that you probably think I'm pulling your leg, but I'm actually serious, and I'm gonna explain that in a second.
[00:01:48] So what's the problem? The problem with the Jasmine DSL is it's not terrifically obvious how to use it idiomatically. You get a lot of constructs, and they're all good stuff. You can describe before each, after each. It does stuff, an expectation library with all these custom matchers, and you can add your own.
[00:02:00] It has a spy API that's a really terrific test double library. But it's not obvious how to order stuff or what the idiomatic way to nest things is. And I was looking for an alternative that would be better. Also the test code tends to become verbose and unwieldy, especially if you're doing characterization testing of deeply nested behavior.
[00:02:17] What I mean is this. And this really drives me crazy, writing all of these what I call crying mustaches at the bottom of my file listings. And I'd love to get rid of those. So what I do instead is I write my specs in CoffeeScript, and I realize that's a contentious thing to say but I find a lot of joy in CoffeeScript, and I think that JavaScript is missing out on a lot of joy generally, so I'll take it where I can get it.
[00:02:37] What that gets me, if you think about this test, there's only three or four lines that really matter here, but if I convert it to CoffeeScript, it's really obvious which three or four lines that is. It gets rid of a lot of the redundant noise in my listing. So some CoffeeScript basics if you're a noob.
[00:02:49] Fir first of all, fear not, it's just plain old JavaScript except for reducing a lot of the visual clutter. First of all, there's no variable keyword functions. You don't use the function keyboard keyword. No more curly braces, no semicolons, and returns are implicit. So an add function like that just looks like that.
[00:03:03] It's much smaller. Also This or this dot can be aliased to the at symbol. So you can say at save instead of this dot save. If you've ever done something like this in JavaScript, where you don't trust the context object getting stolen away from you from some callback function by memoizing it you can actually re lexify the context variable with a fat rocket symbol.
[00:03:21] So that'll make this dot display from whatever was above the save function, continue to work. Also I use this thing called the given domain specific language. What I mean by that is, if you look at the before each, and then the before each, and then the it, what those are really mapping to is the arrange step, the act step, and the assert step of my test.
[00:03:38] But before each is redundantly repeated twice, because I only have the one construct to talk about. And so what I'd much rather do is just use something a terrifically obvious API that everybody knows and understands and that's given, when, then. So given I have some setup, when I take some action, then here's my result.
[00:03:54] And so if you compare to that original listing how verbose vanilla Jasmine looks like, I don't see any reason why not you just jump straight into this. So you can have much shorter, briefer, terser specs that really resonate what their intent is. So Jasmine Given is a library we maintain that provides that DSL.
[00:04:09] It's a port of RSpec Given. RSpec Given was written by Jim Wyrick. I Yeah, absolutely. Woo! Jim was a fantastic teacher and mentor. I was very fortunate to be able to pair with him on RSpec Given a little bit. Which sold me on the concept. And then I ported it to Jasmine excitedly.
[00:04:26] And I haven't looked back. There's also, if you're already a Mocha user, I don't know if there are any Mocha users in the crowd a nice gentleman on the Internet ported Jasmine Given to Mocha quite recently, but you can go ahead and use, start using it today. Let's talk about test runners.
[00:04:39] So a couple things that I don't do to actually run my tests is I don't use the default plain HTML test runner that you get if you just download the framework. In that case, you have an index HTML file and every time you add a file, you're adding a custom script tag. It just, it doesn't scale well to full size applications, obviously.
[00:04:53] I also don't use like server side plugins like Jasmine Maven plugin or Jasmine Rails to integrate with my Rails application or any other server side dependent plugin. Now I'm picking on those two because I wrote them so now I'm saddled with the responsibility of maintaining them even though I don't use them anymore but my recommendation would be instead first talking about problems.
[00:05:10] The problem with those ones, the server side dependent ones, is that the feedback typically isn't fast enough and the tools tend to suck because they're written in languages or in server side environments. Where JavaScript is a second class language. Especially front end JavaScript. There's also an additional friction of server side coupling, like the last thing I want to do is make my server side even more monolithic, my application more monolithic by coupling all of my front end test and build behavior back to the back end so instead what I do is I use Testum, which is a Node.
[00:05:36] js based runner that's very excellent very extensible very pluggable, written by Toby Ho and then I use Lineman, which is a Node. js front end only asset build tool that we maintain at TestDouble that's optimized for conventions default configurations that are sensible and developer happiness.
[00:05:53] Also my goal is that I want to be able to, on every single file change, run all of my tests in under 300 milliseconds. And here's a little bit of what that looks like. So here I might be writing a test to add two functions er, excuse me, write an add method to add two numbers. So here's a real simple specification.
[00:06:10] You can see it's already failing. So I'm going to change the message by creating the function. It takes a couple parameters. I'll just add A plus B. Returns implicitly. Got a passing test. I'll add another test case just to be sure. Three and four. Yes, there's seven. And now I've got passing tests. So that's the kind of workflow I like to work in.
[00:06:25] Now I wouldn't actually normally show the browser. I would just work with the command line there in the interactive interface that Testum provides. Let's talk about AJAX and UI events. Some of the things that I don't do when I'm testing like XHRs is I don't start a fake server that can then stub and verify that the XHRs are taking place because I'm usually writing unit tests with Jasmine and that's like a little too integrated.
[00:06:47] I don't monkey patch the browser's XHR facilities either and I finally I tend not to invoke the code that's under test by say triggering a click event on something in the DOM. The problem with all of those things is that they're just too integrated for unit tests. And now that I've used the unit word, I want to point out this little graph that I made.
[00:07:04] This is a testing pyramid. Test test pyramids are often used to talk about how integrated tests are. The two test suites that I like in any given application are the very top At the very top is a totally integrated test. It's testing a real instance of your application under the most realistic terms possible.
[00:07:19] And it's using your application like a real user. Meaning it doesn't have any awareness that you're using Ember or AngularJS. And none of that coupling. So that way if your implementation changes it continues to work. The very bottom, I call these discovery tests, but these are unit tests that are written expressly for the purpose not of getting regression safety, but of discovering the new objects and reducing big problems into small ones.
[00:07:38] Everything in the middle there is probably every test you've ever seen in a real production application. It's terrible because it's the worst of both worlds. You don't get the regression safety because if you change the implementation and you happen to be somewhat coupled to the framework all your tests will break.
[00:07:52] And then lastly, you can't actually get a lot of discovery feedback from it because the way that most roughly quasi integrated tests are written you it doesn't provide the same pressure to write better APIs. Also, the test pane, when you're mocking at a more integrated level the test pane is less actionable.
[00:08:09] If you mock a thing that you don't own and it's a really cumbersome test to write you don't have any recourse, it's something you don't own, so you can't make it easier to mock or easier to test if you don't own it because you can't change it. And all of these topics, all of these approaches really raise concerns that are better handled by more integrated tests.
[00:08:27] If I'm talking about fake servers and stuff, I want to consider, full API responses, which might be overkill for a particular unit. And finally, they don't provide any pressure to actually improve my private APIs. If I'm testing in a really integrated level, even in my unit tests, I don't really learn about what the usability is like of my particular objects.
[00:08:44] And in JavaScript, they could just be anonymous. I could, I might have no function names at all, which I don't think is a really good style long term. So what I do instead is I tend to wrap native and third party libraries with objects that I do own, and then in my tests I mock those wrappers away.
[00:08:59] That way, if I ever experience test pain, I can just improve the wrapper API, make it easier to use that wrapper, and then nestle some of the annoying drudgery that I have to do with that third party API inside of the wrapper. Wrappers specify the way that we depend on third party libraries, so if you like, you imagine you were isolating yourself from jQuery or something, you could look at all the ways that you're using it, and then if you chose to maybe consider Zepto or some jQuery alternative, you have one file, one point in your system, instead of it leaking all over the place, where you can identify, oh, these are the functions that we use, and you can swap out dependencies way more easily this way.
[00:09:29] And I, finally, I don't typically test wrappers, cause I, like I said, I don't own them, so the unit test feedback isn't gonna tell me that's useful or actionable. And integration testing also sounds inappropriate because most third party libraries probably have their own tests anyway. So an example of this wrap a third party approach, imagine I'm making a new model called a cat box.
[00:09:48] So I instantiate it. Literally a cat box. I'm going to talk about adding a cat to it. So given that I have a cat named Chairman Meow, When I call add on my cat box with the cat, then I expect something, some facility for saving HTTP to have been called with a route and the cat, which means I have to spy on something called app HTTP save to create a fake so that I can observe it.
[00:10:13] So I'm trying to change the message here to get some feedback from my system. I start by creating a a constructor for the cat and then the add method. Now it's complaining it can't spy on something that doesn't exist. So I simply start by delegating, because there's nothing wrong with POST. And then finally, my implementation is just to call my new wrapper with the route and the cat, and then I'm passing.
[00:10:32] Talk a little bit about asynchronous code. First of all, what I don't do is I don't write asynchronous tests of my asynchronous code. And again, I'm talking about unit tests. problem with this is that it yields execution control. A test ideally is a script of this happens, then this happens.
[00:10:48] It's very deterministic. And if you yield execution control, you lose that flow. If you try to regain that flow, you typically add a lot of noise to your test setup. And it can become confusing. Also you introduce speed and timeout concerns. Synchronous really well isolated unit test suites can be super duper fast.
[00:11:03] I've had test suites with 4, specs that run in under one or two seconds in JavaScript. That would not be true if I had really many at all asynchronous tests. Also it can introduce very difficult to debug race conditions. And all of the above really stands to add to the case that there, there become many reasons for those tests to fail.
[00:11:21] And when you have more than one reason for a test to fail, it means that you have to enter this analysis phase before you can go about fixing it. So what I do instead is I'm careful to only, I know everyone in JavaScript line loves async all the things. I only write async APIs when it's actually useful.
[00:11:35] And that usually means when I want to avoid blocking. Like in Phil's talk earlier. Like I'm talking to a network or a file or some such. And in those cases, normally that async behavior belongs in the periphery of my application anyway. Probably not in the core application logic. One way that I get that out of there is that when I have a lot of async behavior riddled throughout my application, I try to pull that into decorator objects or mixins so that my pure logic can be just normal boring functions that are very easy to tec test in the core.
[00:12:01] And I also try to use promises instead of callbacks wherever possible because you could easily introduce like a fake promise implementation that's completely synchronous. Much easier to test than having the deeply nested callbacks. But when I am characterization testing something and I want to actually test something that uses anonymous callbacks you can capture and then re synchrofy things.
[00:12:20] by putting that under discrete test. I've got a tool called Jasmine Stealth that enables that. And here's a really fast paced example. So imagine you have a car with a go method which posts to a route of room and then based on the result if the speed is too high it becomes wrecked.
[00:12:35] So I'm describing my car now and given that I have a new one when I spy on POST so I'm knocking that out so that I can it's not gonna actually call through Then I create a capture object, which is a special type of matcher such that when I call the go method, I can write up an expectation and say, I expect dollar post to have been called with vroom, and then the second thing, capture that.
[00:12:57] That way I can put that function value under explicit test synchronously. I can say, okay, so when I'm going really fast and speed is over 100, then the rect property should be true on the subject. And when I'm not going so fast, here's the other branch. Let's say the speed is 99. then subject rect is false.
[00:13:16] So that's one way I can make synchronous and asynchronous subject code in my test. Let's talk a little bit about the DOM. The first and foremost, the thing that I don't do is I say, wow, that looks hard, so screw this and then I avoid testing DOM interactions. Surprisingly a lot of people, even thought leaders among JavaScript testing folk tend to say the DOM is really messy, it's really hard to test, so just skip that and then test the layer beneath it.
[00:13:36] I think that's a cop out. In fact, if our DOM interactions, they're just functions, and if, you can test anything if you've got a function. So it's not a matter so much that it's impossible it's just difficult, and if it's difficult, maybe it's not just that it's intrinsically complex, maybe we just don't have good enough tools for dealing with it.
[00:13:52] So it's worth exploration, and it's worth trying on your own. Also, I don't use large HTML fixture files. Particularly, I don't use ones that are shared across multiple tests, ever. The reason for going back to the DOM interactions, the reason that it's bad to not test them, is that most JavaScript today, probably 90 percent of the JavaScript written in the world is very DOM interactive front end web JavaScript.
[00:14:12] And that means that if you're not testing that area, you're gonna have very low coverage in your test suites. And the usefulness of a test suite is really diminished greatly when it's, very low. You start to only test in ad hoc circumstances. And it can become confusing like when you test and when you don't.
[00:14:26] In fact, I see that when teams have a sort of a little walled off area of the of the application where they've all agreed not to test, more code gets written there by path of least resistance. So you don't test the DOM because you hate the DOM so much, then you write more DOM code because everyone knows that they don't have to write tests if they're writing DOM code.
[00:14:40] So that's counterintuitive, but bad. And it tends to hamper outside in test driven development. So my goal is to like, start with the entry point, like the user interface and the problem that I'm trying to solve, and then use TDD to break that down. And that, that, that flow gets interrupted when I say that the very top level doesn't get tested.
[00:14:57] I don't use html fixture files when I'm trying to test my DOM interactions because typically those are large and if you have a very large input you'll have larger everything. Larger test methods, larger methods under test. And tests, our test tools particularly are responsible for encouraging good behavior and good behavior should be pushing us towards smaller units, smaller objects.
[00:15:17] Also when you share fixtures across multiple tests it leads to a phenomenon called tragedy of the commons. This means that everyone's totally comfortable adding to that HTML fixture but god forbid anyone ever think about removing something from this gigantic fixture that no one understands anymore because you might break all these other tests and the contract between what the test is specifying and what the code is actually doing is hazily lost into this gigantic fixture file.
[00:15:37] So what I do instead is I treat the DOM like a third party dependency and I minimize my exposure. I keep it out of the periphery just like network requests or third party APIs. But when I do have to write tests against the DOM, I create my HTML fixtures super tiny, inline, with a convenient tool that we wrote called Jasmine Fixture.
[00:15:52] You'll see that in a second. And the goal here is to arrive at single purpose DOM aware functions that, that do just one thing, do it well, and then are composable. So an example here imagine that you have a secret magical door. And so I'm gonna inject it onto the page with this affix. Method and it's just jQuery in reverse.
[00:16:08] So this is promising that to your test that this will be on the page. Maybe a magic phrase input that you can type in. And then when the user shouts, if they get the correct phrase, which is of course Open Sesame, then our assertion is that the door should not be closed. That'll return a Boolean true or false, which will fail the test initially because nothing's defined.
[00:16:28] So I create my shout method. And here it's really obvious. I can actually just totally crib the CSS selectors that I wrote in my test. If the value of that text element is open Sesame, then I want to remove the class closed from the door, and now I'm passing. So less tactically, more broadly, because I'm almost out of time, the most important thing, if you take one thing away from this talk, would be know why you're doing whatever it is you're doing and when it comes to testing, know why you're testing.
[00:16:56] Know what the purpose of any given test is. If you don't know why you're testing and you're just testing cause someone told you or because you're committed to always testing all the things all the time, you're probably gonna have a bad time at least some of the time. It's tremendously more valuable to write tests with a clear concept of why you're doing it.
[00:17:12] Tests that are written for just the sake of having tests tend to lack the. clarity of the purpose for existence, such that somebody reads it later, they can see it, what you're trying to prove, what you're trying to learn. Like I talked about with the DOM, push through the pain before deciding whether something's worth testing or not worth testing, because a certain amount of that pain is just, you don't know how to do it yet.
[00:17:31] And a certain amount of that pain is just intrinsic complexity. So once you've overcome the learning curve, then you can finally make the judgment, yes, this is worth it. No, this is not worth it. Objectively. Also, easy to test code when you're testing like object APIs is easy to use code. And most JavaScript is hard to test.
[00:17:48] So you draw your own conclusions.
[00:17:53] Finally, there's no right way in software. Never listen to anyone who says that they've got the nut cracked on how to build software. Software is just communication. We're all just doing our best to communicate with each other over these temporal gaps. And what I look out for are thoughtful approaches and subtle approaches and embrace of nuance.
[00:18:09] Versus thoughtless approaches of just a prescription to say do this every time. So I gave a lot of prescriptions today because of the the lightning talk format but none of them are binding and I'd love to talk to you more in the discussion track if you're interested. Finally the tool that was used for all of those tests that you saw you can get started in minutes, is a tool we maintain called Lineman.
[00:18:26] You can learn more at linemanjs. com. It includes all of these helpers out of the box. Lastly, again, my name is Justin. It's a pleasure to meet all of you. I hope to get to know you this weekend. And if you'd be so kind, I'd love to start a conversation in the discussion track with you straight away.
[00:18:40] Wonderful. Thank you.
