This talk was delivered at Øredev in Malmö, Sweden. It was a joy to have an entire week to focus on how to best discuss the trends we’re experiencing in fat-client JavaScript.
As a result, I feel like I finally landed at a sufficient explanation of what Lineman is and why it might serve you well. You can find it on YouTube.
Editor's Note: You might want to check out Why did Lineman.js fail while you're here!
Here are the slides, hosted on SpeakerDeck.
Transcript of the talk
[00:00:00] Hi everyone, I'm here to talk about a web application development, and specifically, ostensibly, this is a talk about a specific tool called Lineman, which is something that my team has created, but really this is a much, much more general, hopefully more generally useful and applicable talk than just to describe one tool.
[00:00:18] So to kick things off, I want to let you know who I am. My name is Justin. I would love if you followed me on Twitter, but more importantly, if you engaged with me and sent me feedback about, What you think about the tool, about the talk, about the ideas that we discussed today. And feel free to do that via Twitter.
[00:00:32] Or you could just say hello at testdouble. com, which is my company. And specifically I'm going to talk about a tool today called LinemanJS. So if you want to find Lineman, you can find him on Twitter there. Or at linemanjs. com and via that find him on GitHub. But really this is a talk about three things.
[00:00:49] One, the earth. Two, monoliths. And three, cupcakes. And we may as well start at number one. So the earth. Specifically talk about community. As you may know, the Earth is made up of a certain number of Java developers, a certain number of Ruby developers a whole lot of NET developers, among some other types.
[00:01:10] But when we think of JavaScript developers, JavaScript is the lingua franca. Almost everyone who's a developer in this age does some amount of web development. And Javascript's the only thing that runs in browser. So when you consider Javascript developers, everybody is a Javascript developer. So then why is all the tooling so terrible?
[00:01:28] Why does everyone hate Javascript? And why are there so many problems? And, of course, the reality is that there's no such thing as a JavaScript community. It's very fragmented. And the reason it's so fractured is that there's no such thing, or there's only recently become such a thing, as a first class JavaScript developer.
[00:01:44] Really, what we have is people who herald from one of these other tribes, and they do some JavaScript on the side. Here's some JavaScript. Java developers who dabble in JavaScript, and NET developers, and Ruby developers, and they totally, really don't communicate much.
[00:01:58] More importantly, they're not soluble. So when you mix them up they generally don't co locate. They go to different conferences, and different meetups, and they use different tools. And this creates this incredible inefficiency. On the left maybe is a Ruby team with a couple of JavaScript people, and on the right is a NET team with a couple of JavaScript people.
[00:02:15] Like I said, their tools aren't compatible because they come from totally different server side oriented, environments. But more importantly, even from the human aspect I worked with a front end developer last year who was an incredibly bright front end developer. He knew how browsers worked.
[00:02:28] He was able to make some really great user interfaces in browsers. But because he happened to be on a Ruby team, his workflow included tools like Haml and Sass, which are predominantly Ruby stack only tools. And so he felt like he could, if he was going to look for a job, that he could only join other Ruby teams.
[00:02:47] Even though he didn't know Ruby. And that's a ridiculous notion. I would give talks often on JavaScript testing. And when I presented everything and showed people how to do it, at the very end, people would ask how do I get started? And I'd be like install Ruby.
[00:02:59] And that's a really frustrating thing for somebody, say, on Windows, where Ruby's a pain in the ass. Over time, I have a theory, and based on this theory, I've been driving most of my personal time and most of my open source efforts in the last few years, is that as browsers become more capable, If you were to plot that against the opportunity cost of this lack of communication in the community, that only goes up over time, and it becomes more and more dramatic.
[00:03:22] And so I think, I view this fractured community of JavaScript as a real problem that's only getting worse. Let's talk about monoliths. Now this is obviously the monolith from 2001. But when I say the word monolith, I'm talking about monolithic application architecture. For example, like Ruby on Rails if I'm building a Rails application, I've got all of my server side code and all of my client side code in one directory and one repository.
[00:03:44] It's all deployed and versioned as one thing. And that is extremely convenient. In month one of a project and maybe year one of a project and can really come back to bite me long term. So let's talk about Rails. In my opinion and the opinion of many, Rails won the war of web application development.
[00:04:01] Whether you're a Ruby developer who actually uses Rails, or you're in any other technology, odds are that the premier web application framework for your technology stack stole a lot of really good ideas from Rails. Often we don't ask, which war exactly did Rails win? When we talk about what Rails is, we say it's a web application framework, but that's a very broad statement.
[00:04:26] Really, Rails is really good at a certain subset of web applications, and not so good at all kinds of web applications. DHH, the author of Rails the founder, he said this year at RailsConf, this is a recent quote, that good frameworks are extractions, not inventions. Rails was extracted from a real world application that he wrote at 37signals called Basecamp.
[00:04:47] And that was in 2006 with all the paradigms of web applications of that era where, you know, a workflow of project management, you create a thing, you click through every click is a new page refresh, a very traditional HTML document structure application. Every time that, that Java, that rails has tried to introduce dynamic front end stuff, rich client or dynamic user experience in the front end.
[00:05:12] They've tried to do it in a way that could prevent you from having to write JavaScript. And those were all sort of inventions that were tacked on. And so Rails actually, this quote works both ways that, that to the extent that Rails has tried to be a JavaScript framework, it's been really bad.
[00:05:26] Even the asset pipeline, which I think is their best idea is an implementation that makes life really inconvenient for people who care a lot about front end code. So when we dissect the monolith of rails. The stuff that it's good at is really, better suited to traditional HTML document driven web applications.
[00:05:41] And they're not shy about it. It's just something that we don't think about much. Like routing URLs to specific actions in the code. Modeling. So describing our models and their relationships and the validations they're in. Persistence, how to persist that data, especially to relational databases with database migrations.
[00:05:56] Session management, session storage, mailers, and other kind of conveniences that are traditionally associated with web applications. And then all of this JavaScript free dynamism. Every major version of Rails has had some new headline feature to try to be the final solution to the JavaScript problem.
[00:06:12] Originally there were Ajax helpers built into the templates that would generate all this nasty JavaScript And then they made unobtrusive JavaScript, where it was essentially the same thing but with classes. And then they made RJS, which is Ruby, transpiled onto JavaScript. All these features are dead.
[00:06:28] Now it's Turbolinks and PJAX, where you're just fetching parts of a document at a time. So they're really trying to get people to not have to write JavaScript. It's their goal. Which makes it hard when you're writing a JavaScript application. I like to think of these the way that I discriminate these two types of applications are HTML user interfaces versus JavaScript.
[00:06:45] When I say an HTML user interface, I mean that your web application's primary responsibility is to specify an interface using HTML. Maybe I define a link in my HTML that way. Or I define a form this way in HTML. I'm not actually doing user interface programming. I'm simply handing to a browser a specification of the user interface that I would like, and then it's the user agent's job to go and make that actually interactive.
[00:07:10] But a JavaScript UI is fundamentally different, because in JavaScript, obviously the browser is our runtime. We can do whatever we like without having to go back to the server or have the page refresh or, and rely on the browser for all this behavior. So I could, for example, render dynamically into a particular portion of the page.
[00:07:26] Or I could listen to click events and have custom behavior built in. These things could not look more different in terms of like their declarative nature versus their imperative nature. When I think of HTML UI as a project structure, this is a very oversimplified Rails application.
[00:07:40] Where I've got a controller that handles in URL routing for items, a model that item, grab it from the database, and a view to render it out as a template. Now in reality, most Rails applications also have some amount of JavaScript on top to add a little bit of dynamism on top of the traditional HTML UI.
[00:07:58] Very few applications actually exist in production without some significant amount of JavaScript, even if they're written in the traditional way. Now, nowadays, with the advent of Backbone and Ember and Angular and so forth, A lot of applications are leveling up the craft and the complexity of their front end code, such that they have MVC living in twice at a time in their application.
[00:08:19] Up here this is almost the exact same structure as what down here. So I've got my item controller repeated and two instances of an item model. One in Ruby, one in JavaScript, because one runs on the server and one runs on the client side. This gets really confusing and hard to manage and hard to stick in my brain.
[00:08:34] Among developers who acknowledge at least that much, this right here the views the traditional HTML views, it probably only makes sense to have one primary user interface. So even if we whack that, then we might have something, called maybe a JSON API plus a JavaScript interface.
[00:08:50] And this is a, a sustainable and sensible approach. But if a new person joins your project and sees this structure, and they don't know all of the forensic history of, traditional web application development. They'll be like, why are these two things in the same folder? They have nothing to do with each other.
[00:09:03] They are logically separate applications. In particular, I like to call it, that is a vestigial appendage. These things have no reason to be next to each other. And in order to explain why they are together, we have to talk in terms of it used to be. So what's wrong with vestigial appendages?
[00:09:21] Because certainly, I still have an appendix. Is there anything inherently wrong with them? No, but I think that it causes a couple of bad habits. So in particular whenever any code that we write is still small, it can fit inside of our heads. So whether it's monolithic or split up into lots of different parts, The equation of whether or not it's too big, too small, whether we can conceive and reason about it is really simple.
[00:09:44] Because it's small, and of course we can understand it all simultaneously. But as it starts to grow, eventually every application that survives and is successful reaches a complexity point. There's some sort of cruft constant. Beyond which, we can't fit it all in our brain anymore. And if it's not segmented and modularized, that means that paging occurs.
[00:10:03] So maybe we can only fit the first half of the app in our brain at a time. But, when we're working over here, we have to page out sometimes over to the other half of the application. So I'm working in the first half, but because it's all tangled up with the other half, I just have to know that I have to be diligent and go and check and, grep around and make sure that I didn't break anything else over there.
[00:10:21] But what if we were to split things out into two applications in the beginning? Then we'd have two small things in our brain and they both reasonably fit. And as they grew, they'd still both fit until they reached some point where they didn't anymore. And at that point, now the paging makes cohesive sense.
[00:10:34] The first application can fit into our brain. And when we do have to page out, it's no longer this really high thrash, high churn operation. It's just a simple context, just like anything else, because these applications are physically discreet. We don't have to worry about changes to one totally mucking up the other one.
[00:10:52] On this point. There's a, a debate, I think, between whether or not it makes sense to let things get complex and then to break out a totally separate application after the fact, or whether to start with two small things in the beginning. And my stance on this is that late extraction Costs more than early abstraction.
[00:11:09] That's a confusing statement, which I hope to explain with two balls of yarn. So if I have two balls of yarn here at the outset, and I want to make a big, nasty jungle of tangled yarn, that's really easy to do. So the worst case scenario is I start with two things and I turn out that I only need one thing.
[00:11:26] I can always very easily make a big mess after that. But if I start with that big mess, then it's incumbent upon me to detangle it and nicely ball up these yarns, which of course is very difficult to do because of the basic laws of entropy. And so that's what I mean when I say late extraction costs more than early abstraction.
[00:11:44] Yes, sometimes your abstractions aren't necessary, sometimes they won't be reused. Sometimes you won't gain a lot of benefit. But if you know that ultimately you're going to want two things, you may as well start day one so that you don't end up with a tangle. So really, my rule of thumb is that whenever you see this, Where a an HTML template on your server side has a script tag, and then a whole bunch of global JavaScript variables that are all interpolated server side value.
[00:12:10] Even if you're building a JSON API that talks to a fat client, say, Angular app or something, and you're really proud of this how discreet they are, and how you're building this API that can be used for mobile and so forth, When you see this, your API is broke because what it means is that you weren't able to use your own API to build your front end application.
[00:12:28] So somebody who wants to make a mobile app is going to have to figure out how do I replace all these things? Do I live without them? Clearly they were important to you. And so this physical co location causes tangling, even when we try to be really diligent about it. I've never seen a rails application that had a significant front end component that didn't do that.
[00:12:44] I've seen dozens. Let's talk about cupcakes. So I think that because now obviously I've been using the word Rails a lot. What I really mean when I say Rails is Rails era, or Rails class web application. That meme of convention driven coding of really delightful and fast and convenient web application development, that's seven years old now.
[00:13:08] It's been around for a while. So we tend to think that we're really smart, that we've got web application development figured out. But sometimes we're really just spoiled brats. Sometimes we've, we don't acknowledge that we've acclimated to it. I think that as things become more mature, we start to conflate or associate that with convenience.
[00:13:25] So the maturity of a tool, if the tool is really there to make our life happy like Rails was, Over time, it becomes more and more convenient. When it iterates, they're thinking, how can I make developers happier as they develop? But there's a other side to the coin. Because before Rails was easy, before I had become a Rails developer and was super, delighted with how quickly I could get stuff done, JavaScript wasn't hard.
[00:13:49] Everything was equally terrible before Rails. And so it didn't matter which one I picked. But once Rails made everything so damn easy, then it was a very difficult decision for me to break out and have terrible debugging and have, hard refreshes and just nasty syntax and no helper library.
[00:14:07] So convenience is obviously important from that perspective if you're going to convince people to, to work in a different way or use a different toolset. For example, when I think server side tooling, I think of this toolbox that's immaculately maintained with all these useful, there's four different kinds of screwdrivers there.
[00:14:23] There's almost infinite number of server side code focused tools, no matter what stack you use. But when I think of client side tooling, I come up empty, right? There's just not a lot of tools in these tribal environments in Java and NET for good client side tools. And of course, I'm just kidding because they do exist.
[00:14:45] They're just absolutely terrible and jagged and rusty. Because it's a second class concern. I've written a lot of these. I'm the maintainer of for example, Jasmine Maven Rails, which is a Java, Jasmine is a testing framework, which is for Maven, which is a Java build system. And I'm also the maintainer of Jasmine Rails so Jasmine Maven and Jasmine Rails, which is, obviously a Rails plugin, and they are completely obnoxious, terrible tools.
[00:15:10] And I don't use them anymore. But they're still two of the most popular things that I maintain. Because of this sort of, Tribal fractured community problem. Now I, this talk would be a lie. If I didn't mention JavaScript UI applications are more complex, are heavier and harder to write and take a bigger investment than traditional HTML.
[00:15:30] And the reason is obvious. If you just zoom back to our discussion of the difference, this one over here just has to specify and declare, this is the user interface that I want. And this one actually on the left has to implement both the server side API, as well as the actual user interface behavior of the front end.
[00:15:47] So naturally, Javascript is like going to be a little bit harder and take a little bit more time, even if the tooling is great. In Agile, there's a phrase do the simplest thing that could possibly work. So you pick a story card off the wall, and you try to think, what's the best way to implement this story?
[00:16:04] The answer, in traditional sort of like Agile dogma is, let's think of the simplest possible solution that would satisfy this card. And not worry about the future so much, because we'll refactor, or work the system up into a state that, we'll iteratively let this design emerge.
[00:16:19] This works perfectly fine, except for when you have a case where the simplicity becomes a trap. And then here's an example of that. Let's say that we have a customer who comes to us and he says, I need you guys to show me some data in a table. We're like, alright, that's easy. So we'll pull some stuff from the database and we'll use an HTML UI to render stuff into an HTML table.
[00:16:36] Easy peasy. And then he says, now render it as a graph. And we're like, oh, alright. I guess we're already using an HTML UI, so the easiest thing to do would be to, plot some points and then rasterize an image on the server and then push that down into the HTML as well. At this point, we're feeling pretty good about ourselves because we shipped some code and we probably shipped it really quickly and it was really simple.
[00:16:53] So then the customer comes back and says, Alright, sweet, so now add some filters and some sliders and some zooms to the graph. And then you're just like, God damn it. Because you realize that, that one and two not only didn't logically help you lead to step four but one and two might become waste.
[00:17:10] And the way that they become waste is to be a phenomenon I call the simple cliff. So we, whenever we do anything, we adopt some some costs from the framework that we stand on top of and rails is complex. So we start off at a relatively high complexity and then we build stuff up as we add stories. And that's maybe the complexity of building our HTML user.
[00:17:29] But then there's this chasm where the starting off point is even higher from a complexity perspective, but the gap between them is there's no logical path from the tip of this one to the start of the other. And that's where, JavaScript applies. So what do we do with the HTML side once we decide we have to jump over that chasm?
[00:17:46] Of course we throw away the HTML stuff and redo it all in JavaScript. Which is really wasteful. In fact, we often talk about JavaScript sprinkles, if you ever heard that expression. Like we write an application and we just sprinkle some JavaScript on top. So using the baker analogy, let's say that, you have a customer come in, you're a baker.
[00:18:02] Customer comes in, he says, I want something sweet. So you're like, alright, I'll bake you a cupcake. You bake him a cupcake. And then he says actually I want something even sweeter. So you're like okay, I'll put some JavaScript flavored sprinkles on top. Disgusting as that sounds. And hopefully this will please the customer.
[00:18:16] Then the customer says, on second thought, or as I think about this further, I'd really love it to be hot and even sweeter and filled with fruit. And then you just groan, because you've already baked the cupcake. You can't fill it up with fruit after the fact. And you realize what the customer really wants So, that would be an unfortunate experience if it happened to us as a baker once.
[00:18:38] But what if this was actually our ordained workflow? We keep doing this over and over. He comes back every week wanting another sweet thing, and we keep baking him cupcakes only to throw them away so that we can bake him pies. That is, web development in 2013. And it's absolutely ridiculous. So my goal has been to compete on easy.
[00:18:57] My passion has been let's build really awesome, fun to use a front end tooling that makes it as close to as convenient as server side stuff as we possibly can so that people not only have fun, but can take pride in and really feel a good sense of ownership about their front end project. So when we talk about what goes into the build systems and the productivity and the workflows that we use as developers.
[00:19:25] I'm going to split it up into three categories. One is an application framework. Two is the conventions and the configurations that we use to set everything up that's distinct about our project. And then finally build automation, like tasks that we run either locally or in continuous integration, or to deploy to our servers.
[00:19:42] Again, pushing off of Rails, Rails is the application framework, right? We extend from Active Record Base and Action Controller. So we're, Rails provides us a lot of the code itself. Thank you. Now, in terms of convention and configuration, Rails also sets out, code should go here. And if you name something this way, then it'll just work.
[00:20:01] So it's very tightly integrated in that sense. And when we talk about build automation, Rails uses Rake, but it's very tightly coupled. So really, Rails is the entire stack for all intents and purposes. When you're talking about server side development. And this is true, I think, of a lot of the server side tooling.
[00:20:17] Now, when you talk about front end, I guess you could pick whatever application framework you want. Some people pick Backbone. Some have Ember. Some Angular. Some write their own. I change my mind every six months. So at this point, I'm just like, whatever. I don't want to lock that variable down because I'll look silly a year from now.
[00:20:33] I want people to be able to use whatever application framework they choose. For build automation I've chosen grunt. I like grunt a lot. It's like rake. And we'll talk about grunt a lot more in the future, but it's based on node JS. And then this convention configuration layer, like lineman just lives only in that he just provides a default configuration, sets out some conventions.
[00:20:53] And if you follow those, you'll be really happy. That's our goal. And what's nice for me as an open source maintainer is I can remain really nimble because lineman itself is very tiny. This is an hour long talk and only a little bit of it is actually code lineman itself. Is like I think 600 lines or so, and 400 of that is default configuring.
[00:21:12] It's not big at all. It's mostly about the idea. To install Lineman, you need Node. So you go to nodejs. org and you can download whatever your platform and installable. That'll give you a command line tool called npm. And from NPM, you can suck down packages. So if you tell NPM, Hey, I want to install dash G meaning global lineman will be available for you on your path as a command line.
[00:21:35] And then to create a new app, all you do is you say, Hey, lineman new. Make me an app to demonstrate this. Here's a little screencast. So here's me typing in lineman new into my system. I create an app. It gives me some ask r and some default tasks that I can run to get started. So I CD into my app, and then I'm gonna tree out the files that you get.
[00:21:57] So when you start a Vanilla Lineman project, these are some of the files you get. You have a grunt file, which is the basic build route, and then under app, CSS, images, JavaScript templates, some configuration files. A whole bunch of stuff we help out with specs. And then finally vendor stuff. If I've got third party CSS like Bootstrap, or JavaScript underscore jQuery that goes there.
[00:22:18] That's really all there is to it. Most of these are directories. These directories are empty. So you don't get a lot out of the box, which is good. Like our primary competitor, if you want to call it a competitor, is Yeoman. You may have heard of Yeoman. Also based on Grunt. Also based on Node. Also, a really powerful tool.
[00:22:33] But it's based on a lot of code generation. And when you start a new project, you get a whole lot more than you bargained for free in your repository in terms of files out of the box. Which makes it almost impossible to upgrade this application. And if the generator at all falls out of sync with the framework, then you're married to Yeoman in a way that you probably didn't intend to be.
[00:22:52] So Lyman tries to be as lean as possible. You don't get any configuration dumped into your repository. It's an empty file where you say, hey, if you want to extend this, you can. So let's talk about Earth again. Let's talk about community. Specifically Node. js as it pertains to the community. What I love about Node is that it's the first time in the long life of JavaScript, where JavaScript rides first class.
[00:23:14] People who use Node actually opted in to write JavaScript primarily. They can't complain about the fact that they're writing JavaScript, because they're in a platform that is all JavaScript. So on top of that, it's become business and fancy because it's scalable and they've convinced a lot of, enterprises to invest heavily in it and use it in production because it's so effective at building a small, fast, scalable networking.
[00:23:38] So I imagine node as being first class on this plane. And because it's so fast and because it's so convenient, lots of other people have joined the party. And when I look at this fractured earth that we have Node is successfully connecting all of these previously disparate communities. Like for instance NET and Java people.
[00:23:56] Java and Ruby people, so on and so forth. Like last month, for instance, I was in Bulgaria for a conference called DevReach. It's primarily a NET conference. And any other year, I'm sure that I'd have nothing in common with any of the NET speakers. But this year, we're all talking about the same stuff. We're all talking about Angular, we're all talking about Grunt, and what's happening in ES6, and all the fun stuff in the browser.
[00:24:15] And even though we have our differences, we can at least start speaking the same language. Now there's some, trouble in paradise as well. In my opinion, Node's community is still a little bit immature. There's a lot of thrash. Sometimes stuff breaks. Stuff isn't totally convenient yet.
[00:24:32] It still feels like a little bit crazy. The ties are a little bit loose. But honestly that hasn't proven to be so bad. Sometimes a little bit of like wild west is useful when you need a lot of it. When you look at these different communities, they look at node funny as well because we all come from these very homogenous communities of the server side culture where we've spent a decade all agreeing on everything and the idiomatic approaches to everything.
[00:25:00] And now we've all been jumbled together into this big pot. So somebody from one community might say semicolons, I love semicolons. And somebody from another community might be like, eh, semicolons. And then the Ruby guy is I love CoffeeScript! And they all are blending together and don't have the same comfortable experience of having everyone around to agree with them.
[00:25:18] The most common feedback I get about Node is What's the right way to do something? What's the right way to do X? The answer is, A, there's no right way, and B you probably have to carve out the community that matters to you a little bit, so that you can remain sane. The Earth is still fractured.
[00:25:34] We have not all come together yet, but it's getting better. It's getting closer. And my hope is that within a few years a lot of these kind of, differences will be moved out. Grunt is the build tool that we use. If you're not familiar with Grunt, it's very similar to all of these guys.
[00:25:47] It's very much like Rake, which is based on Make. You define tasks and you run those tasks. It couldn't be easier. So this is an example task that says hello to you. You just export a function that receives Grunt. You register a task and you give it a name and then some action. And in this case, this action that I do is just says hello and writes it to the log.
[00:26:07] So when I run that grunt task, it says, running the hello task, it prints hello at me, and then it exits gracefully. Very easy. Another thing I like a lot about grunt is that there is a brick wall between the task behavior, And the configuration for it. A lot of build tools do this. They separate task behavior from configuration.
[00:26:25] But what makes Grunt special is that the configuration is very mungible. I can actually build Lineman as an abstraction on top of Grunt and give you an opportunity to specify your configuration and then gracefully merge that in with the defaults. I don't think Lineman would have been possible without that sort of design of Grunt.
[00:26:47] Additionally, there's a lot of grunt tasks out there. Virtually everything that you'd want to do, there's probably a grunt task for it in the front end web development. And unlike, the past where Ruby was the first place where we saw a lot of the innovation happen, like Sass, for example.
[00:27:02] Nowadays, if somebody wants to like really share something, a grunt task is not far behind. So really what's great about this stack is that there's so much community involvement that you can just add a new grunt task to your project and then get that, new, sexy, fun behavior basically for, so this is one of the pro sides of the wild west nature is everyone's working, everyone's innovating, everyone's playing and there's a lot of cool new stuff happening all the time.
[00:27:27] Revisiting monoliths. What I've been pitching here is breaking up that one monolith into at least two monoliths client and server.
[00:27:38] Why a lot of people don't do this by default is the client has to talk to the server, and not just the complexity of separating them and versioning them, but if I want to run my client, I've got to run my server too, and then they're on different ports, and it's just really, tedious from a workflow perspective.
[00:27:50] And so Linemen we try to solve that. Taking for example you with a browser, and then running Lineman as your dev server, and also running an instance of Sinatra on some other port on you. This is the workflow that we've adopted. You ping Lineman on port 8000, Lineman can be set up to proxy to whatever API you'd like.
[00:28:09] Maybe 4567. Then that response comes back from Sinatra lineman having proxy. The request feeds it to you. So from the browser's perspective, it's just one cohesive thing. Even your code and all your artifacts are physically separate. So a quick example of that, so this is me, I commenting the basic config to set up API proxying and I'm gonna set the port to 4, 5, 6, 7, save that off.
[00:28:35] I've got this little simple app, the hello text there, just post a get request, get request at high and then the result gets passed to done. And the server side API call, hi, just returns the string, I love Ruby. So I'm going to start my lineman server up, and then tab over to my running Sinatra server.
[00:28:55] And then when I fresh the page, gracefully, invisibly, I'm calling through to my Sinatra applicant. That's a simple example, but it's, it scales. So we've built actually entire applications this way for months and months. We've built small front end components that are, you know, apps within an app against existing APIs.
[00:29:12] And it's really pretty painless. Once you get it set up, once you do that one little setup piece, you There's another aspect, though, too, which is that if you're building too small applicators, there's no reason anymore for development to necessarily proceed in lockstep. You're building one monolithic application, so another Agile dogma is thin vertical slices.
[00:29:29] When you pick up a story card, you should implement from the outside in the user interface, and then, whatever controller stuff, and then model stuff, and then data, and the card isn't done until all of it's done fully vertically throughout the system. It depends on how you define system.
[00:29:42] If we have two separate applications, there's no real reason why we couldn't build the front end code a little bit ahead of the service layer. Except for the fact that if the server stuff doesn't exist, we get blocked in building the front end. So another feature of Lineman we call API stubbing.
[00:29:57] So pretend that server either isn't complete or the current version of the server isn't compatible with what we want to do. We can just stub out individually routes that we care about. And then in that instance, from our browser, we'll hit up Lineman. And instead of calling through to Sinatra in this case, Lineman will pretend that it is.
[00:30:16] And return our stub back to itself and then on through the browser. So an example of this, coming from where we just were, where we have a lineman application that's proxying over to Sinatra, I'm gonna poke a hole in exactly the same route. Doesn't matter whether the route exists yet, it'll always respond.
[00:30:32] So at route high, and this is line express code writing in lineman, just for development. I'm gonna send back the string, I love node even more, which might not be true. I run my server, I refresh the page, and now even though Sinatra's still running, this one wins. So I can stub out and move as fast as I want to on the front end side without necessarily waiting for back end development.
[00:30:55] And in fact, that loop that we saw, you can iterate really quickly and prove out a lot of a front end application. So you don't have to churn of having to redo server side stuff as you learn more about the front end. Another aspect that's really beneficial is testing. So I had an application once that I logically separated that was a 30 minute test build.
[00:31:13] But because all of the tests had to talk to everything in the stack. So I had some tests that were mostly concerned about UI, but that really had to call the database redundantly. I had some tests that were mostly interested in the services, but they had to exercise the UI redundant. It took 30 minutes.
[00:31:27] When I actually physically separated this into two, what I found was really astonishing was that the new integration tests that I wrote, they were clearer. They, they were more focused and understandable and readable, but they were faster, a lot faster. You'd expect a 30 minute build to split up into two 15.
[00:31:43] But the back end in this case sped up to four minutes and the front end also was four minutes. And so I, not neglecting the responsibility of gluing it all together and making sure that it works when they're talking to each other. A smoke test suite that I added to, to test them all integrated only took two minutes every time it ran.
[00:32:00] And so all in all, that's a 10 minute test build and 10 minutes is a lot better than 30 minutes. And if you imagine the graph, the super linear growth of integration test duration over time, this is a really big advantage. Just like constant context. And I think it's habit forming, because once you solve those hard problems once, how do I version these things?
[00:32:19] How do I manage deployment when there's two different versions? Once you do that, once upfront cost you can add another service really cheaply. Because you've already figured out how to maintain that. So you might add, additional mobile clients. You might add additional services proxied by some other service.
[00:32:33] It's a really enjoyable place way to work. And what I really like about it is that eventually, if you learn how to build cohesive, small, focused applications you, they can almost become disposable. The risk around does it work, is the build passing, becomes lower, because each individual thing can build in a little bit of fault tolerance, but also it's like, if nothing ever gets very big, our sunk cost isn't that great, and if we have to, we can rewrite and without epic.
[00:33:03] So I think this is a pretty good start. It's just starting with two. And I don't think it's asking for too much to, if you're building a fat client javascript application, to physically separate the two. Revisiting cupcakes. So this is about convenience. When I think about writing code and then seeing that the code works, I think in terms of feedback.
[00:33:23] So whenever I write some code, these things need to happen for me to know whether or not the code works. I write the code, I save it, It compiles, I read a lot of CoffeeScript as you saw earlier. It compiles it needs to be concatenated with the other scripts, and then I need to be able to play with it.
[00:33:39] With Lineman, our goal all along has been keep this feedback loop at under 100 milliseconds, no matter how big the project gets. And so far, that's been very successful. So here's an example of me editing a file in Lineman and refreshing the page. So I start Lineman server by saying Lineman run, it's already started, I can refresh the page, see my little hello world.
[00:33:59] I can go in here, change the text, as soon as I save, it's already done. And here I reprint. And this is a project with one file, but we scaled this out to hundreds. And it's amazing how fast Node is at file IO. It's worked really well. It's not just about exploratory testing and seeing stuff work. We want the same story to feel very consistent for unit testing.
[00:34:24] We practice TDD often. Not just about playing, but also about testing and running our tests. So here's an example of the same thing. But using Testim, which is the test runner that we've included with lineman. So I switch over into a new tab while the server's still running and I say, lineman, spec testim launches and its interactive runner starts up, captures this browser for me.
[00:34:48] So there, my code's the same and I have a test over here that just a search that result equals hello world. And if I change it to goodbye world here, as soon as I hit save, all my tests have run again. It scales really well, even if I've got a lot of files and here I can fix that test by changing the source.
[00:35:03] And that's really what the story is for unit test. On top of that, I can quit out of everything and run this very easily in a CI format. That's totally headless with phantom JS and the configuration is common and it comes out of the box. So there's my passing test. On top of all this we include a Travis YAML in every new file in every new project.
[00:35:23] So if you upload your project to get hub, all you have to do is turn on Travis and you've got CI set up. We try to just remove all of the friction, all the pain that's typically associated with front end testing. Talk about deployment. So if your server can host static files, you're in luck because that's all this work has been just to produce static files.
[00:35:45] We don't need any fancy server side stuff. When you run lineman build and then we tree out what the dist directory, which is what the bill targets, it's literally three files, a CSS file, an HTML file, and a JavaScript. They're all concated, minified, and ready to be deployed. However you like. And easily, integrated into whatever, your server side tools deploy tools look like.
[00:36:10] And it always starts with this. So even if you add additional stuff like web fonts and images and so forth, that's all your stuff. This is eminently understandable. You can look at it at a glance and know everything that's getting built very quickly. We like that stuff. Heroku's even easier. So if you're familiar with Heroku, it's a cloud based application server that's tied into Git.
[00:36:29] So all we do is one simple configuration. We point to a custom lineman build pack that we built for Heroku. And every time you get push, it'll build on Heroku. It builds with node JS, but then whether are nervous about node or whether you just acknowledge, we don't need node to run and serve static files.
[00:36:46] It runs without node JS. It runs on Apache. And so we host all of a test doubles own websites. They're all lineman applications. They're all producing a static content, our blog and our website, for example. They're all on Heroku, and we don't have to pay Heroku anything. I love Heroku. Pay for Heroku. But we only need one dyno because it's just serving up static content.
[00:37:07] We have some other starter projects too that are really useful. So the vanilla project is easy if you're like a control freak like me and you want to start everything from scratch every time. But if you want to get started with Angular, we have an Angular starter project that deals with a lot of the Angularisms.
[00:37:20] For instance minification of Angular code is a little bit nuanced. So are Angular templates, and that's all built in and pre configured for you. Backbone, same story, we've got a very similar starter project for Backbone that's already set up with templating. Ember, same thing where Handlebars, no, for Grunt Handlebars isn't in Handlebars.
[00:37:39] We've configured all that for you, and we support these things. They won't fall out of sync with the frameworks. If you just clone these repos, you've got a good start on any of those, dominant Additionally, we use Lyman to build web libraries. So we maintain a whole lot of JavaScript libraries, meant to be consumed by the web.
[00:37:58] Especially Jasmine extensions and so forth, and testing. And Lyman can build those too. So we've got a template.
[00:38:07] Workflow, you can do that. And also markdown blogs. So our blog, for example, we use markdown and get hub flavored markdown in particular, as well as highlight JS for syntax highlighting. And if you want to make a blog, you can very easily clone our markdown blog example or alignment blog example, and be up and running with a blog that very quickly compiles all your markdown files into static HTML ready to deploy on top of all of the other front end asset benefits, like using SAS or less or coffee.
[00:38:39] It's very extensible. As an example, a a thing that I've never used and probably won't use is a typescript, which is a transpiler for NET. But if you want to use it, here's how you could set it up. You tell npm, hey I want to install this thing, and please save this off as a development dependency, grunt typescript.
[00:38:55] And then when I look into my package. json, grunt typescript will appear there. It's already all set up, it's already installed. And to configure lineman to know about grunt typescript, we just add a little command into our application. js. It says, load npm tasks. It's an array, and we specify we want to load grunt typescript.
[00:39:14] And then linemen is all about these kind of pre set workflows. When you prepend a task to the workflow common, that means every time you run basically any linemen command it'll have these actions that you specify prepended on. So I can say, every time you run anything, before you do anything else, compile my TypeScript for me.
[00:39:33] And then I can also include in the same file TypeScript's task configuration, however I like. And that's it. And then you're ready to go. When you say line and run from that point on, TypeScript compile will be the first, thing of significance to happen. Really easy. You don't have to understand all of Grunt, you don't have to understand a whole lot about Node.
[00:39:52] Our goal is to make it just as simple and as easy to use as possible. And really, that's been our goal all along. Is front end tooling has been too painful for too long, and as Node has become prominent, as Grunt and these tools have been made available. We just want to provide a little bit of sanity, a little bit of defaults and conventions so that people who are new to this don't have to learn everything in order to take advantage of it.
[00:40:15] Really happy to share this with you today. I hope that like I, I travel to speak about not just this tool, but this mindset cause it's a real sea change in my opinion. And I really thank you for your attention. I hope that you have a, an opportunity to check out Lineman and start engaging in this way.
[00:40:28] I also want to thank an old friend of mine named Marissa Heil. All of the illustrations in this talk that you enjoyed were done by her. The other ones were mine. She it was absolutely fantastic, and she offered just for simple attribution only to do all the illustrations for this talk.
[00:40:43] So I really appreciate her. She's a graphic designer, and she's available for contract. Again, my name's Justin. My company's called Test Double. I'd love to work with you. We love doing JavaScript. If this is the world as you see it in the future, and you'd like to work with us and have a helping hand as you get started migrating to it contact me.
[00:41:02] And we love to help out. We've always got availability to meet new people and to, start new engagement. Thanks again. I really appreciate your time. And this was a lot of fun.
[00:41:17] Thank you, Justin. It's 5 to 12. I'm sure you're all hungry, but A few questions, maybe?
[00:41:22] Hopefully an easy one. Is Lineman, or will it be, opinionated about caching and versioning of those static files that it serves up? Yeah, so those are things that we've built one off right now SHA fingerprinting on the end of files, and we haven't included it into Lineman Core yet, because we're still testing it and making sure that it's sound.
[00:41:40] But yes, that's actually probably the very next feature that's coming. Good question, but don't feel like you have to ask anything guys. If you want to just hurry up, you're also available for discussions after. Yeah, I'll hang out up here for a while. Let's thank the speaker once more and then have lunch.
[00:41:57] Thanks everybody.
