A plea for lean software

Our industry lost one of its great minds on New Year’s Day as Niklaus Wirth passed at the age of 89.

My first real computer programming class in high school used Pascal, which he designed in 1970, and the majority of my college courses used Modula-2, which he also designed in 1975. Both of these languages advanced the field of programming with innovations like modules, one-pass compilation, and concurrent programming.

The type safety, clarity, and compiler support made these languages very approachable and easy to learn, likely keeping me on the path of a career in our industry. One of his greatest contributions to our field, that seems eerily prescient now, is his article “A Plea for Lean Software” published in Computer magazine in 1995.

The article laments the fact that we have been conditioned with significant advances in hardware, leading to software that is unnecessarily bloated. Humorously, the article begins by addressing how memory requirements jump “from several to many megabytes." I am writing this blog post in a browser window that is consuming 700MB while reviewing his article in Preview, clocking in at just shy of 1GB of memory consumption, almost proving his point 29 years later.

We are spoiled

The premise that software projects and programmers have become spoiled by the rapid advances in hardware is evident now more than ever.

I carry around an iPhone 15 that has 2.15 teraflops of processing power. The most powerful supercomputer at the time I graduated college in 1998 could only do 1.6 teraflops. Looking back 25 years, it’s hard to imagine software development on such antiquated hardware, but I can attest that many of the systems we developed in the late 90s and early 2000s are still running to this day.

Many of the biggest businesses with systems that need minimal downtime and no margin of error still run mainframe systems that were built in the 80s and 90s. Contrast that to the build in 3 months, run for 3 years, and then rewrite it all pattern that is so prevalent in businesses today and I think Mr. Wirth may have been on to something.

His article focused almost completely on the advances in hardware that were spoiling us, but 30 years later we also have massive advances in software that provide significant gains in programming efficiency. We operate on programming languages and frameworks that no longer require us to manually manage memory consumption. There’s largely no need for configuring parallelism in our systems for computational heavy tasks. In almost any web development language, we can largely pull in a number of prebuilt frameworks or packages that will solve a huge number of problems that he would’ve had to tackle himself.

How then was he so much more productive than most of the teams I see today?

In the article, he’s trying to distill the lessons they learned while creating the Oberon System, a text-based user interface and operating system that he and a colleague completed in 3 years. It was based upon the programming language Oberon, which they also developed in this time. Keep in mind, they designed and developed this system 37 years ago, when their workstations likely had less than 2MB of RAM and were able to perform maybe 2M instructions per second. I struggle to imagine even the best programmers that I know creating an OS and a programming language under these constraints. What led to their success?

MOAR FEATURES

A point that he makes consistently throughout the article is that, as an industry, we have tendency to build a lot of features that no one really wants or needs.

Uncontrolled software growth has also been accepted because customers have trouble distinguishing between essential features and those that are just “nice to have.”

We see this all. the. time. Software businesses that are bootstrapped or early stage have to focus maniacally on only building the features that are proven to be critical to adoption. Then as adoption picks up, a second or third round of funding comes in, and these same businesses accelerate right out of the lean startup mindset. That first sales hire brings every idea to the table that a prospect may have mentioned into the newer and longer list of critical features. Immature product management does a poor job of validating the features and pruning the list into the set of features that truly drive outcomes.

It results in large systems that are extremely difficult for end users to operate within. Simplicity is inversely proportional to the number of features available for use.

When simplicity goes away, businesses are then forced to invest in other areas like documentation, training, and customer support.

All for a user experience that is more likely to drive away new customers instead of achieving the original goal of bringing more users in. If you don’t believe me, ask yourself how often you see teams prioritize removing neglected features as part of their planning?

Any and all features mean design becomes more complicated and usability more cumbersome.

This incessant accumulation of features results in another cause for inefficiency: larger and larger software organizations.

MOAR DEVELOPERS

In my experience, overly complicated software designs are the number one reason for failure within software focused businesses.

Yet engineering leaders rarely focus on addressing complexity within their systems. I can only assume that they believe the complexity is necessary and not incidental, which leaves them with only one option to pursue: throw more bodies at the problem.

The belief that complex systems require armies of designers and programmers is wrong.

Wirth couldn’t have been more correct on this point. He built an entire operating system and a new programming language with one collaborator. Yet we hear of teams increasing their engineering personnel by 10, 20, 50, even 100s of developers. It’s so common at this point it’s become a badge of honor and a means for proving the value of an organization for the next round of investors. There’s only one problem with this line of thinking: it’s wrong.

If these businesses truly relied upon so many engineers to achieve success, we would not have had 262,582 people laid off in 2023 alone as an industry (credit layoffs.fyi). Businesses feel an inherent pressure to be first to market, otherwise they’ll have missed their window and their idea will be dead on arrival. They will do whatever they can to achieve success, including over-hiring on the basis that 10 developers will complete a project in 1/10th of the time of a single developer. As my co-founder, Justin Searls, describes in this thorough blog post, staffing engineering teams to peak levels has a multitude of bad outcomes. Not just for the business, but as we’ve seen the last couple of years, for the engineering teams themselves.

A plea for lean software teams

If we are to get back to leaner and more efficient software development organizations, we need to solve both of these problems - too many features and too many developers.

Businesses need highly capable product managers who focus relentlessly on driving outcomes, not output. It’s exciting to see many of the agile consultants I used to work with focusing more in this space. It’s one of the reasons we’re extremely excited about working with Pathfinder Product more closely. Great product managers are difficult to find, but when you have highly capable people in these roles, they have an outsized impact on the business.

Further, if we have learned anything in the last 3 years as an industry, it should be to strive for building small, highly efficient teams and avoiding bloated, overstaffed organizations at all costs. Larger teams move slower, create more incidental complexity, and are much more susceptible to the layoffs we’ve all been suffering through. Engineering leaders would be well served to focus on hiring smaller teams and providing them with sufficient time and support to create simple solutions that generate business value.

We’re known for embedding with teams, modeling strong development and product management practices, and helping them get stuff done well. Hit us up if you’d like to talk more about how we can help you build sustainable, lean software teams.