Out of the Software Crisis

Systems-Thinking for Software Projects

Software projects keep failing, not because we don’t have the right team or tools but because our software development system is broken. Out of the Software Crisis is a guide to fixing your software projects with systems-thinking making them more resilient to change and less likely to fail.

Get the ebook for $35

Most software projects fail…

  • If they don’t go over time or over budget, they don’t fulfil their requirements.
  • If they fulfil the requirements, then the requirements were wrong.
  • If we did the user research, got the requirements right, and delivered on time and within budget, then we still fail because bugs spawn out of nowhere as soon as it gets in front of an actual user.
  • The frameworks and dependencies that were supposed to save time keep changing, forcing rewrites.
  • All this causes subsequent sprints and projects to go over time and over budget.

Even when a project goes well, that never lasts because software isn’t just for one project. Software is forever. Or, it feels like it is. It always seems to go wrong eventually.

We try to fix it with better tools and processes. Things improve for a while. Then everything starts to fall apart again, just in slower motion.

Coders and non-coders, designers and non-designers, managers and direct reports, we all try our best, and stick to our field’s best practices—we’re doing the right thing—but success never materialises.

None of our projects work nearly as well as we’d hoped. Our careers are a succession of half-baked software.

We keep writing code, but we never seem to get anywhere. We tick all the boxes and follow all the processes, but the schedule keeps slipping. Bugs keep multiplying. We’ve tried every tool and framework, but nothing seems to help…

We’ve got the design team running smoothly. The team has been churning out great designs that the managers love, but all we got after the release were complaints. The users hate it…

We’re keeping pace, every sprint is technically a success, but it keeps getting harder and harder. I don’t know how much longer we can keep this up. Our core dev has just started flaking on us…

We’re doing everything right—we’re agile and follow all of the best practices—but this still isn’t working.

You think you’re ready to make software

You’re a manager and you’ve got your processes running like clockwork. The deadlines are realistic. The requirements are clear. You’re using a hot new framework. It got you up and running in record time. The designer is cranking out designs like a machine. Your team is fired up. You’re fired up.

Or, you’re the tech lead or senior developer and you’ve been working with your manager on platform choices, testing strategy, language choice, and framework. The project’s technical decisions are sound—based on current best practices. It’s exciting. This time you got to make the right technical choices. You didn’t have to compromise because of managerial whim. It’s going to be great.

Then it turns to shit.

Nothing goes as planned. Everybody underdelivers and underperforms. You keep working on the process. You try everything. You cut back on your technologial ambitions. You try keep the code free of defects but time is running out. You follow all the best practices but aren’t getting the results you expected.

It should have been great, but it’s garbage.

You aren’t alone

You don’t suck.

Your team isn’t uniquely awful.

You aren’t doing any worse than average.

60-90% of all software projects fail. Big projects fail more than small ones. It all comes down to one inescapable fact: most software projects fail.

There must be a process out there that will get us on the right track? A method, tactic, or strategy? Maybe something technological? A programming language, or framework?

None of those are actual solutions. You don’t need a new framework. You don’t need a new programming language. You don’t need a new process. You can’t fix this with better tools because the tools aren’t broken.

The tools didn’t work for you because what was missing was a functioning system for the tools to work in.

Software development tools and processes, old and new, won’t work for you unless you understand software development as a holistic, integrated system.

Software development is an economic system whose primary purpose is to achieve business goals.

Fix your software development process with systems-thinking

Cover for the book 'Out of the Software Crisis

Out of the Software Crisis is a framework-agnostic book for coders and non-coders. Some technical knowledge is helpful for reading but not necessary. If you know what a web server is but couldn’t program one, this book is still for you. If you are a coder, the book’s concepts will help you in your work and collaboration. This book is intended to be read by the entire, cross-functional team.

This book contains everything I wished everybody on every team I’ve worked on knew and understood about software development condensed into 155 pages. It’s for anybody who has authority over processes and technical decisions, even if it’s only partial, even if your ‘team’ is just yourself.

It will explain…

  • The foundation concepts that software development is built on, why processes work for some but not others, which activities are actually responsible for generating value and which aren’t, and what it is exactly that frameworks are doing for you.

  • How all of these disparate elements of the software development process come together to form a system and how you can improve your project outcomes by adjusting the whole—how they integrate—without throwing everything out and starting from scratch.

  • The basics of how to see software development through the lens of systems-thinking and it goes into detail on how the elements of software development differ from most other products or projects.

  • The business value of unit testing and types—why skipping them will have a direct effect on economic outcomes.

  • How design creates value for the business through experimentation.

  • How to understand frameworks as systems, what they are, what they do for you, how they help, and why they can sometimes be harmful.

Get the ebook

The book is 155 pages long and comes in PDF and EPUB formats. You can buy copies with licenses to share with your team or your organisation. Or, you can buy for yourself.

Every edition will get all future corrections to the ebook for free. If I decide to add bonus materials, every edition will get them, for free, in the next update.

Get the Ebook for $35

Excerpts from the book

Introduction

That is because the hardships have been lost in a haze of nostalgia. Software has always been bad. These apps may have been better designed, but they also didn’t have to account for as much variety in screen sizes or inputs. Most of them didn’t support proper multitasking, and stability was next to nonexistent. They crashed, a lot. Accessibility just wasn’t a thing. It’s like admiring the architecture of a classic building while forgetting that it’s a condemned deathtrap and every other building like it burned down, killing every inhabitant.

Try not to kill your software development system

The concept is that the world operates as multiple layers of interconnected systems and that the systems, not the elements of the system, are what defines overall behaviour and output.

A company’s performance isn’t down to the CEO or the employees but the system they operate in. A hospital’s overall effectiveness at healing people isn’t down to the skill of its doctors but how it’s organised and structured.

Much of the field is focused on theories on how you can change or modify systems—the levers you can pull to effect lasting change—but my focus here is simple: let’s try not to murder the system. We quite often begin with functioning development systems in software, and we don’t seem to have problems getting started. But we do have a knack for just absolutely murdering the hell out of the system afterwards.

Processes aren’t about managing the team or the project. They are feedback loops that provide elements of the system—in software that would be the team—with the information they need to adjust Flows and thereby affect Stock. The process’s effectiveness depends on the circumstance and how it connects to the rest of the system. There is no such thing as a prefabricated process. You always need to implement and adjust it to your needs. In software development, all processes work up to a point and in specific contexts. When in doubt, start simple and only add complexity if you absolutely have to. You may be told that you need to implement scrum or SAFe, but how you do it and hook it up to the rest of the system matters almost as much as the process itself. Details matter.

Variability is poison to software development

While defects are very important to any software project, the business benefit of unit tests is to preserve the future economic value of the code. This isn’t quite the same thing. Unit tests make the code resistant to being devalued by change. In software, the biggest threat to the economic value of code is that everything is always in flux. The business purpose of unit tests and types isn’t to prevent defects but to prevent variations in quality. Unexpected improvements in quality are just as undesirable as discovering a bug. Other modules may well be depending on the previous unimproved state. Unit tests serve as an early warning check that tells you something changed when it shouldn’t have.

Research & taste

I don’t care if you’re a backend developer, front-end developer, product manager, project manager, designer, or Swift/Java engineer, you need to collect good software, try it, experience it, and get to know what well-made feels like. You think a boom operator on a film set isn’t a film enthusiast? Half of them, easy, can recite the best lines from Coppola’s The Conversaton from memory.

Programming as theory-building

Software is the insights of the development team made manifest. Software has no life on its own but exists as a kind of cyborg simultaneously in the programmers and the code. To reuse Donna Haraway’s words, software is simultaneously fiercely material and irreducibly imaginary (p. xii, Gray 1995). Software can be real and unreal at the same time. The written code is how software interacts with end users and other software systems. The insights and knowledge that exist in the minds of the developers are how software lives, changes, and grows. Without the code, it has no way to interact with the real world. Without the knowledge in the minds of the developers, it has no way to adapt and survive.

Maturity, Complexity, & Layers

This is the flip side of the maturity question. Many mature frameworks have evolved alongside their exemplars—the apps and software that exemplify how the framework should be used. When you attempt to build something similar using that same framework, you will often fail miserably because you are trying to extend a complex system with complex interactions. The exemplary apps won’t have the same problem because the system was simple when their projects began and grew with it. This is a frequent issue with single-vendor frameworks. Whenever you have a framework whose primary purpose is to support a single product or service, you often end up with interwoven complexity that makes using it for a new service much harder.

Table of Contents

INTRODUCTION

PART ONE: SOFTWARE DEVELOPMENT SYSTEMS

  1. It was great until it wasn’t
  2. Try not to kill your software development system
  3. How do you make good software?
  4. Variability is poison to software development
  5. Design practice requires experimentation
  6. Intrinsic versus extrinsic innovation
  7. Programming as product design
  8. Research & taste
  9. Programming as theory-building

PART TWO: HOW WE FAIL

  1. The first Software Crisis
  2. The agile reaction
  3. The many kinds of software failures
  4. Silver bullets

PART THREE: FRAMEWORKS

  1. WTF do you mean when you say ‘framework’?
  2. You need to study exemplary software
  3. Check your needs
  4. Understand the paradigms
  5. Maturity, Complexity, & Layers
  6. In the end, what else is there but work?

WHAT NEXT?

  1. Further reading
  2. References

Refund Policy

If you aren’t happy with the purchase, I’ll give you a refund, no questions asked, even beyond the EU’s 14-day statutory requirement, up to a year from purchase.

The Author

Baldur Bjarnason

My name is Baldur Bjarnason.

I’ve been a web developer for over twenty-five years. I’ve worked as a lead developer in the past. Lately, I’ve been working as a consultant. My clients in the past have included companies small and large, startups and enterprises, and both not-for-profits and for-profits.


Terms of Use

Mastodon