Here’s some guesses as to why waterfall is a thing and why it doesn’t work well for developing software.
I don’t know much about making hardware but that won’t stop me from speculating.
Building a physical thing takes time. Mass-producing a lot of physical things takes a lot of time, and also a lot of up-front work. You need to:
Because of all this effort, before you start mass-producing a thing, you need to be really sure you’re doing it right. Finding out part-way through the process of mass-producing that you got something wrong is really expensive.
Faced with this, it makes sense to break your process up into two steps:
Note that the output from step 1 here isn’t a “rough plan” of what to do. It’s a very specific, precise blueprint. Getting even minor details of things wrong in step #2 is very expensive.
Thus, the first step is probably going to involve a lot of trial and error to make sure that the blueprint actually makes sense. If you’re doing it well, it may involve a lot of speaking with the people who are going to actually use the blueprint to mass-produce things and iterating with them.
But the magic of software is that there’s no step #2.
The act of describing a program in unambiguous detail and the act of programming are one and the same.— Kevlin Henney (@KevlinHenney) August 17, 2009
Writing software is exactly producing an unambiguous description of how to do a thing. Source code is a blueprint: it’s just a blueprint implemented not by factories and people on assembly lines but by CPUs.
Any kind of English-language/block-diagram based specification of a software system that tries to be unambiguous but can’t actually be executed by a CPU is:
Once you do have the software, “implementing” it in a mass-production kind of way is just a matter of copying bits. You obviously still need machines to execute the code, devops is definitely a thing, etc., but it’s:
Nonetheless, we had this “waterfall-y” process that worked for hardware, and it had two steps: first design, then implement. So we tried to map both these steps to software development, despite the fact that it’s really only the “design” part.
I think this is the real, core failure of waterfall. It’s not necessarily a bad system in the abstract: it’s applied to software incorrectly.
Going in to step #2, we only have a vague plan of what to do. It hasn’t been battle-tested down to the details (if it really had been, you’d already have the source code – because that’s how you’d battle-test it). So of course, there’s going to be issues that come up. But the waterfall is not at all engineered around these conflicts and/or how to resolve them.
It also results in a lot of duplication: the details of your software system now encoded in both source code and one (or several) documents.
↩ February 4, 2020