Problem with many business people is that they don’t hear “he estimated the deadline”. They hear “he committed to the deadline”.

Thanks for sharing. I’ve added your site to my RSS reader

Personally I’m not really convinced estimates in software are possible at all. I think we might need to simply take everyone along for the ride so they can see first-hand how the design is unfolding.

I worked for many years in companies developing mechanical and electronics products, and I can tell you that there is nothing special about software in that regard. The most important difference between software and other artifacts is its mutability and (relatively) much lower cost of engineering changes, especially in the early iterations, which carries over much better in the later iterations too!

Top-notch companies with people engaged in multi-disciplinary product development work also know by now to prototype early, and delay decisions until the gremlins have gotten worked out from the concept, the system architecture, and its design/implementation in hardware, whether that’s problems with turbine-blade resonances, casting process tolerances, RF interference, warping of parts during plastic injection molding, or durability of components due to vibrations. In those cases, once you “hit hardware”, even in prototype form, you are subject to supply-chain lead-times and loops that are far more expensive than the cost of the prototype’s production.

Having said that, there are certainly still many non-software businesses that develop in what many deride as “waterfall”–and even then, it’s practically never waterfall. It’s most likely Stage Gate / phase-gate, and even then, it’s untenable for such a business to launch a new product every 5-7 years as they once were used to. So, most non-software companies have already for years now adopted some ideas of agility (not mentioning capital-A Agility, that’s a whole different story, with rituals and certificates and other nonsense), or introduced design processes and rules of thumb / internal guidelines for the design of key components.

Everything that has uncertainty has / must have a design process behind it, that reduces the uncertainty to an ALARP degree for the product to work as expected. From the “cartridge concept” of how some mechanical components (turbochargers, rotary valves are two I’m aware of) are designed to suddenly (after decades of design stagnation) prioritize repairability/maintainability, to figuring out the trade-offs behind the bounding box of the product, the size of the FPGA, the available space for the battery, etc., e.g. in this one of which I was in charge of.

The “software exceptionalism” is misplaced, sorry.

Edit: estimation is similarly an issue in non-software businesses, too. Generally, pointy-haired bosses who don’t understand or don’t want to understand something that goes against what they’re being pressured to enforce, are what ruins things. I once was asked to develop the shaft size of new turbochargers. Eh, we have design rules for that, how hard can it be? Original estimate (by my decidedly non-PHB supervisor) was 3 weeks. Within these 3 weeks it became obvious that the design rules were useless, as they targeted shafts of single-stage turbocharging systems. My investigation for updating the design guideline with new rules for higher pressures and 2-stage turbocharging systems took 13 months. Good thing that it wasn’t on the critical path, and thanks to parametric 3D CAD we could delay decisions about the shaft’s interfaces and converge as late as possible. 3 months into the investigation the formulas for the first 2-stage system’s parameters were available (after millions of Monte-Carlo simulations), so no harm was done. But 3 weeks and 3 months is still a crazy difference. Making bold estimations without understanding what you’re really estimating is hare-brained, regardless of software or non-software work. And most people in managerial roles are far removed from the actual system, to know what they’re really asking for, or what they can expect, or what slack they should allow for.

I actually think that the program is the design, which, uniquely to software amongst engineering disciplines, is also the product.

Consider what happens if you throw away all the design and instructions once the product is complete. With a production line producing widgets, if you throw away the (possibly CA) design, you still have the widgets. With software, you just have a computer. This is why software differs from other engineering disciplines, IMO.

I might be triggering people by making comparisons to other engineering disciplines that I’m not qualified to do. I do think that the output of software development being a set of instructions makes it unique, and that estimating that process (my original comment to the blog post!) is almost impossible. I’m talking about the topics covered in publications like:

The emperor’s old clothes (1980)
https://dl.acm.org/doi/10.1145/358549.358561

Mythical man month (1975-1995)

Peopleware (1987-2016)

Or if you prefer something that’s not decades old (reading back over it, I have almost parroted this author’s points above in the comments above about building bridges and script-writing ):
Out of the software crisis (2022)

Same, and that’s proven by mine and many others’ professional practices. I had calls, on multiple occasions in the past, with colleagues about why weren’t some 2-3 problematic items inside a PR surfaced earlier during a tech planning stage. Mine, and many others’, answers were stereotypical: “Because even if we sat on a call for two hours and discussed a task breakdown, we still wouldn’t guess X and Y will become a problem. The only way for this to surface would be if we started coding it together during a screen share session, and even that does not guarantee we’d have spotted it earlier than we have in this PR”.

I get the need to want to reduce chaos but this is similar to trying to observe molecules and atoms – the deeper you go, the blurrier it gets. You have to stop while the clarity is at a satisfactory (== not ideal) level and not pursue further because you’ll hit the “last 20% of the task take 80% of the energy” part of the Pareto principle head-first, and it’s going to hurt.