One major “secret” to advancing in a technical career is learning how to give accurate estimates. It certainly has been for me...
Seems like there's a hint of survivorship bias. The author will eventually give a bad estimate. There's no secret or trick otherwise it would be widely known by now.
Even the video games industry has been coming around on this in the last decade. This is a sector of the software industry famous for making aggressive, impossible deadlines for itself. It has ruined countless lives trying to hold to them. The smart ones talk about milestones and road maps. They don't announce release dates until they're basically done and ready to cut the release.
This is the conclusion you come to after you churn staff year after year and people leave in droves and never come back.
An aggressive sales team can ruin a small company. If what they sell is a deadline and promises they can't keep your team has no control or autonomy. People feel good when they have autonomy over their work and feel in control. They get burned out when their company/career is on the line when an estimate they were forced to make blows past due to forces outside their control.
I always recommend selling on what you can control. Promise only what you can deliver: your skills, experience, and knowledge. You can try to estimate how long it will take you but you will be wrong 66% of the time. The people in those studies were also as smart, or smarter, than you. There is no secret.
There are ways to control estimates - just have good controls on time, scope and cost and make sure every stakeholder is aware of and accountable for the impact of their actions in changing expectations.
Someone (sales team, developer or anyone else) wants to radically change the scope half way through? They have to cut the scope or adjust the schedule to compensate. Software is infinitely malleable so it's tempting to just accept any change that comes along, but with unmanaged changes and complexity come missed schedules and blown deadlines - it's all very predictable and avoidable and usually caused by a dysfunctional organisation without proper communication or accountability.
This is not rocket science and while there are no secrets or perfect estimates there are certainly ways to break down most work until estimation is trivial. Sure there are exceptions (research, v. difficult new problems) but for the majority of business/consumer software I've encountered, a proper schedule is possible and software can be delivered on time and on budget, as long as the scope is properly controlled, the work is properly subdivided early on and someone is managing the entire process and keeping communication open with stakeholders so that when things change/go wrong the appropriate action is taken and everyone is aware of why.
Nothing about what you've said wasn't known and employed by people at companies who've made budgets and estimates that went over. This is why estimates and budgets go over. Everyone thinks they have it figured out.
The Taylorist principles of management don't apply to knowledge work in my experience. Refine your requirements gathering and estimation processes all you want. Your estimates will still be wrong most of the time. They're educated guesses and we don't have the foundation to make accurate ones.
One thing I think you nail though is that communication is key. A lot of good comes from being honest, transparent, forthcoming, and supportive.
I never recommend software teams and companies make estimates. I say break down tasks, make mile stones, set learning goals, and get to work. Communicate progress frequently and keep feedback coming back to the team. Working software talks. When you can see the goal in sight that's the time to start talking about release dates. Once you have that first couple of releases you then you can start developing a cadence. It's all based on evidence and what you know and making promises you can keep.
The Taylorist principles of management don't apply to knowledge work in my experience.
The simple principles above do apply in my experience, presumably there is some difference in practice.
Maybe everyone thinks they have it figured out but some people demonstrably do, because they deliver software on time which meets requirements. I agree with delivering software early and often, and that is a solid basis for delivering reliable estimates and promises you can keep.
Where estimates fail IME it's down to lack of accountability and communication among stakeholders, which leads to constantly shifting and unclear priorities and requirements.
Seems like there's a hint of survivorship bias. The author will eventually give a bad estimate. There's no secret or trick otherwise it would be widely known by now.
Even the video games industry has been coming around on this in the last decade. This is a sector of the software industry famous for making aggressive, impossible deadlines for itself. It has ruined countless lives trying to hold to them. The smart ones talk about milestones and road maps. They don't announce release dates until they're basically done and ready to cut the release.
This is the conclusion you come to after you churn staff year after year and people leave in droves and never come back.
An aggressive sales team can ruin a small company. If what they sell is a deadline and promises they can't keep your team has no control or autonomy. People feel good when they have autonomy over their work and feel in control. They get burned out when their company/career is on the line when an estimate they were forced to make blows past due to forces outside their control.
I always recommend selling on what you can control. Promise only what you can deliver: your skills, experience, and knowledge. You can try to estimate how long it will take you but you will be wrong 66% of the time. The people in those studies were also as smart, or smarter, than you. There is no secret.