Looking at the metrics used in the publication[0], it seems most of them focus on the absolute number of engineers working on a given component. This makes sense — more engineers touching a component introduces more opportunities for bugs. (Edit: as other commenters have pointed out, total lines of code, highly correlated to number of engineers, is likely the best first-order predictor of bugginess.)
I bet we can improve predictive power by considering the degree of overengineering, i.e., the number of engineers working on a task (edit: or lines of code) relative to the complexity of the task they’re working on. 100 people working on a task that could be accomplished by a single person will result in a much buggier product than 100 people working on a task that actually requires 100 people. The complexity of code expands to fill available engineering capacity, regardless of how simple the underlying task is; put 100 people to work on FizzBuzz and you’ll get a FizzBuzz with the complexity of a 100 person project[1]. Unnecessary complexity results in buggier code than necessary complexity because unnecessary components have inherently unclear roles.
Edit: substitute "100 people" with "10 million lines of code" and "1 person" with "1000 lines of code" and my statement should still hold true.
My own (current) personal hell is essentially the inverse of this assumption. I work in an area that was essentially ran by 1 extremely overworked developer. The result? Now there are 40 people (including management, PMs, Scrumlords, QA and devs) doing the work originally done by a single person. The tech-debt and cruft is unbelievable (considering the domain is also quite complex). Every decision made before the 40 people were hired was made just to check off 1 of 1000 things on this person's plate... I could probably write a book about this if we are ever able to turn things around, which requires explaining to management why the things that have been 'working' for over a decade are longer working.
The sad part is, it would seem like all the engineers we have are overkill, but in my little silo, we could easily split our work into even more sub-teams, hire 12 more people, and still keep churning just to stay afloat. Sorry for the rant, I'm not sure exactly what I'm driving at. I guess I'm just trying to give a cautionary example of how not to manage large-scale software projects.
Often times, I've found situations that you describe (ones where you need to throw infinite developers at to fix) to be a smell that what your "product" does isn't something that isn't line with how your business delivers value. Such things are almost always candidates to be replaced by third party software of some kind.
Maybe I'm wrong though. When you are charting new ground, building new shit that has never been build before--which is what your product teams should be doing--you don't have years long backlogs because you can't see that far out. Good, productive feature work is iterative.
If you can see with a high degree of clarity what you will be working on 5 years from now, it probably means it's been done before and you are better off cutting a check for it.
I agree. I've even suggested to my manager that we'd be better off utilizing a 3rd party. Problem is, this is a corporate gig. We don't have a 'product', we're just a cost center for the business. Even if we outsourced to a 3rd party, that would likely take years to coordinate.
>What would be a good way to measure the comlexity of a task in some objective way?
From an existing codebase this would be very difficult to objectively assess. I think you’d have to study it empirically — come up with a set of tasks (“A”) that each takes a single programmer “P” on average a week to complete. Then come up with a set of tasks (“B”) that each takes a team “T” of 10 programmers on average a week to complete (ensure that 10 programmers is a lower bound, i.e. decreasing the number of coders causes the project to take longer). Across multiple solo programmers and teams, compare the quality of the code produced by programmers P on tasks A, teams T on tasks B, and teams T on tasks A. I’d bet P/A > T/B > T/A.
I wonder if number of engineers is strongly correlated with lines of code. This would indicate maybe lines of code is still the best predictor... will read the report to see if they bring it up!
I agree that lines of code is probably the overall best predictor to first order (didn't mean to imply otherwise; I've edited my original post to clarify). I just meant that x lines of overengineered code will almost always be buggier than x lines of non-overengineered code.
I like the "surface area" metaphor. Every where I've worked took a divide & conquer approach.
Some day, I'd love to participate in the NASA / JPL style. Everyone reviews the entire code base together. Bugs are assumed a failure of process. I guess the thinking is all bugs are shallow given enough eyeballs.
Realizing now that I'm a hypocrite (again). I hate pair programming. But do kind of enjoy code reviews. Now I don't know what I believe.
I bet we can improve predictive power by considering the degree of overengineering, i.e., the number of engineers working on a task (edit: or lines of code) relative to the complexity of the task they’re working on. 100 people working on a task that could be accomplished by a single person will result in a much buggier product than 100 people working on a task that actually requires 100 people. The complexity of code expands to fill available engineering capacity, regardless of how simple the underlying task is; put 100 people to work on FizzBuzz and you’ll get a FizzBuzz with the complexity of a 100 person project[1]. Unnecessary complexity results in buggier code than necessary complexity because unnecessary components have inherently unclear roles.
Edit: substitute "100 people" with "10 million lines of code" and "1 person" with "1000 lines of code" and my statement should still hold true.
[0] https://www.microsoft.com/en-us/research/wp-content/uploads/...
[1] https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...