> The performance test has a reasonable amount of noise. At first, 10% of test runs fail it, and people just rerun them. Eventually 90% of test runs fail, and people need to start dealing with it, but the effort required to get it all the way back down to 10% is unrealistic for any one person.
This is a tricky thing to deal with - (imo, again) how you deal with this is twofold:
1.) The automated test that measures performance needs to be rock solid - whatever you measure needs to be deterministic. Wall-clock time isn't always the way to go (although it's the easiest), maybe you profile and measure cpu cycles consumed by your application.
2.) If you capture performance over time then hopefully you can capture a trend-line that shows performance degrading from other changes that push the app past the performance bar. If your dev loop is fast enough you can isolate performance degradations to a small set of changes. Ideally if you handle #1 above this won't be much of an issue. In my experience changes that tank performance usually do so beyond the threshold of noise.
Regarding "newbie developer needs to learn all about performance" and "asking developers to delete code", that's why you need organizational buy-off. If your changes are regressing performance in a core area that is complex with a lot of legacy, maybe you shouldn't be making changes there, or the org needs to be aware of and sign off on letting your change go through. It doesn't always need to be a hard and fast "reject all changes", but there needs to be a discussion.
This is a tricky thing to deal with - (imo, again) how you deal with this is twofold:
1.) The automated test that measures performance needs to be rock solid - whatever you measure needs to be deterministic. Wall-clock time isn't always the way to go (although it's the easiest), maybe you profile and measure cpu cycles consumed by your application.
2.) If you capture performance over time then hopefully you can capture a trend-line that shows performance degrading from other changes that push the app past the performance bar. If your dev loop is fast enough you can isolate performance degradations to a small set of changes. Ideally if you handle #1 above this won't be much of an issue. In my experience changes that tank performance usually do so beyond the threshold of noise.
Regarding "newbie developer needs to learn all about performance" and "asking developers to delete code", that's why you need organizational buy-off. If your changes are regressing performance in a core area that is complex with a lot of legacy, maybe you shouldn't be making changes there, or the org needs to be aware of and sign off on letting your change go through. It doesn't always need to be a hard and fast "reject all changes", but there needs to be a discussion.