Back when I was hacking on DF ~4 years ago I believe it was actually the item tracking that was ruining performance in big forts. The main loop had to touch every item in the game every frame, including things like every rock mined by your dwarves. You can usually get significant speedup in big forts just by destroying all the spare rocks (I think the best way to do this in-game was put them under a bridge and then close it on top of them).

I ended up having more fun hacking on DF than playing it, but it is an interesting game.