It'd be a useful feature for github and bitbucket to have a default pre-commit that scans for probably-sensitive files and reject commits like that (maybe with a repo setting to opt out on the website, if you think you know what you're doing)
edit: In fact, they should probably run a full search for such files now and email a heads-up to users with repos that produce hits.
They might think of that as crossing a line - from just hosting and enabling you to do what you want to do to actually telling you what you want to do.
How exactly does this happen? I can understand that some people don't understand the need to keep their private key offline, but I would have thought those people aren't really aware of the existence of the private key...so how does it end up in a github repo?
In all likelihood, these are NOT the user's private SSH keys, but a key generated for the service that the user is developing - such as a key to run backups and issue remote commands. Thus I can well see that the key could be committed as part of an application stack.
edit: In fact, they should probably run a full search for such files now and email a heads-up to users with repos that produce hits.