That makes a lot of sense, right? Stack exchange is designed around making it easy to find and get good answers to questions, not to have more involved or less focused discussions, even if they're more interesting to read.
The problem with this is the Law of Internet Monopolies (I just made the name up): monopolies on the internet form very naturally and are tough to dissolve.
The sheer size of stackoverflow/mathoverflow means it attracts many mathematicians in one place, unlike almost any other online forum I know. If you restrict some subset of questions there, it is both impossible to look for such questions elsewhere (nobody goes elsewhere) and to create a trivial fork of mathoverflow where such questions are allowed (you'll never get critical mass because people will not spend their time on two similar websites).
It's the same reason why nobody can create a better Wikipedia, no matter how much pointless arguing is on the original and how clunky is the input editor.
---
To sum up, restricting questions on MO actually forbids such questions to be asked by a large online mathematical community for a few years, until some better Q&A forum appears. And that's what makes us upset, I think.
I recently had the experience of listening to Joel Spolsky talk on the subject of seemingly arbitrary and unintuitive rules in large internet communities. Basically what I gathered from it is that in order to keep the quality of the content at a higher level, some of these rules are the best ways that this can happen. Some of these rules sound dumb, but they come from the experience of actually moderating very large communities and still trying to maintain focus on the actual goal of the site. As for your concern on this seemingly monopolizing the space of mathematicians I think this is untrue. If the demand for a place to discuss this content exists enough to have a critical mass community in the first place, surely it can exist outside of stack exchange. There is no overlap there because that is not at all the same content that stack exchange is hosting. I can certainly imagine a scenario in which I would frequent one site to answer and receive answers on questions in mathematics, but also frequent a board for casual community driven math topics.
> I recently had the experience of listening to Joel Spolsky talk on the subject of seemingly arbitrary and unintuitive rules in large internet communities. Basically what I gathered from it is that in order to keep the quality of the content at a higher level, some of these rules are the best ways that this can happen.
I understand the creators must have done something right, but I believe it is chiefly the tight UI (focus on questions instead of posts that get hidden over time) along with the strong gamification (you get a lot of power if you amass points) that kept the early adopters hooked, thereby gaining enough mass for the site to become popular.
The tightness of the questions is for me just a quirk of the creators' personalities. Look at Reddit -- also partially gamified, very loose with its rules, and massively popular. If the rules were the magic that makes or breaks fora, Reddit would never have made it.
> If the demand for a place to discuss this content exists enough to have a critical mass community in the first place, surely it can exist outside of stack exchange.
I disagree. If you look at this discussion and many others, you'll see people expressing their discontent about questions being forced out of MO/SO. Do you think the people that complain do not want such a place?
In response to the comment on Reddit, Joel mentioned this in one of his points. His response related to the part where I mentioned maintaining a high level of quality in the content. It's not that everything is bad on Reddit, but it sure is easier to filter out the bad when we have these rules on what belongs and what doesn't. In other words, things must be sacrificed if you want to have lower troll density. Even HN has it's rules, even if they are community driven at this point, but that's how many of these rules begin on other sites (citing Joel's experiences with SO and his knowledge about wikipedia).
I don't think casual fun interactions should be banned on the internet, it's just more likely to involve undesirables that certain websites are a million times more effective by avoiding. Imagine if you searched up a question on MO or SO and the first number of responses were bloated indefinitely by other answer threads discussing the topic, but maybe not in the same context, their website becomes that much more ineffective for it's desired purpose.
As far as not seeing a place and the discussion on it, I think that is more indicative of the fact that people would like a place, but nobody has been particularly motivated enough to make it happen yet. Not to mention, at least currently you can get a similar discussion by linking the historic thread on a comments section with upvotes, so it seems another competitor for that community would be sites like HN and Reddit rather than MO where you can't post on it anymore.
There are basically two approaches to designing things on the Internet. One is where you pay attention to what your users want and find ways to satisfy those needs. The other is to run around shouting NO NO NO YOU'RE DOING IT WRONG and whacking the users on the nose with a newspaper any time the do something other than you intended.
Both sorts of approaches can work; Stack Exchange is decently successful. But I find the latter hopelessly arrogant. An example is Friendster, the social networking site that preceded MySpace and Facebook. The founder wanted something where he could get dates, so that's what he built. When people started creating identities for non-people things (bands, movies, places, groups, political parties) they could have said, "Hey, look, our users are trying to tell us something. Let's built those features right!" Instead, they ran around with the banstick trying to force everybody back to the intended behavior. That worked until MySpace and Facebook came along, and now people barely remember Friendster.
Stack Overflow could have created features that channeled the "let's talk about interesting things" energy properly. It would have served their community well, and it would have given them more eyeballs on things that are questions with definite answers. Instead, they've been unrepentant dicks about it and have driven many people away.
The problem is, whatever the intentions are I still see people posting and getting away with karma magnets while interesting questions which have real answers are closed on technicalities.