Most of my reading about this was in Russian, and I don't remember the exact sources that were cited there. This particular stuff was mentioned kinda in passing in a broader conversation on evolution of human behavior, as an explanation of how it could get so complicated so fast. But searching around, it looks like there are some very specific papers on it - e.g. this looks interesting:
"This paper presents theoretical arguments and simulation results in support of our view that sexual selection creates new fi tness peaks (and thus new niches), helps species escape from
old local optima to find new, better peaks, and promotes speciation to increase the number of lineages searching for peaks."
http://www.unm.edu/~gfmiller/new_papers2/todd%201997%20biodi...
"This paper presents theoretical arguments and simulation results in support of our view that sexual selection creates new fi tness peaks (and thus new niches), helps species escape from old local optima to find new, better peaks, and promotes speciation to increase the number of lineages searching for peaks."