I think that the successful genres of a particular period are reflections of the needs and thoughts and social struggles of that time. When you see a bunch of similar projects meeting with success, you've found a place in the social landscape where a particular story (or moral or scenario) speaks to readers. You've found a place where the things that stories offer are most needed.
And since the thing that stories most often offer is comfort, you've found someplace rich with anxiety and uncertainty.
On his blog, talking about genre