I think there's a few things that contributed to the steady decay of RTS:
Phone games - A.D.D. generation gets their gaming fix through all the quick playing games on your phone. No need for a high-end gaming rig, which brings me to my next point.
These unholy "gamers" are new. Even if we add one gazillon marsians starting to play phone games tomorrow, the number of CoH players are the same.
Technology escalation - RTS has often been one of the most demanding/least optimized gaming vehicles and I think it turns a lot of people off. You could just go get a whole console and a couple of games for the price of a cutting edge graphics card that lasts 2-3 years before you have to upgrade again. Seems like every time a new RTS is released the graphics emphasis is way overblown.
The same goes for many other PC games and the number of people (not percentage) prepared to buy more expensive hardware to play has increased.
Complexity - While shooters have gotten ADS/crouch/cover systems etc over the last 10 years and RPGS have gotten crafting(and a bunch of other stuff), it was all implemented more smoothly than all the new additions to RTS. RTS has gotten increasingly complicated, more difficult to balance, and less accessible.
Plenty of other more complex games around with steady fanbase: Civilization, Crusader Kings 2...
If RTS is going to make a come back a developer out there needs to take it back to it's roots. Simpler combat with a very solid rock-paper-scissor balance system , satisfying base building, and a solid minimally buggy game engine. Graphics also need to be moderate with ways to scale it way up or way down (without the game looking awful). I'd also like to see a new RTS released that only starts with 2 factions. Each faction added makes the game exponentially more complicated to balance.
This section is horrible. You wanna dumb things down and make it uglier? WTF. Would get you executed for RTS Treason if there was any law like that.
I don't get this defeatist, desillusioned thinking about "dead genre" and "dead game". If I went into gaming I would wanna be Relic: Unchallenged in RTS with a good following (millions games sold & 57th online ranked).
I guess you guys would try to sell a phone game or a FPS and the business case would be "look at all the people".