In the song, "Stay With Me", is Rod Stewart singing to a woman or a crossdressing man?

In "Lola" Ray Davies pretty much hits you over the head with the idea of the song, just in case you didn't get it but in "Stay With Me" it seems more subtle. Or maybe I'm imagining it?