Which setup works best for an early-stage PHP web application?

That’s really going to depend entirely on your specific needs and on your development team (whether that’s just you, or a group, or whatever).

Personally I’ve enjoyed tinkering with Nginx recently.

As for hosting - I’d definitely recommend VPS most of the time. Unless you’re big enough that you need dedicated, and to be honest, there’s an argument for remaining with more easily scalable solutions anyway.

With as cheap as VPS are getting now, if you have the knowledge necessary (or have someone who does on your team) to set up a small hosting VPS, I’d never recommend shared hosting. But that’s just my preference.