For some things I use native LAMP on an ubuntu box or native mysql and whatever server I'm playing with in rails/django, but for the last couple of months I've just been using XAMPP locally on my machine-- it's pretty easy to get going.
I usually develop entirely locally, unless it's something super trivial like a WP based marketing site that will be just a couple of template customizations.
Generally, I will create a fresh install locally, update it, install the plugs I will use and either the bare theme I will be developing or the theme that I will be starting with. When I get to a point where I am ready to show it to the client (assuming that I'm not using git or something), I then do aan export from phpmyadmin locally and then import that and the files into the host, and I just add the constants for the site URL. Then I either work mostly on the remote site or I hand sync the files between my local the the remote... usually it's no big deal to just, say, delete a plugin or theme I'm working on in the remote and and upload my current version, which I find easier than remembering which 5 files I've been working on for the last 4 hours.
That's the basic gist of what I do. There are some things to watch out for-- like, the paths change between your environments. So I have a shortcode I wrote that basically puts the blogURL wherever I need it. Between that and making sure to never hard code the blogURL in my files, it's not really a problem.
However, one cool thing that I've been doing off an on is to simply put the final domain name into my hosts file-- that way I can be sure that all the URLS work on the remote without having to take the site live.... and you can do the same thing on your test hosting wihtout taking the site live by pointing the A record for the domain to the hosting.