I'm afraid that I don't fully agree with this but I'm not going to start a war here. I personally haven't seen IE being faster than FF, at least in any of my computers.
How fast is a browser should depends on its enigne but I'm afraid that plug-ins and add-ons may slow it down, depending on what the plug-ín does and how much memory it takes. So deactivating/uninstalling the plug-íns that you don't use will increase its performace.
Also, you may want to tweek the configuration. To do so, write "about:config" in the address bar and then hit enter.
For each entry, you can find information here http://kb.mozillazine.org/Firefox_:_FAQs_:_About:config_Entries
or something a bit more direct http://www.wikihow.com/Make-Firefox-Load-Pages-Faster
Some other tips, different tricks can be found here http://www.techradar.com/news/software/applications/8-hacks-to-make-firefox-ridiculously-fast-468317
Or you can find other browsers based on FF which are lighter and, therefore, faster, such as swiftfox.
And, of course, you can also try to use some other completely different browser as suggested, like Opera or Chrome