For purposes of consistency, it would make sense to use SEF URLs on everything, though you don’t necessarily need to. That’s what I generally do. My rewrite in htaccess sends requests to a file which then uses a switch statement to determine which resource is being requested then loads the appropriate include files. Adding SEF support for another page pretty much is a simple process requiring only an additional entry in the switch statement. The choice is yours whether or not to do it depending on your preferences.
Personally, I’d check for existing directory and file before redirecting on “pretty URIs.” The other “technique” would be to disallow the dot character (and /'s) in your redirections to eliminate filenames.