That's not a perfect way to tell. Not all files are needed at all times. Some are for different sections of wordpress that you might not necessarily visit during testing.
Wordpress does a good job at depreciated core files with each upgrade. There's no need to do anything.
However--if you are seriously concerned about old files hanging around, backup all your files, then delete everything except the files in your root folder and your wp-content subfolder (or any other special custom files/folders). Then, download a fresh copy of wordpress unzip it in your root folder. However, I would not recommend this if there is actually nothing wrong with your site.