It's definitely worth changing it. The good news is that it's very easy to do.
I use SSI (server-side includes) on several of my websites. You just include a line like
<!--#include virtual="/menu.htm" -->
at the appropriate point, and the server will insert the contents of
/menu.htm at that point. The included file should just be an HTML fragment, not a complete page, so that it makes sense in the context of the page(s) it's being dropped into. Then you just need to configure your server to parse .htm files as SHTML in order for the SSI to work (if you call them .stm or .shtml they will automatically be parsed as SHTML, but that involves renaming or redirecting all your current files, so it's easier to change the server settings); just add the following lines to your
AddHandler server-parsed .htm
You can also use PHP to achieve the same result, but the code setup is different.