Will Google follow those links to other pages. or will Google only follow the links on the initial loading of the webpages html.
Short answer: yes.
Somewhat longer answer:
Saying that however, there are certain steps that you can take yourself to make your AJAX site more search engine friendly.