From what I can see at the moment, you are loading hammer twice. Once as a stand-alone version, and again as a jQuery version.
Things are likely to perform a bit better if you remove the stand-alone version of hammer.
Also, some of your files are being requested from domains outside of your control. That can slow down the loading of the page too, such as with the bootstrap file.
With the request animation polyfill, I am left with several questions - but don't worry. I then realised that it's been obtained from a separate external source.
Also, I would move the different parts of your scripts out to separate non-minified files, which makes development easier to achieve.
Then - when you're ready to move from development to production, you can combine them all in to one minified script.
I'm running your scripts through jslint - and am finding some interesting issues there. I also notice that there are some conventions from other programming languages (such as underscore prefixing to designate private variables) , which aren't necessarily as effective as when using other techniques.
It has also helped to highlight too that you have several global variables that the pullToRefresh script relies on, which really should be resolved in some way before further optimisations are looked in to.