In the same way that a SEO company can, up to a point, boost your raking, it can do all kinds of stuff to take it down... but for what purspose? if you're a customer, they want to keep you happy and that means doing everything possible to increase that ranking.
I agree with everything that's been said above up to a point. I wouldn't give username and password to access to all my site. But creating a FTP account just for them, doesn't necessarily mean that they can access the whole site.
Let me explain myself. FTP means File Transfer Protocol. Its name is very self-explanatory, I think. When you download a file, or you upload it, or any transfer files, you're using this protocol.
When you contracted your hosting, the hositing company gave you a FTP account so you could upload you website files. Else, your site would not be visibile because the website wouldn't be in the hosting computer at all. So it is necessary.
Now, if your hosting allows you to create more that one FTP account then you can create a second account for them. The thing is that when you create that account, you have the choice to decide which folder they have access to. And this is the difficult decision. If you give them access to the root folder, then they will be able to change everything. If you give them access to a subfolder, they will only have control over the files and folder under it.
Now, why a SEO company would want to access your files and make changes to them?
Because the way you code your page, how semantic is your HTML, the words that you use in your title and in your content, the number of HTTP requests needed to load the full page... everything, every little thing, affects your ranking. Therefore, they want to speed up results by taking control over code and content.
SEO is done in a page-per-page basis.
As it's been said, this is extremely dangerous and can have some nasty consequences. As an example, if you're using a CMS, what needs recoding is the templates used and not one page itself. They can mess up big time.
But it is also true that things run faster if they take that control. And still... would I give them so much power?
This is a question of trust, reputation and honesty. I think I would follow a third path... let them send me the changes that they want to do and I will approve those changes and then implement them myself, more than given them full access.
And only if I see it necessary and depending on what this company wants to do... I may create a FTP account for them that will allow them to access a specific subfolder... if the file they load needs moving to some other folder, I may do it myself.
Just in case
Better to prevent