Thanks for your answer. I've tried your solution, it worked well but as I received new specifications, it couldn't work anymore.
My problem is that the inputs can be of different size and the buttons needs to be aligned with the largest input.
Here's an example:
To achieve this, I had to set my label to a width of 50% and text-align right, which placed my labels pretty much in the middle of the fieldset. Then, I simply placed my inputs next to it and it was considered to be fine so far, even with multiple size inputs. However, the button is placed using a margin-right directly on its HTML tag since EACH page needs a different margin for this button, this is where it gets troublesome.
I tried putting it back inside a div to be able to do this easily, but I had some trouble when the inputs were not of the same size, when I tried to simply play with the width of my labels, they would break since they would take 50% of the new div instead of the whole fieldset...
Would you have any idea how to make the input part of my form be as a whole ? So the button can easily be placed at the bottom right according to the largest input ?