Some selected buttons should go back being unselected

JavaScript
1

Please follow these steps in my jsfiddle:

  1. click on the (open grid) link and select option “13”, buttons A to N would appear

  2. click on the select all answers button, all buttons A to N would turn green.

  3. Now click on the open grid again and select option “5”. As you can see all the buttons from A to 5 are now selected (Have all turned green).

  4. Now go back to grid and choose option 13. As you can see all buttons A to N would be selected. I don’t want this to happen. What should happen is that the 5 buttons selected before the option change should remain selected and the other 8 buttons should go back being unselected, but I do not know how to do this.

I tried including this in the selectAll() function but it did not work:

$('.answerBtns:hidden').removeClass('answerBtnsOn').addClass('answerBtnsOff');

Does anyone have any ideas? Code is in jsfiddle, click here

Thank You