One giant sprite would be just as bad an idea as 100 small images.
Basically, I group related items together which have no real meaning (decoration only). I generally also only put together those that are small (since that's where you get the real boost). And, just for the sake of simplicity, I'll group related icons together.
If icons are slightly different sizes you can group those (just position them so they have an appropriate amount of white space around them to show only them in their given area).
The two reasons you use sprites are:
- reduce the total file size of your images.
The reason you can put together images and get a smaller next size even with the same amount of information is because each image has a certain fixed amount of "header" information that's attached to it. This means that 100 images have 100 headers, whereas one image with them all together only has 1 header, reducing the size a bit.
- reduce the number of HTTP requests to the server
If you have 100 images, 100 different connections and transactions must take place to get all of these across. If you have one bigger image, you only need one connection to transmit the image.
If your sprite isn't helping one or both of these problems, it isn't worth using a sprite. Sprites can be quite difficult to deal with in many situations, so you don't want to overuse them or you'll be pulling your hair out.