Take this with a grain of salt - I am not a controls expert, but from my long experience I can repeat what I've learned.
Surfaces that are primarily horizontal need to be balanced. The behavior of these surfaces (elevators and ailerons) is a product of aerodynamic loads, control system friction and free-play, and vertical gust loads. If a horizontal control surface is not balanced, a vertical gust creates a control deflection which, typically, is dynamically unstable.
Imagine, for illustration, that all the weight of an elevator is behind the hinge line. An upward acceleration of the airplane due to a gust will cause an elevator DOWN deflection due to the inertia of the mass behind the hinge line. This, in turn, causes a nose down pitch, creating a negative vertical acceleration of the airplane which will cause an upward elevator deflection. The cycle will repeat. Depending on the other factors - primarily true airspeed - the elevator will oscillate up and down and, with the right conditions, do so with such frequency and force that it can destroy itself.
Ailerons, even though they are "balancing" each other because they are interconnected, still need to be balanced at the hinge line. Control system slop, cable stretch, or push-rod bowing under load, gives a bit of independence to the ailerons, allowing them to suffer the same dynamic instability due to vertical gusts, if not balanced at the hinge line.
For the rudder, the gust loads are much less than the horizontal surfaces, so the deflections due to gusts is a much smaller problem. Some airplanes don't have any mass balance at all for the rudder, but some mass balance can be beneficial. However, I think the problem is small enough that perfect balance is not required.
As I said at the beginning, I'm not an expert. I'm hoping someone that is will chime in and correct the things I have misstated. I'd like to learn more!