You can rule out electric forces by just grounding the masses to have them at equal potential, at least as long as your test masses are conductive. Magnetic forces seems harder, use a non-magnetic material does not really help, how would you know that it is not ever so slightly magnetic, maybe even just due to contamination? Maybe try different materials and find that the effect scales with mass but seems independent of the material, that could increase confidence as it seems unlikely that different test masses made of different materials would exhibit the same tiny magnetic forces. Also I would first do a calculation, I have no idea how the gravitational forces compares to say paramagnetic effects you might encounter, for all I know they could be of similar size or orders of magnitude apart.
How about deliberately putting a strong magnet close to the rotating test masses, then repeating the experiment with the magnet on the other side and verifying that it doesn't make a difference?
Of course, that might be the easiest way. Given that you can get tiny magnets that are pretty strong but only a millimeters or so in size and therefore also light, you could just stick them to your different test masses in various configurations and see if it makes any difference while leaving the mass distribution pretty much unchanged.
