I had these shimmed as close as one can get given the working area, but there is always a chance I was slightly off. I even went with two thin washers instead of one thick washer in one case because of the thousandths of difference between the two was what was needed. Being an engineer myself, I have an attention to detail that usually means I aim closer to perfection rather than good enough. Lube definitely wasn't the issue either, that center ball was turning freely and still was partially lubed since last annual. There is no play in the center ball and its race, only between the middle ring and the press fit outer ring. This appears to be due to the lateral load on the bearings, of which I'm pretty sure they are not designed for.
I likely do many more barrel rolls than most, maybe a contributing factor, maybe not. I'd be really surprised if I am the only one with this issue, it was small enough of play that I almost missed it...luckily these were not that difficult to remove from the plane, and I will be watching what happens with the new/fixed set even more closely. Replacing is easy enough other than the work involved in either prepping completely new F-633s, or figuring out how to press fit new bearings. Van's also suggested staking the bearings might be good enough to fix this.
Thanks for all the replies!