Aileron trim

I can see that in straight and level flight the right aileron is up and the left one is slightly down without a passenger. It's been this way since I bought it. At higher speed the rt is even with the trailing edge but the left is still deflected downward just a little. The RV never goes as faster than 172-5 MPH is level flight WOT. Is this what I should expect when flying with an empty seat?

NO!

This is a clear out of rig indication... at least that is the first place to start... ultimately there are potential underlying issues with inconsistent wing incidence, twist, or misaligned fairings and tail surfaces but a systematic and detailed review of the rigging of all flight controls and surfaces is in order as a benchmark.

I would not jump to the conclusion that you are seriously out of rig yet. Check your Wing tip and Flap alignment. It is possible that one or the other are not correct and the ailerons are aligned.
As Stephen suggests, a base line comparison should be made. One thing to try is loading up the passenger seat so it weighs the same as you and insuring you have the same fuel level in each tank to start. Fly it and see.