You definately want to correct that now. I corrected mine (same side) last night. I removed the rudder and took a blue marker and ran a reference line about 1/32" on the skin. Then I took a die grinder and a 2" sanding disc (fine) and removed the line. Long steady slow strokes using the reference line as a "depth of cut" reference. On the final pass angle the die grinder to chamfer the inside surface to give the skin even more clearance. You should be able to get a match book cover thick piece of paper in there when the rudder is in place and FULLY moved to the left, or the paint WILL rubb off and it will look bad.
Use your hands on the skin to steady your tool. Lite pressure, using the down side of the rotating disc to sand. Practice on a scrap piece and you'll get the hang of it.
Another technique on other RV's is to simply bend the skin outward, but on the -12 there just is not the room for a bend and it would look bad. After grinding off 1/32" mine looks great, and NO rubbing.
Good luck.