If you really want to get a good adhesion with primer here is the routine.
1. After all work has bee completed (drilling, deburring and dimpling) rough the parts up with red scotch bright pads. Just take the luster off the alum. All you want here is a little mechanical adhesion.
2. Clean all parts. Blow off first with air.Then use acetone or another cleaner. Do not use a wax remover or degreaser. Alum is porous and will soak this stuff up which could lead to pinholes or fisheyes. Also, us CLEAN towels. I use the blue shop towels that can be bought at Home Depot. Then I just toss them in the trash.
3. Static wipe down the parts. These can be purchased at the auto paint store. Wait for the static wipe to evaporate.
4. Primary tack cloth-don't push too hard, just a gentle brush over the surface.
5. Secondary tack cloth the parts-both of these can be purchased at the paint store also.
6. Paint the part. If you get a spot you don't like. Wait till the paint dries, then scotch bright is again and repeat the process except for the acetone part. That will just make another mess.
Hope this helps.