How I did it
Mine turned out fine the first time.
I think the trick is to make sure it is held firmly in position when you drill and rivet, from the inside of the fuselage.
I taped mine firmly in place (Duct tape or packing tape) and then strapped it down with ratcheting straps.
Once inside, make sure the ribs and inner door are correctly positioned and then drill and cleco. I used clear packing tape to hold everything in place. It's like having an extra set of hands.
In my opinion, it's better to do this after the top skin is riveted on. If you do it prior to riveting, you run the risk of the skin being pulled tighter and the baggage door may not match as well.
Using this method worked for me and should work for you, too.
Make sure you have the hinge installed as part of the assembly.
Good luck.