It may be that you're under-dimpling in the first place. When I started my empennage, my technical counselor visited my shop. He saw the kind of depressions you describe, around most of my rivets. I had seen them, too, and had managed to fix many of them the way you describe. He told me this was the most common thing he saw among new builders and that I was under-dimpling. He showed me how he did dimpling and I was surprised to learn a few things.
Using a C-frame, he hit each dimple twice with the hammer. Once to form the dimple, and a second good solid whack to really finish it. I was surprised how hard he hit it. "Beat that aluminum into submission. Don't be shy."
I have a DRDT-2 dimpler, and to use it correctly, I learned to adjust the ram downward enough so that the two dies come together solidly by the time the handle is pulled about 3/4 of the way down. This is with no material in the dimpler. When I'm dimpling, you can literally see the frame of the DRDT-2 flex from all the pressure. I hit each dimple twice, just like he did on the C-frame. I've had good results ever since, with no more depressions around the rivets. Try it on some scrap and see if you notice a difference.
