As others have said, if it is going to receive the rivet directly, then it is too deep. If a dimpled piece goes over it, it may be Ok depending on how snugly the dimpled part fits; try it with a piece of scrap dimpled but if the scrap wiggles around then it is probably too deep.
However, in that case, you can use some structural epoxy (Hysol, liquid aluminum, etc.) to fill the countersinks, re-drill and countersink again, and it will be Ok. You just want support under the dimpled part and a thin layer of epoxy will provide enough. The parts themselves have not lost significant strength. You can also use this trick on single countersunk holes where the rivet head goes directly in the countersink, but I would replace a series of holes like that, or if the countersinks were in thinner material (chance of opening up the rear face by countersinking too deep). Even then, if it's just one hole, you can make a tab to go behind the enlarged hole, fill and re-drill as before, and use a slightly longer rivet and still maintain structural integrity.
All that said, you will almost certainly feel best if you replace a part where you have made an error in several places.