You don't need to use a countersink-style tool to deburr holes. Most of us are too agressive with those anyway, beveling the edge of the hole. Use a scotchbrite pad or a small wheel in a dremel tool. Don't try to work just over the hole or you will make a shallow divot. Instead, polish the entire flange surface.