Just cut the rubber plunger cover, a steel rod about 6" long pulls right out. The most difficult part is then removing all the glued-in remnants of the rubber cover from the hole. Go to Lowe's or Home Depot an get either a plastic or metal plug to cover the hole.
Or you could just make a small slit in the cover and remove the rod. No matter what you do, disable that pushbutton today.