My take
From the picture it appears you might have didferent socket contacts than what I am used to. All the socket contacts I have used are a complete straight cylinder without that step in the diameter yours have.
Regardless, they should push in the back like you are showing, but normally after the wire is soldered on. I noticed you are using soldered type contacts.
They make an insertion and removal tool, but these are just cylinders with a slit down the side.