Cost and weight
Leaving it off will save weight, cost and potential failure point.
Weight: fuel lines, nozzles, solenoid, fittings brackets
Cost: nozzles and solenoid are not cheap
Potential failure point: primer line fails, causing a rough running engine
With out a primer you pump raw fuel into the carb (and drips into the airbox). It evaporates as it sits and gets drawn into the engine as you crank. With real cold temps you will not evaporate the fuel as easily, making starting harder but not impossible without primer. You may just have to wait to let the fuel vaporize and may be give it a pump while cranking. When you get in pump the throttle 1 or 2 times in cold weather by the time you strap in and are ready to "CLEAR" the fuel is ready to do its thing. Most of the stories about hard cold start problems are from the fact engines crank over slower. Slow cranking is not because there is no primer. In fact primers lead to flooded engines and raw fuel in the cylinder washes the good oil away, which is not good for the engine.
The big down side of no primer, fire danger
If you are not careful and realize you are blowing raw fuel into the carb (and into the airbox), over do it, have a backfire, you can have a fire. One or two pumps are plenty. If you do have an induction fire you need to keep cranking to suck the flame into the engine. No problem. However I have seen people over due it to where they are blowing fuel out the air box drain, down the cowl and on the ground.
Use common sense and fire is not an issue.
I am going with no primer on the O-360A1A (180HP) in my RV-7. Keep in mind some small Lycomings like the O-235 (O-320?) don't have accelerator pumps in the crab to get that shot of fuel for starting. They need a primer system. Also to add a primer later would be a easy. G