Scratches that may penetrate the alclad layer of the metal may allow for corrosion to form, so leaving it on as long as possible is also an attempt to help prevent that from happening. I agree that it is a double edged sword, in that the longer you leave it on the harder it is to remove. However, if you are making steady progress on your build it should not be too much of a concern. It is totally your option to take it all off or leave some of it on. I chose to keep as much of it on for as long as possible, removing only the sections needed for riveting. One of my reasons for doing that was to limit the amount of primer I used as part of a weight reduction strategy.
An added tip - if you use the soldering iron trick to cut the rivet sections out of the vinyl, be sure to dull the tip of the iron on a scotch brite wheel or sand paper, and keep the iron at a proper temp and use a slow steady speed to keep from scratching the metal with the soldering iron. Ask me how I know...