It depends on the primer you use. If you go to walmart and buy an 88 cent can of spray paint primer then it's not going to last and you're gonna mess some stuff up. If you spray some automotive paint primer on it or some epoxy primer it should hold up quite well. Don't worry about primer on the car. I'll try and get some pics of my car right now lol....I'm driving it around totally primered with grey and then where I did body work there is a lighter grey primer

It's your car and you know what it will look like in the end so don't worry about what others think.