I'm glad you stood your ground. I've had guys follow me to my car and it's terrifying! He wasn't "just being polite" he was intimidating you, or rather, trying to and I'm sure he knows it.