You are so NOT the jerk. The only story you're the jerk in is his little BS narrative. You describe a man who has basically survived through emotionally blackmailing and managing people by intimidation to get what he wants. You owe him nothing. Blocking him was a good move.