I don't think the idea is to get with you or date you, at keast not exactly. I think they want to force you to interact with them. It ultimately doesn't matter what you would even have said to him, whether positive or negative, what I think he wanted was for you to be forced to acknowledge his presence.

I've encountered quite a few men who feel utterly entitled to a woman's attention, so much in fact, they act like if a woman doesnt give them any form of attention or interaction, that it somehow impacts their manhood! Sometimes it's not even about you. It's to impress the other men watching the two of you. Interacting with him legitimizes him in front of the other men.