I actually second Deadpoet's suggestion. Drop a note to HR and keep your contact with him strictly within a business context.
This guy obviously can't get the clue even if it hits him in the face, so I don't see much point in being too "considerate". He is obviously interpreting your "niceness" as hope for him and his behaviour is at least borderline harassment. Stand up for yourself, you really shouldn't need to hide and be embarrassed about the whole thing when you're not the one at fault.
I haven't had the "good fortune" to be wooed so fervently but if that jacket thing ever happened to me, I would have marched straight to HR and lodged a complain.