Do you think I would scare him away?

Why don't you and a girlfriend go for lunch at the restaurant, if he pops up at you table, be surprised and see where it goes....you could trade numbers by the end of the meal.