Polly's is still around although it has lost a lot of its charms since changing owners a few years ago. Last time I was in there the new owners couldn't be bothered to fix the spotty satellite radio reception, which kept cutting out unless the bartender stood in a specific spot, and they'd stopped accepting credit cards -- often a sign of a business on its last legs. That was many months ago, hopefully things have changed, because it was and could be again a great little hangout.
Don't know about Duffy's... I've been there once, and maybe it was a bad day, but the food was "meh," the atmosphere totally lacking.
Personally I'd walk another block down to DC9 or over to the Velvet Lounge, both are nice downstairs hangouts.
I second the Saloon as a choice too -- good beer selection and the cheese steaks are divine. One must be warned that the owner has some rather quirky policies in place, such as no standing, and no loud talking. It's not like it's a library in there, though. The rules mainly seem aimed at giving the owner some leverage in minimizing the types who think that having a drink requires them to act like they're freshmen at a college frat party, so it's all good.