Generation Records in NYC is great. It's about a block from Washington Square Park and has a lot of indie and classic punk stuff, plus a lot of live bootlegs. Even better, there are a huge assortment of used live bootlets in the basement, some going for five or six bucks per disc. I dropped about 100 bucks there last weekend, but now have about 15 new discs to check out.
In west Seattle, Easy Street Records has a great selection, plus you can get a kick-ass breakfast there too.