awesome...new zealand music discussion!
the bats are indeed a fantastic band, and i am jealous of bookert because i have still not seen them live. daddy's highway is considered their best, but all of their albums are solid. my personal favorite is couchmaster (1995), which really reminds me of yo la tengo. in fact, it is safe to say that YLT is highly influenced by the bats, the clean, and the verlaines (who were aussie, but might as well have been kiwi).
the clean were, arguably, the velvet underground of new zealand. every single thing they've ever recorded is great, no exaggeration. when they opened for built to spill in 2001, i thought they were easily the best band that night. if you're looking for a place to start, i completely agree with booker's suggestion.
another great NZ pop band was the chills, whose bassist, justin harwood, was in luna.
i love all things flying nun and expressway; new zealand bands have been responsible for some of the most amazing music of the last 25 years, from jangly indie-rock to drony space-rock and abstract noise. i could go on for another hour about tall dwarfs/chris knox, roy montgomery, dead c, jean paul sartre experience, gate, and bailterspace, among others.