If you're going to make this an on-going thing, you could just spend $99 on a USB mic from Best Buy and it'll come with recording software. They're actually pretty decent broadcast mics...
He's right about very affordable USB mics. In fact, even the ones that cost a lot less may very well serve your purpose if you are only using it for speech. Just stay away from those dime a dozen ~$14 standard comes with your computer pieces of trash. Trash is what they sound like. Again, it really does depend on your final purpose. If it's to be used strictly for online listening, you have even less to be concerned with sound quality wise as most online listeners are using nothing but crappy speakers.
If you haven't already, download the Audacity software. It's free and there isn't any adware or other crap hidden in there. You'll probably end up using it to slice, dice and tweak your files. You can do all kinds of nifty things with it. Someone once said on here that they couldn't use it to make a DIY DJ mixed tape without it sounding like it was just songs back to back. (Or something like that.) Au contraire! I've done it on occassion, especially when I was making radio IDs with sound effects for my other radio show or for other DJs who liked that kind of stuff. You just have to know how to use the controls.
One little tip regarding making sound files: stick with a sample rate of 44,100 hertz as it's most compatible with the average MP3 and whatever podcast program you use might only work with that rate. (That's an issue I always have to be concerned with using Live365.) It's pretty standard but if you go to layer something with one and each have a different rate, you'll hear one's speed change. Can make for interesting sound effects but will be very frustrating otherwise.
Be aware that if you decide to lace any music within your files, you will have to be very careful about royalty issues. Any halfway decent podcast server will have a page explaining how their license works. I had looked into using one several years ago but finally decided against it for that very reason. There was no way that I wanted to stick with airing unsigned bands only along with the impossible task of any individual permissions and royalty paying for the others I would have to do on my own.
If you do want to include some music, there are websites that offer free MP3s to podcasters. Generally, you have to sign up and get an approval for them but it's pretty easy. The down side is that you have to weed through a bunch of crap to find something that suits your programming and it being something you actually like. Not easy with a load of unknowns.
Would be easier to help you if we knew a little bit more. If it's just to use as some kind of mobile tour thing like they have in museums, there are all kinds of ways to do this and should end up being very easy once you've done a couple of them.