This is my first proper website, so I can't claim to be an expert on hosting etc. I am an IT professional, but you don't have to be to set up a server like mine. I'd just like to document what's worked for me, and a few ideas. Basically, to set up your own self-hosted website, you'll need to take care of the following:
Dedicated server hardware can be expensive, but for a personal site, that doesn't host any critical resources or applications, you don't need it. A second-hand PC will happily run web server software. The demand for new high-end machines, to run increasingly resource-hungry desktop applications, has resulted in a good second-hand market for machines that are quite adequate for our purpose here. Check your local small-ads, and, if you live near a university, they may well have second-hand kit for sale. You might even be able to pick up an old PC for free - mine was from Freecycle. You don't need a sound card, or a fancy graphics card (or even any graphics card at all, though it helps at the installation stage).
You don't need to pay anything for a good operating system. GNU/Linux and various BSD variants are free, stable and secure. Installation is much easier than it used to be - a beginner-friendly Linux distribution like Ubuntu is, I'd say, no harder to install than Windows. Speaking of Windows, it's also an option if you're that way inclined, but be aware that you'll have to pay for a licence, and there are many more viruses available for this OS.
Again, no need to spend money on web server software. One of the most popular web servers, Apache, is completely free, and runs on a wide variety of operating systems, including those mentioned above. There are many other options, but Apache seems to be the obvious choice - at the time of writing, Apache has a market share of over 50% (if a free application can be said to participate in a "market").
Windows users have the option of IIS, which is also very popular
If you want your server to deliver truly interactive content (whether it's a simple guestbook or a full-blown web application), you'll need to use server-side scripting. There are many languages to choose from, and again, it shouldn't cost you anything. Perl and Python are both useful scripting languages which perform well when used for server-side scripting, while PHP was developed specifically for this purpose. Windows users can use ASP (not that I'd recommend it). Ruby is apparently also very good, but I have little familiarity with it, so can't really advise you.
Java, while not a scripting language, also has its role for web development, and is used widely for enterprise web applications and services. A full discussion of this interesting language is beyond the scope of this page.
A Database server may also prove useful, and as usual, there are some good free ones to choose from. MySQL, PostgreSQL and Firebird are possibilities, among others.
You can create web pages using a simple text editor such as Notepad, or for old-school credibility, with a command-line text editor like Nano, Vi or Emacs, but a more fully-featured editor makes the job easier. For Linux users, Gedit and Bluefish are available (I'm writing this using Bluefish). For Windows, Notepad++ is a good choice (and is free, of course).
Alternatively, you may want to use a graphical web page editor. Adobe Dreamweaver is very well regarded (but costs money), while Amaya is a free browser/editor created by the World Wide Web Consortium, and the Mozilla / Seamonkey suite also has an editor built in. Of course, MS Frontpage also exists, and I have no way to stop you using it...
Of course, none of the above resources are any use without the knowledge to use them. Good-quality web tutorials exist for most (probably all) of the above. There are some excellent (if a little Windows-biased) tutorials at W3Schools. For Perl scripting, Perlmonks.org is a mine of useful information, while the Python website has its own tutorial. When it comes to books, you can't go too far wrong with the O'Reilly press
I hope this little guide has been of use to you. Good luck with any projects you have planned.