Most of the things that you should be aware of you will figure out through trial and error. Problems are just waiting to happen in every corner, and as much I would hope for you to avoid them, until you actually run into them, I'd suggest not driving yourself crazy with all the things you need to be careful with.
The first steps would be to setup your "playground" ...
1) Register the domain they'd like to use (if they haven't already).
2) Sign-up with a reliable host, especially if you are doing something for the government, I'd recommend a host with 24/7 support, a host who makes frequent backups, 99.9% uptime, and feature enabled so that whatever comes to your mind you can build.
3) Put up a index.html page on the new server (coming soon), and link the domain to the new host.
4) Now create a directory on the host, just for you, and start with the website in whatever program you decide on. Most of your time will be spent here.
The only thing I can suggest is to test your site frequently. Start with the main layout, structure, design, style sheets, colors... design just the first page until you are compeltely happy... get some feeback from the people who hired you. Like I said, upload frequently and check your website with at least IE7 and Firefox. You can also run your website through (I'd recommend it) here http://validator.w3.org/ and then fix any recommendations.
Also, come back here if you run into any problems and we'd be more than happy to assist you along the way with any questions you may have!
Also, for the website, start of by keeping things SIMPLE... the best websites have a simple structure, easy navigation.... you can always expand on things later, but keeping it simple = less problems.
Good Luck!
~Sovereign