When you install SP2, the new Security Center in the Control Panel will show you if it's on.
If you don't have SP2, then it's not "on" by default and you need to install Zonealarm or read how to use and configure ICF. ICF is in Start, Help and Support so I don't have to write further.
I have Windows XP installed on my computer. How can I tell whether Windows XP installed a firewall on my computer. Thank you.