No amount of overclocking is really going to help the performance of a Celeron, so why risk frying a perfectly good non-gaming CPU?
It also often leads to system instability, and the excess heat generated can damage other components in your system.
There are a couple of different ways you can overclock a CPU if you don't mind causing massive damage to your system. The easiest is to just increase the FSB speed, which forces an overclocking of the CPU to keep everything in sync. You can also fiddle with the voltages, but not all motherboards have any mechanisms for this sort of thing.
You can probably find far more specific information using a search engine, or a hardware oriented site like Tom's Hardware, than coming to an XP oriented forum like this. I'd just leave well enough alone. If you don't game, then that Celeron is likely to be considerably more than adequate, and if you do game, the extra on-die cache on a Pentium 4 chip would improve performance well beyond what overclocking a Celeron could ever hope to. Leaving very little point to overclocking a Celeron aside from just saying you did it.
I have a celeron D prossesor (i think 335) running at a standard 2.8 gigs (and yes i know you all grimaced when you read celeron D, i do to, but i has always done me well) anyway, i can build computers with pretty much no problem, but im for the most part ignorant when it comes to cpu's. I know you have to change jumper setting on the mobo, and kick up the voltage in the bios, but i really dont know where to start. It seems like just about everyone does it that is knowledgeable in computers. My cpu fan is not stock but its not much better for the most part, but i only wanna do to 3.0 maybe 3.1 so that shouldnt be a problem. Anyway, i just wanted to know where do i get started with this. Also, im not gonna be mad if i fry my cpu, if i wanna overclock it, thats my own fault, so you dont have to tell me your not responsible