You don't need a memory card right away, I still haven't bought one for my DSlite.
I suggest buying him the New Super Mario Bros. and Mario Kart DS. Those are probably the two best games for the DS and are not hard to play for kids. You will even want to play them.
I appreciate all the feedback on the PSP and DS Lite. We decided to get the DS.
Now the only question is, what do we need for my son to be fully functional on Day 1??
Should we buy a starter pack?
A memory card? Which kind and how much mem?
Anything else?
Thanks to all the previous input. I think we'll consider the PSP for him when he is older (12 or so). Who knows what they'll be offering by then--3d hologram systems???

Chowhound
Comic Vine
GameFAQs
GameSpot
Giant Bomb
TechRepublic