**I'm speaking only of Windows Mobile Pocket PCs...Palms are devices I simply won't touch with a ten-foot pole.**
The first step, like Bob said, is password-protecting the device. However, you must know the limitations.
The password will protect all data stored on the PDA's main memory for as long as it takes the lucky finder to crack it (hope that it's a long password). I don't know of any programs currently available to crack a Windows Mobile password if the device is locked. (Just have to crack them the old fashion way...sacrafice the next several years of your life.)
The storage card, obviously is not protected by the WM password, and neither is the built-in storage. Aside from guessing the password, the device cannot be used unless the lucky one performs a hard reset, deleting the primary memory and restoring it to factory-like condition. However, any data that was stored in the built-in storage remains, and is now open for public view. (Thus it makes sense not to store any personal info there.)
For added security, try downloading Nice Start from Trancreative. It replaces the standard WM password utility with one that allows customization. Aside from the customizable skinned start screen, you can set the device to lock after so many incorrect attempts for a certain period of time. You can also set it to perform a hard reset after a certain number of correct attempts, ensuring that the finder does not have the opportunity to try the dictionary. The trial version is unlimited access, but with a small nag screen after you enter your password...just click ok. Or, you can just buy the full version for $10.
In addition, download the free program Mobile Encrypter from AirScanner. It allows you to encrypt and decrypt individual files using a secure password, even on a storage card. It also lets you drag your files into a secure folder which can be opened with a password, then sealed by "dismounting." (It creates a duplicate of your files here, so be sure to erase the originals.)
As far as using wireless services, it's pretty much an open book unless you are using an encrypted connection (WEP or WPA for WiFi). Clicking on a connection and looking under "Network Key" should tell you what type of network you are connected to. There are some good articles on CNET for information on setting up your own secure conenction. (Otherwise someone may use your internet to perform illegal activities and it's traced back to you.)
Finally, I'm assuming you mean latest PDA features in general, not specifically security. The fastest is 624MHz, with a standard of 64MB RAM and 64MB ROM. (Many have more memory, especially those media-oriented.) VGA screens are faily new (came out late-fall to early-winter), and offer enhanced resolution at the expense of price and battery life. WiFi and Bluetooth are standard in non-budget models. Watch for battery life and easy-to-use buttons. Last, but not least, some PDAs come only with SD memory slots, while most support CompactFlash as well.
Hope this helps,
John