Out of the box, XP doesn't do that. Or rather Explorer does not do that. Later when we install Windows Media Player it adds that to Explorer.
Registry cleaners, iTunes and more can break this functionality but you are right. This is something (Windows) Explorer does not do out of the box.
It's hard to fault this since it's not assured as a feature in a fresh install.
Let's add it back in with a tool. Try AudioShell at http://www.softpointer.com/AudioShell.htm
When I open Windows file explorer, for my .mp3 files, I cannot get track title, number, artist, etc. It only displays file name, size, type, date created and date modified. When I right click on the file and select properties, summary, advanced, the information is not there either. This only happened in the last month or so because before I did get this missing track info for the same files on the same disk using the same Windows file explorer.
When I play them on my mp3 player or on Windows Media Player, all of the information is displayed there.
Dell Latitude D420, Windows XP.