My general understanding is that when nVidia bought the company that originally developed PhysX, they basically killed the silicon product that was being offered and reimplemented the API to use the programmable shader language.

But, assuming they did keep the silicon chip around, the basic process would be that the driver would probe the card and figure out whether or not it had this PPU. If it did, then anything using the PhysX API would be directed to the PPU. Otherwise it would be directed to the GPU or CPU. The game wouldn't really know or care if you had a PPU or not. If you had one, then physics related calculations would be done by the PPU.

What you might want to do, is look into the history of 3D graphics cards, since they too started out as secondary add-in cards before 2D and 3D functions were all folded into a single chip. It's the same basic idea with the PPU, and you can also look into MPEG-2 video decoding cards that used to be sold. The basic process will be the same in both cases, so you should be able to apply the things you learn from one to the other.