The Bookmarks file is just an html file and is very small compared to others on your system, in the KB range, not MB or GB. In essence it is just a database, and so it just sits there until you call it up when you list your bookmarks.
Personally I don't have my bookmarks on the toolbar, but only because I prefer my browser window to be uncluttered.
Does having a lot of bookmarks(a lot of which are in folders on the toolbar) slow down Firefox performance? What do you think?