Available hard drive space, block sizes, and size terminology

You purchase a new hard drive, install it in your system, and run through the necessary steps to partition and format it according to your needs. Then you look at the space available and it's far less than the space advertised for the drive. This might no

You purchase a new hard drive, install it in your system, and run through the necessary steps to partition and format it according to your needs. Then you look at the space available and it's far less than the space advertised for the drive. This might not be apparent at first, but in some cases, various errors have had people questioning why this happens.

Apple discussion poster "macnut26" writes:

"I have a new iMac 2.93ghz with the 640GB drive. I made a 32GB BootCamp partition, but later chose to restore it to the full Mac OS partition. BootCamp crashed during the restore. I rebooted the iMac using the install DVD, and saw that there was 596GB in the Leopard partition and 32GB in the Windows partition. The windows partition was in blue and was unable to be changed. Being an idiot, I decided to change the partition scheme to one partition, and repartition over it. Doing this effectively erased 32GB from my drive. Disk Utility shows the full drive size as 596GB."

Many manufacturers use the literal interpretation of "thousand", "million", and "billion", for the prefixes Kilo, Mega, and Giga, respectively. This results in their calling a Kilobyte 1,000 bytes, when in reality a kilobyte to a computer is 1,024 bytes, which is based on the binary object size of 2^10 (closest number to 1000 in powers of 2), which is how the operating system and applications see a kilobyte. This results in a loss of 2.4 percent of the drive space based on definitions, which would report that 1 kilobyte as described by the manufacturer as ~0.98 kilobytes as interpreted by the computer. This scales up as you get into larger byte-size descriptions (Mega, Giga, Terra, and so on). For megabyte descriptions, the percentage lost is 4.8 percent, and for gigabyte descriptions the percentage lost is about 7.4 percent. As such, a drive that's advertised at 500GB using exactly 1 billion bytes per "gigabyte" in the description will actually be about 463GB to the computer. It's the same number of bytes, but different definitions for the terms used.

In addition, manufacturers sell drives based on how many GB can fit onto the platters, and not the amount that will be available to you after you have formatted it, because this amount may vary depending on the type of format and partition scheme you use. For the HFS format, you should see about a 6 percent to 9 percent reduction in size after formatting a drive with one partition. In addition to the the size conversions we mentioned above, extra space is used for the directory and index files used to locate and manage files on the disk.

Sometimes when errors occur during partitioning, drive sizes can be incorrectly reported which can lead to the confusion that "macnut26" experienced. Depending on the nature of the problem, this can go away after rebooting, or running disk repair routines with Disk Utility or other utility software.

In addition to the format reducing the size by a given amount, the file system's "block size" will also change the efficiency of space use on the drive. Blocks are small chunks of the drive that are written to (like "cubby holes" for data), and in most formats (including HFS ) are evenly sized. Though with some utilities you can designate the block size to use, in a standard HFS file system the block size is 4KB by default for partitions above 1GB. This 4KB size is the minimum size that a file can hold "on disk", regardless of the actual size of the file. This means that regardless of the actual size of a file, it will take up a multiple of 4KB of actual disk space (note that one block can only hold a single file, but a single file can span multiple blocks). You can test this out by opening the terminal and entering the following command:

touch test.txt

This will create a file in your home folder called "test.txt" that has nothing in it (nothing so far because we have not written anything to disk, so the file just exists as a directory entry). If you go to the Finder and get info on the file you will see it contains 0KB (is not physically "on" the disk yet--it contains no data to store). However, if you open the file and enter a single character and save, despite the single character making the file be only 1 byte in length, the file will now take up 4KB on disk; it is using one block. As you add more information to the file, the file size will grow but that 4KB will remain the "used" disk space until the file size surpasses 4KB, in which case the drive will allocate another block to the file and the "used" disk space will be 8KB.

What does this mean? Overall, as an example, if you have a drive that is 40KB in usable size, and formatted into 4KB blocks, you can "fill" it by creating 10 separate files that are only 1 byte each, and, essentially, wasting the remaining ~39,990 bytes that you could be using for new files. Granted, you could always add more information to each file and still "use" the free space that way, but you will not be able to add more files to this disk. In this sense, 4KB block sizes are exceptionally inefficient for a 40KB drive. In most instances, this will not be a problem, especially since you're using 4KB blocks and have disks so large that the number of available blocks far exceeds the number of files one would put on the disk.

System administrators will change the block size according to how disks are used, mainly taking into account the minimum size files on the disk and how many small files will be on the disk. If you have block sizes of 1MB, every time you create a small file, it will take up 1MB of space, and will fill the drive rather fast. The trade-off in creating larger disk blocks generally is faster disk access. For most uses, however, a 4KB block size is perfectly adequate and offers the best balance of space efficiency and speed for Mac OS X.

UPDATE: Small math correction. Thanks, "Wiz" ;)

Resources
  • macnut26
  • More from Late-Breakers
  •  

    Join the discussion

    Conversation powered by Livefyre

    Show Comments Hide Comments
    Latest Galleries from CNET
    The best tech products of 2014
    Does this Wi-Fi-enabled doorbell Ring true? (pictures)
    Seven tips for securing your Facebook account
    The best 3D-printing projects of 2014 (pictures)
    15 crazy old phones from a Korean museum (pictures)
    10 gloriously geeky highlights from 2014 (pictures)