On dynamically allocated vdisks, freed space on a guest is never released back to the host once freed.
This is by design.
The procedure for shrinking (compacting in VirtualBox parlance) is straightforward and consists of a series of steps.
- Delete files on the guest to achieve the desired amount of free space
- Zero free space out with an appropriate utility
- Shut down guest
- Compact disk
Zeroing guest free space is simple: Microsoft provides an excellent utility through its Sysinternals group called SDelete. Download the program, extract it from the archive, and execute it on the disk to be zeroed. _Note: this procedure only zeroes free space._
One the zeroing procedure is completed, you may power off the guest and compact the virtual disk from the command line.
VBoxManage modifyvdi /path/to/machine.vdi compact
Note: This was tested against VirtualBox 3.0.4 r50677