Very simple this one: Boot into your BIOS and disable hyperthreading!
Your processor will then appear to your system as the correct number of cores running at full clock speed, rather than double the number of cores running at half speed. This means VirtualBox will be able to allow virtual machines to use a core for up to the full time, instead of only half the time. Virtual machines will become much snappier, and so will the host if the VM was hanging VB, which was then hanging the host.