Your guest is limited not by the hardware or its drivers, but by VBox, and the best video drivers the guest can use are built into the Guest Additions. In some cases with graphics cards that have very high resolution, you simply cannot access some higher resolutions because VBox just doesn't go there.
I've had guest resolutions in VirtualBox way above the capabilities of my physical hardware! The VBox driver is certainly capable (if the physical hardware is willing ).
I stand by the monitor setting. In the guest. Try it.
And I agree, the monitor needs to set to the right one, too.
However, setting the graphics card resolution still depends on the Guest Additions being configured properly, and that is not an ATI or Nvidia setting. When you go to PCC, you will be setting a VBox Graphics Adapter. When the Guest Additions aren't right, VBox defaults to VESA graphics settings.
It's also very difficult to get VBox to go beyond the video settings that the host has to begin with, but easy to get it to match. So, based on this...
The problem is that I can't get a resolution of 1280x1024 in the virtual machine even after installing the guest additions. PCC won't let me set anything above 1024x768.
There was some kind of error message when I installed the guest additions which ended kind of "if ...... you can safely install this file." Maybe I can post that in the next days.
... I'm still wondering if the Guest Additions were actually installed correctly, if the host resolution is set at 1280x1024. If changing the monitor setting doesn't work, that's the next thing I'd look at.