I don't know what desktop you're using. I'm using LXDE, so the first screenshot will be an LXDE response.
1) Put the USB drive connector into a USB slot. You will probably get a popup requestor window like the one below. Don't mount the drive. In my example, I clicked the Cancel button.
2) Start VirtualBox. Highlight the XP VM, then select Settings. Go to the USB section. Your USB drive should be showing after clicking the Add filter from device button.
3) Put an X in the checkbox to the left of your USB drive. The drive should then show in the USB devices window.
4) Start XP in VirtualBox. After logging in, your drive should be showing in "My Computer". If it doesn't show up immediately, give it a minute.
Here's the disk usage. (Drive is empty of any files, so far.)