it is possible that this card using compiz is not friendly with vlc, if you are using kde desktop effects turn them off to compare(same with compiz if used)
in this case you could need to install the ati proprietary drivers available on repos
i haven't done it recently but you can find other posts in this forum explaining what packages are required and what to do after you install them
another nice player is smplayer, you can configure the video options menu in it to make it use a certain video output plugin, also with vlc, usually x11 or xv are good options