What do you mean by "stops working?" I have a folder view widget on the panel and have no problem with it. My folder view lists the files in a particular folder. I did try a widget linked to a folder containing other folders and each folder listed opened in the file manager, still without any problem.
You could try removing the widget and adding a new one.