Author Topic: Dolphin, Samba, network test please.  (Read 1774 times)

Offline ThirdOfSix

  • Hero Member
  • *****
  • Posts: 745
Dolphin, Samba, network test please.
« on: January 06, 2010, 02:07:17 AM »
Would someone who has hardware installed kde4, a network with samba shares, and movie files on the samba share please do a test.

On four separate computers and multiple different machines containing movie files on the network, I am getting a very annoying condition that nobody else has yet mentioned.

If I access a samba share with icons for movies in various formats by using Dolphin the system starts many unintended downloads into the kde4 machine from the open share folder.

To duplicate this, first open a Konquerer window and set it to show icons sorted by date. Also set it to show file size below the icons.

In that konqueror window, open  /temp/kde-username.

This is just so that you can see the effect of the problem with dolphin and samba.

Now open dolphin and navigate to a samba share containing movies..preferably 1G to 4G in size so that the effect lasts long enough to see.

I have not checked to see if the display format matters, so be sure to have dolphin showing the contents of the share folder as icons.

Now, just hover the cursor over any movie icon and watch the Konqueror window. Or, if you have a network switch connected to your kde4 machine with lights, watch the lights then the Konqueror window.

If yours behaves as all of my installs behave, whatever file you hover over will start to be downloaded into a temp folder and will be visible in the Konqueror window as the last item showing and with an incrementing file size indication.

It looks as though it is doing a read ahead of the file. The only catch is that once it starts the download, it will not stop until it has downloaded the whole file.

You can tell that it is downloading that particular file by clicking on the icon in the Konqueror window at which time the movie will play.

Now this alone would not be too bad. The thing is that if instead of hovering over the icon, you click on it or drag in order to copy it to your desk top or another folder, it will still start the undesired download to the temp folder and sometimes several at once will start in addition to the intended copy.

Now you have the same file being copied to up to five destinations at the same time which really thrashes the hard drive of the source.

It also loads down the network. While trying to figure out what was happening, I had my 100Mbps network loaded to 87% capacity with unintended transfers.

This is not good. It really ties up the drive on the source machine as well as the drive on the kde4 machine.

Now, if this were not bad enough, just moving the cursor around in the source window will start the same thing happening with several source files while it attempts to make multiple copies of each.

The first noticeable effect of all of this if you are not monitoring with a network switch with lights or with Konqueror watching the temp folder is that an intended file transfer from the source folder that should take 4 minutes will show as needing hours to transfer due to all the undesired activity.


Also, once you get a lot of these transfers going, it continues even if you shut down the Dolphin window that you started it all from.

Needless to say, this makes kde4 have very random response times to everything that you tell it to do just because the local hard drive is so busy writing all these files.

I do not know if this only happens with movie files. I have only tested with them because that is the first place I saw it happening and they are the only files large enough for the condition to build to disastrous levels.

Smaller files just simply finish their temp folder copy before enough other copies start in order to slow the system down too much before any of the copies can finish.

The files I have tested with are various movie files that show as AVI, MPEG, and others that I have forgotten.

I have eliminated almost everything except the backbone of the network itself.

But, in truth, I believe that I have even eliminated that.

I installed the Mepis 8.5 beta3 distribution and it works flawlesly on the same machines accessing the same folders and movies.

By the way, these are all installed copies of PCLOS and Mepis. They are actual hardware installs not virtual installs on VirtualBox.

I even installed the latest test kernel from testing which is the same kernel that Mepis is using. It had no effect.

Please understand that I have tried this on four different machines, two were Intel machines and two were AMD.

While this is an extreme nuisance to me, I can get around it on my own system (I can access everything normally using Konqueror). But, in my efforts to convert family members to PCLOS, it is an absolute show stopper as they all have networks with many stored movies.

Offline Roc4fun

  • Hero Member
  • *****
  • Posts: 620
  • The life so short, the craft so long to learn
Re: Dolphin, Samba, network test please.
« Reply #1 on: January 07, 2010, 02:19:32 PM »
Hi ThirdOfSix,  I'm not at my PCLOS machine, and can't test your issue right now, but I do have a similar setup with a network server and media files...

One thing that comes to mind is if Dolphin is trying to "preview" the media files from the server to your desktop when you hover the cursor.

You might try turning off previews for some file types, and see of this helps your problem.

Dave 

Offline ThirdOfSix

  • Hero Member
  • *****
  • Posts: 745
Re: Dolphin, Samba, network test please.
« Reply #2 on: January 07, 2010, 05:22:00 PM »
Thanks for the reply.

Yes, I have suspected that maybe it is a preview or read ahead function that has gone astray.

I'll check into what you suggest. But, the tmp files are not being created in the desktop folder where I would expect them to be for a preview...but, what do I know?

An example is, right now, I told it to transfer a 3.1 GB file from the net drive to the desktop.

It is creating one file in the desktop folder and three in the tmp folder.

What is most annoying is that the process that is writing to the desktop is doing so at half the speed of the first one that started writing to the tmp  folder. The other two being created in the tmp folder are doing so at the same speed as the one in the desktop folder.

So, it appears that the first of these transfers is always being done with a process that has a higher priority than the intended transfer.

What is more, these unintended files copy to completion and then are deleted... I think. They are not used as a cache for the creation of the file on the desktop.

If you do not go and look in the tmp folder, there is no indication to the user that this is even happening...other than the very slow transfer to the intended location.

Anyway, I will go and poke around in the verious configuration areas and look for anything referring to preview or read ahead.

Thanks for the suggestion.

At this point, I just want for somebody else to be able to confirm that it happens to them too. That way I can rule out anything weird happening on my network that is causing it.

Offline ThirdOfSix

  • Hero Member
  • *****
  • Posts: 745
Re: Dolphin, Samba, network test please.
« Reply #3 on: January 07, 2010, 05:59:49 PM »
Thanks rudge, I'll check that. I found that I could delete the tmp files and the transfer to the desktop continued. But that was before I changed Konqueror to sort by date and I may have missed a valid tmp folder.

I'll try it again later tonight. When I was experimenting before, I could not tell if the desktop transfer was still working right because it only shows a fixed size on the desktop of a few hundred kbytes until the transfer is done. Since then, I have found that if I look in the desktop folder with Konqueror or Dolphin set to show size, I can watch the size increment.

Offline Rudge

  • Hero Member
  • *****
  • Posts: 9676
  • I'm Just A Dog.
Re: Dolphin, Samba, network test please.
« Reply #4 on: January 07, 2010, 06:10:56 PM »
I removed my post because I am an idiot, but I know that back in my old unix days it worked that way. A .tmp file was always the primary working file until the process was complete.
« Last Edit: January 07, 2010, 06:13:21 PM by rudge »


-If you wish to make an apple pie from scratch, you must first invent the universe-  Carl Sagan

Offline Rudge

  • Hero Member
  • *****
  • Posts: 9676
  • I'm Just A Dog.
Re: Dolphin, Samba, network test please.
« Reply #5 on: January 07, 2010, 06:19:28 PM »
It could also be that opening any directory automatically starts a calculation of file sizes and property's. That could be what you are seeing.  ???  Having another beer.


-If you wish to make an apple pie from scratch, you must first invent the universe-  Carl Sagan

Offline ThirdOfSix

  • Hero Member
  • *****
  • Posts: 745
Re: Dolphin, Samba, network test please.
« Reply #6 on: January 07, 2010, 06:31:44 PM »
rudge,

I don't think you are an idiot. I fully expected it to work as you said too.

Apparently though, it does not.  I watched as the downloads came to completion. The only file in the tmp folder that was as large or larger than the one in the desktop folder finished and was deleted while the desktop file was still incrementing.

Also, it is not just a calculation that I am seeing. I can click on any of these files in the tmp folder and Dragon Player will play them while they are being written and they are all different sizes and will play up to the point represented by the size indication. They are real files.

You can pretty much tell this anyway by counting them and multiplying the normal transfer time for that size of file and come up with the time that the overall transfer takes.

Offline Rudge

  • Hero Member
  • *****
  • Posts: 9676
  • I'm Just A Dog.
Re: Dolphin, Samba, network test please.
« Reply #7 on: January 07, 2010, 06:44:39 PM »
"The thing is that if instead of hovering over the icon, you click on it or drag in order to copy it to your desk top or another folder, it will still start the undesired download to the temp folder and sometimes several at once will start in addition to the intended copy.

Now you have the same file being copied to up to five destinations at the same time which really thrashes the hard drive of the source.

It also loads down the network. While trying to figure out what was happening, I had my 100Mbps network loaded to 87% capacity with unintended transfers."

As I understand it, this the root of your problem?


-If you wish to make an apple pie from scratch, you must first invent the universe-  Carl Sagan

Offline ThirdOfSix

  • Hero Member
  • *****
  • Posts: 745
Re: Dolphin, Samba, network test please.
« Reply #8 on: January 07, 2010, 07:47:02 PM »
It is really a combination of things.

If this only occurred when I clicked on an icon or dragged it, I could live with it.

The problem is that if I have a share window open on the desktop for whatever reason and I forget and let the cursor travel across that window, it will start a random number of these transfers without clicking on anything.

Before I got a handle on what was happening, I would end up with two machines on the ends of the transfers working to capacity and unusable. They would show that there were transfers going on that were going to take several days to complete. At the same time, the network would be so overloaded that using a browser on an other machine on the network became just about impossible due to delays.

There are things about Dolphin that I really like. So, if I am being forced into kde4, I would prefer to use it. But this situation is just one more huge thorn in my side.

If the version of Konqueror on kde4 had all the functionality  that it has with kde3, I would probably just forget about Dolphin until some fix in the future cures this problem.

In particular, I have a number of machines with multibooting and varying numbers of drives. This is a bad combination when you have a really bad short term memory.

I always used the "my Computer" function from the desktop with kde3 in order to jog my memory as to how many drives were on a machine and how they were partitioned and whether the machine was a multiboot machine.

Konqueror with kde4 does not have that and the way that Dolphin displays information about the drives is so ambiguous to me that I don't have a clue which drives are on the machine and what is on them. It is fine for the USB external drives. But, it does not show internal drives that are not mounted. I assume that there are ways to set things up to do this. But at this time, I am just not to that point. I know that it is real easy with CLI.  Remembering a CLI command from one day to the next is simply not possible. The truth is that even when I look up the commands, I can not remember them long enough to type them without errors. I have to look at a phone number twice while dialing.

I have had this memory problem for 30 years and can always come up with ways to get around it. The real problem is that I can not do that when things are in a state of flux and changing constantly.

All will be fine once the base of kde4 stops changing so much and I can set my machines up to use a very basic set of functions that are likely to stay the same for a while.

Offline Roc4fun

  • Hero Member
  • *****
  • Posts: 620
  • The life so short, the craft so long to learn
Re: Dolphin, Samba, network test please.
« Reply #9 on: January 08, 2010, 06:28:24 AM »
Hi Again ThirdOFSix,

Well, I'm not working today; was able to test your issue and I can't duplicate it.

No file is created in my Konqueror /temp/kde-dave folder, and no network activity to speak of while hovering over samba shares in Dolphin.

This is my PCLOS kde4 fully updated desktop to a PCLOS Bel server.  I looked at several movies up to 22G in size.

I wish I had a smart solution to offer, Good luck and post your results.

Dave

Offline pags

  • Hero Member
  • *****
  • Posts: 2515
  • Keep it clean.
Re: Dolphin, Samba, network test please.
« Reply #10 on: January 08, 2010, 07:16:00 AM »
I am seeing what appears to be similar behaviour (although, I only have small files to test with, a 3Mb and 10MB)...

I would suspect it is suposed to be for the preview, but the preview doesn't work...even clicking the file opens kaffeine, but it fails to play (across the network).  Local files play fine, and preview fine.

This is reminescent of behaviour I noticed when playing with HTPC setups in the past...PCLOS/KDE had to have the network location mounted, otherwise (depending on the software used -- kaffeine, mplayer and I forget...) it either wouldn't plat at all, or it would copy the entire file across the network before starting to play (this was back in KDE 3.5.x).  I also played with U****u (no names  ;) ), and it would the same videos direct across the network (Gnome/Totem).  You knew it was streaming, because even files that were hundreds of Mb would start within seconds.  Ended up using XBMC, which handles everything internally (very well, I might add...but, I digress).  However, I think this was an issue with the backend streamer (gstreamer vs. whatever PCLOS/KDE was using at the time ... bad memory, I apologize).

Does anyone know what is currently being used as a backend streamer in PCLOS/KDE4?  Is there an easy way to switch it (and, be able to switch it back) to test if this has any impact?

Offline pags

  • Hero Member
  • *****
  • Posts: 2515
  • Keep it clean.
Re: Dolphin, Samba, network test please.
« Reply #11 on: January 08, 2010, 07:49:46 AM »
Additionally, I noticed other file types have similar behaviour (again, you probably need large enough files to notice it, but not so large that Dolphin would decide to not preview it  ;) ).
I had a .pdf file that previewed in Dolphin which was a few Mb, and I noticed the temp being created and deleted.
Also, when the mouse went over a directory, a file named "dolphinANNNNN.tmp" was created, where A=an alpha character (upper or lower case), and N=a number.  A new file was create for every mouse over (even if you went from a directory to a file and back to the same directory, a new file gets created).  The files were all 0 bytes.  Deleting them from the Konqueror window (leaving the Dolphin window open), appears to have no deleterious effect.
Closing the Dolphin windows does not result in these .tmp files being removed (not yet, any way...I just closed it).

I opened a Knoqueror File Browser, and went to the network location (smb://servername.or.IP in the address bar), and it does not exhibit the same behaviour (and, it doesn't have a preview pane to the right, either).  Clicking a file directly brings up a requestor to SAVE or OPEN (with KMPlayer, in my case).  Selecting OPEN results in the creation of another temp file ("konqueror-crash-L19668.log") with no program execuiting...hold on...

It's probably because of this:
Code: [Select]
[jpaglia@core2pclinuxos ~]$ kmplayer
kmplayer: error while loading shared libraries: libkmediaplayer.so.0: cannot open shared object file: No such file or directory
[jpaglia@core2pclinuxos ~]$

So...I'll see if I can fix KMPlayer, and try this part again...

EDIT.

Hmmm...
Neither kmplayer nor libkmediaplayer are in Synaptic.  A casualty of the KDE4 upgrade?  Curious that the executable remained on my system ???

EDIT 2.

I changed the default player to kaffeine, and tried opening the file from the network, again.  The requestor now offered to use kaffeine, which ran, but no video played...
« Last Edit: January 08, 2010, 08:11:50 AM by pags »

Offline ThirdOfSix

  • Hero Member
  • *****
  • Posts: 745
Re: Dolphin, Samba, network test please.
« Reply #12 on: January 11, 2010, 09:02:36 PM »
I have identified the culprit in this matter.

If I turn off the information panel that shows on the right hand side of the Dolphin window, the problem goes away.

I can consistently transfer large movie files from the network shares at speeds of 2.3 to 7.5 MBps  to the KDE4 desktop or local folders.

Pags could not reproduce it because my procedure to do so was not complete.

I took notes and now know how to reproduce the problem consistently.

I will post that information when I remember what I did with it so that the code gods can fix the problem.

Unfortunately, there is what appears to be a similar problem when transferring files the other direction.

Sometimes sending the same file that I copied at several MBps back to the same folder that it came from will go the other direction at a few hundred Kbps.

It looks like a similar problem because the network monitor is showing a much higher overall transfer than the speed that the particular file or folder is transferring at.

So far, I have not figured out a way or place to monitor this.

It appears to also have to do with whether I am transferring a folder or an individual file.

I will keep at it until I figure out how to reproduce this one at will also.

As it is, anyone trying to use this on a small office network with both Linux and Windows machines would be very frustrated by the inconsistent transfer times.

Last night I tried to transfer a 128 MB folder of pictures from my digital camera that I had downloaded to the KDE4 desktop to the network drive. It took a half hour to do it. A bit ago, I did it again and it took a few seconds.

Oh well, I tend to not let go of a problem until I figure it out. I am certain that eventually, I will figure out how to reproduce this one at will also.

Anyway, for now, if you have network shares and use Samba and Dolphin and you have any large files in the folder, turn off the Dolphin information panel or you are likely to have a nasty surprise.