Script Question

[Was_Just19]

This refers to something I recall happening some time ago, but would like to have some input for learning purposes.

In a Bash script, copying a large file to a USB flash drive, the copying process completes ......  but of course the write of the file to the drive is not completed, as the file is actually cached I understand, and will continue to be written to the drive long after the command completes.

This presented a problem, as the size of the file is unknown, as is the speed of the write to the drive, so no allowance could be made for the time it would take.

Now if that copy command was followed by another command which used that file (I think that was the case), the second command will kick in while the writing is still happening, as the copy command itself has completed.

I think that was the situation that presented itself.

IIRC, it was worked around by doing another small write to the drive immediately after the copy, which did not begin until the drive was free to be written to, and completed without needing cache.
Thus the real second command could be executed as the write of the file was definitely completed.

I hope I have remembered that correctly ......  and been able to explain it properly.

My question then is this .......  is there a 'proper' means of determining that a copy and write to such a drive has completed before the next command is executed?

Maybe something obvious that I just do not know about, or remember?


regards.

[Joble]

Maybe something like this?
http://linux.die.net/man/8/sync

[vjeko]

you could do safe remove, it ll make sure that data write and cache clear is done, but will leave you in a limbo as far as the future actions

[Was_Just19]

Maybe something like this?
http://linux.die.net/man/8/sync

Thanks.

mmmmmn ......  seems to be using a sledgehammer to crack a nut .......  unless I missed something that could be used for specific file/s rather than the whole system ......

[vjeko]

could you instruct the sync to apply only to a specific drive, by including that in your script?

[Was_Just19]

you could do safe remove, it ll make sure that data write and cache clear is done, but will leave you in a limbo as far as the future actions

Yes thought of that ......  but the present solution is neater I think ....  just force the wait for the write to complete, by issuing another small write and delete the temporary file.

I thought there might be something along the lines of 'wait' ..... or something like that ....  that I have no knowledge of ...

Thanks for the suggestion.  

[Was_Just19]

could you instruct the sync to apply only to a specific drive, by including that in your script?

I have not seen that option .....

Code: [Select]

$ sync --help
Usage: sync [OPTION]
Force changed blocks to disk, update the super block.

      --help     display this help and exit
      --version  output version information and exit

Report sync bugs to bug-coreutils@gnu.org
GNU coreutils home page: <http://www.gnu.org/software/coreutils/>
General help using GNU software: <http://www.gnu.org/gethelp/>
Report sync translation bugs to <http://translationproject.org/team/>

[Was_Just19]

Maybe fsync could be used ......  it seems the method previously used is as simple as it gets   

I like simple   

[kjpetrie]

I would have thought if you simply chain the copy command to the read with '&&' that would require the copy to be completed successfully before the read is attempted.

That wouldn't necessarily mean it was all written to the drive though, as the end of the file might still be in cache, but it should be accessible for reading. As others have said you would have to eject or unmount the drive to guarantee a complete write to the hardware.

[Was_Just19]

I would have thought if you simply chain the copy command to the read with '&&' that would require the copy to be completed successfully before the read is attempted.

That is exactly the problem I have described .......  the copy is completed but the writing of the file is not .......  so there is an incomplete file on the drive when the next command attempts to run.

That wouldn't necessarily mean it was all written to the drive though, as the end of the file might still be in cache, but it should be accessible for reading. As others have said you would have to eject or unmount the drive to guarantee a complete write to the hardware.

The other method I described works fine without having to eject the drive.


So far nothing easier has presented itself ........