Thursday, 15 October 2015

rsync -avzhP -e ssh usability (1.2) resume after network interrupt requires temp file rename AND (2.) ~delayed~ rsync error about permissions problem if transferring large files

rsync is great
very useful
current favourite invocation:

rsync -avzhP --bwlimit=1000 -e ssh <from_files> <touser@host:/path/>

-P is short for --partial --progress
-avhz .. h=humanoid, v=verbose, a=archive, z=compression
.. archive instructs it to maintain time_t values so even if clocks are out rsync knows the true date of each file

2(kind of 3) usability caveats:
   (may depend on version of rsync at source and dest?)
   (may be related to using ssh?)


1.1 user interrupt leaves truncated file with full file name on destination

IF the user hits ctrl-c on rsync and interrupts transfer then the temporary file at destination (by default named .<from_file>.<6randomchars> is renamed to <from_file>. This could cause a user to think file transfer was successful but file is truncated. If the rsync is resumed then rsync uses the partially transferred file and resumes with new temporary file.

It would be better if the temporary file was kept. (so user knows transfer was not successful).


1.2 resume after network interrupt starts from scratch - doesn't resume from temporary file

IF there is a network interruption and rsync is interrupted then the temporary file at destination is not renamed. If the rsync is resumed then rsync doesn't use the already partially transferred file. rsync starts from scratch. If you rename the (truncated) temporary file to original file name and then resume rsync the rsync uses the partially transferred file.

It would be better rsync resume checks for temporary file and resumes transfer from there.


2. permission to write on destination error not reported until files fully transferred over network

IF touser cannot write to /path.
(and if from_files are large and network is slow)
I have seen rsync transfer 100% of all files over network.
Then - possibly hours after transfer start - rsync reports permission denied.
There is no temporary file on destination.

$ rsync -avzhP TEST -e ssh touser@host:/path/
sending incremental file list
TEST
           5 100%    0.00kB/s    0:00:00 (xfer#1, to-check=0/1)
rsync: mkstemp "/path/.TEST.zXCDQB" failed: Permission denied (13)

sent 74 bytes  received 31 bytes  14.00 bytes/sec
total size is 5  speedup is 0.05
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1042) [sender=3.0.7]
$

### rsync versions at source and dest:
$ rsync --version
rsync  version 3.0.7  protocol version 30

$ rsync --version
rsync  version 3.0.9  protocol version 30


This is a KNOWN rsync bug/issue:
https://bugzilla.samba.org/show_bug.cgi?id=6788
Bug 6788 - Skip file transfer if destination file cannot be opened for writing


Other bugs:
https://bugzilla.samba.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=RESOLVED&bug_status=VERIFIED&product=rsync




No comments: