torios team mailing list archive

[Nio Wiklund <nio.wiklund@xxxxxxxxx>]

Den 2015-05-24 02:31, Israel skrev:
> Hi Jack,
> Sure!  This ISO is having trouble uploading... I keep losing connection,
> so I am going to rsync this one using:
> rsync --progress -avze ssh ToriOS-rsync.iso israel@xxxxxxxxxx:oldISO
> to Phill's server
> 

Hi Israel,

I will review your rsync-options with references to the manual.


--progress (should not affect the file transfer, only the output to the
user's screen)

       --progress
              This  option tells rsync to print information showing the
progress of the transfer. This gives a bored user something to watch.
Implies --verbose if it  wasn’t  already  specified.


-a (I use this option almost always, and I think it is a good option)

       -a, --archive
              This is equivalent to -rlptgoD. It is a quick way of
saying you want recursion and want to preserve almost everything (with
-H being a notable omission).  The only exception to the above
equivalence is when --files-from is specified, in  which case -r is  not
implied.

Note that -a does not preserve hardlinks, because finding
multiply-linked files is expensive.  You must separately specify -H.


-v (should not affect the file transfer, only the output to the user's
screen)

       -v, --verbose
              This  option  increases  the  amount  of  information the
daemon logs during its startup phase.  After the client connects, the
daemon’s verbosity level will be controlled  by the  options that the
client used and the "max verbosity" setting in the module’s config section.


-z (maybe this option was creating your problem, if you used it when the
transfer failed)

       -z, --compress
              With this option, rsync compresses the file data  as  it
is sent to the destination machine,  which reduces the amount of data
being transmitted -- something that is useful over a slow connection.

              Note that this option typically achieves better
compression ratios than can be achieved by using a compressing remote
shell or a compressing transport because it takes advantage of the
implicit information in the matching data blocks that are not explicitly
sent over the connection.

              See the --skip-compress option for the default list of
file suffixes that will not be compressed.


-e ssh (this is normally the default. I do not use it, maybe necessary
in your case).

       -e, --rsh=COMMAND
              This option allows you to choose an alternative remote
shell program to use for communication between the local and remote
copies of rsync. Typically, rsync is configured to use ssh by default,
but you may prefer to use rsh on a local network.

-o-

My personal comment about -z (compression)

The main part of our iso file is already compressed: the tarball and the
squashfs file. Only a minor part is not compressed (those files needed
to boot the live system). So I don't think it is worthwhile to spend
effort for compression and expansion. I think rsync's compression should
be well debugged and resistent to disturbances (for example because of a
bad internet connection), but if I should suspect any of your rsync
options to have caused your problem, it is this one.

There might be other problems, outside your control, for example some
extraordinary condition in Phill's server, so that it could not manage
the information received.

In my computer running Ubuntu 12.04.5 LTS iso files are in the default
list of file types, that are not compressed. Maybe the default lists are
different in your production computer and Phill's server. In that case
the file might be borked after using the -z option.

-----
      --skip-compress=LIST
              Override  the list of file suffixes that will not be
compressed.  The LIST should be one or more file suffixes (without the
dot) separated by slashes (/). You may specify an empty string to
indicate that no file should be skipped.

Simple character-class matching is supported: each must consist of a
list of letters inside the square brackets (e.g. no special classes,
such as "[:alpha:]", are supported, and ’-’ has no special meaning).

The characters asterisk (*) and question-mark (?) have no special meaning.

Here’s an example that specifies 6 suffixes to skip (since 1 of the 5
rules  matches 2 suffixes):

                  --skip-compress=gz/jpg/mp[34]/7z/bz2

The  default list of suffixes that will not be compressed is this (in
this version of rsync):

7z avi bz2 deb gz iso jpeg jpg mov mp3 mp4 ogg rpm tbz tgz z zip

This list will be replaced by your --skip-compress list in all but one
situation: a copy from a daemon rsync will add your skipped suffixes to
its list of non-compressing files (and its list may be configured to a
different default).
-----

Best regards
Nio