Got a Hack?

Got a non-obvious solution to an interesting problem? Contribute your hack and share it with others online. We'll consider it for publication in future titles as well. View Sample Submission

Sorry, this form has been closed down.

Please visit the new weblog.

Thank you!

Sample Submission

Hack title: Using rsync over ssh
Summary: Keep large directory structures in sync quickly with rsync
Topic: Linux Server

<p>While tar over ssh is ideal for making remote copies of parts of a filesystem, rsync is even better suited for keeping the filesystem in sync between two machines. Typically, tar is used for the initial copy, and rsync is used to pick up whatever has changed since the last copy. This is because tar tends to be faster than rsync when none of the destination files exist, but rsync is much faster than tar when there are only a few differences between the two filesystems.</p>

<p>To run an rsync over ssh, pass it the <code>-e</code> switch, like this:</p>

<pre>root@rover:~# <b>rsync -ave ssh greendome:/home/ftp/pub/ /home/ftp/pub/</b></pre>

<p>Notice the trailing <b>/</b> on the file spec from the source side (on greendome.) On the source specification, a trailing <b>/</b> tells rsync to copy the contents of the directory, but not the directory itself. To include the directory as the top level of whatever is being copied, leave off the <b>/</b>:</p>

<pre>root@village:~# <b>rsync -ave ssh bcnu:/home/six . </b></pre>

<p>This will keep a copy of the <em>~root/six/</em> directory on village in sync with whatever is present on <em>bcnu:/home/six/</em>. </p>

<p>By default, rsync will only copy files and directories, but not remove them from the destination copy when they are removed from the source. To keep the copies exact, include the <code> -- delete</code> flag: </p>

<pre>six@jammer:~/public_html# <b>rsync -ave ssh -- delete greendome:~one/reports . </b></pre>

<p>Now when old reports are removed from <em>~one/reports/</em> on greendome, they're also removed from <em>~six/public_html/reports/</em> on jammer, every time this command is run. If you run a command like this in cron, leave off the <code>v</code> switch. This will keep the output quiet (unless rsync has a problem running, in which case you'll receive an email with the error output). </p>

<p>Using ssh as your transport for rsync traffic has the advantage of encrypting the data over the network and also takes advantage of any trust relationships you already have established using ssh client keys. For keeping large, complex directory structures in sync between two machines (especially when there are only a few differences between them), rsync is a very handy (and fast) tool to have at your disposal. </p>

See Also:

<li>man rsync</li>
<li>"Quick Logins with ssh Client Keys" (Linux Server Hacks #66)"</li>
<li>"Using ssh-Agent Effectively" (Linux Server Hacks #68) </li>
<li>"Automated Snapshot-Style Incremental Backups with rsync" (Linux Server Hacks #42) </li>

Return to:

Popular Topics

Browse Books & Videos

International Sites

O'Reilly China O'Reilly Germany O'Reilly Japan