lftp: backup and mirroring
This is a wiki page. Be bold and improve it!
If you have any questions about the content on this page, don't hesitate to open a new ticket and we'll do our best to assist you.
You can use the lftp mirror command to make a backup of a web site.
Since the whole command can get very long very quickly, it would be burdensome to have to type it in its entirety each time, so we are going to write the command once into a file, and tell lftp to execute the content of that file.
Let's call that file
lftp_command.txt. It can then be executed this way:
lftp -f /home/user/lftp_command.txt
Now, let's start creating the content of lftp_command.txt. The first thing we need to do is to connect to the remote server. You can do so easily by explicitly setting the site, username and password:
lftp -u username,password ftpsite
However, in our example, we'll assume that you have a bookmark called remote_site with the proper credentials, so that you can connect this way:
Remember: put this within the lftp_command.txt file and execute it as expressed above.
Next, we are going to actually mirror the content of the remote server onto our local machine. See the man page for all the arguments and flags that the command
mirror can take. The most basic usage is:
mirror path/to/source_directory path/to/target_directory
Here are some of the most useful switches that you can use:
--delete delete files not present at remote site
--only-existing download only files already existing at target
--only-newer download only newer files
--no-recursion don't go to subdirectories
--reverse reverse mirror (put files)
--include RX include matching files
--exclude RX exclude matching files
--verbose verbose operation
Check the lftp man page for all the advanced flags that
mirror can take.
lftp_command.txt has the following:
lftp remote_site -e "mirror --verbose --only-newer path/to/source_directory path/to/target_directory"
The -e switch tells lftp to execute the command that follows.
If you want the command to be executed at a specified time, use the
at switch and add an ampersand at the end of the command, so that the process is detached from your console window (which you can then close). Here, the command will be executed at midnight:
lftp at 00:00 remote_site -e "mirror --verbose --only-newer path/to/source_directory path/to/target_directory" &
And to upload a site from a local source, simply user the
lftp remote_site -e "mirror --verbose --only-newer --reverse path/to/source_directory path/to/target_directory" &
Note that it would be best to ensure that your ftp connection is secure. Check the section using lftp on a secure network within this site's wiki.
Also, note that other tools offer more or same the same feature, secured by default, e.g.
rsync -a -e ssh email@example.com /backup/to/location/locally
Check also: unison, bacula, amanda.