lftp
This is a wiki page. Be bold and improve it!
If you have any questions about the content on this page, don't hesitate to open a new ticket and we'll do our best to assist you.
|
|
|
| Check the local lftp issue tracker
|
|
|
Important links
LFTP is sophisticated ftp/http client, file transfer program supporting a number of network protocols. Like BASH, it has job control and uses readline library for input. It has bookmarks, built-in mirror and can transfer several files in parallel.
Information
Official web site: http://lftp.yar.ru/
Wikipedia: http://en.wikipedia.org/wiki/Lftp
Mailing lists
Mailing lists and archives (since July 2011):
http://univ.uniyar.ac.ru/mailman/listinfo/lftp-devel
http://univ.uniyar.ac.ru/mailman/listinfo/lftp
List archives (including prior to July 2011):
http://www.mail-archive.com/lftp@uniyar.ac.ru/
http://www.mail-archive.com/lftp-devel@uniyar.ac.ru/
Code
Browse the source code:
https://github.com/lavv17/lftp
http://fossies.org/dox/lftp-4.3.3/
Bug trackers
http://linux.overshoot.tv/project/issues/lftp
https://github.com/lavv17/lftp/issues
Basic usage
Here are a few tutorials covering basic usage. Further down this page, you can find information that is complementary to these tutorials.
Configuration files
On startup, lftp executes /etc/lftp.conf and then ~/.lftprc and ~/.lftp/rc. You can use those files to set system-wide and user-specific settings.
The main configuration file, /etc/lftp.conf
comes with various examples and comments. Check it out.
set
In your configuration file, simply use the command set
followed by the name of the setting, followed by its value. Use on/off for boolean values.
set ftp:ssl-force on
set ssl:verify-certificate on
set ftp:use-feat off
set ftp:ssl-protect-data on
Please refer to the man page for the description of each command.
Note: the settings are all nicely prefixed by the protocol they apply to, or a different prefix to note that it applies in a more generic way to all protocols. E.g. ftp:ssl-force
applies to ftp, http:user-agent
is for http, and so on.
Set for specific servers
You can set specific settings for specific servers only. Simply append a slash (/) and the server name to the setting.
set ftp:use-feat off /example.com
set ftp:ssl-force on /example.com
In the man page, it it documented thus:
The closure for `dns:', `net:', `ftp:', `http:', `hftp:' domain variables is currently just the host name as you specify it in the `open' command (with some exceptions where closure is meaningless, e.g. dns:cache-size). For some `cmd:' domain variables the closure is current URL without path. For other variables it is not currently used. See examples in the sample lftp.conf.
debug
By default, lftp is silent about the operations it performs in the background. If you need some more output, e.g. for debugging purposes, you can use the 'debug' command (without the 'set' command):
debug 5
debug 5
will display the full debug output.
debug 3
will only display greeting messages and error messages.
Note that if you use the -d
switch from the command line, the debug setting will be overruled and the full debug output will be displayed.
alias
If you prefer some commands to be called by another name, you can set aliases:
alias dir ls
alias less more
alias zless zmore
alias bzless bzmore
alias reconnect "close; cache flush; cd ."
alias edit "eval -f "get $0 -o ~/.lftp/edit.tmp.$$ && shell \\"cp -p ~/.lftp/edit.tmp.$$ ~/.lftp/edit.tmp.$$.orig && vim ~/.lftp/edit.tmp.$$ && test ~/.lftp/edit.tmp.$$ -nt ~/.lftp/edit.tmp.$$.orig\\" && put ~/.lftp/edit.tmp.$$ -o $0; shell rm -f ~/.lftp/edit.tmp.$$*"
The last alias (edit) should be all in one line. It allows you to get a remote file, edit it with vim (change as needed), and place the edited file back on the server. This is very convenient!
Tips and tricks
Editing remote files
The alias settings above shows one way you could use lftp to edit a remote file and put it back on the server. It's very convenient for small fixes and for debugging an application on a remote server or simply to change the values in a settings file.
You can also check out the lftp_vi plugin. The page is in German and the instructions a bit dated. Here is the updated instructions:
- Download and unpack the source code of lftp.
- Download and unpack the source code of the lftp-vi plugin from the page above. (lftp-vi.1.1.tar.gz or newer version)
- Move the file cmd-edit.cc to the top of the lftp source code directory.
- Run the command:
lftp-4.2.3 $ gcc -fPIC -shared -nostdlib -I./src/ -I./lib/ -I. -I./trio/ cmd-edit.cc -o cmd-edit.so
$ mkdir ~/.lftp
$ cp cmd-edit.so ~/.lftp/
- Add the following line to your ~/.lftprc:
module ~/.lftp/cmd-edit.so
Multipart Transfers
Below is an example of a mult-part download over the SFTP protocol:
$ lftp -e 'pget -c -n 5 /path/to/file' sftp://username@server
short explanation:
-e: lftp option to execute command
pget: is the command for partial download
-c: pget option to resume
-n: pget option for number of parts
Backup and mirroring
See the article: CLI Magic: Quick and easy backup with lftp.
Using lftp on a secure network
SSL and TLS support
lftp supports the SSL and TLS protocols (HTTPS and FTPS URI schemes) using either the GnuTLS library or the OpenSSL library. Because of small licensing issue discussed in the wikipedia OpenSSL article, linux distributions usually prefer using GnuTLS.
There are two ways to make sure that lftp has been compiled with SSL/TLS support.
The easiest way is to check the output of the command version
within lftp. You might get something like:
Libraries used: Readline 6.2, Expat 2.0.1, OpenSSL 0.9.8o 01 Jun 2010, zlib 1.2.3.4
or something like:
Libraries used: Readline 6.2, GnuTLS 2.8.6, zlib 1.2.3.4
The other way is to use the command line tool ldd
. First, check the full path to lftp:
$ which lftp
/usr/bin/lftp
Then check whether either libgnutls or libssl is among the libraries lftp has been compiled with:
ldd /usr/bin/lftp
linux-vdso.so.1 => (0x00007fffa83ff000)
libexpat.so.1 => /lib/libexpat.so.1 (0x00007f943b485000)
libssl.so.0.9.8 => /lib/libssl.so.0.9.8 (0x00007f943b236000)
libcrypto.so.0.9.8 => /lib/libcrypto.so.0.9.8 (0x00007f943aeaf000)
librt.so.1 => /lib/librt.so.1 (0x00007f943aca7000)
libreadline.so.6 => /lib/libreadline.so.6 (0x00007f943aa67000)
libutil.so.1 => /lib/libutil.so.1 (0x00007f943a864000)
libncurses.so.5 => /lib/libncurses.so.5 (0x00007f943a621000)
libdl.so.2 => /lib/libdl.so.2 (0x00007f943a41d000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f943a206000)
libc.so.6 => /lib/libc.so.6 (0x00007f9439e96000)
/lib64/ld-linux-x86-64.so.2 (0x00007f943b6ae000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00007f9439c7a000)
libz.so.1 => /lib/libz.so.1 (0x00007f9439a63000)
The line starting with libssl
in the output above indicates that you are good. lftp compiled with GnuTLS would show a line like this one:
libgnutls.so.26 => /usr/lib/x86_64-linux-gnu/libgnutls.so.26 (0x00007fc68cf20000)
If you need to compile lftp with OpenSSL, you may need a variety of packages from your distribution (e.g. On Ubuntu, libreadline6-dev and other packages are needed).
Then do:
./configure --with-openssl=/usr/lib
make
sudo make install
A multitude of protocols
A new user may be confused by the profusion of protocols for secure file transfers. To make matter worse, the commonly used acronyms are confusingly similar. Throughout this document, we'll endeavour to use a strict naming convention as defined in the table below.
The following wikipedia articles may also be of interest:
URI Scheme | Port | Protocol | Supported | Secure layer |
---|---|---|---|---|
sftp:// | 22 | SSH File Transfer Protocol (a.k.a. SFTP) | Yes | SSH |
ftp:// | 21 | File Transfer Protocol (FTP) | Yes | None |
ftp:// | 443 | FTPS (explicit) (a.k.a. FTPES) | Yes | SSL |
ftps:// | 443 | FTPS (implicit) (Deprecated) | Yes | SSL |
? | FTP over SSH (i.e. neither SFTP nor FTPS) | ? | SSH | |
http:// | FTP over HTTP proxy protocol | Yes | None | |
https:// | HTTP Secure (a.k.a HTTPS) | Yes | SSH? | |
hftp:// | HFTP | Yes | None | |
fish:// | Files transferred over shell protocol Fish is a protocol working over an ssh connection to a unix account. | Yes | SSH | |
file:// | File URI scheme | Yes | None | |
? | BitTorrent | Yes | None |
Encryption and trust
There are two additional things to note in regards to using TLS with ftp. 1
The first is certificate verification, same as when visiting an https web site. It's of little comfort that the password was sent with strong encryption if it is sent to the wrong guy. TLS uses certificates to help ensure you are connected to who you intended to.
A basic setup is to make sure certificate verification is turned on (these too are on by default in the current version):
set ssl:check-hostname yes
set ssl:verify-certificate yes
set ssl:ca-file "path to your a certificate bundle file, containing the certificate authorities you choose to trust"
An easy answer to what bundle of certificates authorities to trust is to just take what your browser vendor (eg. Mozilla) or operating system vendor supplies. Then you'll be generally as safe as you'd be accessing https web site in your browser.
More paranoid users might hand-pick what certificate authorities to trust on their own.
The second important part for ftp with TLS is unique for the FTP protocol's peculiarity of using multiple connections, one as a control channel and a separate one for transferring data.
lftp by default is set to encrypt only the control channel and leave the data channel in the clear.
Computers and Internet connections are plenty fast enough nowadays to afford encrypting everything, so just turn it all on:
set ftp:ssl-protect-data yes
set ftp:ssl-protect-list yes
As can be seen from all this, everything is a whole lot simpler if you just connect with SFTP to an SSH2 server instead. Everything is always encrypted no matter what, no separate control and data channels to worry about, no certificate authorities to trust (a host fingerprint is verified instead).
Server features
Given the plethora of protocols given above, it is customary for a explicit FTPS client to ask the server which protocols it supports. When debugging is turned on, you may see the FEAT
command and the reply from the server. E.g.:
---> FEAT
<--- 211-Features:
<--- AUTH SSL
<--- AUTH TLS
<--- EPRT
<--- EPSV
<--- MDTM
<--- PASV
<--- PBSZ
<--- PROT
<--- REST STREAM
<--- SIZE
<--- TVFS
<--- UTF8
<--- 211 End
In the example above, the client asked
FEAT
and the server responded, advertising the fact that, among other features, it supports AUTH TLS. Thus the client chose among the supported protocols which one to use.
Problems with use-feat
Some badly configured server may not advertise all of the supported protocols, tricking the client into using an insecure protocol instead 2
In such a case, if you are sure the server actually does support the said feature, you may turn the FEAT process off and force lftp to use a secure protocol.
For example, a server that was known to support FTPES answered thus when ftp:use-feat
was set to true (the default):
---> FEAT
<--- 211-Extensions supported:
<--- EPRT
<--- IDLE
<--- MDTM
<--- SIZE
<--- REST STREAM
<--- MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;
<--- MLSD
<--- ESTP
<--- PASV
<--- EPSV
<--- SPSV
<--- 211 End.
---> OPTS MLST type;size;modify;UNIX.mode;UNIX.uid;UNIX.gid;
<--- 200 MLST OPTS type;size;sizd;modify;UNIX.mode;UNIX.uid;UNIX.gid;unique
Note the absence of
AUTH SSL
.
The following two settings solved the problem for that specific server (example.com):
set ftp:use-feat/example.com off
set ftp:ssl-force/example.com on
The first turned the FEAT process off, while forcing lftp to use the secure SSL protocol over ftp.
ftp:ssl-force is set and server does not support or allow SSL
If you get an error like the following:
cd: Login failed: ftp:ssl-force is set and server does not support or allow SSL
but you know that the server does support SSL, then turn off ftp::use-feat as described above (problems with use-feat). It is another symptom of a badly configured server. You can force the use of SSL and in such a case, there is no need to use FEAT (which determines extended features of ftp server).
Being the best
This article is featured in the blog entry "The best combination of Linux users: the lftp example".
- 1. This whole section is based on this message from the lftp mailing list, used with permission.
- 2. See for example this message from the lftp mailing list.
Issues related to this page:
Project | Summary | Status | Priority | Category | Last updated | Assigned to |
---|---|---|---|---|---|---|
lftp (project) | Compiling/ using lftp-vim | active | normal | bug report | 12 years 23 weeks | |
lftp (project) | lftp 'edit' comand auto-completion patch | active | normal | feature request | 12 years 23 weeks | |
lftp (project) | lftp mirror -n does not work | active | normal | bug report | 12 years 48 weeks | |
lftp (project) | Git source code incomplete | active | normal | support request | 13 years 3 weeks | |
lftp (project) | How to compile lftp with openssl? | active | normal | support request | 12 years 10 weeks | |
lftp (project) | How to use the lftp -c option? | active | normal | support request | 13 years 2 weeks | |
lftp (project) | ftp:FEAT does not work | active | normal | feature request | 12 years 11 weeks | |
lftp (project) | lftp reverse mirror silently skips files in sub… | active | normal | bug report | 8 years 47 weeks |