posted on 2014-07-26 18:59:26
For file transferring, there exists three protocols that are usually used. All these have in common that a FTP daemon (ProFTPd, vsftpd, ServU, ...), or in case of SFTP, an SSH daemon (i.e. OpenSSH), has to run on the server. (For FTPs you'd need even both of them.) To daemon/service the user can then connect with a FTP client (i.e. Filezilla) running from his machine.
First some comments on authentication, because this topic often causes confusion. Although this is mostly due to users' misunderstanding SSH, having no idea what their firewall is doing and them trying wildly all the options they can find in their client. ;)
There exist several 'logon modes' that can be used:
- anonymous FTP: The server provides an anonymous account, with which the connection can be established. No authentication is done.
- password: user credentials in terms of a username and password have to be supplied
- public-key: having exchanged public key halves out-of-band before, these can be used for encryption later
Technically, SFTP can be set up to provide an anonymous account, too.
How much use this idea is, having no user access control present but securing the data transmission channel, is left as a thought exercise to the reader.
But if id Software can use FTPS for its anonymous account, so why should you not use SFTP for your own server? :)
For anonymous SFTP make sure to use rssh (a restricted secure shell), to prevent shell access but provide sftp access to the user.
Then just creating a user named 'anonymous' without a password will do the trick.
rssh is usually available through the package manager of your linux distribution.
In Filezilla (the most-used FTP client AFAIK), these options are available:
anonymous: no user credentials are submitted, to be used for anonymous logins normal: user:pw credential combination ask for password: just the user is provided, the pw will be asked for once interactive: just the user is provided, the pw will be asked for every time account: if the server uses the ACCounT command, it can be passed here
Standard protocol, but no encryption is used. All data is sent unencrypted over the network. This includes user credentials as well as the data being sent.
First a visual representation I stole from RFC 959:
------------- |/---------\| || User || -------- ||Interface|<--->| User | |\----^----/| -------- ---------- | | | |/------\| FTP Commands |/----V----\| ||Server|<---------------->| User || || PI || FTP Replies || PI || |\--^---/| |\----^----/| | | | | | | -------- |/--V---\| Data |/----V----\| -------- | File |<--->|Server|<---------------->| User |<--->| File | |System| || DTP || Connection || DTP || |System| -------- |\------/| |\---------/| -------- ---------- ------------- Server-FTP USER-FTP
Two kinds of connections are opened. One is the
command, one the
The data channel is solely used for payload transfers, and is opened when needed and closed when transfers are completed.
Whereas the command channel stays open all the time, and handles things like authentication, starting transfers, closing the connection.
FTP uses the TELNET protocol for exchanging control messages.
The command channel is an established TCP connection between a random port on the client (Port N, where N is an available port number.) and by default on port 21 of the server. "Random" usually means a port above 1023.
The data channel is established on port 20 on the server and channel N+1 on client side, as soon as one is needed for data exchange. Once the transfer is done, the channel is closed. It will be reopened, in case it is needed again.
This is just a matter of direction. Since usually ingres connections are filtered by the firewall, but egress traffic not so, passive mode can be used to work your way around restrictive firewalls on the client's side.
In short: (standard ports are used in this example.)
The connections are build up in a different order, control channels (
---) and data channels (
===) shown below:
(ports above 1024 are arbitrary ones, depending on settings and current OS state.)
ACTIVE ====== client (1027) connects to server (21), establishing control channel - client sends PORT command with its ip and opened port 1028 through control channel - other commands may follow SERVER (20) connects to CLIENT (1028), establishing data channel. CLIENT SERVER port 1028 1027 21 20 | | | | | |------ 1: PORT 1028 ----->| | | | | | | |<----- 2: ACK ------| | | | | | | | | | | | | | |<=========== 3: data transfer =============| | | | | | | | | |============ 4: ACK ============>| | | | | | | | | PASSIVE ======= client (1027) to server (21), establishing control channel - client sends PASV command through control channel - server answers which port is has available for data channel through control channel - other commands may follow CLIENT (1028) connects to SERVER (2033), establishing data channel. CLIENT SERVER port 1028 1027 21 20 2033 | | | | | | |------ 1: PASV ----->| | | | | | | | | |<----- 2: PORT 2033 ------| | | | | | | | | | | | | | | | | | |============ 3: data transfer ===================>| | | | | | | | | | | |<=========== 4: ACK ====================| | | | | | | | | | |
Digits in brackets denoted ports, of course.
So while in passive mode the server admin has to open the high ports in his firewall, in active mode to client admin has to do so in his. Pretty simple stuff, one might guess. Real world tells: 'not so much'.
Further there can problems arise when FTPS/FTPES is used. This is also firewall-related, see the FTPS section at its end.
Available commands are, according to RFC 959:
Note, not all these are implemented in all FTP clients/servers. Still all of them are listed, to get a better grasp on FTP and what happens beside what you see during its usage.
USER <username> pass username to server PASS <password> pass password to server ACCT <account-information> pass account information to the server CWD <pathname> change working directory CDUP change working directory to parent SMNT <pathname> Structure MouNT, allow mounting of a different file system QUIT quit connection REIN REINitialize connection to state immediatly after opening control channel, prior to USER PORT <host-port> data port on host, can be several, comma-delimited PASV enable passive mode TYPE <type-code> ascii or ebcdic plaintext, image (bytestream), local proprietary format for data during network transfer STRU <structure-code> set file, record, page for file structure MODE <mode-code> choose transfer mode (stream, block, compressed) RETR <pathname> download data STOR <pathname> STORe / upload data STOU STORe Unique, save data under unique filename in CWD APPE<pathname> upload and APPEnd data ALLO<decimal-integer> [<SP> R <SP> <decimal-integer>] sometimes needed to ALLOcate space on server prior to upload REST <marker> restart, only works for block and compressed modes RNFR <pathname> old name to be renamed RNTO <pathname> new name, old name to be renamed to ABOR abort currently running operation and associated data channels DELE <pathname> delete file on server RMD <pathname> delete directory MKD <pathname> create directory PWD show current directory LIST [<pathname>] transfer filename list or file information from server NLST [<pathname>] Name LiST, transfer filename list SITE <string> send specific commands to remote server SYST return system type of server STAT [<pathname>] show ftp status HELP [<string>] show help
Often you also find custom ones, ones that are not listed here. That is all dependant on the actual implementation of the client you use:
After all, RFC's are just specifications. These describe, how software/systems should look like.
Specifications are not Documentations, which describe how systems actually are shaped in reality. (In case the documentation is up-to-date...)
So you very likely might encounter these:
BYE same as QUIT LS same as LIST DIR same as LIST
Another command actually is the
!. It runs local commands, see:
[sjas@beckett ~]% ftp ftp> help ! ! escape to the shell ftp>
There are also other commands specified (There exists more than one RFC for FTP stuff.), that take on encryption or other functionalities. In case you really are curious, see:
RFC 697 - CWD Command of FTP RFC 959 - File Transfer Protocol (FTP) RFC 1639 - FTP Operation Over Big Address Records (FOOBAR) RFC 2026 - UTF-8 Option for FTP (Draft 2002)  - draft-ietf-ftpext-utf-8-option-00.txt RFC 2228 - FTP Security Extensions RFC 2389 - Feature negotiation mechanism for the File Transfer Protocol RFC 2428 - FTP Extensions for IPv6 and NATs RFC 2640 - Internationalization of the File Transfer Protocol RFC 3659 - Extensions to FTP RFC 5797 - FTP Command and Extension Registry
A list on all its return codes (200, ...) can be found here.
Here the transfers are encrypted using certificates within a public key infrastructure (PKI), see X.509 certificates.
explicit modes are possible:
Implicit: forces TLS, or quits the connection. Explicit: uses TLS if possible, but works unencrypted in case encryption is not possible.
Implicit forces encryption for all commands/data being transferred.
Explicit lets the client choose, what he wants to have encrypted. (data channel? control channel? both? none?)
Usually implicit is not used anymore, if I am not mistaken.
Besides having SSL/TLS encryption, FTPS/FTPES is standard FTP.
There are occasions where FTPS will fail between two computers, whereas FTP works. This is usually due to 'intelligent' firewalls, watching out for FTP control messages. Upon finding these (PORT command!), they know which of their ports they have to open up for data exchange.
If however the connection is encrypted, this packet analysis will no longer work. So the needed ports for the data channel stay closed, causing SFTP to fail.
Since SSHv2, SFTP is available. (SSHv1 implementations only provided 'secure copy'.)
Not much left to explain (except maybe SSH in depth, which is no use here.), so it will be cut short:
FTP/FTPs is an entirely different protocol from SFTP.
I.e. where FTP needs two connections (1x command, 1x data channel), SFTP only needs one. Which is from a random port on the client side, to port 22, the SSH port, on the server side. All data exchange is done through ssh specific commands. These may be similar to the ones specified above. But besides similarities they have nothing in common, they don't stem from the same specification.
On a sidenote,
This one is a rare occurance and usually seldom used. (Except by these notorious people that will do things, just because they can.)
Comparisons for easier understanding, spot the differences:
======================================================= FTP ======================================================= | plain | encrypted ---------------------+---------------+------------- control channel | FTP | data channel | FTP | ======================================================= FTPs, FTP-over-SSH ======================================================= | plain | encrypted ---------------------+---------------+-------------- control channel | | FTP-over-SSH data channel | FTP-over-SSH | ======================================================= FTPS / FTPES, FTP-over-SSL ======================================================= | plain | encrypted ---------------------+---------------+-------------- control channel | | FTP-over-SSL data channel | | FTP-over-SSL ======================================================= SFTP, Secure FTP (only one channel!) ======================================================= | plain | encrypted ---------------------+---------------+-------------- control+data channel | | SFTP
Instead of using the regular FTP (the 'plaintext' version, if I may say so), where both control and data channel are unencrypted, a SSH tunnel is established prior to FTP usage and its encryption is used for the control channel. The data is transmitted without encryption.
So USER and PASS commands cannot be read in plain text anymore, which makes FTPs better than regular FTP.
Since the initial overhead of creating the SSH tunnel is to complicated, people stick to FTP/FTPS/SFTP instead of FTPs. Also as noted at the beginning, you'd need both a ftp server and a ssh server running and and a tunnel previously and properly set up.
But why bother and not just either go the easy way with FTP or the secure with SFTP or FTPS?
This one has nothing at all to do with encryption, at least per se. It's also only possible with the FTP variants, not the SFTP one.
Scenario: Say you have a client, and two servers. (Instead of regular FTP where you just have a client and a single server.)
With regular FTP used, to get files from server1 to server2, the transfer would have to work over the client (and it's possible bad connection). No direct connection is possible from server1 to server2 over the 'good'/fast network.
With an FXP client, direct transfers are possible, if the FTP servers support this functionality. It is achieved by exchanging the ip and ports of server1 to server2 via the client machine.
Basically, the client connects to both servers at once. He negotiates connection between himself and server1 in passive mode, to know which port server1 has open for data exchange. He tells server2, and calls for data transmission. server1 will be contacted by server2, data channel is established, and files will flow directly.
The first server here was contacted via passive mode. The second was passed the first servers ip and port.
Of course this should also work with active/passive switched, but the client has to have to know the second server's open ports prior, without being told by the server, which complicates things. Without this information, passing the first server in the first step the ip of the second server and a wrong port makes just no sense.
Lastly two sidenotes:
Usually when customers are unable to use SFTP/FTPS it's due to them forgetting to adjust the protocol in their FTP client (i.e. Filezilla). Or they miss the proper port. Or they changed encodings/transfer modes/passive while trying to get things to run...
View posts from 2017-09, 2017-08, 2017-07, 2017-05, 2017-04, 2017-03, 2017-02, 2017-01, 2016-12, 2016-11, 2016-10, 2016-09, 2016-08, 2016-07, 2016-06, 2016-05, 2016-04, 2016-03, 2016-02, 2016-01, 2015-12, 2015-11, 2015-10, 2015-09, 2015-08, 2015-07, 2015-06, 2015-05, 2015-04, 2015-03, 2015-02, 2015-01, 2014-12, 2014-11, 2014-10, 2014-09, 2014-08, 2014-07, 2014-06, 2014-05, 2014-04, 2014-03, 2014-01, 2013-12, 2013-11, 2013-10