Wednesday, August 20, 2014

FTP i-to-i part 1

ftp get put quote rcmd

FTP, File Transfer Protocol, is probably the most widely used way to transfer objects between computers. It should come as no surprise to you that FTP has been available since the AS400 era.

I work in an environment with 11 IBM i partitions and servers, and I can spend a lot of time moving objects between. To move objects between partitions on the same server I use the SAVRSTOBJ and SAVRSTLIB commands. To move objects from one server to another, especially if they are in different geographical locations, I use FTP.

Why FTP rather than SNDNETF or SAVRSTOBJ? In my experience when moving very large amounts of data between servers it can be several hours quicker to FTP the file rather than use the other methods.

This post is going to first of several posts explaining how to FTP a file or files from one IBM i server to another. This method is not secure and I only do this within my company's network or while part of a VPN. I do not recommend doing this in an unsecure environment, i.e. over the internet while not being part of a VPN, as it can be "sniffed" and the data compromised. If you do want to send data via an unsecure environment then use SFTP, Secure FTP.

In all FTP "conversations" one server is the FTP server and the other is the FTP client. On the FTP server the TCP/IP FTP server will need to be started, the command to do so is:

   STRTCPSVR SERVER(*FTP)

Before you start the FTP server it is wise to check if it is already started. You can do this by using this command to display the FTP server jobs:

   WRKACTJOB JOB(QTFTP*)

If any jobs are displayed the FTP server has been started. If none are displayed you will need to start the FTP server.

There are many FTP client subcommands, in this post I am only going to discuss the following:

  • cd - Change directory/library on the FTP host from the default to another.
  • lcd - Change the directory/library on the FTP client.
  • put - Copy a file from the FTP client to the FTP host.
  • get - Copy file from the FTP host to the FTP client.
  • bin - Change transfer type from ASCII to binary.
  • quote - Execute a command on FTP host.
  • quit - Quit FTP.

There is a link to all of the FTP subcommands at the bottom of this article.

In this post I am going to explain how I would transfer a number of objects from the FTP client to the FTP host. Rather than transfer the objects one at a time I have saved them into a save file using the SAVOBJ command, notice how I have used the DTACPR to use data compression when saving the objects:

   SAVOBJ OBJ(TEST*) LIB(MYLIB) DEV(*SAVF) SAVF(MYLIB/SAVEFILE) 
              DTACPR(*YES)

To start FTP you use the FTP command followed by either the FTP host's name or IP address:

  
  FTP QSRV2

  FTP '173.194.117.192'

The FTP command entry screen will be opened, asking you to enter your log ID for the FTP host.

                    File Transfer Protocol

 Previous FTP subcommands and messages:
   Connecting to remote host 173.194.117.192 using port 21.
   220-QTCP at qsrv2.
   220 Connection will close if idle more than 5 minutes.






 Enter login ID (simonh):
 ===> _____________________________________________________
___________________________________________________________
___________________________________________________________

 F3=Exit     F6=Print       F9=Retrieve
 F17=Top    F18=Bottom    F21=CL command line

When you have entered your login ID you will be prompted for your password. When you have entered your password the following additional messages are displayed:

> simonh
  331 Enter password.
  230 SIMONH logged on.
    OS/400 is the remote operating system. The TCP/IP version is "V7R1M0".
  250  Now using naming format "0".
  257 "QGPL" is current library.

The above gives me two important pieces of information:

  1. FTP is currently using: naming format "0". This is the standard format we are all use to, library/file. The other format, "1", uses the hierarchical format, I will discuss that later.
  2. The current library on the FTP host is QGPL. When I transfer the file I will be changing the default library.

I am going to start by changing the default directory (library) on FTP host by using the cd subcommand. The save file that will be receiving the data is in the library SAVFLIB, therefore the command I enter is:

  cd savflib

When I press Enter the following is displayed in the message section:

> cd savflib
  250 "SAVFLIB" is current library.

I also need to change the default directory (library) on the FTP client by using the lcd subcommand:

  lcd mylib

The following is displayed in the message section:

> lcd mylib
  Local working directory is MYLIB

I want to clear the save file on the FTP host before I send the data, therefore, I use the QUOTE subcommand followed by rcmd to submit a remote command to the host:

  quote rcmd CLRSAVF SIMONHSAVF

When the line above is executed the following is displayed in the message section:

> quote rcmd CLRSAVF SIMONHSAVF
  250 Command CLRSAVF SIMONHSAVF successful.

If SIMONSAVF is not in the library I gave when I used the cd subcommand the following would have been displayed:

quote rcmd CLRSAVF SIMONHSAVF
550-Error occurred on command CLRSAVF SIMONHSAVF.
550 File SIMONHSAVF in library *LIBL not found..

I do need to change the transfer type. The default transfer type is ASCII, which is fine for plain text files. A save file needs to use the binary transfer. I just enter bin to change the transfer type to binary:

  bin

The following is displayed in the message section:

> bin
  200 Representation type is binary IMAGE.

Now the command to transfer the data from the save file on the FTP client to the FTP host using the put subcommand. In this scenario the save file in both locations is called SIMONHSAVF:

  put simonhsavf

But if the save files had different names, for example: MYSAVF on the FTP client, then put would be:

  put mysavf simonhsavf

I could even qualify both files with the library names:

  put mylib/mysavf savflib/simonhsavf

When the transfer completes the number of bytes transferred will be displayed. I need to check for the line with the status 226 to know that the transfer was successful.

> put  mysavf simonhsavf
  250  Now using naming format "0".
  257 "SAVFLIB" is current library.
  227 Entering Passive Mode (173.194.117.192).
  150 Sending file to member SIMONHSAVF in file SIMONHSAVF in library SAVFLIB.
  226 File transfer completed successfully.
   21120 bytes transferred in 0.004 seconds. Transfer rate 5406.720 KB/sec.

I mentioned the hierarchical name format earlier. In our example the hierarchical name of our two save files would be:

  • FTP client: /qsys.lib/mylib.lib/mysavf.file/mysavf.mbr
  • FTP host: /qsys.lib/savflib.lib/simonhsavf.file/simonhsavf.mbr

In my opinion that is far too complicated and more prone to keying errors than the conventional, name format "0".

It is also possible to transfer files in the IFS from one server to another. I have the test_file.txt in the IFS folder SIMONH on QSRV1 and I want to copy it to the same location on QSRV2:

  put /simonh/test_file.txt /simonh/test_file_1.txt

I find it interesting that in the messages section it shows that the name format was automatically changed to "1" as the files are in the IFS:

put /simonh/test_file.txt /simonh/test_file_1.txt
227 Entering Passive Mode (173,194,117,192).
150-NAMEFMT set to 1.
150 Sending file to /simonh/test_file_1.txt
226 File transfer completed successfully.
     9 bytes transferred in 0.009 seconds. Transfer rate 1.024 KB/sec.

The opposite of put is get subcommand. If I wanted to get the save file from the FTP host I would just use:

  get simonhsavf mysavf (replace

The (replace option replaces the contents of MYSAVF with the data from SIMONHSAVF, thereby, removing the need for me to CLRSAVF MYSAVF before using the get.

When you are finished using FTP you can either press F3 or use the quit subcommand

In the next part of this series I will describe how you can copy "data" files using FTP without using a save file.

 

Other posts in this series:

 

You can learn more about this on IBM's website:

 

This article was written for IBM i 7.1.

9 comments:

  1. The "original" or "native" QSYS.LIB file system is part of the IFS (as explained in the IFS Concepts manual online). It is true that specifying FTP names in QSYS.LIB using NAMEFMT 1 is a bit more "wordy" so possibly more error prone, but it can be done.


    It is also possible to have FTP "automatically" create a savefile on the target system by using NAMEFMT 1 syntax with the ".savf" suffix. Tom Liotta and I discussed this back in 2007 in this USENET thread (now obfuscated by Google Groups):
    http://compgroups.net/comp.sys.ibm.as400.misc/how-to-write-a-saved-file-to-ifs/1347607

    ReplyDelete
  2. I have been using DDMF in order to trasnfer data between machines instead of save and restore and FTP.

    ReplyDelete
  3. Depending on the speed of your network and the amount of data using DDM can be slow. Also if you are doing multiple files, you could get files out of sync with one another (files in parent/child type relationships.) Using save files give you a snapshot of things you are moving

    ReplyDelete
  4. Ìsn't that the same as SAVRST, SAVRSTLIB and/or SAVRSTOBJ?

    ReplyDelete
  5. If transferring from one partition to another then SAVRSTOBJ/SAVRSTLIB are good to use. I have found when transferring files from one server to another, especially if they are in different locations, FTP is faster.

    For example I had library that I had to transfer from USA to Germany. It took 6 hours using SAVRSTLLIB, and about an hour using FTP.

    ReplyDelete
  6. There are additional compression options in V5R4 and later: *LOW, *MEDIUM and *HIGH. In our internal testing, we found that *YES and *LOW resulted in about 2:1 compression of database files. *MEDIUM and *HIGH compressed around 3.5-4.5:1. We found that *HIGH is significantly more CPU intensive than *MEDIUM for only a small gain in compression so we typically use the *MEDIUM option.

    ReplyDelete
  7. This a wonderful, concise yet complete guide to FTP. It's perfect for those of us that do not use FTP very often.

    ReplyDelete
  8. Simple way for a save file

    cd /QSYS.LIB/MYLIB.LIB
    bin
    put savefile savefile.savf

    ReplyDelete

To prevent "comment spam" all comments are moderated.
Learn about this website's comments policy here.

Some people have reported that they cannot post a comment using certain computers and browsers. If this is you feel free to use the Contact Form to send me the comment and I will post it for you, please include the title of the post so I know which one to post the comment to.