I received a question from a reader asking if there was a way to copy data from a central IBM i to a number of remote IBM i using SNADS, and then have the central server call a program on the remote ones. Fortunately this is easy to do just using a few simple commands and a bit of common sense.
Distributed Data Management (DDM) files have been available on IBM computers for as many years as I can remember. DDM files are a quick and easy way to copy data to and from one IBM computer to another. Most of my experience with them has between AS400 and later IBM i, but I have also worked receiving data from an IBM mainframe via DDM files.
There are restrictions to what tools and utilities you can use with DDM files:
(only IBM utilities listed, I will not vouch for non-IBM products)
| Works with DDM files | Does not with DDM files |
| RPG/RPGLE using native I/O Most CL commands DFU |
RUNQRY/WRKQRY STRSQL SQL embedded in RPG/RPGLE EDTF |
While all DDM files are created using the Create DDM File command, CRTDDMF, there is a difference in the way they work depending upon the desired communication type:
CRTDDMF FILE(QTEMP/DDMFILE1) RMTFILE(RMTLIB/TFRFILE) +
RMTLOCNAME(OTHERSVR)
CRTDDMF FILE(QTEMP/DDMFILE2) RMTFILE(RMTLIB/TFRFILE) +
RMTLOCNAME('10.000.000.000' *IP)
|
The command's parameters are straight forward.
FILE: This is the file name and library where the DDM file will be placed on this IBM i.
RMTFILE: Name of the file and the library where the file occurs on the remote system, in our case another IBM i.
RMTLOCNAME: The first parameter is the remote system's name. In the first example I have used the network id, OTHERSVR, and in the second its IP address, it makes no difference which I use.
The second parameter is important. There are two values to choose from depending upon what I want to do:
- *SNA this is the command's default, which is why is it not shown in the first example. The job at the remote end will use the user profile QUSER. I will have to give QUSER authority to all files accessed via DDM, which means that any user using DDM files could access any of the files that QUSER has been authorized to.
- *IP if used then the user profile used on this server will need to exist on the remote server. Well, not actually true, there is another way to "map" a user profile on the local server to one on the remote server, but I will have to cover that in another post. This is the more secure way of controlling access to file on the remote server as you can authorize only certain profiles to the various files on the remote server.
I can build the DDM file to any Physical or a Logical file on the remote system.
In RPG I use the DDM files just like I would a local one:
dcl-f DDMFILE1 keyed extfile('QTEMP/DDMFILE') ;
dcl-f DDMFILE2 keyed usage(*input:*update)
rename(TFRFILER:TFRFILER2) ;
read TFRFILER ;
update TFRFILER2 %fields(TFR003) ;
|
Now comes one of those philosophical questions I argue all the time with other IBM i developers. Do I...
- Create the DDM file in a specific library so it can be used by any program at any time
- Create the DDM file when it is needed in QTEMP (where else would I put work files?)
In my opinion there is not one correct answer. It depends on how the DDM file is going to be used. If it is going to be used by a lot of programs and users, then I would put it in a library that these users can access. If it is only going to be used once a day, I would build it in QTEMP. I will leave it to you to come to a decision for your own DDM files.
Now we have to deal with an issue that plagues all DDM files, what happens when...
- The remote system is down, it could be for something as simple as for IPL or backup.
- If communications fails between the local and the remote. This is unlikely to happen if they are IBM i partitions on the same server. But if they on different servers in different locations this will happen.
If the remote system is down the RPG program will start, open the DDM file, and when it performs the first read to the DDM file it will hang almost indefinitely waiting for the data from the remote. I have seen programs that have been in this state for hours, even when communications had been reestablished.
What I need is a quick way to test that the "other end" of the DDM file is OK before I use it:
01 OPNDBF FILE(QTEMP/DDMFILE) OPTION(*INP)
02 MONMSG MSGID(CPF4125) EXEC(DO)
03 SNDMSG MSG('Cannot connect to remote system') TOUSR(&USER)
04 RETURN
05 ENDDO
06 CLOF OPNID(DDMFILE)
07 CALL PGM(PGM1)
|
If can open the DDM file with the Open Database File command, OPNDBF, line 1, then the connection to the remote system is good, and the logic jumps to line 6 to close the file.
But if I cannot open the file as there is an issue with the DDM file, I issue a message to the user and quit the program. Feel free to put looping logic here to check to see if the link is reestablished, etc.
There are a couple of other useful DDM file commands:
CHGDDMF: Change an existing DDM file to "point" to another file or IBM i. I do use this command a lot, especially when I need to copy the latest data from the Production server to the Development server. For example:
01 CRTDDMF FILE(QTEMP/DDMFILE) RMTFILE(PRODLIB/ORDHDR) +
RMTLOCNAME(PRODUCTN *IP)
02 CPYF FROMFILE(QTEMP/DDMFILE) TOFILE(MYLIB/ORDHDR) +
MBROPT(*REPLACE) FROMRCD(1)
03 CHGDDMF FILE(QTEMP/DDMFILE) RMTFILE(PRODLIB/ORDDTL)
04 CPYF FROMFILE(QTEMP/DDMFILE) TOFILE(MYLIB/ORDDTL) +
MBROPT(*REPLACE) FROMRCD(1)
|
I have not given the RMTLOCNAME on line 3. As the two files are at the same location I can use the command's default, which is *SAME.
The other command I want to bring to your attention is the Display DDM file command, DSPDDMF. As you would expect this displays all the information about the DDM file. When you find a DDM file you did not know of, this is the way to find out what server and which file it links to.
DSPDDMF FILE(QTEMP/DDMFILE) DSPDDMF FILE(MYLIB/*ALL) OUTPUT(*PRINT) |
After all that information about how to create, change, and display DDM files, how can I call a program on another IBM i using DDM files?
The command Submit Remote Command, SBMRMTCMD, allows me to submit any IBM i command to a remote system using a pre-existing DDM file. For example:
SBMRMTCMD CMD('call mylib/pgm1') DDMFILE(QTEMP/DDMFILE)
|
This is where the type of connection comes into play. If you have a DDM file created using *SNA then the program will run under the user profile QUSER, with all of its authorities, etc., which I would hope is not much. If I had used *IP then the program will run under the same user profile on the remote system as I am using on the local one. This allows me to authorize only certain users to certain files on the remote system, and prevent everyone from accessing information that is not relevant to them.
I do not recommend you call a program on the remote system that does "stuff". The job on the local system will wait for the job on the remote system to finish. When I use the SBMRMTCMD I use it to SBMJOB a command to the remote system. This way I can set all the things like library list, output queue, etc. in the submit job command, and the job on the local system will continue without waiting for the processing on the remote system to finish.
Here is a simple example of what I was describing:
01 CRTDDMF FILE(QTEMP/DDMFILE) RMTFILE(PRODLIB/TFRFILE) +
RMTLOCNAME(OTHERSVR *IP)
02 CPYF FROMFILE(ORDTRS) TOFILE(QTEMP/DDMFILE) +
MBROPT(*REPLACE) FROMRCD(1) +
INCREL((*IF SITCOD *EQ '015') (*AND PFLAG *EQ 'G'))
03 CHGVAR VAR(&SBMJOB) +
VALUE('SBMJOB CMD(CALL PGM(PRODLIB/TFRPGM1)) +
JOB(TFRUPDATE) +
JOBD(PRODLIB/TFRJOBD) +
JOBQ(QBATCH1)')
04 SBMRMTCMD CMD(&SBMJOB) DDMFILE(QTEMP/DDMFILE)
|
Another place to store data you might want to get to and from a remote system is in a data area. It is also possible to make a DDM data area and then use it on the local system.
01 CRTDTAARA DTAARA(QTEMP/DDMDTAARA) TYPE(*DDM) +
RMTDTAARA(PRODLIB/TFRDA) RMTLOCNAME(OTHERSVR)
02 RTVDTAARA DTAARA(QTEMP/DDMDTAARA *ALL) RTNVAR(&DATAAREA)
|
You can learn more about this from the IBM website:
This article was written for IBM i 7.2, and should work for earlier releases too.
Nice! Very helpful in this age of networked systems. What do you have about sharing files not on IBM?
ReplyDelete<shocked>Even though DB2 for i is the best database in the world and IBM i is the best operating system in the world, people still put their data in other databases and on other types of servers!</shocked>
DeleteGreat tips sir, thanks for sharing.....
ReplyDeleteWhenever there is DDM communication failure , our admins always suggest us to change from SNA to IP. Does it really make any difference?
ReplyDeleteAs I mention in the post the advantage for using DDM with IP is it is more secure.
DeleteUsing SNA means that all DDM communications are done using QUSER. Which means you have to give QUSER authority on the remove server to all the files, etc it could possibly need.
If you use IP then you need to have the same profile on the remote system as the one on the local system. This means I just have to authorize certain profiles to certain files. Everyone else would not be authorized to the files, and will receive an error message on the local system. This is a much better security situation.
There is a better way to do the authorization part for IP, but that is too much information for a comment. I needs to be its own post.
I do not know of any other reason why it would be advantageous to switch from SNA to IP. In my experience they are both equally fast (or slow depending on the connection).
On our 7.1 TR10 level system we can access the remote target system using DDM file. The target server job runs under QRWTSRVR job name. You should see a message "Target job assigned to handle DDM connection started by source system over TCP/IP." (CPI9162).
ReplyDeleteMatt
I am going to join a new project where First time I am going to work where Production and development has different Iseries box. So I think DDM will be useful for me to take data from Production server to development server for testing or debugging etc.
ReplyDeleteBut usually production files has different profile(Not user profile) and in development files has usually user profile, so will I able to copy the data from production file to development lib file or I will be require some special authority.
Also In my new project they widely used MQ for getting data from other application(like POS etc) can you please tell me something about working with MQ, that will be very helpful as I never worked with MQ before.
For the network people complaining about SNA vs. IP - are they saying that your SNADS networking is NOT using IP? Back around 2011, IBM came up with Enterprise Extender (EE) connections for high-performance routing (HPR) . . . This is IP over Ethernet and is very efficient.
ReplyDeleteIf you have things already built in SNADS, continue using them. The SNADS apps just sits on top of EE/HPR and is more efficient than plain IP. EE/HPR can be used with IPv4 or IPv6. And can be between System-i boxes or a hybrid network with Mainframes, too.
http://www.ibm.com/support/knowledgecenter/SSLTBW_1.13.0/com.ibm.zos.r13.halz002/considee.htm
http://www.cisco.com/en/US/tech/tk331/tk336/technologies_design_guide09186a00802149da.shtml
- JV
Rob,
ReplyDeleteWith IBM as well as all ISV2s talking about how only DDLs (SQL Tables/Indexes/Views etc)are the go forward route and DDS not being improved upon, will the DDM concept also be sunsetted?
If an NEP program connects via DDM and then later loses connection because remote box was brought down, how can the connection be reestablished when box is brought??? issue another CHGDDMF or just continue and connection will be reconnected???
ReplyDeleteYou are describing a scenario my employer had. There was a remote site and their connection to the WAN was "fragile". This caused problems bacause if a DDM file is open and the connection in interrupted the program using it will just "hang".
DeleteI overcome the problem the problem by not using the DDM file in a RPG program. I created a CL program that looped (like a NEP). This CL would...
1. Create the DDM file,
2. CPYF the contents to a file on the local server.
3. Delete the DDM file.
4. Call the RPG program.
5. DLYJOB
6. Goto 1
If the CL program could not establish a connection to the remote server a message would be sent to the appropriate people notifying that the connection was down again.
You might want to consider using that approach to your situation.
Might want to reclaim the DDM connection, RCLDDMCNV.
DeleteGetting error while using data area.
ReplyDeleteCannot establish DDM connection with remote system.
Operation on DDM data area DDMDTAARA in QTEMP failed.
Operation on DDM data area DDMDTAARA in QTEMP failed.
Unmonitored exception at line 6
Looks like you cannot connect to the remote IBM i. Check the connection status.
DeleteVery nice article..!
ReplyDeleteDo we have way to check remote system msgw jobs and reply them from local?
Also, can we show output of any remote command like wrkobj xxxx on local ststem only. Dont want to login remote system to check object present there or not.
Talking about command line commands and Wrkobj is just example.
Rather than use commands, like WRKOBJ, use the SQL equivalent. You can use the three part name to interrogate the remote partitions.
ReplyDeleteAs for monitoring jobs in MSGW you can use what I describe here, and use the three part name for the remote partitions. There is no way I know to answer message from one partition from another.
Can we do DDM and use Mainframe file on AS400 system?
ReplyDelete