Someone asked me how I would write a program to copy all of the members in source files to separate files in the IFS, where they could then me imported into a GiT. I can see a program like this being more useful overtime as more people move from using source files to using a GiT repository.
In this example I will be copying all the source members from the source file DEVSRC in the library MYLIB to the IFS directory /home/MyFolder. The files in the IFS would all be named in the following way: source_member_name.source_member_type, for example: TESTRPG.RPGLE
My first attempt at creating a program to do this was to use RPG. I was going to use the CPYTOIMPF command within the QCMDEXC Scalar function. Alas, several hours of creating program I received the following message when running the RPG program for the first time:
Additional Message Information Message ID . . . . . . : CPD0104 Severity . . . . . . . : 30 Message type . . . . . : Diagnostic Date sent . . . . . . : DD/DD/DD Time sent . . . . . . : TT:TT:TT Message . . . . : Expression not allowed for parameter TOSTMF. Cause . . . . . : Expressions are only allowed in a control language (CL) program. |
Which means that my example program is CL:
01 DCL VAR(&SRCLIB) TYPE(*CHAR) LEN(10) VALUE('MYLIB')
02 DCL VAR(&SRCFILE) TYPE(*CHAR) LEN(10) VALUE('DEVSRC')
03 DCL VAR(&IFSPATH) TYPE(*CHAR) LEN(100) VALUE('/home/MyFolder/')
04 DCL VAR(&STRING) TYPE(*CHAR) LEN(1024)
05 DCLF FILE(QTEMP/WORKFILE)
06 DLTF FILE(QTEMP/WORKFILE)
07 MONMSG MSGID(CPF0000)
08 CHGVAR VAR(&STRING) VALUE('+
09 CREATE TABLE QTEMP.WORKFILE +
10 (LIBRARY,FILE,MEMBER,IFS_FILE) AS +
11 (SELECT CHAR(A.SYSTEM_TABLE_SCHEMA,10),+
12 CHAR(A.SYSTEM_TABLE_NAME,10),+
13 CHAR(A.SYSTEM_TABLE_MEMBER,10),+
14 CHAR(A.SYSTEM_TABLE_MEMBER || ''.'' || +
15 A.SOURCE_TYPE, 200) +
16 FROM QSYS2.SYSMEMBERSTAT A CROSS JOIN QSYS2.SYSFILES B +
17 WHERE (A.SYSTEM_TABLE_SCHEMA,A.SYSTEM_TABLE_NAME) = +
(B.SYSTEM_TABLE_SCHEMA,B.SYSTEM_TABLE_NAME) +
18 AND B.FILE_TYPE = ''SOURCE'' +
19 AND A.SYSTEM_TABLE_SCHEMA = ''' || %TRIMR(&SRCLIB) || ''' +
20 AND A.SYSTEM_TABLE_NAME = ''' || %TRIMR(&SRCFILE) || ''') +
21 WITH DATA')
22 RUNSQL SQL(&STRING) COMMIT(*NC)
23 DOWHILE COND('1')
24 RCVF
25 MONMSG MSGID(CPF0000) EXEC(LEAVE)
26 CHGVAR VAR(&IFS_FILE) VALUE(%TRIMR(&IFSPATH) || &IFS_FILE)
27 CPYTOIMPF FROMFILE(&LIBRARY/&FILE &MEMBER) +
28 TOSTMF(&IFS_FILE) +
29 MBROPT(*REPLACE) +
30 STMFCCSID(*PCASCII) +
31 RCDDLM(*CRLF) +
32 DTAFMT(*FIXED)
33 ENDDO
|
Line 1: Variable that contains the name of the source library.
Line 2: Variable that contains the name of the source file.
Line 3: Name of the directory in the IFS I want to create files within. I made this variable deliberately long in case, in the future, I want to put the files in a "deeper" subdirectory.
Line 4: This variable will be used to contain the SQL expression that will be executed.
Line 5: I will be reading, RCVF, this file.
Line 6: First I want to delete a file of this name if it already exists in the QTEMP library.
Line 7: The MONMSG stops the previous command from erroring if the file does not exist.
Lines 8 – 21: This is string that contains the SQL statement that will be executed to create the work file. Thew work file will contain the following columns:
- LIBRARY: Library the source file is within
- FILE: Source file's name
- MEMBER: Source file member's name
- IFS_FILE: The file name that will be used for the file in the IFS
Line 9: The file, WORKFILE, will be created in QTEMP.
Line 10: The name of the columns.
Line 11: I have used the CHAR scalar function to convert the column SYSTEM_TABLE_SCHEMA from variable character to just character. CL can cope with variable length character columns, it is just easier if all the columns are fixed width character.
Lines 12 and 13: I do the same with the SYSTEM_TABLE_NAME and SYSTEM_TABLE_MEMBER columns.
Lines 14 and 15: Here I make the IFS file name by concatenating a dot to the member's name, and then concatenate the source type to that.
Line 16: I am joining the SYSFILES View to the SYSMEMBERSTAT View so I can check if the file the members are in is a source file.
Lines 17: These are columns that I use to join the two Views, File library and name.
Line 18: If the file type in SYSFILES is source then this is a source file.
Lines 19 and 20: Here I am selecting only those records where the library name is the same as the value in the program variable &SRCLIB and &SRCFILE. Rather than use the shortcut "|<" I thought I would use the TRIMR as everyone should be familiar with that Built in Function.
Line 21: I want the created file to contain data.
Line 22: I executed the SQL statement I created above using the RUNSQL command.
Line 23: I am going to "read", RCVF, work file, therefore I need a Do loop, which will be executed until I choose to leave it.
Line 24: I "read" work file.
Line 25: If I reach end of file, or there is some problem with the file, I exit this Do loop.
Line 26: I append the directory, from the variable &IFSPATH, to the start of the variable &IFS_FILE.
Lines 27 – 32: I use the CPYTOIMPF to create and copy the contents of the source member of a file in the IFS.
Line 33: End of the Do loop.
After I compiled the above program I called it, and after it completed, I used the following SQL statement to see if the source file members had been copied to the IFS folder with the names, I expected:
01 SELECT SUBSTR(PATH_NAME, 16) AS "File",
02 FROM TABLE(QSYS2.IFS_OBJECT_STATISTICS('/home/MyFolder/','NO','*STMF'))
|
Line 1: I am using a substring scalar function starting at position 16. The path name is in the first 15 characters and I don't care about that, I only want to see the names of the files.
Line 2: I am using the SQL Table function IFS_OBJECT_STATISTICS to get the results. The first parameter tells it which directory to get the results for. The second I do not want information from any subdirectories. And third I only want to list stream files.
The results are:
File -------------------- TESTRPG2.SQLRPGLE TESTFILE.PF TESTSQL.SQL TESTDSPF.DSPF TESTRPG.RPGLE TESTTABLE.SQLTABLE TESTCL.CLLE TESTVIEW.SQLVIEW |
The files are exactly what I wanted: each source member is its own file.
This article was written for IBM i 7.5, and should work for some earlier releases too.
I would strongly suggest, if the final destination is a system like github or similar, to use UTF8 to avoid losses , so STMFOPT(*REPLACE) STMFCCSID(1208).
ReplyDeleteI have a basic (but effective) nightly setup that get the modified source members since say 20 days in the past. Then a directory with the same name of the library, and a subdir with the same name of the file gets created in case, and the relative source member gets copied from the source file to IFS.
...This provide a good "natural" structure to browse than in a revision control system. Then git is called in PASE to push the modified source to github.
ReplyDeletePretty effective as a backup and a tool to see the evolution of a source, with little effort (and not changing local dev habits).
Hey Simon,
ReplyDeleteIf that may help, I did similar function with QAFDMBRL to list all the source members. The lib is passed as parm
PGM PARM(&LIB)
DCL VAR(&LIB) TYPE(*CHAR) LEN(10)
DCL VAR(&FRPATH) TYPE(*CHAR) LEN(50)
DCL VAR(&TOPATH) TYPE(*CHAR) LEN(50)
DCL VAR(&IFSPATH) TYPE(*CHAR) LEN(14) VALUE('/home/flavoie/')
DCLF FILE(QAFDMBRL) RCDFMT(QWHFDML)
/**------------------------------------------------------**/
/** Main Process ----------------------------------------**/
/**------------------------------------------------------**/
DSPFD FILE(&LIB/*ALL) TYPE(*MBRLIST) +
OUTPUT(*OUTFILE) OUTFILE(QTEMP/MBRL) +
OUTMBR(*FIRST *REPLACE)
OVRDBF FILE(QAFDMBRL) TOFILE(QTEMP/MBRL)
LOOP1: RCVF
MONMSG MSGID(CPF0864) EXEC(GOTO CMDLBL(EXIT))
IF COND(&MLSEU = ' ') THEN(GOTO CMDLBL(LOOP1))
CHGVAR VAR(&FRPATH) VALUE('/QSYS.LIB/' *CAT &MLLIB +
*TCAT '.LIB/' *CAT &MLFILE *TCAT '.FILE/' +
*CAT &MLNAME *TCAT '.MBR')
CHGVAR VAR(&TOPATH) VALUE(&IFSPATH *TCAT &MLLIB *TCAT '/')
CHKIN OBJ(&TOPATH) SUBTREE(*ALL)
MONMSG MSGID(CPFA0A9 CPFA0BE) EXEC(MKDIR DIR(&TOPATH))
CHGVAR VAR(&TOPATH) VALUE(&TOPATH *TCAT &MLFILE *TCAT '/')
CHKIN OBJ(&TOPATH) SUBTREE(*ALL)
MONMSG MSGID(CPFA0A9 CPFA0BE) EXEC(MKDIR DIR(&TOPATH))
CHGVAR VAR(&TOPATH) +
VALUE(&TOPATH *TCAT &MLNAME *TCAT '.' *CAT &MLSEU2 *TCAT '.txt')
CPYTOSTMF FROMMBR(&FRPATH) TOSTMF(&TOPATH) STMFCCSID(*PCASCII)
GOTO CMDLBL(LOOP1)
/**------------------------------------------------------**/
/** Exit Process ----------------------------------------**/
/**------------------------------------------------------**/
EXIT: RETURN
ENDPGM
Thank you for that. I knew there had to be an API to do the same thing. I just thing SQL makes it easier/simpler to do the same.
DeleteIf you happen to have comments in fixed form in 1 to 5, and/or, like we, hex codes to bring colors to SEU, do yourself a favor and write a program to copy the member to the IFS instead of CPYTOSTMF.
ReplyDeleteThen you can remove all that stuff and do more, like rewrite /COPY or /INCLUDEs to a path and whatever might help you later to convert to **FREE.
And if your member names start oder contain signs like #, $ or §, there might be trouble waiting, too, although these are correct object names, many tools have problems with these. If you can, get rid of them.
Hi Simon,
ReplyDeleteYou can do this in RPG or PL/SQL too.
Just make sure you embed the target IFS-file name in quotes on the CPYTOIMPF command.
Declare vIfsDir varchar(512);
Declare vIfsFile varchar(1024);
SET vIfsFile = ''''
|| vIfsDir
|| system_table_member
|| Case When source_type IS NOT null
THEN Concat('.', lower(trim(source_type)) )
Else ''
End
|| '''';
Set vCmd = 'CPYTOIMPF '
|| ' FROMFILE('
|| 'QTEMP'
|| '/'
|| 'SRCMBRTBL'
|| ')'
|| ' TOSTMF('
|| vIfsFile
|| ')'
|| ' MBROPT(*REPLACE)'
|| ' STMFCCSID(*PCASCII)'
|| ' DTAFMT(*FIXED)'
|| ' RCDDLM(*CRLF)'
;
CALL Qcmdexc(vCmd);