The idea for this post comes from one I found upon the Seiden Group's website. I need to thank Alan Seiden for giving me permission to take the contents of his post and tweak it in a way I would use it.
The post gave an example of using QShell to grep, search files for a string of characters, to search source members in source files. As it is possible to use wild cards in a grep statement I can search many libraries and/or many different source files for the string I desire. For example if I want to search every RPG source file for the string "free". I would start QShell by using the following command:
qsh |
And when the "QSH command entry" screen is displayed I can enter the following:
/usr/bin/grep -i -n "free" /QSYS.LIB/*.LIB/QRPG*.FILE/*.MBR |
-i will ignore the case of the search sting.
-n means that the line number in the file or member the string is found on is returned.
"free" is the search string I am searching for.
/QSYS.LIB/*.LIB/QRPG*.FILE/*.MBR is where I am searching. This statement includes wild cards (asterisk, *). I am searching every library, that I am authorized to, for any file that starts "QRPG", and all the members in those source files.
If I just wanted to search all the members in my source file, DEVSRC, in my library, MYLIB, the statement would look like:
/usr/bin/grep -i -n "free" /QSYS.LIB/mylib.LIB/devsrc.FILE/*.MBR |
If I don't say this I am going to get emails from people explaining I could enter the grep string in the QSH command:
QSH CMD('/usr/bin/grep -i -n "free" /QSYS.LIB/mylib.LIB/devsrc.FILE/*.MBR')
|
I just prefer to use QShell the first way I described.
I wanted to change a few things to make Alan's example to, IMHO, make it more user friendly. The two things that spring to mind are:
- Have a screen where I could enter the parameters I desire
- Direct the output to a file, rather than display all the results on a screen at once
Let me start by showing you the display file I created for myself. As this is an example it is very simple:
01 A DSPSIZ(24 80 *DS3)
02 A PRINT
03 A ERRSFL
04 A CA03(03 'F3=Exit')
*-------------------------------------------------------------------------
05 A R SCREEN
06 A 2 3'Search pattern .'
07 A ZPATTERN 20 B 2 20
08 A 3 3'Source file . .'
09 A ZSRCFILE 10 B 3 20
10 A 4 3'Source library .'
11 A ZSRCLIB 10 B 4 20
12 A 5 3'Output library .'
13 A ZOUTPUTLIB 10 B 5 20
|
The display, that I have called TESTDSPF, has only one record format, SCREEN, and four input fields:
- ZPATTERN: This is a 20 character field I use to enter the characters I want to search for
- ZSRCFILE: Name of the source file I want to search, can include a wild card
- ZSRCLIB: Name of the library the source file(s) are found in, can include a wild card
- ZOUTPUTLIB: Name of the library the output file will be in
Next comes the first of two CL programs. This one, I am calling TESTPGM1 looks like:
01 PGM
02 DCLF FILE(TESTDSPF)
03 SNDRCVF
04 IF COND(&IN03) THEN(RETURN)
05 SBMJOB CMD(CALL PGM(TESTPGM2) +
PARM((&ZPATTERN) (&ZSRCLIB) (&ZSRCFILE) +
(&ZOUTPUTLIB))) +
JOB(QSHSEARCH)
06 ENDPGM
|
As I said before as this is an example the programs are deliberately simple. This means that there is no validation, which I will add to the version of this I use for myself. If you are going to be using this in an environment where you will not be the only user I recommend you add some basic error checking.
Line 2: The definition for the display file.
Line 3: I display the display file's only record format, and after enter is pressed control returns to the program.
Line 4: If F3 was pressed to exit I quit the program.
Line 5: I am submitting the program that does all the "work" to batch as I have no idea how long it takes to gather all the results for the entered selection. I have decided to call the submitted job QSHSEARCH, you could call yours any name you like.
Next is the program that is submitted to batch, TESTPGM2, does all the "work":
01 PGM PARM(&PATTERN &SRCLIB &SRCFILE &OUTPUTLIB)
02 DCL VAR(&PATTERN) TYPE(*CHAR) LEN(20)
03 DCL VAR(&SRCLIB) TYPE(*CHAR) LEN(10)
04 DCL VAR(&SRCFILE) TYPE(*CHAR) LEN(10)
05 DCL VAR(&OUTPUTLIB) TYPE(*CHAR) LEN(10)
06 DCL VAR(&STRING) TYPE(*CHAR) LEN(100)
07 CHGVAR VAR(&STRING) VALUE('+
/usr/bin/grep -i -n "' || &PATTERN |< '" +
/QSYS.LIB/' || &SRCLIB |< '.LIB/' || +
&SRCFILE |< '.FILE/*.MBR')
08 OVRDBF FILE(STDOUT) TOFILE(QTEMP/QSHOUTPUT) +
OVRSCOPE(*CALLLVL)
09 QSH CMD(&STRING)
10 CHKOBJ OBJ(&OUTPUTLIB/QSHOUTPUT) OBJTYPE(*FILE)
11 MONMSG MSGID(CPF9801) +
EXEC(CRTPF FILE(&OUTPUTLIB/QSHOUTPUT) RCDLEN(240))
12 CPYF FROMFILE(QTEMP/QSHOUTPUT) +
TOFILE(&OUTPUTLIB/QSHOUTPUT) MBROPT(*ADD) +
FMTOPT(*CVTSRC)
13 ENDPGM
|
Line 1: The names this program will use for the variables of the passed parameters.
Lines 2 – 5: Definitions for those variables.
Line 6: I will be using this variable to contain the grep string.
Line 7: I am building the string that grep will be using. I am concatenating in the passed variables. I always use the CL short cuts when I do this. The || is just a straight forward concatenation. |< is a truncated concatenation, which removes the trailing blanks from the variable, thereby, removing the need to right trim them.
Line 8: To direct the output from QShell to a file I need to override the standard output, STDOUT, to a file. The output is directed to a source file, rather than a data file. If the file does not exist one is created when QShell runs. When QShell creates it I want it in QTEMP as this is not my final output file, just a work file.
Line 9: I execute the grep statement in the variable String.
Line 10: I check if the final output file, QSHOUTPUT, exists in the library contained in the &OUTPUTLIB variable.
Line 11: If it does not I create it with a Create Physical File command, CRTPF, with a record length of 240 characters.
Line 12: I copy the contents of the source member in the generated source file to the file in my library.
I don't bother to delete the generated source file as when the job ends the contents of its QTEMP are deleted.
When I have compiled everything I am ready to go. I enter the following on a command line and press the Enter key:
CALL TESTPGM1 |
The display file's record format is displayed, and I enter the following into it.
Search pattern . FREE Source file . . QRPG* Source library . * Output library . MYLIB |
I press Enter and program TESTPGM2 is submitted to batch.
When line 7 in the program has completed the value in the variable &STRING is as I expect:
'/usr/bin/grep -i -n "FREE" /QSYS.LIB/*.LIB/QRPG*.FILE/*.MBR' |
When the job has finished I can look at the results in the output file using the Display Physical File member command:
DSPPFM QSHOUTPUT |
The results on the partition I use look something like:
/QSYS.LIB/LIB1.LIB/QRPGLESRC.FILE/PGM001.MBR:1:**FREE /QSYS.LIB/LIB1.LIB/QRPGLESRC.FILE/PGM002.MBR:1:**free /QSYS.LIB/LIB2.LIB/QRPGLESRC.FILE/PGM003.MBR:5: /FREE /QSYS.LIB/LIB2.LIB/QRPGLESRC.FILE/PGM003.MBR:17: /END-FREE /QSYS.LIB/LIB3.LIB/QRPGLESRC.FILE/PGM004.MBR:15: D*Qp0lSaveStgFree()... /QSYS.LIB/QSYSINC.LIB/QRPGSRC.FILE/QTAFROBJ.MBR:5: I*Descriptive Name: Free... |
The line number where the search string is found is displayed after the path name, followed by the line from the source member that includes the search string.
If I wanted to search for information I would use SQL to do so. For example if I wanted to return only where the string "/FREE" was found:
SELECT * FROM QSHOUTPUT WHERE UPPER(QSHOUTPUT) LIKE '% /FREE %' |
Notice that I used the UPPER scalar function to convert the contents of the record to be upper case before I perform the comparison to the string " /FREE ".
The partition I made this example upon makes this difficult. Its default CCSID is 65535, but my job uses CCSID 37. This means that the source file was created with CCSID 65535, and when I copy the data into my final output file, which is CCSID 37, I just see "rubbish". That is easily corrected by using a cast to convert the CCSID of the field that contains the results:
SELECT CAST(QSHOUTPUT AS CHAR(240) CCSID 37) FROM QSHOUTPUT WHERE UPPER(QSHOUTPUT) LIKE '% /FREE %' |
As the output file, QSHOUTPUT, was created the way I did it contains only one field that has the same name as the file, QSHOUTPUT.
A sample of the results from the about SQL statement would look like:
00001 ---------------------------------------------------------------- /QSYS.LIB/LIB1.LIB/QRPGLESRC.FILE/PGM010.MBR:5: /FREE /QSYS.LIB/LIB1.LIB/QRPGLESRC.FILE/PGM011.MBR:5: /FREE /QSYS.LIB/LIB2.LIB/QRPGLESRC.FILE/PGM012.MBR:1: /free /QSYS.LIB/LIB3.LIB/QRPGLESRC.FILE/PGM013.MBR:6: /FREE /QSYS.LIB/LIB4.LIB/QRPGLESRC.FILE/PGM014.MBR:1: /free |
This is fantastic as now I can search for strings across many libraries, making this a lot easier than using PDM's Find String command, FNDSTRPDM.
This article was written for IBM i 7.5, and should work for some earlier releases too.
For source file searches, I use the iSphere RDi plugin functionality. I like it for speed and versatility, it also allows to open the source members right off the printed list and, optionally, point the view right at the string of interest. I see the add-on value with the QShell grep approach in the ability to search multiple source files at once. Thank you!
ReplyDeleteHi Simon. It's very interesting. However I have some problems: The example, as it is, does not work in V7R4 but in general it does the job. In V7R5 it works perfectly (as indicated in the article)
ReplyDeleteI have tried the QShell statement on partitions with the following:
Delete7.4 TR9 = successful
7.4 TR8 = successful
7.4 TR5 = successful
I would check if your 7.4 partition is up to date with PTFs.
Thanks Simon. Didn't know grep could do that. Based on your inspiration I just modeled a new CL command called: GREPSRCLIB off this that uses the QSHEXEC command from my QShell on i Utilities. Look for it here soon: https://github.com/richardschoen/qshoni
ReplyDeleteThank you Simon. This is spectacular.
ReplyDelete1. When I searched for "**free", it took the "*" as a wild card and included /free and /end-free. How can I search for the literal "**free"?
2. When I use the 2nd method and include the command in the QSH command, it works perfectly. When I use the 1st method and try the command in the QSH Command Entry, nothing happens. How can I troubleshoot this?
The way I would start to troubleshoot this is to look in the job log, and check if any spool files were generated that might give you an idea or two.
DeleteTypo.
ReplyDeleteLine 11: If it does not I create it with a Create File command, CRTF, with a record length of 240 characters.
should be:
Line 11: If it does not I create it with a Create Physical File command, CRTPF, with a record length of 240 characters.
Thank you for bringing that to my attention. I have made the correction.
Delete