Build report
IBM Dependency Based Build (DBB) provides APIs to create and generate a build report that contains summary information of the build commands executed during the build. The build report is generated as a JSON file on the local file system at the end of the build and can be rendered as an HTML file that can be custom formatted and viewed in a browser.
Image of the default rendered build report provided by DBB
Build report design
At its most basic level, the build report is simply a collection of records, each of which contains information about a build command that was executed during the build. These records are automatically created and added to the build report when the command is executed. This assumes that the build report exists when the command is executed. If the build report has not been created when the command is executed, then no record is created or stored. Each command creates a specific type of record to be added to the build report.
Record Type | Creation Method | Attributes |
---|---|---|
Record (Base Class) | new *Record() |
|
CreatePDSRecord | CreatePDS.execute CreatePDS.create |
|
CopyToHFSRecord | CopyToHFS.execute CopyToHFS.copy |
|
CopyToPDSRecord | CopyToPDS.execute CopytToPDS.copy |
|
ExecuteRecord | MVSExec.execute ISPFExec.execute TSOExec.execute JCLExec.execute |
|
BuildResultRecord | MetadataStore.createBuildResult |
|
DependencySetRecord | DependencyResolver.resolve |
|
VersionRecord | BuildReportFactory.createDefaultReport |
|
* All record types inherit from the base Record class.
** ExecuteRecord and CopyToPDSRecord build reporting attributes
Initializing and accessing the build report
At the beginning of the DBB build process, the user can use the BuildReportFactory static method createDefaultReport()
to create a BuildReport instance. The factory maintains a single static instance of the BuildReport that exists
for the entire build process. Once the BuildReport instance is created, the user can directly access it by using the BuildReportFactory static method getBuildReport()
.
// During build initialization
BuildReportFactory.createDefaultReport()
. . .
// To access the BuildReport during the build
def buildReport = BuildReportFactory.getBuildReport()
Setting ExecuteRecord build report attributes
In the record types table above, most of the attributes shown for each record type are populated automatically by the DBB command class generating the record based on command API arguments. However, in the case of the ExecuteRecord, the attributes marked with '**' are not required by the execute command APIs for execution. They are there for capturing useful information specifically for displaying in the build report. As such, the user will need to explicitly add the attribute information by using optional command API arguments.
File attribute
The file attribute refers to the source file being built. It is used by the default DBB rendering script as the key when collecting ExecuteRecords to display together showing all of the commands executed to build a source file. This attribute is part of the MVSExec, ISPFExec, TSOExec, and JCLExec command classes.
// Build a BMS Map
def copybookGen = new MVSExec().file("MortgageApplication/bms/epsmlis.bms").pgm("ASMA90").parm("SYSPARM(DSECT),DECK,NOOBJECT")
. . .
def compile = new MVSExec().file("MortgageApplication/bms/epsmlis.bms").pgm("ASMA90").parm("SYSPARM(MAP),DECK,NOOBJECT")
. . .
def linkedit = new MVSExec().file("MortgageApplication/bms/epsmlis.bms").pgm("IEWBLINK").parm("MAP,RENT,COMPAT(PM5)")
In the example above, it takes three commands to build a BMS map. By setting the file()
argument on each command to the same file, the ExecuteRecord for each command can be collected for file MortgageApplication/bms/epsmlis.bms
and displayed together in the build report HTML file as shown in the beginning of this document.
Data sets attribute
The data sets attribute is a list of data sets used by the execute command. It is set by the DDStatement utility class by adding a report(true)
attribute to a DDStatement that has the dsn
attribute set. While any
DDStatement used by the execute command can be flagged as a report data set, the MortgageApplication sample application only flags input data sets in its build report.
// Flag input datasets for inclusion in the build report
compile.dd(new DDStatement().name("SYSIN").dsn("$cobolPDS($member)").options("shr").report(true))
Outputs attribute
The outputs attribute is a list of data sets that have been marked as the output of an execute command. It is set by the DDStatement utility class by adding an output(true)
attribute to a DDStatement that has the dsn
attribute set.
// Flag output datasets for inclusion in the build report
linkedit.dd(new DDStatement().name("SYSLMOD").dsn("$loadPDS($member)").options("shr").output(true))
DeployType attribute
The deployType attribute is an additional attribute of a data set marked as an output data set (see Outputs attribute above). It is designed to be used primarily by post build processing tools such as UrbanCode Deploy that deploy build artifacts
to production systems. It is set by the DDStatement utility class by adding a deployType
attribute to a DDStatement that has the output(true)
attribute set. It is only valid when used with a DDStatement flagged
as output(true)
and is ignored otherwise. The value of the deployType attribute can be any valid String value required by the post build processing tool.
// Provide a deploy type for output datasets that can be used in post build tools like UrbanCode Deploy
linkedit.dd(new DDStatement().name("SYSLMOD").dsn("$loadPDS($member)").options("shr").output(true).deployType("LOAD"))
LogFiles attribute
The logFiles attribute is a list of zFS files created or updated by the execute command. These files are created or updated by the CopyToHFS commands added to the execute command and are assumed to be SYSPRINT files copied to zFS as log files. This attribute is set automatically for each CopyToHFS command added to the execute command.
// Copy subcommands automatically contribute to the logFiles attribute
compile.copy(new CopyToHFS().ddName("SYSPRINT").file(logFile))
Setting CopyToPDSRecord build report attributes
Another common build step is copying the JCL file required to execute the built binary. Similar to the ExecuteRecord, you can indicate that a CopyToPDS command should contribute to the build report and display not just source and destination but also the deployment type so that data sets can be correctly handled by an automated deployment manager like UrbanCode Deploy.
Key attribute
The key attribute refers to the source file being built. It is used by the default DBB rendering script as the key when DBB collects ExecuteRecords and CopyToPDSRecords to display together to show all of the commands executed to build a source file.
new CopyToPDS().file(new File("${properties.sourceDir}/$file")).dataset(cobolPDS).member(member).key(file).execute()
Outputs attribute
The outputs attribute contains the target data set when the CopyToPDS command has been flagged as an output for the build report. To set it, add an output(true)
attribute to the CopyToPDS command.
NOTE: The
output(true)
option is required to generate build report attributes to the CopyToPDSRecord. Ifoutput(true)
is missing from the command, the other build report attributes are not generated.
// Flag the target data set for inclusion in the build report
new CopyToPDS().file(new File("${properties.sourceDir}/$file")).dataset(cobolPDS).member(member).output(true).key(file).execute()
DeployType attribute
The deployType attribute is an additional attribute of the target data set. It is used primarily by post build processing tools such as UrbanCode Deploy that deploy build artifacts to production systems. The value of the deployType attribute can be any valid String value required by the post build processing tool.
// Provide a deploy type for the target dataset that can be used in post build tools like UrbanCode Deploy
new CopyToPDS().file(new File("${properties.sourceDir}/$file")).dataset(cobolPDS).member(member).output(true).key(file).deployType("JCL").execute()
User-defined record type
All of the build report record types discussed so far are generated automatically by DBB command APIs and are designed with specific attributes for the command that generates them. To enable users to create and add custom records to the build
report manually, DBB provides a PropertiesRecord record type. The PropertiesRecord is a simple record that contains an internal properties table in which users can add their own property names and values as record attributes. The PropertiesRecord
is in the com.ibm.dbb.build.report.records
package.
// Add a build report record containing build information
def buildInfo = new PropertiesRecord("buildInfo")
buildInfo.addProperty("group", "module_1")
buildInfo.addProperty("label", "build-${new Date()}")
buildInfo.addproperty("description", "Online processing module #1")
BuildReportFactory.getBuildReport().addRecord(buildInfo)
Generating and rendering build report
The DBB build report exists only in memory while the DBB build process is running and will cease to exist once the build process terminates. At the end of the build before the process terminates, the user must save the build report to a file on the local file system.
// get the build report
def buildReport = BuildReportFactory.getBuildReport()
// create build report data file
def jsonOutputFile = new File("usr1/build/BuildReport.json")
def buildReportEncoding = "UTF-8"
buildReport.save(jsonOutputFile, buildReportEncoding)
The content of the saved build report is in JSON format so the convention is to add a .json
file extension to the save file name. This file is also sometimes referred to as the build report data file.
Sample content of build report JSON file (formatted)
{"records":[
{"type":"VERSION","id":"VERSION_1","date":"28-Feb-2018 21:58:25","build":"145","version":"1.0.0"},
{"type":"COPY_TO_PDS","id":"COPY_TO_PDS_1","destination":"USER1.MORTAPP.BMS(EPSMLIS)","source":"\/u\/user1\/test\/jenkins\/workspace\/User1sMortgageApp\/MortgageApplication\/bms\/epsmlis.bms"},
{"type":"EXECUTE","id":"EXECUTE_1","file":"MortgageApplication\/bms\/epsmlis.bms","command":"ASMA90","options":"SYSPARM(DSECT),DECK,NOOBJECT","rc":0,"outputs":[{"id":"USER1.MORTAPP.COPYBOOK(EPSMLIS)","type":"OUTPUT","dataset":"USER1.MORTAPP.COPYBOOK(EPSMLIS)"}],"datasets":["USER1.MORTAPP.BMS(EPSMLIS)"],"logs":["\/u\/user1\/test\/jenkins\/workspace\/User1sMortgageApp\/BUILD-9\/EPSMLIS.log"]},
. . .
Once the build report data file is saved, a second HTML file can be created which displays a summary of build events in a human readable table.
// get the build report
def buildReport = BuildReportFactory.getBuildReport()
// create a default build report html file
def htmlOutputFile = new File("usr1/build/BuildReport.html")
buildReport.generateHTML(htmlOutputFile)
Using the build report in post-build steps
While the main purpose of the build report is to provide a nice formatted HTML summary of the build for people to look at, the build report JSON data file also allows post build processes a convenient way to obtain information about the preceding
build. Additionally, users can pass specific information required by the post build process via the build report by creating and adding custom PropertiesRecords. Use the BuildReport.parse
method to parse a build report JSON file
for post build processing.
// read build report data
def buildReportDataFile = new File("user1/build/BuildReport.json")
def buildReport= BuildReport.parse(new FileInputStream(buildReportDataFile))
// parse build report to find the build outputs to be deployed.
// the following example finds all the build output with deployType set
def executes = buildReport.getRecords().findAll{ record ->
record.getType()=="EXECUTE" &&
!record.getOutputs().findAll{ o ->
o.deployType() != null
}.isEmpty()
}