Running a test from a command line

You can run a test without opening the product by using the command-line interface.

Before you begin

You must have set the Web Reports preferences in the desktop client to view the status of the test run from the command line. For more information, see Accessing reports remotely.

Procedure

  1. Navigate to the directory that contains the cmdline.bat and cmdline.sh files. On Windows operating systems, this directory is typically found as productInstallationDirectory/cmdline. For example, C:\Program Files\IBM\SDP\cmdline.
  2. Issue the following command:

    cmdline -workspace workspace_full_path -project proj_rel_path -eclipsehome eclipse_full_path -plugins plugin_full_path -suite suite_rel_path -importzip file_name.zip -varfile variable_file_full_path -configfile file_full_path -results result_file -overwrite {"true" | "false"} -quiet -vmargs JVM_args -publish serverURL#project.name=projectName -publish_for {ALL,PASS,FAIL,ERROR, INCONCLUSIVE} -exportlog log_full_path -exportstats local_dir_path -exportstatshtml local_dir_path -compare "result_path1, result_path2" -exportstatreportlist stats_list -execsummary local_dir_path -execsummaryreport reportID -usercomments "any user comment" -publishreports "FUNCTIONAL, MOBILE_WEBUI, STATS, TESTLOG" -stdout -swapdatsetsexisting_dataset_file_path:new_dataset_file-path

    Note:
    • The workspace is locked after you issue the command. To check the progress of the test during the run, invoke another workspace and open the project through that workspace.
    • On Linux operating system, the command must start with cmdline.sh.
    If a value contains spaces, enclose the value in quotation marks. To see the online help for this command while you are in the directory that contains the .bat file, type cmdline -help.

    The following table explains each option:

    Option Description
    -workspace Required. The complete path to the Eclipse workspace.
    -project Required. The path, including the file name of the project relative to the workspace.
    -eclipsehome Optional. The complete path to the directory that contains eclipse.exe.
    -plugins Optional. The complete path to the folder that contains the plugins. Typically, on Windows operating systems, this folder is located at C:\Program Files\IBM\IBMIMShared\plugins.

    Required. This option is required only if the folder is at a different location.
    -suite Optional. However, in a command, it is mandatory to use one of the following options:
    • -suite
    • -aftsuite

    You must not use the -suite option along with the other options. The path includes the file name of the suite to run relative to the project.

    Starting from Rational® Functional Tester version 9.2.1.1, you can execute multiple tests simultaneously.

    For example, -suite test1:test2:test3.
    -aftsuite Optional. However, in a command, it is mandatory to use one of the following options:
    • -suite
    • -aftsuite
    You must not use the -aftsuite option along with the other options. The -aftsuite option accepts aft XML as the parameter value. It supports only one aft XML as input.

    For example, -aftsuite aftinput
    -importzip Optional. To import the project as test assets with dependencies into your workspace, use the -importzip option. This command is available from 9.2.1.1 and later.
    -varfile Optional. You can use this option to specify the complete path to the XML file that contains the variable name and value pairs.

    To run a Web UI test on a different browser than that was used for the recording, specify the predefined variable. For more information, see Defining a variable to run a test with a selected browser.
    -configfile Optional. You can use this option to specify the complete path to a file that contains the parameters for a test run. Each parameter must be on a single line. To create a configuration file, you must use an editor that does not wrap lines. Any parameters, whether required or optional, can be set in the configuration file. Command line parameters override the values in this file.
    Notes:

    • If you are creating a config file manually, the file must be in the UTF-8 format. You must not use quotation marks in this file even for values that contain spaces.

    • Starting from V10.0.2, you can create command line config file from Rational Functional Tester, which you can use while running tests from the command-line interface and Maven. See Creating a command-line config file. This option is available only for Web UI and compound tests.
    -results Optional. You can use this option to specify the name of the results file. The default result file name is the test name with a time stamp appended. Specify a folder name that is relative to the project to store the test results. For example, -results folder/resultname.
    -overwrite Optional. Determines whether a results file with the same name is overwritten. The default value, false, indicates that the new results file is created. If the value is true, the file is overwritten and retains the same file name. You must use double quotes for values "true" or "false".
    -quiet Optional. Turns off any message output from the launcher and returns to the command shell when the run or the attempt is complete.
       
    -vmargs Optional. To specify the Java™ maximum heap size for the Java process that controls the command line playback, use the -vmargs option with the -Xmx argument. For example, when you use -vmargs -Xmx4096m, specify a maximum heap size of 4096m. This method is similar to specifying -Xmx4096m in the eclipse.ini file for the workbench when playing back the test from the user interface.

    To capture resource monitoring data, use -vmargs "-Drm.collect=true -Drm.collect.interval=numeric value more than 1000".

    To collect the response time data for the app itself and for the server and network and display them in different bar charts, use -vmargs "-De2e.collect=true". For desktop-based web applications, the response time data is captured and displayed by default.

    To execute tests in parallel on all mobile devices, which are in passive mode, connected to the workbench and ready for playback, use -vmargs "-Dall.available.targets.in.parallel=true".

    To execute tests in parallel on all supported desktop browsers and connected mobile devices, use -vmargs "-Dall.available.targets.in.parallel=all".

    To execute tests in parallel on selected desktop browsers and connected mobile devices, use -vmargs "-Dall.available.targets.in.parallel=browser1,browswer2,browser3". You must separate browser names with a comma. For example, firefox, ff, chrome, ie, ie64, safari, "-Dall.available.targets.in.parallel=browser1,browser2,browser3". Separate browser names with a comma, for example, firefox, ff, chrome, ie, ie64, safari, edge.
    - protocolinput Optional. You can use this parameter to run a Web UI test in parallel on different browsers.

    -protocolinput "all.available.targets.in.parallel=all"

    -protocolinput "all.available.targets.in.parallel=chrome,ff,ie"

    You can also specify the Web UI preferences in this argument.

    For example, -protocolinput "webui.highlight=<value>;webui.report.screenshots=<value>"

    where

    webui.highlight specifies whether the page element must be highlighted.

    webui.report specifies whether the screenshots must be captured while playing back the test in the browser.

    You can use this parameter to run only the failed tests from a previous playback in an Accelerated Functional Test suite.

    cmdline -workspace workspacename -project projectname -aftsuite aftsuitname -exportlog exportlogpath -results autoresults -protocolinput "runfailedtests=true"

    where

    runfailedtests=true specifies whether the failed test from a previous playback must be rerun in Accelerated Functional Test suite.

    Note: If you use the -protocolinput argument, you must not use the following equivalent -vmargs arguments:
    -vmargs "-Dall.available.targets.in.parallel=all"
-vmargs "-Dall.available.targets.in.parallel=browser1,browswer2,browser3"
    -publish Optional. You can use this parameter to publish test results to the IBM Rational Test Automation Server. You can use the following options along with the -publish parameter:
    • serverURL#project.name=projectName: You can use this option if to specify the project name. If the project name is not specified, the name of the current project is used.
      Note: If you provide the server and the project details under Window > Preferences > Test > Rational Test Automation Server in the product, and if you use serverURL#project.name=projectName along with the -publish parameter, the server details in the command-line interface takes precedence over the product preferences.
    • no: You can use this option if you do not want to publish test results after the run. This option is useful if the workbench preferences are set to publish the results, but you do not want to publish them.
    Note: You must provide the offline token for the server under Window > Preferences > Test > Rational Test Automation Server and test the connection before using the -publish parameter in the command-line interface.
    -publish_for Optional. You can use this option to publish the test results based on the completion status of the tests:
    • ALL - This is the default option. You can use this option to publish test results for any text execution verdict.
    • PASS - You can use this option to publish test results for the tests that have passed.
    • FAIL - You can use this option to publish test results for the tests that have failed.
    • ERROR - You can use this option to publish test results for the tests that included errors.
    • INCONCLUSIVE - You can use this option to publish test results for the tests that were inconclusive.
    You can add multiple parameters separated by a comma.
    -exportlog Optional. You can use this parameter to specify the file directory path to store the exported HTTP test log.

    Starting from IBM Rational Functional Tester V10.0.1, by using the -exportlog parameter, you can provide multiple parameter entries when running multiple tests. You must use a colon to separate the parameter entries.

    For example: -exportlog c:/logexport.txt:c:/secondlogexport.txt

    If there are multiple -suite parameter entries with a single -exportlog parameter entry, the -exportlog parameter generates the appropriate number of test logs by appending 0, 1, 2, and so on to the -exportlog parameter entry name.

    For example: -suite "sampletest1:sampletest2:sampletest3" -exportlog c:/logexport.txt The command generates the following test logs:

    • logexport_0.txt
    • logexport_1.txt
    • logexport.txt

    The last test log generated has the same name as that of the initial -exportlog entry.

    Note: If there are multiple -suite and -exportlog parameter entries, the number of suite entries must match with the number of -exportlog entries. Otherwise, the following error message is displayed:

    Error, number of -suite and -exportlog entries do not match.
    -exportstats Optional. You can use this option to export reports in comma-separated values (CSV) format, with the file name derived from the report name. This directory can be relative to the project or a directory on your file system. If the -exportstatreportlist option is not specified, the reports specified on the Export Reports page of the Performance Test Report preferences are exported.
    -exportstatshtml Optional. You can use this option if you want to export web analytic results only. The results are exported in the specified directory. You can then analyze the results on a web browser without using the test workbench.
    -compare You can use this argument along with -exportstatshtml and -execsummary to export the result in compare mode. The value can be paths to the runs and are relative to the workspace. You must separate the paths by a comma.
    -exportstatreportlist Optional. You can use option to specify a comma-separated list of report IDs along with exportstats or exportstatshtml to list the reports that you want to export in place of the default reports, or the reports selected under Preferences. To view this setting, navigate to Window > Preferences > Test > Performance Test Reports > Export Reports.

    To copy the report IDs list into your command line, navigate to Window > Preferences > Test > Performance Test Reports > Export Reports. UnderSelect reports to export, select the required reports, and click Copy ID to clipboard. You can then paste the clipboard content on to your command line editor
    -execsummary Optional. You can use this option to export all of the reports for the test run in a printable format, also known as an executive summary, to the local computer. You must specify the path to store the executive summary.
    -execsummaryreport Optional. You can use this option to export a specific report as an executive summary for the test run to the local computer. You must specify the ID of the report to export. For example, to export an HTTP performance report, specify http. You must use this option along with -execsummary.

    To copy the report IDs list into your command line, navigate to Window > Preferences > Test > Performance Test Reports > Export Reports. Under Select reports to export, select the required reports, and click Copy ID to clipboard. You can then paste the clipboard content on to your command line editor
    -usercomments Optional. You can add text within double quotation mark to display it in the User Comments row of the report.
    Note: You can use the file CommandLine.exe to run the command to add comments in a language that might not support Unicode characters on Windows operating system.
    -publishreports Optional. You can use this option to publish test results in Rational Test Automation Server. The parameters that you can use with it are the following:
    • FUNCTIONAL
    • MOBILE_WEBUI
    • STATS
    • TESTLOG
    For example, -publishreports "STATS, TESTLOG".

    Prefix with “!” to publish all the reports except the specified one.

    For example, -publishreports "! TESTLOG".

    All the reports except the TESTLOG report is published to Rational Test Automation Server after executing the command.
    -stdout Optional. You can use this option to display the information about the test on the command line.

    After you run a test from the command line, the following outputs are displayed to give you the overall information of the test :

    • --VERDICT: The verdict of the test .
    • --REMOTE_RESULT: The URL of the result published to Rational Test Automation Server.
    • --REMOTE_RESULT_UI: The URL of the result published to Rational Test Automation Server and can be opened in a browser to analyze the result.
    • --LOCAL_RESULT: The path of the result saved locally.

    For example, -workspace workspace_full_path -project proj_rel_path -publish publish_url -stdout
    -swapdatasets Optional. Use this option to replace dataset values during a test . If a test is associated with a dataset, you can replace the dataset at run time while initiating the run from the command line.

    You must ensure that both original and new datasets are in the same workspace and have the same column names. You must also include the path to the dataset when you run the -swapdatasets command. For example, -swapdatsets /project_name/ds_path/ds_filename.csv:/project_name/ds_path/new_ds_filename.csv

    You can swap multiple datasets that are saved in a different project by adding multiple paths to the dataset separated by a semicolon. For example, -swapdatsets /project_name1/ds_path/ds_filename.csv:/project_name1/ds_path/new_ds_filename.csv;/project_name2/ds_path/ds_filename.csv:/project_name2/ds_path/new_ds_filename.csv
    To stop the test run, you can open another command prompt window and use one of the following options with the cmdline option:
    Command Description
    -stoprun Optional. Stops the test run after the specified number of seconds. The block is executed, and the test log is transferred before stopping the run. You must use the -workspace command and specify the location of the workspace.
    -abandonrun Optional. Stops the test run immediately. You must use the -workspace command and specify the location of the workspace.
    Note: Messages are displayed to indicate when the test is launched and when it is completed unless you include the -quiet option.

Example

Note: The configuration file must contain information in option=value pairs. Although paths that contain spaces must be enclosed in quotation marks on the command line, you must not enclose spaces in quotation marks in the configuration file. A sample configuration file is as follows:
workspace=D:\My Workspace
eclipsehome=C:\Program Files\IBM\SDP
plugins=C:\Program Files\IBM\IBMIMShared\plugins
project=myProject
suite=mytestsuite

What to do next

After you run the test , you may want to export the results for further analysis. For more information, see Exporting report counters automatically.

Feedback