Debugging MapReduce applications using live service debugging

Debug MapReduce applications remotely on the service side using the live service debugging feature.

About this task

Live service debugging refers to debugging a service while it is running in its test (Developer Edition) or production environment. Use live service debugging to debug the map, combine, and reduce functions in MapReduce.

Before debugging your MapReduce application, ensure that the following criteria is complete:

To debug your MapReduce application using live service debugging, follow these steps:

Procedure

  1. Define the port which the service can use to listen on the incoming debugger's connections using an environment variable in any one of the following ways:
    • Set the PMR_SERVICE_DEBUG_PORT from the cluster management console:
      1. From the cluster management console Dashboard, select Workload > MapReduce > Application Profiles.
      2. Find your application and click Modify.
      3. In the application profile window, add the PMR_SERVICE_DEBUG_PORT variable and set its value to any port number.
      4. Unregister and reregister the application profile using the soamunreg and soamreg commands. For example:

        To unregister the MapReduceversion application profile, enter:

        soamunreg MapReduceversion -f

        To reregister the same application profile, enter:

        soamreg MapReduceversion.xml

    • Set the PMR_SERVICE_DEBUG_PORT in the pmr-env.sh configuration file:

      1. Locate the pmr-env.sh file located under $PMR_HOME/conf/.
      2. Disable and enable the application.
  2. Configure Eclipse:
    1. Select the Open Debug Dialog from the Run menu to open the launch configuration dialog, displaying all configurations that support the debug mode.
    2. Click new Remote Java™ Application.
    3. Configure the following:
      • Project that identifies the MapReduce program.
      • IP address of the host running IBM Spectrum Symphony.
      • Port configured for live service debugging in step 1.
  3. Run the MapReduce application using mrsh.

    For example, to run the WordCount job, enter:

    • Hadoop version 2.4.x

      mrsh jar $PMR_HOME/version/os_type/samples/hadoop-mapreduce-examples-2.4.x.jar wordcount input output

    • Hadoop version 1.1.1

      mrsh jar $PMR_HOME/version/os_type/samples/hadoop-examples-1.1.1.jar wordcount input output

    In this live mode, only one service instance (slot) will be started even if your settings specify more than one.

  4. Set breakpoints inside the map() or reduce() methods to discover your bugs.
  5. If you want to disable live service debugging, remove the PMR_SERVICE_DEBUG_PORT setting:
    • If you set the port from the cluster management console, remove the setting from the application profile, then unregister and reregister the profile.
    • If you set the port in the pmr-env.sh configuration file, remove the setting from the file, then disable and enable the application.

    Now, when you submit a job, all service instances will start on your host.