Logging and containers

Information about supported log levels, container logs and some related errors.

Redirecting container logs to syslog

While you can use the docker logs service command to access the logs for a container, you can also configure Docker to redirect output to system logs.

  1. Edit or create the /etc/docker/daemon.json file.
    {
      "log-driver": "syslog",
      "log-opts": {
        "syslog-address": "unixgram:///dev/log",
        "tag": "docker/{{.Name}}" 
      }
    }
  2. Restart the docker daemon:
    systemctl restart docker
  3. Edit the core.yml and service.yml files located at /opt/aspera/faspex/conf/docker_compose_templates:
    1. Change the driver value to syslog in the docker-compose YAML file of any Faspex container.
    2. Remove the max-size: "10m"and max-file: "10"options under the driver parameter.
    For example, after editing core.yml and service.yml :
    Note: This template is only an example, you should not copy and paste it.
    # cat core.yml
      core:
        image: icr.io/ibmaspera/faspex-core:5.0.5
        container_name: faspex-core
        env_file: [ docker/core.env, docker/db.env ]
        environment: [ SECRET_KEY_BASE=$FASPEX_CORE_SECRET, OLD_SECRET_KEY_BASE=$FASPEX_CORE_OLD_SECRET ]
        ports: ["$FASPEX_CORE_PORT:$FASPEX_CORE_PORT"]
        logging:
          driver: "syslog"
     
        networks: [ docker_network ]
        depends_on: [ db ]
     
    #####################################################################
     
    # cat service.yml
      service:
        image: icr.io/ibmaspera/faspex-service:5.0.5
        container_name: faspex-service
        env_file: [ docker/service.env, docker/db.env, docker/core.env ]
        ports: ["$FASPEX_SERVICE_PORT:$FASPEX_SERVICE_PORT"]
        logging:
          driver: "syslog"
     
        networks: [ docker_network ]
        depends_on: [ core ]
  4. Run faspexctl setup to implement the changes.
  5. Read the /var/log/messages log file to see container logs.

    Faspex containers begin log entries with docker/faspex_container_name. For example, the Faspex core container begins log entries with docker/faspex-core.

    You can use grep and the container name to filter for or filter out specific log entries.
    Filter for the core container:
    tail -f /var/log/messages | grep "docker/faspex-core"
    Filter out the core container:
    tail -f /var/log/messages | grep -i "docker/faspex-core"

Viewing container logs

Use docker logs service to access the logs for a container.

Supported log levels

You can set the desired log level to filter out messages. They are listed in increasing order of severity.

FASPEX_SERVICE supported log levels:

  • debug
  • info
  • warning
  • error
  • fatal
  • panic

FASPEX_CORE supported log levels:

  • debug SQL
  • debug
  • info
  • warning
  • error

Setting log levels

You can configure log levels for Faspex by adding the corresponding variable to each .env file(s) located in /opt/aspera/faspex/conf/docker/ For example: FASPEX_SERVICE_LOG_LEVEL=debug

Variable .env file
FASPEX_UTILITY_LOG_LEVEL=debug utility.env
FASPEX_CORE_LOG_LEVEL=debug core.env
FASPEX_SERVICE_LOG_LEVEL=debug service.env
FASPEX_ROUTER_LOG_LEVEL=debug router.env
FASPEX_UI_LOG_LEVEL=debug ui.env

502 Bad Gateway error in the Web UI

For this error it is important to ensure consistency across the port numbers that the Faspex 5 containers use. If you want to make sure all environmental variables have the same port number, you can update the values in the /opt/aspera/faspex/conf/docker/router.env file. Here's an example of what your file should look like:

FASPEX_ROUTER_CORE_URL=core:3000

FASPEX_ROUTER_UI_URL=ui:3000

FASPEX_ROUTER_SERVICE_URL=service:3000

FASPEX_ROUTER_UTILITY_URL=utility:3000

Make sure that all four parameters (FASPEX_ROUTER_CORE_URL, FASPEX_ROUTER_UI_URL, FASPEX_ROUTER_SERVICE_URL, and FASPEX_ROUTER_UTILITY_URL) have the same port number to avoid any inconsistencies and potential 502 Bad Gateway errors. After making these changes, you'll need to restart faspex to apply the new configuration.

faspexctl restart

Faspex package debug helper

Follow these steps to increase logging levels to debug in core and service containers and send a new package.

Note: We're using pkg_id 123 as an example.

Steps for core container:

  1. To see the stats of all packages run:
    docker logs -f faspex-core 2>&1 | grep 'Processing transfer_stats'
  2. To see the stats of a specific package run:
    docker logs -f faspex-core 2>&1 | grep 'Processing transfer_stats: pkg_id: 123'

Steps for service container:

  1. To see the stats of all packages run:
    docker logs -f faspex-service 2>&1 | grep 'PKG_ID'
  2. To see the stats of a specific package run:
    docker logs -f faspex-service 2>&1 | grep 'PKG_ID: 123'

Steps in the UI:

Go to Admin app > Job queues and search for "delivery_id":123. If the search returns multiple pkg_poller records, click every record and look for direction: upload. If you don’t find it, that means Faspex finished polling on the package.