QRadar: Application migration fails caused by existing app data on target host

Troubleshooting

Problem

While migrating QRadar applications (from apphost to console or console to apphost), you can see an error if the target host has application data from previous application migrations.

Symptom

Clicking "Migrate to [Console][Apphost]", a warning might be displayed:
```
There is already app data on the target host
```
The migration process stops at Stage 3 in System and License management:
```
Stage 3 of 8: Migrating app data 5 of 10 to target host
```

In /var/logqradar.log, the following error can be observed:

[hostcontext.hostcontext] [pool-2-thread-3] com.ibm.si.application.migration.tasks.StartAppMigrationTask: [ERROR] [NOT:0000003000][<IP>/- -] [-/- -]An error occurred while attempting to execute task to perform migration of type [console]: [com.ibm.si.application.migration.exception.AppMigrationMigrateDataException: There was a problem transferring app data. [Status of transfer to destination [/store/docker/volumes/qapp-2303] is [ERROR] Status code: [24] Metadata: [Process exited with an error: 24 (Exit value: 24)]]]
[hostcontext.hostcontext] [pool-2-thread-3] com.ibm.si.application.migration.exception.AppMigrationMigrateDataException: There was a problem transferring app data. [Status of transfer to destination [/store/docker/volumes/qapp-XXXX] is [ERROR] Status code: [24] Metadata: [Process exited with an error: 24 (Exit value: 24)]]
[hostcontext.hostcontext] [pool-2-thread-3]    at com.ibm.si.application.migration.AppMigrationServiceImpl.migrateAppData(AppMigrationServiceImpl.java:803)
[hostcontext.hostcontext] [pool-2-thread-3]    at com.ibm.si.application.migration.tasks.StartAppMigrationTask.performStandardAppMigration(StartAppMigrationTask.java:208)
[hostcontext.hostcontext] [pool-2-thread-3]    at com.ibm.si.application.migration.tasks.StartAppMigrationTask.runTask(StartAppMigrationTask.java:97)
[hostcontext.hostcontext] [pool-2-thread-3]    at com.ibm.si.frameworks.taskmanagement.Task.run(Task.java:108)
[hostcontext.hostcontext] [pool-2-thread-3]    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:522)
[hostcontext.hostcontext] [pool-2-thread-3]    at java.util.concurrent.FutureTask.run(FutureTask.java:277)
[hostcontext.hostcontext] [pool-2-thread-3]    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
[hostcontext.hostcontext] [pool-2-thread-3]    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[hostcontext.hostcontext] [pool-2-thread-3]    at java.lang.Thread.run(Thread.java:825)
[hostcontext.hostcontext] [pool-2-thread-3] Caused by:
[hostcontext.hostcontext] [pool-2-thread-3]    java.lang.Exception: Status of transfer to destination [/store/docker/volumes/qapp-XXXX] is [ERROR] Status code: [24] Metadata: [Process exited with an error: 24 (Exit value: 24)]
[hostcontext.hostcontext] [pool-2-thread-3]    at com.ibm.si.application.migration.AppMigrationVoltranService.waitForMigrationToComplete(AppMigrationVoltranService.java:172)
[hostcontext.hostcontext] [pool-2-thread-3]    at com.ibm.si.application.migration.AppMigrationServiceImpl.migrateAppData(AppMigrationServiceImpl.java:799)

Cause

During an application migration, the content of /store/docker/volumes/ is copied from the current host to the target host.
If the target host already has data in that same location, the migration might fail.

Note: Application data can be left in /store/docker/volumes/ after previous migrations.

Environment

Qradar version 7.5

Diagnosing The Problem

SSH in to the QRadar console as the root user.
Connect to the target host by using ssh.

Run the following command to list the content of /store/docker/volumes/.

ls -l /store/docker/volumes/

Example output:

[root@hostname~]# ls -l /store/docker/volumes/
total 0
drwxr-xr-x 3 nobody nobody 103 <date- time> qapp-2702
drwxr-xr-x 4 nobody nobody 203 <date- time> qapp-2704
drwxr-xr-x 6 nobody nobody  94 <date- time> qapp-2705
drwxr-xr-x 3 nobody nobody  17 <date- time> qapp-2751
drwx------ 5 nobody nobody 108 <date- time> qapp-2801

Folders named in the format qapp-XXXX are old applications data that might cause the issue.

Resolving The Problem

SSH in to the QRadar console as the root user.
Connect to the target host by using SSH.
Create a backup folder.
```
mkdir -p /store/ibm_support/old_apps/
```
Move the application data in /store/docker/volumes/ to the newly created backup folder.
```
mv /store/docker/volumes/qapp* /store/ibm_support/old_apps/
```

Important: Make sure to run these commands on the target host, do NOT run the commands on the host that is hosting the applications.

Result:
The application migration process no longer fails at Stage 3, while the application data is being moved to the target host.

Related Information

Changing where apps are run

Migrating an App Host from one deployment to another

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwt3AAA","label":"QRadar Apps"}],"ARM Case Number":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"7.5.0"}]

Tips