How do I remove or reduce retention dates of documents to clean an Automation Document Processing deployment?

Answer

You can configure and run a retention update sweep job in the Administration Console for Content Platform Engine to update retention for unexpired Automation Document Processing documents so that the Clean utility can complete successfully.

See Updating object retention with sweeps for additional details on creating a sweep job to update retention. Note the required security needed to create and run this job.

To create the sweep job in the Administration Console for Content Platform Engine:

1. Log in with a credential that is able to create sweep job.
2. Look for the deployed project to be clean up.

   From the navigation panel, go to the selected Object Store and use the Search functionality. Create a ‘New Object Store Search’ to query for ‘Deployed Project’ objects.

   

3. From the deployed projects that are returned, select the one that you intend to clean.

   

4. Get a list of classes that are created when this project is deployed.

   A project can have multiple versions deployed. Gather all the classes across all project versions. You must modify the retention dates for all documents across all of these classes. To do so, issue another Object Store query based on the Deployed Project GUID.

   Note: this query might have a limit on the number of returned records so make sure that you collect all the classes.

   

   Example of the query in text format:

   SELECT This, * FROM ClassDefinition cd WHERE EXISTS (SELECT v.* FROM DbaDeployedProjectVersion v WHERE v.DbaDeployedProject = OBJECT({}) AND v.DbaDeployedArtifactVersions AS a SATISFIES (a.DbaDeployedClassDefinition = cd.This))

   The search result will look like this example. You can potentially have more than 1 Document class.
   

5. Check for documents in the class with retention date set.

   This step is optional. It is used mainly to check the result of running the sweep later on. To do so, issue an Object Store query based on the Class.
   

   Filter output column to show Retention Date.
   

   Example of query result:
   

6. Make sure that the ‘Retention sweep job’ can modify the retention date property on each document of the class.

   This step needs to be done once and will apply to all subsequently created retention sweep jobs.

   Retention update sweeps are supported in two modes: RetentionReductionAllow and RetentionReductionPrevent, controlled by the AllowRetentionReduction property. When the mode is set to RetentionReductionAllow, then the sweep updates the retention on all affected items, even if doing so results in the retention period being reduced. When the mode is RetentionReductionPrevent, then the sweep does not update the retention on any objects for which the update would result in reducing the retention period. The AllowRetentionReduction property is read-only, which means that the only way to change the value is to update the default value given in its property definition. The default value can be changed by modifying the PropertyDefaultBoolean property on the Property Definition for the AllowRetentionReduction

   To get to the PropertyDefinition, navigate to OS > Data Design > Classes > Other Classes > Sweep > Sweep Job > Retention Update Job. There you can view the Class Definition for the Retention Update Job. The ‘Class Definition’ contains a list of ‘Property Definitions’. Look for the PropertyDefinition of AllowRetentionReduction property in order to change its default value from ‘False’ to ‘True’.

   Important: You will need to set this property value back when done.
   
   
   Select the ‘Allow Retention Reduction Definition’ to get to a Property Definition screen.
   On the new screen, look for the field ‘Property Default Boolean’. Change its value from False to True. Remember to reset this back when you are done.
   

7. Create a sweep job for each gathered class:
   1. To create a new sweep job, on the left hand side tree panel, navigate this path: OS > Sweep Management > Job sweeps > Retention Update Jobs. Right click to create a new ‘Retention Update Job’.

      Complete the wizard screens.

      For the first screen, give the job a display name to indicate which project and which class this job is for. Change the ‘Sweep Mode’ from the default setting ‘Preview Only Counters’ to ‘Normal’ so when this job runs, it actually changes the retention date, not just tell you how many elements will be updated.
      
      

       
   2. Click Next to take you to the ‘Define Sweep Targets’ screen. For the ‘Target class’ field,  type in the ‘display name’ of the class whose retention date will be updated. If you have multiple entries with the same display name, since display name is not unique, you might want to rename the displayName of the class. Check the ‘Include subclass’ Enabled flag.

      You can also check the ‘Record failures’ to review errors in the ‘Sweep Result’ page instead of the p8 error log.
      

   3. Set the new retention date to be less than today.
      
   4. The summary screen should look like this before you click Finish.
      
   5. Click Finish to create a ‘Retention Sweep Job’.
8. Verify and Run the sweep job that you just created.
   1. Make sure that the owner of the sweep job has permission to modify the retention date on the Object Store.

      Retention updates sweeps run with the access rights of the security principal specified as the owner of the sweep. 

      Access the sweep job just created. Check who is the owner. You can update the owner at this point.

      Make sure that the Owner has permission to ‘Modify retention’ on the whole Object Store.
      

   2. Perform the last check of the sweep job before running it. Make sure the following fields are set properly before running it.  On the General Page, make sure that:
      • Include subclasses is checked
      • Record failures is checked
      • Target class specifies the correct class
      • Retention date is set to expected value
      • Sweep Mode is set to Normal
   3. Run the job manually by clicking on Enable or create a schedule.

      Trigger once by checking ‘Enable retention update job’, Save and Refresh.
      

      Or schedule it.

      Note: After the job runs to completion, it cannot be re-run. You must create a new job if you want to run it again. You might need to wait for 15 minutes to schedule a new job.
      

      The sweep start and end date will be updated when it runs and it will also report the number of elements updated.

9. Check the output of the sweep job. Use the query shown from step 4.
10. After updating retention dates on all documents across all classes of the project, delete all sweep jobs that were created in step 7.
11. Reset the value for the PropertyDefinition of the AllowRetentionReduction property from ‘True’ to ‘False’.
12. Rerun CPDS clean project.