TM1 Server Feeders Guidelines and Best Practices

Content

Basics of Feeders

TM1 Server uses feeders to identify leaf cells that are exempt from the sparse calculation algorithm.  The TM1 Server uses two different algorithms to determine the value of consolidated cells.  When the SKIPCHECK statement is placed in rules the sparse calculation algorithm is used by default to determine consolidated values.  If the SKIPCHECK statement is not placed in rules then all consolidated values are determined using a slower calculation algorithm.  Without the SKIPCHECK statement all feeder statements are ignored.

The sparse calculation algorithm cannot calculate the value of consolidated cells if leaf cells below the consolidation are rule derived.  Therefore rule derived leaf cells must be fed for consolidation based on those leaf cells to correctly evaluate.  If a consolidated cell incorrectly reports a value of 0 (or a lower than expected value) the likely cause of the problem is missing feeders on rule derived cells that are used to determine the consolidated value.   

The sparse calculation algorithm is the fastest method to determine the value of consolidated intersections during view calculation.  This means that overfeeding (unnecessary feeding) should be avoided to achieve optimal view calculation time for views that contain consolidated intersections.   Avoiding overfeeding also helps to reduce the overall memory footprint of the TM1 Database as feeders have a memory cost.

When authoring feeder statements the objective is to make sure all consolidated cells calculate the expected value by feeding the minimum number of cells.  It’s possible to have consolidated cells that are unused.  In this case it may be unimportant that the value in the consolidated is correct.

Feeder Evaluation

Feeder statements in cube rules are used to determine which cells are fed (have a feeder).  The feeder statements in a cube are evaluated into feeders.  This is also known as feeder processing.  A feeder statement in a cube may feed cells in that cube, or cells in another cube if the DB function is used on the right side of the feeder statement.

The time to evaluate feeder statements for each cube depends on the number of feeder statements, complexity of the feeder statements, and the number of non-empty cells in the cube.  The more non-empty cells in a cube, the longer it will take to process feeders. 

Feeders are evaluated at the following times:

• If during startup, the TM1 Server is missing (or identifies a problem with) a .FEEDERS file in the TM1 database directory.
• Restarting the TM1 database with PersistentFeeders=F (or PersistentFeeders not set in tm1s.cfg)
• Saving a change to rules
• Running a TI process with the CubeProcessFeeders TI function
• Adding a value to an intersection that contains no value (null value).  This only evaluates the feeders for the one intersection.
• Changing the value in an intersection when ForceReevaluationOfFeedersForFedCellsOnDataChange=T.  This only evaluates the feeders for the one intersection. 

The use of the ForceReevaluationOfFeedersForFedCellsOnDataChange parameter is discouraged as it can make data input very slow and have a very noticeable impact on TI processes that load data into cell that are referenced by the left side of a feeder statement.  This parameter is only required if loading a new value into a cell that already contains a value is expected to cause feeders that reference the cell to be reprocessed.  This would only be required when using conditional feeder statements.  Consider avoiding using conditional feeder statements or using a CubeProcessFeerders TI function at the end of the TI process.It’s also important to understand the impact of processing feeders while the TM1 model is in use.  While feeders are being reprocessed for a cube the end users of the TM1 model will not be able to use that cube.  The TM1 model may effectively be locked to end users.

Removing Feeders

Once a cell is fed it remains fed.  This is true even if the feeder statement that caused the cell to be fed is removed or modified.  Reprocessing feeder statements does not remove existing feeders, it only adds new feeders. 

It is possible to remove feeders on a cube by using the cube unload feature (CubeUnload TI function).  This only removes feeders on the cube being unloaded.  It will not unload feeders that are generated in other cubes based on feeder statements in the cube being unloaded.

Feeders may be completely removed and reprocessed on a restart of the TM1 database.  This depending on the PersistentFeeders parameter.  Consider the following cases:

1 - PersistentFeeders=F (or PersistentFeeders is not set).  In this case feeders are lost from memory when the TM1 database is stopped and reprocessed during the start.

2 - PersistentFeeders=T.  In this case the feeders (not feeder statements) are written to feeder files.  The feeder files are read by the TM1 Server on startup and loaded into memory without reprocessing of feeder statements.

It may be occasionally desirable to remove feeders and reprocess feeder statement on a restart when PersistentFeeders=T.  This can be done by running a TI process with the DeleteAllPersistentFeeders TI function before restarting.  Remember that reprocessing feeder statements will typically take much longer than reading feeders from the feeder files.  A common technique is to schedule a weekly or monthly restart where the DeleteAllPersistentFeeders is used, but have more frequent TM1 Database restarts where feeders are not reprocessed.  TM1 Database restarts with feeders reprocessing should be scheduled at a time when users do not require use of the TM1 Database.

Removing and reprocessing feeders is only necessary when feeder statements have changed, feeder statements are conditional and the condition has changed, or data has been unload from a cube.  In many use cases it is not productive to consistently remove feeders and reprocess feeder statements.

Conditional Feeders

Feeders statements can be written to conditionally feed a cell.   The condition may be used to determine what cells are fed based on another cell value in a cell, attribute value, or other conditions.  Conditional feeder statements are evaluated like all other feeder statements.  It’s important to understand that once a cell is fed it will remain fed until a restart with PersistentFeeders=F, a restart where DeleteAllPersistentFeeders TI function is used, or a cube unload.  Accumulation of feeders may explain why a view becomes slower over time and then performance improves with a restart (where feeders are removed, and feeder statements are reprocessed).

Another very important consideration is the performance impact of feeder processing compared to the performance of view calculation.  Feeder reprocessing to support conditional feeder statements while the TM1 model is in production use is typically less desirable than the overhead of some unnecessary feeding. 

Conditional feeder statements can often be avoided by refined the into one or more non-conditional feeder statements.  Unnecessary feeding may also be preferable to conditional feeder statement.  Consider the case where the feeder statement causes cells that are not used in any view to be fed.  If the unnecessarily fed cells and consolidations on top of those cells are not used in any view than the unnecessary feeders will have no impact to view performance.