Pipeline Stages

Imagine a stage as a small black box inserted into the plumbing pipeline described in Table 1. Also, imagine a series of such boxes, all connected serially, one after the other, throughout the length of the pipeline. Furthermore, imagine that each box performs one specific task on the water passing through it: adjust temperature, remove salt, or add chlorine. Even though each box does only one thing, the cumulative result is salt-free, temperature-controlled, chlorinated water.

Something similar happens in a NetView® pipeline: data passes through a stage which performs some action on the data. In Figure 1 you can see several stages linked together to form a pipeline that takes data from a disk, processes it, and displays it on an operator console.

Data in the pipeline is viewed as a series of discrete records called messages. They are so called because, when read into the pipeline, each record becomes a message consisting of message text and message attributes.

Figure 2 shows an example of a stage processing messages. Unprocessed messages enter from the left, the stage reads and processes them, and the output appears on the right. This is analogous to the operation of one black box in our earlier plumbing metaphor. Just as in the earlier metaphor, you can string together several stages, each one driven by the output of a preceding stage and each one performing some unique operation on your data.

Figure 2. Messages Flowing through a Stage

In this example, practically anything can happen to the messages: they can be modified, discarded, split apart, joined together, and so on. Precisely what happens depends on the stage that is being used. Many stages generate one output message for each input message; some commands do not. In this example, three messages went in, but only two came out. Without knowing exactly what stage was in effect, we cannot say for sure what happened to the third message, but we do know that such disappearances can be legitimate.

Figure 3 shows a more explicit example. In this case, the stage specification is:

LOCATE /Bob/

LOCATE is a stage and its purpose here is to locate every occurrence of the string Bob in the data passing through the stage. Here, we see three messages flow into the stage. LOCATE looks at the content of each incoming message. If the incoming message contains the string Bob, the message remains in the pipeline. Otherwise, the message is removed from the pipeline.

Figure 3. Messages Flowing through a LOCATE Stage