Tokenization

You can configure how events are tokenized using the ParseAsLines property.

The probe processes the data acquired from a process either one character at a time or one line at a time, depending on the value specified by the ParseAsLines property. The probe separates tokens according to the characters specified by the BreakCharacters property. Delimiters are specified by the WhiteSpaces property.

The following three lines are examples of three events received by the probe from the monitored process:

user;host;port; 0;1;2;3;4;this is an example;1
user;host;port; 0;1;2;3;4;this is an example;2
user;host;port; 0;1;2;3;4;this is an example;3

The following examples show how the probe parses the events differently depending on the value of the ParseAsLines property.

Example 1

If you use the default ParseAsLines property value of 0, the probe parses the events one character at a time and separates the tokens as follows:

Header = <header>
Token 00 = user;host;port;
Token 01 = 0;1;2;3;4;this
Token 02 = is
Token 03 = an
Token 04 = example;1
Token 05 = user;host;port;
Token 06 = 0;1;2;3;4;this
Token 07 = is
Token 08 = an
Token 09 = example;2
Token 10 = user;host;port;
Token 11 = 0;1;2;3;4;this
Token 12 = is
Token 13 = an
Token 14 = example;3
Footer = <footer>

Example 2

If you specify a value of 1 for the ParseAsLines property, the probe parses the events one line at a time and separates the tokens as follows:

Header = <header>
Token 00 = user;host;port; 0;1;2;3;4;this is an example;1
Token 01 = user;host;port; 0;1;2;3;4;this is an example;2
Token 02 = user;host;port; 0;1;2;3;4;this is an example;3
Footer = <footer>