Aggregate Operator: Predicate-Based Dimension Options

Contents

Run the Sample
Related Topics

In the Edit Dimension dialog, from the Type dropdown list, select Predicate to specify a predicate-based dimension and give it a name.

With a predicate-based dimension, each new window is opened, emitted from, and closed based on the evaluation of predicate expressions (which must evaluate to boolean true or false). Expressions evaluating to null are treated as false. In the Open, Emit if, and Close if fields you can enter expressions that determine the behavior of the dimension named in the Name field.

As with other dimension types, when the emit conditions are satisfied for any open window, the operator calculates the value of the aggregate expressions specified in the Aggregate Functions tab, and emits a tuple whose fields contain those calculated values, along with the values of any output fields specified by the Group Options tab. Unlike other types of windows, however, a predicate-based dimension lets you explicitly specify arbitrarily complex expressions to evaluate, rather than limiting you to evaluating counts of tuples (tuple-based dimensions), counts of seconds (time-based dimensions), or numeric field values (field-based dimensions).

The following illustration and table describe the options available in the Edit Dimension dialog for predicate-based dimensions. Values displayed in the dialog are from the StreamBase Studio sample Aggregate by Predicate.

In each of the five expression fields in this dialog, input tuple fields can be identified as input1.fieldname or as just fieldname without the path qualification.

Field Expression Required? Expression Type Description
Windows in the current group
Open a window in the current group when No Simple Enter a simple expression that is to be evaluated for each input tuple. The expression must be simple, as the decision to open a new window takes place outside a window. If you enter an expression in this field, an aggregate window is opened when the expression evaluates to true. If you have defined a Group-by expression, a new window opens within the group it belongs to. Leaving the Open field empty causes an aggregate window to open whenever a tuple arrives and there is no active window (that is, when the first tuple arrives or after the previous window closes, as there must always be an open window to receive tuples).
Emit if No Simple or Aggregate Enter a simple or aggregate expression to be evaluated. If you enter an expression in this field, the current window emits a tuple when this expression evaluates to true. If this field is left empty, each window emits when it closes.
Close if Yes Simple or Aggregate Enter a simple or aggregate expression to be evaluated. Enter an expression in this field. The current window closes when this expression evaluates to true. If the Emit field is empty or evaluates to true, a tuple is emitted. If the Emit field contains an expression that evaluates to false when the Close expression is true, the window closes but does not emit a tuple.
Windows in all groups
Emit if No Simple or Aggregate Enter a simple or aggregate expression to be evaluated. If you enter an expression in this field, all currently open windows in all groups emit tuples when this expression evaluates to true.
Close if No Simple or Aggregate Enter a simple or aggregate expression to be evaluated. If you enter an expression in this field, all currently open windows in all groups close when this expression evaluates to true. If the Emit All field is empty, then the windows closed when the Close All predicate evaluates to true also emit a tuple on closing.

You can use dynamic variables and module parameters in any of the above fields. Note that if no Group By fields is specified on the Group Options tab, there is no distinction between "current group" and "windows in all groups". In such cases, "current group" means all the open windows for that Aggregate operator instance.

When using this dialog, keep in mind that:

  • The optional Open a window in the current group when predicate must be a simple expression. All other predicates can be either simple or aggregate expressions.

  • Supply a Close if predicate for the current group of windows. All other predicates are optional.

  • Windows in the current group means all windows in which the current tuple is eligible to be processed.

  • If two Emit if predicates are specified and both evaluate to true, only one tuple is emitted for the current group.

  • When the Emit if predicate for Windows in all groups is true, all windows emit regardless of the truth of any other predicate.

  • To close or emit after a specific number of tuples have entered a window, use the count() aggregate function. For example, the expression count() == 5 will close a window or emit once it contains 5 tuples. This has the same result as a Tuple dimension with a window size of 5 has, but can be part of a more complex expression that meets other conditions.

  • Predicates entered under Windows in all groups apply to all open windows in all groups, regardless of the group to which the current tuple belongs. For the two fields in this section:

    • If the predicate in either the Emit if or Close if field is a simple expression, it is evaluated once for the current tuple and all open windows in all groups are affected equally if the result is true.

    • If the predicate in either the Emit if or Close if field is an aggregate expression, the expression is evaluated separately for each open window in all groups and only those windows for which the result of the evaluation is true are affected.

      For example, a Close if predicate under Windows in all groups is a simple expression such as Price * Volume < 1000 closes all open windows in all groups whenever the values in the input tuple are too small by that criterion. If that predicate were an aggregate expression, such as volume-weighted average price (VWAP), for example, vwap(Price, Volume) < 100, only those windows for which the expression evaluates to true would close.

When you need to ensure that all windows emit or close, use a simple predicate expression. For example, your input stream might signal the end of the trading day by setting a field {done boolean} to true. Setting the Close if predicate under Windows in all groups to the simple expression input.done will cause all windows that are open in any group to emit and close.

If, instead, you set the Close if predicate to an aggregate expression, such as input.time - lastval(time) > seconds(30), when a new tuple arrives, all windows are tested. Some windows might close and some might not, depending on the time value of the last tuple that arrived in each window.

Evaluate Close and Emit predicates before adding the new tuple to the window.

Aggregate operator windows can process tuples in slightly different ways depending on certain options you select for a dimension. In predicate-based dimensions, setting the Evaluate Close and Emit predicates before adding the new tuple to the window checkbox beneath the five predicate fields affects the sequence of processing tuples. The following paragraphs explain how this option affects how Predicate dimension windows respond to input events.

For this discussion, a triggering tuple is one whose values cause a predicate in an Emit if or a Close if expression field to evaluate to true, thus triggering a window to close or emit a tuple. Conditions in the operator might or might not cause a new window to be opened when one is closed.

With this checkbox cleared (the default state), the values of the triggering tuple are included in the calculation of the value emitted from the window. This is the default behavior. Field, Tuple, and Time dimensions do not have this option.

When the checkbox is cleared, processing of predicate-based dimensions occurs in the following order:

  1. Open new windows as needed. (A window is opened if none exist.)

  2. Accumulate the aggregate expressions in the Aggregate Functions tab.

  3. Accumulate any aggregate expressions in the Emit and Close fields.

  4. Evaluate the Emit and Close expressions, and emit tuples or close windows accordingly.

When this checkbox is selected, processing occurs in this order:

  1. Accumulate any aggregate expressions in the Emit and Close fields.

  2. Evaluate Emit and Close expressions, and emit tuples or close windows accordingly.

  3. Open new windows as needed. (A window is opened if none exist.)

  4. Accumulate Emit and Close expressions for the new windows.

  5. Accumulate the aggregate expressions in the Aggregate Functions tab.

In either case, the Emit and Close predicates are always evaluated when processing the triggering tuple. When the Close expression evaluates to true, a tuple is emitted if the Emit predicate is either true or is empty, but not when it is false.

For discussion on what the terms accumulate and calculate mean in this context, see Using the Aggregate Operator.

Run the Sample

You can better understand predicate-based dimension behavior by running the Aggregate Operator Predicate Dimension Sample provided in the Operator sample group. Remember to open a window by enqueuing a tuple with the OpenWindow field set to true to the WindowControl input stream before sending tuples on the TradesIn stream. Then enqueue another WindowControl tuple with CloseWindow set to true to close the window and emit a result tuple. Try changing the Emit predicate from input1.EmitWindow || input1.CloseWindow to a simple expression such as Volume > 100 or an aggregate expression such as vwap(Price, Volume) > 100.

Related Topics

Back to Top ^