Aggregate Operator: Time-Based Dimension Options

Contents

Run the Sample
Related Topics

In the Edit Dimension dialog, from the Type dropdown list, select Time to specify a time-based dimension.

With a time-based dimension, a new window is managed and evaluated based on the time a tuple arrives. For example, a tuple containing the aggregate results may be emitted, and the current window closed, when a tuple arrives outside the period allowed for the currently open window. The triggering tuple is received by a new window.

Caution

Using a time-based dimension can cause unnecessary waits in testing or operation, makes impossible any deterministic tuple-by-tuple playback in unit tests and feed simulations, and complicates high availability design patterns.

In general, it is a best practice to avoid the use of the time-based dimension for most applications. Instead, use a field-based dimension with a timestamp field, or a predicate dimension with an expression such as lastval(time_field) - firstval(time_field) > seconds(30) to create a 30-second window.

The Edit Dimension dialog for a field-based dimension has the following appearance.

The following example clarifies the behavior of the time-based dimension option:

  • An aggregate is set with the dimension for window opening and closing to 60 seconds (Advance 60, Close and Emit 60), with no intermediate output.

  • Because there must always be an open window to receive tuples, this dimension's window opens when the first tuple arrives (call this time 0).

  • Subsequent tuples arrive at 3, 8, 17, 31, 42, 49, and 58 seconds.

  • At time 60 seconds, the window emits an aggregate tuple and closes with a lastval of 60.

  • A tuple arrives at time 62 seconds. A new window with its firstval set to 60 opens to receive it.

The following describes the options available in the Edit Dimension dialog for time-based dimensions.

Category Options and Meaning
Opening policy Select one of these options:

  • Do not open window based on this dimension—Default. If selected, this dimension cannot cause the opening of a new window for this Aggregate operator. Note that a window will in fact open when the first tuple arrives if no other dimension has opened a window, as there must always be an open window to receive tuples.

  • Open per:

    • Advance—An integer representing the number of seconds by which to advance the window. Think of this setting as a slider for multiple windows.

    • Offset—A value by which to increment the start of windows. Windows start at the offset plus positive integer multiples of the Advance value. By default, the value of the offset is 0.

      You can specify Advance and Offset as a number, a constant, or any simple expression that evaluates to a time interval expressed in seconds.

      To understand offsets, consider a time-based dimension where the window size is 1 hour (3,600 seconds). With no offset, windows advance by multiple of the window advance. So with an advance of 3600 we expect to see windows at exact hourly intervals, like 8:00, 9:00, 10:00. But suppose you want windows to start at 15 minutes past the hour. If you set the offset to 900 seconds, the preceding window start times would change to 8:15, 9:15, 10:15, and so on.

Window size Select one of these options:

  • Do not close window based on this dimension—Default. If selected, this dimension cannot cause the closing of a new window for this Aggregate operator. If a new window is never opened, and Do not close is selected, this creates an unlimited size window that never closes for the life of the StreamBase Server instance.

  • Close and emit after [number] seconds—If selected, specifies the number of seconds the window remains open. When the number of seconds elapses, the window closes and an aggregate tuple is emitted. number can be a number, a constant, or a simple expression that evaluates to a time interval expressed in seconds.

Emission policy Select one of these options:

  • No intermediate emissions based on this dimension—Default. If selected, this dimension does not force an intermediate emission of a results tuple, after applying the function's calculation to the tuples in the window.

  • Intermediate emission every [number] seconds—If selected, allows tuples to be emitted before the window closes. For example, an operator could emit a tuple every second during the 30-second window, instead of waiting for the window to close. number can be a number, a constant, or a simple expression that evaluates to a time interval expressed in seconds.

Optional windows Select one of these options:

  • Open only a single window for the first event or following a gap in event arrival times—Default. If tuples arrive intermittently to this operator, with long gaps between tuples, select this option to prevent emitting a long series of empty tuples during the gap periods: one for each time event independent of arriving tuples. The operator emits tuples for the first time event after the Server starts, and for the first time event after a gap. Values for the aggregate operators in the Aggregate Functions tab continue to increment during the gap.

  • Open all windows for the first event or following a gap in event arrival times—Select this option to have the operator emit tuples for every time event, including after the first time event and during gaps between the arrival of tuples.

Back to Top ^

Run the Sample

To learn more about field-based aggregation works, run the Aggregate Operator Time Dimension Sample. It contains an Aggregate operator that uses 2-second windows to compute the average price per share of stocks, grouped by the Symbol field. The following table shows output from the sample. The output fields are:

  • Symbol—The stock symbol, used to sort tuples into groups

  • Average—Time-based Average price per share, calculated with the aggregate function avg()

  • start_time—The time the window opened (its openval())

  • end_time—The time the window closed (its closeval())

  • first_time—Time at which the first tuple arrived in the window (firstval(time))

  • last_time—Time at which the last tuple arrived in the window (lastval(time))

  • n—The number of tuples over which the average price per share was computed (count())

Each table row is one window's emission when it closed. Timestamps have been truncated to only show hours, minutes, and seconds.

. Symbol Average Start_time End_time First_time Last_time N
1 MSFT 26.033 08:03:06.000 08:03:08.000 08:03:06.494 08:03:07.583 3
2 AAPL 43.966 08:03:06.000 08:03:08.000 08:03:06.582 08:03:07.583 3
3 AAPL 43.950 08:03:08.000 08:03:10.000 08:03:08.083 08:03:09.583 4
4 MSFT 26.050 08:03:08.000 08:03:10.000 08:03:08.283 08:03:09.483 4
5 AAPL 43.950 08:03:10.000 08:03:12.000 08:03:10.083 08:03:11.683 4
6 MSFT 26.050 08:03:10.000 08:03:12.000 08:03:10.283 08:03:11.583 4
7 AAPL 44.200 08:03:12.000 08:03:14.000 08:03:12.183 08:03:13.487 2
8 MSFT 26.150 08:03:12.000 08:03:14.000 08:03:12.383 08:03:13.487 2

When you run the sample with input from the feed simulation resource AggregateByTime.sbfs, you will find that averages, first and last times, and number of tuples per window all vary, depending on the rate of input.

Related Topics

Back to Top ^