Using the BSort Operator

  The BSort operator performs an approximate sort of an input stream. It is useful when you anticipate that the input stream might be slightly out of order with respect to some field. Of course, a complete sort is not possible over a potentially unlimited stream of data. However, you can use the BSort operator to order the tuples that are temporarily stored in its buffer, as data flows through the application.

You sort on a specified input tuple field, using one of these two methods:

How you use these options depends on how you want to balance latency and ordering. Setting a high range of values or a large buffer size favors complete ordering, at the expense of higher latency while input is buffered. Conversely, a low setting favors low latency at the expense of making the output less ordered.

An optional third method is to reorder tuples by grouping on one or more input data fields.

You might use the BSort operator to order all tuples as they arrive, or only those meeting a certain criteria. For example, for a stock ticker feed that contains a time field and a symbol field, you could sort on time for all stocks, or sort on time for stocks of each symbol.

The BSort operator accepts a single input stream and produces a single output stream whose output fields match the fields of the input tuples. The BSort operator is not order sensitive.

Name: Use this required field to specify or change the name of this instance of this component. The name must be unique within the current EventFlow module. The name can contain alphanumeric characters, underscores, and escaped special characters. Special characters can be escaped as described in Identifier Naming Rules. The first character must be alphabetic or an underscore.

Storage method: Specify settings for this operator with the same in heap and in transactional memory options described for the Annotations tab of the EventFlow Editor. The default setting is Inherit from containing module.

Enable Error Output Port: Select this checkbox to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports to learn about Error Ports.

Description: Optionally, enter text to briefly describe the purpose and function of the component. In the EventFlow Editor canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.

Use this tab to the field by which to sort, and the set of data in the BSort buffer that is to be sorted. See the Introduction for a general description of sorting.

Sort by

Choose the kind of data that you want to sort by, and specify the bounds of the sort.

Values, with a range of n

Choose this property if the selected sort field is numeric (for example, not a string), and if you want to sort on a range of values in the sort field. Specify the range as a number that matches the data type of the selected sort field. For example, if the sort field is an integer, express the range as an integer.

Each time a tuple arrives, BSort evaluates its order among the tuples in the buffer, based on the values in their sort field. If the new tuple would cause the range of sort field values to exceed the specified range, BSort dequeues the lowest tuple or tuples in the group, as required to keep the remaining values within the specified range. Consider the following example:

• Your BSort's sort field, Count, is an integer. You choose the option to sort by Values, with a range of 4.

• Two tuples arrive whose Count fields have the values 2 and 3. These values are within the specified range, so they are stored in the BSort buffer, and nothing is dequeued.

• Next, a tuple arrives with a Count field of 20.

• BSort dequeues the first two tuples (in order), because they would cause the range of sort field values to exceed the specified limit.

There is no limit to the number of tuples in the buffer as long as their sort fields are within the specified range.

Tuples, using a buffer size of n

Choose this property if you want the sort to be bounded by the size of the buffer instead of the value of the sort field. Specify the buffer size as an integer. To see how a BSort by tuple works, step through the Example in this topic.

Note that for both types of Bsort, tuples with duplicate key fields are stored in the buffer just like any other tuple. As sorting occurs, duplicate tuples are released in the order they arrived.

The following figure shows a selection for sorting on the Time field, with a buffer size of 3:

Optionally, you can use the Group Options tab to specify one or more groups of sorted tuples in your output. (For a description of sorting, see the Introduction section.) Instead of sorting all tuples as a single group, the Bsort operator sorts each group separately.

In the Group By table, add a row for each group that you want to sort on. Identify the group in the Expression column. The definition is frequently the name of an input field.

For example, suppose that your input schema included a timestamp and stock symbol field, and you selected the timestamp field to sort on (as shown in the preceding Sort Settings tab. If you did nothing else, BSort would sort all of your input in timestamp order. Now, suppose you still wanted to order by timestamp, but you were interested in the order of each separate stock symbol. To accomplish this, you could use the Group Options tab, specifying the Symbol field as shown in the following figure:

Use the Concurrency tab to specify parallel regions for this instance of this component, or multiplicity options, or both. The Concurrency tab settings are described in Concurrency Options, and dispatch styles are described in Dispatch Styles.

The following series of diagrams traces the flow of unsorted data through the BSort operator. In this example, assume that we want to sort by tuples, with a buffer size of three. No grouping is specified, so all fields are sorted together. In each step, the single box on the left represents a tuple enqueued on the input port of our BSort operator. When a sort occurs, the output tuple is shown flowing out of the BSort operator's output port on the right.

The preceding steps demonstrate how this BSort operation compares each inbound tuple with the current values in the buffer and, if the buffer is full, emits the tuple with the lowest value.

Finally, compare the order of inputs on the left (by reading the sequence from the top down) to the order of outputs on the right. The order is approximate because, on a potentially unlimited stream of data, the inbound tuples are compared only with the tuple values currently in the BSort buffer. Also notice the latency: no output occurs until the fourth tuple is input. Keep in mind that in your application, sort results depend on the size of the buffer, and the degree to which the incoming values are unsorted.

Back to Top ^