StreamBase Execution Order and Concurrency

By default, when StreamBase processes data in an application, operations are executed in a predictable order and input tuples are each processed individually to completion. In EventFlow applications, the predictable order is from left to right in the EventFlow canvas.

If you know that portions of your StreamBase application can run without dependencies on the other streaming data in your application, you may be able to improve the overall throughput of the application by specifying that some portions of the application run in their own processing parallel regions. This can result in faster performance on multiprocessor machines, because multiple regions are automatically processed in parallel on the operating systems supported by StreamBase.

The StreamBase rules of execution order are as follows:

A parallel region is a collection of running StreamBase code, rooted at a particular instance of a module, that is guaranteed to process one tuple at a time, and to do so asynchronously from other parallel regions. (There is an exception for synchronous container connections, as described in Rule 9 of the previous section.)

Parallel regions and the Run this component in its own parallel region concurrency setting are closely related.

The following examples clarify the concept of parallel regions:

See Operator Support for Concurrency Options for a table of Spotfire Streaming operators and adapters and the concurrency options supported by each.

Most operators have an associated Multiplicity setting that defaults to single, but which can be changed to multiple. In the case of single multiplicity, each tuple arriving at one of an operator's input ports is processed once.

In the case of multiple multiplicity, you also specify a number of instances. Each module instance is a separate copy of the module specified in the Module Reference, and each module instance has its own independent copy of the module's state, including Query Tables.

Each tuple arriving at one of a multiple component's input ports is passed to one instance or all instances, depending on the dispatch style of the input port. The dispatch style is a property you specify in the case of multiple multiplicity, or when using an Extension Point. Dispatch styles are discussed in Dispatch Styles.

In general, using multiple multiplicity results in improved overall application runtime performance only when the runtime cost of the following sequence of events is significantly less than performing the computation that occurs within the multiply-instantiated instances themselves:

These constraints suggest that the granularity of the work performed within a multiply-instantiated instance should be relatively large — usually much larger than the amount of work performed by a single operator or a small number of operators. Therefore, although the Multiplicity = multiple setting can be used with any operator that supports multiplicity, it is usually only used for Module References.

If you know that portions of your StreamBase application can run without dependencies on the other streaming data in your application, you may be able to improve the overall throughput by allowing those portions to run in their own parallel regions. Parallel regions are usually implemented as separate threads in the JVM running StreamBase Server, but do not think of parallel region and thread as exactly synonymous terms. For example, it is possible for more than one parallel region to execute, serially, in the same thread. It is also possible for separate scheduled executions of the same parallel region to occur in different threads. See Parallel Region Defined.

In StreamBase Studio, in the Properties View for most operators, the Concurrency tab includes an option to Run this component in a parallel region (which changes to Run each instance of this component in a parallel region when Multiplicity = multiple).

This option is available for most operators, except those that need to be processed sequentially. The option to run in a separate parallel region is not available for the Sequence operator, or for Query operators connected to Query Table data constructs. By contrast, the option is available for Query operators that are connected to JDBC Table data constructs.

Let's say you have two Module References for separate subsets of an application. Each Module Reference is set to run in its own parallel region. In addition, when the StreamBase application runs, one additional region is created for the processing of the rest of the application; that region is named main.

You can specify the parallel region option for an individual operator, which would then get its own parallel region. If the individual operator you mark to run in a separate region resides in a module referenced by a parent application, the operator does get its own region, but the other operators in the module run in the containing module's region. Instead, the best practice is to group a set of operators (and any data constructs they use) into a single module, make a Module Reference to that module, and then mark the Module Reference to run in its own thread. Do this only if the Module Reference can run concurrently without data dependencies on other components in your application.

Under heavy load, modules running in parallel regions may not be able to immediately process a tuple. In this case the tuple is queued in the parallel region until it can be processed later. If this queuing continues without bound, memory resources will also be exceeded and the performance of the application will be significantly impacted as the queue depth grows. To avoid this congestion, the maximum number of tuples to queue on a parallel region can be configured. When this queue depth is hit back-pressure is exerted preventing additional work from being queued in the parallel region. When back-pressure is in affect, the thread queuing the inbound tuple will block until the queue depth drops below the configured maximum value, releasing the back-pressure.

When back-pressure is exerted all locks are released to avoid holding locked resources that might prevent work from progressing.

For example, a parallel region as a multiplicity of four with a path of default.SlowModule, the runtime will generate four paths:

Set the streambase.runtime.stream-queue-max-tuples-<parallel-region-module-path> property in the system:

jvmArgs =
    [
        "-Dstreambase.runtime.stream-queue-max-tuples-default.SlowModule.*=10"
        "-Dstreambase.runtime.stream-queue-max-tuples-default.SlowModule%3A0=100"
    ]

The first example uses a regex to set 10 as the maximum number of tuples allowed to be queued for all instances of default.SlowModule. The second example sets the maximum number only for the first instance (default.SlowModule:0) and sets the value to 100. The other instances will use the default of 0 (no limit).

Multiplicity and the ability to run in a separate region are distinct and independently controllable properties of operators. When you enable both separate regions and multiple multiplicity, each instance runs in its own region. (The use of both multiple multiplicity and separate regions together was formerly known as data parallelism.)

You might use single multiplicity and separate regions when you only want to spin off some computation in a module to run asynchronously from the rest of your application's processing.

You might want to use multiple multiplicity and separate regions when you not only want some computation in a module to run asynchronously, but the module's design makes it possible to spin off more than one region and to divide the work across them arbitrarily, independent of the values of the incoming tuples. In this case, set the dispatch style of the input port on which tuples arrive to Round Robin.

Another scenario for using both multiple multiplicity and separate regions is where you not only want a module to process asynchronously, but the module's design makes it possible to dispatch multiple instances using an expression that uses values in the incoming tuple. For example, the incoming tuple might have an integer customerNumber field and the processing might involve a computationally expensive analysis of all the orders received from the identified customer. In this case, set the dispatch style of the input port on which the tuple arrives to Numeric and set the number of instances to a number that reflects how much concurrent computation power you want to devote to the analysis. (This number of multiple instances should not exceed the number of CPU cores available.)

As a guideline, suppose further that the analysis depends on the current temperature, such as for a local business that sells ice cream. In this case, the Module Reference could have another input port on which tuples arrive reporting the current temperature as it changes. For this input port, set the dispatch style to Broadcast because all the module instances require the same temperature information.

Using multiple regions can improve runtime performance on multiprocessor machines. However, using too many regions can actually hurt performance. And the inherent asynchrony that results from using multiple regions makes the behavior of your application harder to describe and predict.

Before you enable multiple multiplicity for a component, read this section and determine whether any of the following cautions applies to your StreamBase application.

Back to Top ^