System Container and Control Streams

The System Container

When an EventFlow engine process starts up, a StreamBase container named system is always included alongside the container, usually named default, that holds the top-level EventFlow module. The system container emits continuous streams of information and statistics about the currently running server engine.

There are several system streams emitted from server, described in sections below:

system.connections
system.control
system.subscriptions
system.error
system.statv2
system.statv3

You can see the containers in a running fragment by issuing an sbc list command or

epadmin --ad=adminport display container

Connecting to System Streams

In general, do not make a long-running direct connection to a system stream with an sbc dequeue command. (The epadmin command does not allow dequeuing from system streams.) To monitor control streams, use a container connection between StreamBase containers with an asynchronous connection type, or use the StreamBaseMonitor API for Java or .NET.

Do not design a StreamBase application architecture that assumes you can make long-term connections to any system stream with the synchronous connection type.

Caution

It is not recommended and supported directly connecting to any system stream with a synchronous container connection. Doing so is likely to hamper the functioning of the system container or of the StreamBase server itself.

Synchronous connections to the control stream are not prevented, but produce a warning message instead of an error.

StreamBase provides a number of ways to connect to and interpret system streams, especially the busy system.statv2 stream:

system.connections

Emits a tuple every time a client connects to or disconnects from this server. The schema of the system.connections stream is the following:

Field Data Type Comments
connectionid string A global ID for this connection. This is first field returned from an sbadmin listConnections command, and is used as the argument for the getClientIP() function.
connect bool Shows true for a connection event or false for a disconnection event.
time timestamp The date and time of the connection or disconnection event.

system.control

Tuples are emitted from the control stream in response to certain system-level events. These tuples have the following schema:

Field Type
subsystem string
id int
param0 string
param1 string
param2 string

The use and meaning of fields in the control stream schema differ according to which control stream subsystem sends the tuple. The subsystems are: CONTAINER, CLUSTER, HEARTBEAT, and HA (deprecated). You can identify the subsystem type from the string sent in the first field.

Control Stream, CONTAINER Subsystem

The control stream schema has the following meanings for tuples emitted from the CONTAINER subsystem, which tracks start and stop events for StreamBase containers:

Field Type Description
subsystem string String literal container
id int

Contains one of:

-100 for STARTING events
0 for START events
100 for STOPPING events
200 for STOPPED events
param0 string

Contains the name of the container whose start or stop event is being reported.

param1 string Contains one of STARTING, START, STOPPING, or STOPPED. The meanings of these event states are described in the table below.
param2 string Not used.

The CONTAINER subsystem sends the event tuples described in the following table:

id Field Param1 Field Description
-100 STARTING

A request to start the specified container has been received by the Server, but the container has not completed startup. While the STARTING tuple is sent on the control stream, the container being added does not see this tuple, as it has not yet finished starting. Instead, the STARTING tuple is seen by other containers.

For example, add container X with a container connection to the control stream; container X itself does not see its own STARTING tuple. Now add container Y, also with a container connection to control; container Y also does not see its own STARTING tuple. However, container X does see container Y's STARTING tuple.

0 START The specified container has completed startup.
100 STOPPING A request to stop the specified container has been received by the Server, but the container has not completed stopping.
200 STOPPED The specified container has successfully stopped.

If the specified container is removed and readded, another pair of event tuples are sent.

Note

The container START event is not guaranteed to be the first event processed by modules being started in the newly added container. In most cases, it is the first event, but not all cases.

Control Stream, CLUSTER Subsystem

The CLUSTER subsystem of the control stream emits tuples in response to state changes in the containing node's participation in a StreamBase Runtime cluster. For example, a tuple is sent when a partition becomes active on the local node, or when quorum among nodes is attained or lost.

The control stream schema has the following meanings for tuples emitted from the CLUSTER subsystem:

Field Type Description
subsystem string String literal CLUSTER
id int

Contains the event target, one of:

0 for QUORUM events
1 for NODE events
2 for PARTITION events
param0 string

Contains the event type:

for QUORUM events the cluster name
for NODE events the fully qualified node service name
for PARTITION events the partition name
param1 string

Contains the event reason:

for QUORUM events lost quorum
for NODE events active or unavailable node
for PARTITION events current partition state
param2 string

Contains the event description, one of:

for QUORUM events quorumLost(description)
for NODE events N/A
for PARTITION events previous partition state

Control Stream, HEARTBEAT Subsystem

The control stream schema has the following meanings for tuples emitted from the HEARTBEAT subsystem:

Field Type Description
subsystem string String literal heartbeat.
id int

Contains the heartbeat period in seconds.

param0 string

Contains the sequence number of the current heartbeat event since this EventFlow engine started, converted to a string.

param1 string Not used.
param2 string Not used.

Heartbeat tuples are emitted every heartbeat period for the benefit of StreamBase clients that have enabled heartbeat detection with the TupleConnections.enableHeartbeating() method in the StreamBase Client API. The heartbeat period is 10,000 milliseconds (10 seconds) by default. You can manage the heartbeat period with the clientHeartbeatIntervalMilliseconds parameter in a configuration file like the following example. Set the period to zero to disable heartbeating for all clients.

name = "clientapilistenersettings"
version = "1.0.0"
type = "com.tibco.ep.streambase.configuration.sbclientapilistener"
configuration = {
  ClientAPIListener = {
    apiListenerAddress = {
      portNumber = 10000
    }
    ...
    clientHeartbeatIntervalMilliseconds = 60000
  }
}

Control Stream, HA Subsystem

The HA subsystem of the control stream is deprecated. Migrate your legacy applications to use the high availability services of the StreamBase Runtime.

system.subscriptions

The system.subscriptions stream emits a tuple every time a client starts or stops dequeuing from any stream on this server. The schema of the system.subscriptions stream is the following:

Field Data Type Comments
connectionid string A global ID for the connection, the same ID described for the connections stream.
path string The path to the stream that had a dequeue event, using StreamBase path notation. Examples: system.control, default.outputstream
logicalPath string The path to the stream specified in a filtered subscription, or the path field repeated, if not filtered. See Narrowing Dequeue Results with Filtered Subscribe.
subscribe bool Shows true when a stream starts dequeuing or false when a stream stops dequeuing.
time timestamp The date and time of the dequeue event.

system.error

The system.error stream is part of the EventFlow try-catch error streams system. This system and the schema of this stream are described on Using Error Ports and Error Streams.

system.statv2

The system.statv2 stream continuously emits tuples containing EventFlow engine monitoring statistics for operators, queues, threads, client connections, and memory use in the running server.

As with all system streams, do not make a synchronous container connection to the system.statv2 stream. The statistics tuples are not designed to be human readable or parseable; use one of the tools listed in Connecting to System Streams to interpret this stream.

For reference when developing a custom monitor app, the following is the schema of the statv2 stream:

Field Data Type Comments
cycle int The snapshot number of a group of statistics emitted together from a snapshot. Tuples from the same snapshot are grouped between a tuple whose what value is begn and a tuple with all null values except time.
what string A four-letter string that identifies the type of statistic in that tuple.
name string The name of the feature whose statistic is reported, such as the name of stream, queue, or operator, or a connection ID
index int Used for Java garbage collection statistics (when the what field is gctm) to report the collection count; otherwise null.
value long The value of the statistic reported in the tuple, if appropriate; otherwise null.
status string A status string, if appropriate; usually null.
time timestamp The date and time of the reported statistic event.

Note

Do not confuse the system stream's statistics output from within a running EventFlow engine with the node-level statistics that you can enable and display with the statistics target of the epadmin command. See epadmin help statistics at the command prompt for instructions.

system.statv3

The system.statv3 stream is an optional alternative to the statv2 stream described above. An EventFlow engine emits either a system.statv2 or system.statv3 stream, and must be configured to emit the latter. Without configuration, the statv2 stream is the default.

The statv3 stream reports the same statistics information as statv2, but in a different, binary format that is not human readable. All of the StreamBase tools that interpret a statv2 stream, listed in Connecting to System Streams above, can detect and interpret a statv3 stream, if encountered. The statv3 stream can result in around a ten times improvement in network bandwidth usage, especially with large LiveView projects.

To enable the statv3 stream for an EventFlow engine, set the Java system property streambase.appgen.stats.schema.version equal to 3. The following example configures to use the statv3 stream and also slows down the frequency of each statistics tuple emission:

name = "sbengine"
type = "com.tibco.ep.streambase.configuration.sbengine"
version = "1.0.0"
configuration = { 
  StreamBaseEngine = { 
    systemProperties {
      "streambase.appgen.stats-schema-version" = "3" 
    }   
    streamBase = { 
      engineMonitor = { 
        statsFrequencyMilliseconds = 10000
      }   
    }   
  }
}

For reference, the schema of the statv3 stream has three of the same fields as statv2, but adds a blob field to contain the binary statistics information:

Field Data Type Comments
cycle int The snapshot number of a group of statistics emitted together from a snapshot. Tuples from the same snapshot are grouped between a tuple whose what value is begn and a tuple with all null values except time.
what string A four-letter string that identifies the type of statistic in that tuple.
time timestamp The date and time of the reported statistic event.
data blob Binary statistics data.