Using Error Ports and Error Streams

Error Handling Overview

By default, an error encountered at runtime interrupts the processing of data for the current tuple that caused the error. StreamBase Server reacts to the error according to the type of problem encountered. One error type is fatal and requests the Server to shut down. Other error types are logged while processing continues. The table in Processing Error Tuple Data shows the default response for different types of error conditions.

You can add error handling to your EventFlow and StreamSQL applications with three features:

  • Error Ports, which you can enable for individual operators or adapters to provide a catch mechanism specific to that operator or adapter.

  • Error Input Streams, which provide a catch mechanism at the module level.

  • Error Output Streams, which provide a throw mechanism at the module level.

Errors are reported as a tuple in a standardized error tuple format, which can be manipulated as data. You can apply local logic to error processing just as when processing normal tuples.

Error tuples flow from an inner module to its containing module, and flow eventually to the StreamBase Server instance running the application.

Note

Error output streams are considered intermediate streams for purposes of dequeuing. To allow dequeuing directly from an error output stream, you must enable ISD, as described in Intermediate Stream Dequeuing. Note that this is useful only for debugging: in a well-designed application, your application does not need to read the tuples output on error streams, which are passed automatically to the next higher module, or to the hosting Server.

Your StreamBase installation includes an Error Streams sample and an Eclipse Cheat Sheet that interactively walks you through several error handling scenarios. See Error Streams Sample for instructions on running this sample.

Error Ports

An Error Port is a special purpose output port that you can optionally enable for all operators, adapters, and module references. The Error Port emits an error tuple for every processing error that occurs inside this operator or adapter, including Java Exceptions that are thrown in custom operators or adapters.

In EventFlow applications, enable Error Ports on the General tab of the Properties view for operators, adapters, and module references. Select the check box next to Enable Error Output Port. This places a red output port on the component's icon in the palette. The red port is a visual reminder that this port is a special case, and that the schema for this port is the standard StreamBase error tuple. The Error Port is always the last output port of each operator or adapter.

In StreamSQL applications, enable error handling with the ERROR INTO stream-identifier clause. You can place this clause just before the closing semicolon of any of the StreamSQL data manipulation statements, as listed on StreamSQL DML.

Error Input Streams

An Error Input Stream is a special purpose Input Stream that serves as the error catch mechanism for all otherwise unhandled errors in the containing module. There can be only one Error Input Stream in each module. Error Input Streams have a predefined schema, which is the schema of the StreamBase error tuple described below.

In EventFlow applications, add an Error Input Stream to a module by dragging the Error Input Stream icon from the Streams drawer of the Palette view to the canvas. The Error Input Stream icon has a single red output port, which is a reminder that the schema for this port is the standard StreamBase error tuple. In StreamSQL applications, add an Error Input Stream with the CREATE ERROR INPUT STREAM statement.

The output port of an Error Input Stream must be connected to another component. The simplest use of an Error Input Stream is to connect its output to a Map or Filter operator, whose output is then connected to an Error Output Stream. This configuration of three components can remain in the canvas alongside the other operators of your application or module. This configuration, illustrated in the following figure, means: enqueue all otherwise unhandled errors in this module at ErrorInput_A, and map its output unchanged to the input of ErrorOutput_B. Error tuples received at ErrorOutput_B are sent to the containing module or application's Error Input Stream.

Error Output Streams

An Error Output Stream is a special purpose Output Stream that serves as an error throw mechanism for the containing module. There can be more than one Error Output Stream per module, but the tuples emitted from all Error Output Streams in the module are sent to the single error output of the module.

In EventFlow applications, add an Error Output Stream to a module by dragging the Error Output Stream icon from the Streams drawer of the Palette view to the canvas. In StreamSQL applications, add an Error Output Stream with the CREATE ERROR OUTPUT STREAM statement.

It is important to distinguish between:

  • an Error Output Stream

  • a standard Output Stream connected to an operator's Error Port

These cases are illustrated in the following figures. In Case 1, the Output Stream named LoggingOutputStream is not an Error Output Stream. It does have the standard StreamBase error tuple schema as its schema, but that is because it is connected to an Error Port. Connecting a stream to an operator's Error Port does not create an Error Output Stream. That is, such a port does not participate in the throw mechanism for the containing module or application. It is only a standard stream to which you are directing error tuples from the operator or adapter with the Error Port, perhaps for processing farther downstream.

By contrast, the stream named ErrorOutputThrow in Case 2 is a true Error Output Stream. Error tuples from the Astonisher operator are passed on to the next error handler in the sequence of error handlers (that is, to the containing module's error handler, or to the handler for the calling module, or to StreamBase Server.

Case 1: Errors sent to Output Stream for Logging

Case 2: Errors are re-raised to the next error handler

Error Handling in Modules

A module's Error Output Stream, if any, flows to the containing module. This process continues up the chain of modules to the top-level StreamBase application module, and then to the StreamBase Server instance running the application. It is the Server that performs the action designated in the action field of each error tuple.

All unhandled errors in a Module Reference go to the same place, regardless of whether they come from an Error Output Stream or are simply unhandled errors. Errors from a Module Reference appear in the same manner as errors from an operator:

  1. If the Module Reference has an Error Port, then errors are sent there.

  2. Otherwise, if the containing application has an Error Input Stream, then errors are sent there.

  3. Otherwise, error tuples continue up the chain of modules.

Init Errors in Operators and Adapters

Errors that occur during the processing of the init() method in operators and adapters are exceptions, and are not subject to being caught by the error stream mechanism described on this page.

Schema of the StreamBase Error Tuple

The standard StreamBase error tuple has the schema shown in the following table.

Field Data type Comments
description string Contains the error message string. For exceptions thrown in Java operators and adapters, the description field includes the result of Exception.getMessage().
originalTuple string Contains the original failing tuple as a string encoded as a JSON object, which is a set of comma-separated field-value pairs, with pairs separated by a colon, and with the entire object surrounded by braces. Strings in JSON are surrounded by double quotes.

The StreamBase expression language provides the parsejson() function that you can use to extract the field-value pairs of the failing tuple from this error tuple field. Using this function, you can reconstruct the failing tuple by combining the extracted field-value fields into a single field of type tuple. See the Error Streams sample for an example.

operatorName string Contains the name of the operator in which the error occurred, prefixed with the name of its container.
inputPort int Contains the input port number of operatorName on which the failing tuple arrived.
nodeName string Contains the host name of the machine running StreamBase Server.
type string Contains the error classification type according to the Server, from among the error types listed in the table in the next section.
action string Contains either shutdown, continue, or ignore. This is the action ultimately passed to the StreamBase Server instance running the application.
time timestamp The time the error occurred.

The following is an example of an error tuple generated when running the dividemodule.sbapp application from the Error Streams sample delivered with your StreamBase installation. The error tuple is shown for clarity with its fields on separate lines of a table, but the error tuple itself is normally one long comma-separated string of values.

Field Value
description Evaluation exception: Division by zero
originalTuple "{""dividend"":234,""divisor"":0}"
operatorName default.DivisionModule.DoDivision
inputPort 0
nodeName cerberus
type eval-error
action shutdown
time 2011-04-21 13:02:43.527-0400

Processing Error Tuple Data

Your error handling code can read the fields in each error tuple, extracting certain fields for logging purposes. But your code can also manipulate the error tuple data, which can dramatically change how your application operates. The following sections contain a starting point collection of ideas for managing error tuple data. You are by no means restricted to the ideas presented here.

Add an ID String to the Description

Your error processing code might prepend an application-specific ID string to the description field, and then pass the modified errors downstream. When error tuples arrive downstream, a second error processing operator can extract or sort all tuples with the same ID string.

Extract and Correct Failing Tuples

Your error processing code can extract the failing tuple from the originalTuple field using the parsejson() function. For simple cases, your code could insert corrected or default data in a field of the failing tuple, and then re-send a corrected tuple downstream.

Change the Action Value of the Error Tuple

The most dramatic change you can make to your applications is to trap and prevent Server shutdowns on error conditions. Only the fatal-error error type has a default action of shutdown. Your error processing code could look for fatal-error in the type field of error tuples, and if detected, change the action field from shutdown to continue.

The action field can contain the following values:

  • shutdown — Server error handlers are called, and the shutdown error handler (which always runs last) shuts down the Server.

  • continue — Error handlers are called, but the shutdown error handler does not stop the Server.

  • ignore — Error handlers are not called. The error is simply dropped.

The following table lists the StreamBase error categories. The default action for all error types is continue, except fatal-error, whose default action is shutdown.

Error Type
unknown-type-error
fatal-error
eval-error
mem-error
network-error
xml-error
typecheck-error
xmlrpc-error
db-error
authentication-error
expr-error
usage-error
plugin-error
eval-plugin-error
tuple-error
config-error
cluster-network-error
checkpoint-error
checkpoint-setup-error
checkpoint-creation-error
checkpoint-consumption-error
page-pool-error
ha-io-error
ordering-error
adapter-error
container-error
security-tag-error
non-fatal-error