Using StreamBase Logging

Logging Overview

For all supported operating systems, StreamBase uses the industry-standard SLF4J logging system with the Logback framework. See Windows Event Logging for additional considerations for Windows.

All subsystems in EventFlow and LiveView fragments use Logback logging. You can configure the level of logging information emitted, and can direct logs to different appenders using standard Logback configuration files in either XML or Groovy formats.

Log files are written by default into a subdirectory named logs in a fragment's node directory:

  • For fragments run in StreamBase Studio, the node directory is removed on Studio exit for all successful fragment launches. If a launch fails, its node directory is preserved so that you can inspect log files.

  • For fragment launches made from the command line, you specify the location of each fragment's node directory as part of the epadmin node install command. In this case, the node directory and its logs subdirectory is preserved in the location you specify.

To configure a custom Logback configuration files see Custom Configuration and StreamBase Logback Extensions below. The default logs subdirectory location can be overridden as described in StreamBase Logback Extensions.

Logging Configuration Processing Order

The StreamBase Runtime processes log configuration in the following order:

  1. Default logging.

  2. Logging configuration specified in fragment archives generated from fragment projects.

  3. Logging configuration specified in StreamBase Application archives generated from an Application project.

Logging configurations do not merge. That is, a later configuration in the processing order sequence overrides an earlier one. For this reason, any custom logging configuration file should specify settings for both RootFileAppender and RootConsoleAppender to avoid error messages.

Logging settings are ultimately specified per engine. If your fragment has more than one engine, each engine can inherit the settings made for the fragment and application as a whole.

The logging level and logging configuration can be changed at runtime per engine using the logging target of the epadmin command, as described in Managing Logging at Runtime.

The Log Output Adapter

In addition to the logging messages emitted by EventFlow and LiveView as a whole, you can add logging instructions to any EventFlow module with the Log Output adapter. The adapter has a sample described in Log Output Adapter Sample that illustrates formatting options you can perform with the Log Output adapter.

Do not confuse the Java logging facility discussed here with the StreamBase error stream feature. Logging is for writing your own logging messages and exception stack traces to one or more log files. By contrast, the error stream feature is like a module-level throw that ascends up the call stack to the containing module, and from there to the top-level EventFlow module instance running the fragment. StreamBase sends an error tuple to the error stream when it encounters an exception during a tuple processing method. See Using Error Ports and Error Streams for further information.

Logging in Custom Code

If you write custom operators or adapters, you can use an import org.slf4j.Logger statement in your code to define a logger for your operator or adapter. The base class, Operator, provides the convenience method getLogger(), which returns an org.slf4j.Logger object that should thereafter be used for logging messages and exceptions in your code. For the full list of methods available on Logger objects, refer to the Logger class in the SLF4J Javadoc.

The StreamBase installation includes a logging sample that illustrates the difference between using a Log Output adapter and using getLogger() methods in your custom code. This sample compares both methods with a third module that does not log.

Logback Log Levels

StreamBase uses the standard log level meanings provided by the Logback framework, as defined in the Logger API.

Method Notes
ERROR A node level error caused by a resource problem (memory, disk, file descriptors, and so on), a corrupted product installation with missing required resources, or host machine failures. The node is terminated and must be reinstalled to restore service.
WARN An operational error that impacts a specific operation, application performance, or availability of certain services. An operational error is caused by invalid parameters, temporary resource unavailability, or authentication and/or authorization failures. No other node services are affected. The node may automatically recover from this error, but in some cases operator intervention is required for full recovery.
INFO A normal status message for node and engine life cycle, resource start-up information, and environment details such as library version numbers, class path contents, or location of configuration files. No action is required.
DEBUG Mid-level diagnostic debug messages. DEBUG messages should only be enabled on production systems at the request of TIBCO Technical Support. These messages are only of interest to the product development team.
TRACE Low-level diagnostic trace messages. TRACE messages should only be enabled on production systems at the request of TIBCO Technical Support. These messages are only of interest to the product development team.

Legacy Log Levels

The following table maps the log level integer settings used in StreamBase releases before 10.2.0 to Logback message levels.

Level Method Notes
–2 ERROR  
–1 WARN  
0 INFO The default setting. (Legacy StreamBase releases called this level notice.)
1 or 2 DEBUG Levels 1 and 2 are equivalent ways to specify DEBUG level logging, and produce the same number of log messages.
3 TRACE (Legacy StreamBase releases called this level debug2.)

Separate Adapter Log Levels

Most StreamBase adapters allow you to set a log level independently for the adapter than for the hosting EventFlow module:

  • The Wombat MAMA Input adapter has a set of log levels that derive from the Wombat MAMA API.

  • Most other adapters allow you to set the log level for log messages emitted from that adapter. To determine whether an adapter of interest has independent logging, look for a Log Level property in the Adapter Properties section of the documentation page for that adapter in the Adapters Guide.

For example, an adapter might set the log level to WARN or ERROR for messages emitted from the adapter while the hosting fragment is still set to its default INFO level. This allows you to troubleshoot adapter connectivity issues without having to wade through hundreds of verbose server messages from the fragment.

Logback Default Configuration

StreamBase's default settings for Logback logging are as follows:

  • Two appenders are configured, the console appender and a file appender that writes to the logs subdirectory of the node directory.

  • Log messages from third-party libraries used in StreamBase are set to minimize their chattiness.

  • The default log level is INFO.

  • The default log message pattern is internally defined as:

    %d{yyyy-MM-dd HH:mm:ss.}%usecs%d{Z} [%process:%thread] %-5level %logger: %msg%n
  • One log file is written per engine in a running fragment, with the log file name enginename.log.

  • Log file rollover is set to a maximum size of 10 MB for ten files.

For fragments launched from Studio, the logs subdirectory includes three log file types:

  • .log extension — log messages from the StreamBase Runtime

  • .out extension — log messages sent to standard output

  • .err extension — log messages sent to standard error

The components of the default log message pattern have the meanings shown in the following table.

Message Pattern Component Meaning
%d{yyyy-MM-dd HH:mm:ss} Formatted timestamp.
%usecs%d{Z} Adds the microseconds and time zone values to the timestamp. (%usecs is a StreamBase extension.)
[%process:%thread] The process ID and the name of the thread that generated the logging event. (%process is a StreamBase extension.)
%-5level The log level in five characters, left-justified, with one or more padding spaces added on the right.
%logger The qualified name of the class or component from which the logging event was emitted
%msg%n Everything after the hyphen represents the text of the log message emitted by an instance of the Logger class in code. %n represents the line ending character or characters appropriate for the current operating system.

Consult the Logback documentation for PatternLayout for the available log message pattern components.

You can override any of the default settings with a custom Logback configuration.

Custom Configuration for Logback Logging

To configure Logback logging, specify a Logback configuration file. StreamBase accepts files in either the traditional XML format, often named logback.xml, or in the alternative Groovy format with .groovy extension. In either case, place your configuration files in the src/main/resources folder of your Studio project.

You can also use the src/test/resources folder for a logging configuration you only need when running Maven tests. To have one configuration used both for standard fragment launches and tests, use src/test/resources.

The StreamBase logging sample includes an example logback.xml configuration file that illustrates a setup for both a console appender and a rolling file appender.

StreamBase Logback Extensions

In general, StreamBase Logback configuration files use standard Logback patterns as described in the Logback documentation for PatternLayout. StreamBase provides the following extensions to the standard Logback patterns:


A pattern that specifies the microsecond portion of the current second, used with a timestamp pattern:

%d{yyyy-MM-dd HH:mm:ss.}%usecs%d{Z}

A pattern that specifies the process ID of the currently running engine.


A Logback substitution variable with a default value of node-directory/logs.


A Logback substitution variable with a default value of the current engine name.

You can define different values for these variables in your configuration file:

<variable name="com.tibco.ep.dtm.logging.logDirectory" value="/Users/sbuser/logs"/>
<variable name="com.tibco.ep.dtm.logging.logFileName" value="testrun"/>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
. ...

Search for "Variable substitution" on this Logback documentation page to learn more about Logback variables.

Command Audit Log

StreamBase creates entries in an audit log for every epadmin command issued per node, stored by default in the same logs subdirectory of the node directory.

The audit log:

  • Is named audit.log.

  • Has a default log level of INFO.

  • Has the following Logback message format:

    %d{yyyy-MM-dd HH:mm:ss.SSSZ}: %msg%n

Command audit messages are saved only in the audit log, and are not duplicated in the other node logs.