Using StreamBase Logging

Logging Overview

The StreamBase Java Client library uses SLF4J, the Simple Logging Facade for Java, for all internal logging. TIBCO encourages customers to use the same logging system when developing your custom embedded Java adapters and Java operators.

SLF4J is a successor project to Log4J and is written by the same team. It provides a front-end facade for various Java logging system back ends that can be loaded at deployment time. The back ends SLF4J supports include JDK 1.4 Logging, Log4J, and Logback, among other systems. StreamBase uses two logging back ends under different circumstances:

  • Logback. For sbd running in the foreground, or for sbd launched from StreamBase Studio, StreamBase uses the Logback back end for SLF4J. Logback is the default logging mechanism for the StreamBase Java Client library.

  • Log4j. For StreamBase Server running in the background (that is, when launched with sbd –b or when running as a Windows service), the logging back end switches to Log4j. The reason for the different logging mechanism when running in the background is that Log4j includes appenders that know how to write to the Windows Event Log. (The Logback back end lacks a Windows Event Log appender.) See Server Background Mode and Logging for further details.

Logging Configuration Samples

The StreamBase installation includes two samples that illustrate the settings to configure logging and to enable an alternate logging system. The samples are:

sample/logging-logback

This sample provides an example logback.xml configuration file that configures both a console appender and a rolling file appender. This sample illustrates customizations you can apply to the Logback logging back end.

sample/logging-log4j

This sample provides an example log4j.xml configuration file that configures a console appender and a rolling file appender, and configures the same log message format as the logging-logback sample, except for the prefix "Log4J" in the log message. This sample illustrates the Studio, configuration file, and runtime environment changes required to replace the default Logback back end with the Log4j back end.

Environment Variables That Affect Logback Logging

The default Logback logging back end can be configured with environment variables, Java system properties, or with a configuration file as described in Configuration for Logback Logging. If both an environment variable and system property are specified, the system property takes precedence. The settings that affect Logback logging are shown in the following table.

Environment Variable Java System Property Default Value Description
STREAMBASE_LOG_PATTERN streambase.log‑pattern (See Logback Default Configuration) Controls the output of log messages to standard output. See the logback PatternLayout documentation for the available conversion sequences.
STREAMBASE_LOG_LEVEL streambase.log‑level 0 (INFO) This setting controls the severity of log messages sent to standard output. See the table in the next section for a mapping of integer keys to log levels.

You can further configure Logback with a configuration file, as described in Configuration for Logback Logging below.

Note

These environment variables are only used by the Logback back end. If you replace Logback with an alternative back end as described below, these environment variables (and equivalent Java system properties) are not honored.

Log Levels

The following table maps STREAMBASE_LOG_LEVEL integer settings to message levels in the Logger API.

Level Method Notes
–2 ERROR  
–1 WARN  
0 INFO The default setting. (Older StreamBase releases called this level notice.)
1 or 2 DEBUG Levels 1 and 2 are equivalent ways to specify DEBUG level logging, and produce the same number of log messages.
3 TRACE (Older StreamBase releases called this level debug2.)

Separate Adapter Log Levels

Most StreamBase adapters allow you to set a log level independently for the adapter than for the hosting StreamBase Server:

  • The Wombat MAMA Input adapter has a set of log levels that derive from the Wombat MAMA API.

  • Most other adapters allow you to set the log level for log messages emitted from that adapter, using the same log level settings used by the server, described above in Log Levels. To determine whether an adapter of interest has independent logging, look for a Log Level property in the Adapter Properties section of the documentation page for that adapter in the Adapters Guide.

For example, adapters can set the log level to WARN or ERROR for messages emitted from the adapter while the hosting server is still set to its default INFO level. This allows you to troubleshoot adapter connectivity issues without having to wade through hundreds of verbose server messages.

Embedded adapters participate in the logging back end currently used by the hosting server. Thus, independent adapter log level settings are passed through to the SLF4J front end and Logback back end when running the server in the foreground or from Studio, and are passed to the Log4j back end when running the server in the background.

Logback Default Configuration

StreamBase's default settings for Logback logging are as follows:

  • One appender is configured, the console appender.

  • The default log level is INFO.

  • The default log message pattern is internally defined as:

    %d{yyyy-MM-dd HH:mm:ss.SSSZ} [%thread] %-5level %logger{36} - %msg%n

The following sbd startup message, taken from the Console view in StreamBase Studio, illustrates the components of the default log message pattern. (The message is shown on two lines for publication clarity.)

2011-02-22 13:30:13.721-0500 [ThreadPool-1] INFO  c.s.sb.sbd.net.StreamBaseHTTPServer - 
   sbd at Argyll:10000; pid=4048; version=7.0.4; Listening

The components of the default log message pattern have the meanings shown in the following table.

Message Pattern Component Meaning Results in this example
%d{yyyy-MM-dd HH:mm:ss.SSSZ} Formatted timestamp. 2011-02-22 13:30:13.721-0500
[%thread] The name of the thread that generated the logging event. [ThreadPool-1]
%-5level The log level in five characters, left-justified, with one or more padding spaces added on the right. INFO
%logger{36} The qualified name of the class or component from which the logging event was emitted, truncated as necessary to fit in 36 characters. The final name is never abbreviated, even if it is longer than 36 characters. c.s.sb.sbd.net.StreamBaseHTTPServer
- A literal hyphen. -
%msg%n Everything after the hyphen represents the text of the log message emitted by an instance of the Logger class in code. %n represents the line ending character or characters appropriate for the current operating system. sbd at Argyll:10000; pid=4048; version=7.0.4; Listening

Consult the Logback documentation for PatternLayout for the available log message pattern components.

Configuration for Logback Logging

You can configure Logback logging in simple ways without resorting to a configuration file. You can:

  • Specify a different log message pattern with the environment variable STREAMBASE_LOG_PATTERN (or the system property streambase.log-pattern).

  • Specify a different default log level with the STREAMBASE_LOG_LEVEL variable (or the system property streambase.log-level).

For more complex Logback configuration, specify a Logback configuration file, which is often an XML file named logback.xml. There is an online wizard for generating Logback configuration files that makes basic logging setup straightforward that you can access at http://wizard4j.org/pc?action=languageExamples. Use the wizard to generate a Logback XML configuration file to change the log destination, configure filename rolling rules, specify log filename patterns and log string patterns, and so on. You can also download a logback.xml template file to modify.

If you inadvertently specify both STREAMBASE_LOG_PATTERN and a Logback configuration file, any message pattern specified in the configuration file is the one used by StreamBase Server.

The StreamBase sample logging-logback includes an example logback.xml configuration file that illustrates a setup for both a console appender and a rolling file appender.

StreamBase incorporates Logback version 1.1.2. Documentation on the Logback website may describe configuration elements added in later versions of Logback. To obtain accurate documentation on the Logback version included with StreamBase, follow these steps to obtain version-specific Logback documentation:

  1. From the Logback distribution site, download either logback-1.1.2.zip or logback-1.1.2.tar.gz.

  2. Expand the archive file to a local directory that you identify in the environment variable LOGBACK_HOME.

  3. Point your browser to $LOGBACK_HOME/docs/manual/index.html (UNIX) or %LOGBACK_HOME%\docs\manual\index.html (Windows).

StreamBase supports configuring Logback using any of the configuration rules and methods documented for Logback 1.1.2.

It is common in development environments to have Logback locate its configuration file on the Java runtime classpath. When developing with StreamBase Studio, you can place your logback.xml file in the java-src directory of your Studio project. This makes the file a Java resource on the Java Build Path when running your application from Studio and when deploying your application with a StreamBase bundle file generated from your Studio project. To specify using java-src/logback.xml when running sbd from the command line, make sure your server configuration file includes elements like the following example:

<streambase-configuration>
...
  <java-vm>
    <dir path="java-src"/>
  </java-vm>
...
</streambase-configuration>

Another way to configure Logback is to specify the path and name of the Logback configuration file using the logback.configurationFile Java system property. Using this property allows you to separate the Logback configuration file's location from your development environment, which allows you to locate the file independently in the development and deployment environments using an environment variable or other method. This allows you to use separate Logback configuration files for different situations.

Specify this system property in the server configuration file with the <sysproperty> child element of the <java-vm> element:

<streambase-configuration>
...
  <java-vm>
    <sysproperty name="logback.configurationFile" 
                 value="/path/to/logback.xml"/> 
  </java-vm>
...
</streambase-configuration>

Using Logback Logging in Custom Code

In the StreamBase Java Client library, the base class, Operator, for Java operators and embedded adapters provides the convenience method getLogger(). This returns an org.slf4j.Logger object that should thereafter be used for logging messages and exceptions in your code. The logError() method from previous StreamBase releases and all related convenience methods have been deprecated. TIBCO recommends replacing any instances of logError() in your existing StreamBase Java code.

Replace instances of logError() with calls to comparable logging methods on the Logger object returned by Operator.getLogger(). For the full list of methods available on Logger objects, refer to the Logger class in the SLF4J Javadoc.

Code generated by the StreamBase Java code generation wizards includes examples of calling and using getLogger().

Note

The StreamBase Java Client library has internal methods with names such as logError() and fatalError(). These are not part of the public API, and should not be called by end users.

StreamBase Studio bundles the Javadoc for the Logger class so that when you are typing Java code in a Studio editor, you see code assistance pop-ups to help you understand the methods available for logging.

Do not confuse the Java logging facility discussed here with the StreamBase error stream feature. Logging is for writing your own logging messages and exception stack traces to one or more logs. By contrast, the error stream feature is like a module-level throw that ascends up the call stack to the containing application, and from there to the StreamBase Server instance running the application. To send an error to the error stream, you only need to throw an exception during a tuple processing method. See Using Error Ports and Error Streams for further information.

Two Sources of Logging Messages

The SLF4J logging mechanism captures most, but not all debug and console messages from the running server. There are two sources of logging messages emitted from StreamBase Server:

  1. The SLF4J front-end logging mechanism and the logging back end together capture everything that runs in the server's Java virtual machine (JVM), including output from all running StreamBase applications, as well as much of the server infrastructure.

  2. StreamBase Server uses a small amount of native code for each platform to start and manage the JVM. Messages from this sbd startup code convey information about server startup and shutdown, plus some miscellaneous output.

    When running sbd in the foreground or within Studio, these messages from the native startup code are sent to standard output, like other log messages. When running sbd in the background, these startup messages are not directed through Log4j, but are delivered directly to syslog for UNIX and to the Event Log for Windows.

Advanced: Overview of Changing Default Logging

If your site has standardized on a different Java logging mechanism, you can configure StreamBase to use a different logging system. You can:

StreamBase provides three versions of the Java Client API JAR file. This JAR file is used by StreamBase Server and by all custom client, operator, and adapter code. The three client JAR file versions are:

Java Client Library JAR File Logging Features
sbclient.jar The default case. Includes the SLF4J front end and Logback back end. Use the Logger API to emit logging messages in your custom code. Can be configured with a logback.xml file.
sbclient-no-logging-backend.jar Includes the SLF4J front end but no logging back end. Use this version of the file to replace the default Logback back end with an alternative back end supported by SLF4J. The alternative back end must be configured with its appropriate mechanism, such as with a log4j.xml file when replacing the back end with Log4J.
sbclient-no-logging.jar Does not include SLF4J or any logging mechanism. Use this version to suppress all log and debug messages (which is not recommended) or to substitute a different version of SLF4J used by your site.

In general, to use an alternative logging back end, you must:

  1. Add the alternative Java Client Library's JAR file (sbclient-no-logging-backend.jar) to your classpath, replacing sbclient.jar.

  2. Include the JARs that implement your alternative logging system to the classpath. This generally means:

    1. Adding the alternative JARs to the Java Build Path for your Studio project (for running your application in Studio).

    2. Adding the alternative JARs in one or more <jar> child elements of the <java-vm> element of the server configuration file (for running your application from the command line).

Advanced: Replacing Logback with Log4J

As described in Logging Overview, the Log4J back end is automatically used when StreamBase Server is run with sbd -b on the command line, or when run as a Windows service (which requires the use of sbd -b). This section describes the configuration steps that allow you to use Log4J for all StreamBase logging, including when running from Studio or running in the foreground with the sbd command.

There are four criteria that must be met to successfully replace the Logback back end with the Log4J back end. The criteria can each be implemented in different ways to impact the different ways to run StreamBase applications.

Criterion 1: Direct StreamBase Server to run with the no-logging-backend client JAR

To implement criterion 1, you can either:

  • Set the environment variable STREAMBASE_USE_CUSTOM_LOGGING to true in the environment that will run StreamBase Server.

  • Run the sbd command with the --use-custom-logging command line option.

Implement criterion 1 for different StreamBase environments as follows:

Running in StreamBase Studio with Log4J:

Edit a Launch Configuration for your application as described in Editing EventFlow Run Configurations. Add STREAMBASE_USE_CUSTOM_LOGGING=true to the Environment tab of your Launch Configuration.

You must also add the JAR file sbclient-no-logging-backend.jar to the Libraries tab of the Java Build Path dialog for the Studio project. Locate this file in $STREAMBASE_HOME/lib.

Running with sbd at the command prompt with Log4J:

Either set the environment variable for the shell environment from which you run the sbd command, or use sbd --use-custom-logging to run the server. This applies both to running sbd and running sbd -b at the shell command prompt.

Criterion 2: Remove the default sbclient.jar from the classpath

Criterion 2 may not be required for your environment. In general, this is required when running sbd interactively on Windows using a StreamBase Command Prompt, which automatically appends the path to sbclient.jar to the CLASSPATH environment variable. In this case, you must edit the CLASSPATH variable to remove the reference to sbclient.jar. If CLASSPATH has sbclient.jar as its only entry, just remove the variable by setting it to null: set CLASSPATH=

Criterion 3: Add the JARs that implement Log4J to the classpath

To run Log4J, you must add two JAR files to the classpath:

  • slf4j-log4j12-version.jar, a file that bridges the SLF4J front end with the Log4J back end, obtained from the SLF4J distribution.

  • log4j-version.jar, the JAR that implements Log4j itself, obtained from $STREAMBASE_HOME/lib/ext or from the Log4j web site.

Implement criterion 3 for different StreamBase environments as follows:

Running in StreamBase Studio with Log4J:

Add both JAR files to the Java Build Path for your Studio project, using the Libraries tab of the Java Build Path configuration dialog.

Running with sbd at the command prompt with Log4J:

Add both JAR files with <jar> child elements of the <java-vm> element in the server configuration file. Then run the sbd command with its -f option: sbd -f sbd.sbconf ... Example:

...
<java-vm>
  <jar file="${STREAMBASE_HOME}/lib/slf4j-log4j12-version.jar"/>
  <jar file="${STREAMBASE_HOME}/lib/ext/log4j-version.jar"/>
</java-vm>
...
Criterion 4: Specify the location of your Log4J configuration file

When StreamBase Server automatically uses the Log4J back end, as described in Logging Overview, it reads its Log4J configuration instructions from $STREAMBASE_HOME/etc/log4j.configuration. To configure Log4J run automatically, either edit this file, or provide a system property that points to a different configuration file, as described next.

When explicitly replacing the default Logback back end with Log4J, no configuration file is used by default. You must explicitly specify the path to a Log4J configuration file, using the log4j.configuration system property. You can use the <sysproperty> element of the server configuration file for this purpose, as shown in this example:

<java-vm>
...
  <sysproperty name="log4j.configuration" value="log4j.xml"/>
...
</java-vm>

You can specify the path to a configuration file in either the older .configuration format, or the newer XML format.

The alternative JAR files must be on the classpath at runtime. It does not matter what is on the classpath when you build against the StreamBase Client library, as long as you do not use classes from the ch.qos.logback package in your custom code, only classes from org.slf4j.

Further documentation on SLF4J is available at the SLF4J web site.

The StreamBase sample logging-log4j implements examples of all the above implementation methods for all four criteria, and runs equally well in Studio or at the Unix or Windows command prompt with sbd -f sbd.sbconf. The sample can be bundled and the resulting bundle can be run as an argument to the sbd command.

Additional Information About Criterion 1

If you use the sb-config command as part of a classpath-setting script, use it with the --no-logging-backend option when configuring for Log4J.

That is, you might normally use a command like the following to set the classpath for running StreamBase. The sb-config --classpath command returns the path to the sbclient.jar file:

export CLASSPATH=`sb-config --classpath`:$CLASSPATH

When using an alternative logging back end, use the following version of the command instead:

export CLASSPATH=`sb-config --no-logging-backend --classpath`:$CLASSPATH

The --no-logging-backend option must be specified before the --classpath option. This version of the command returns the path to the sbclient-no-logging-backend.jar file.

Advanced: Replacing Logback with JDK 1.4 Logging

To replace the Logback back end with JDK 1.4 logging, use the same criteria and implementation steps as described in the previous section for Log4J.

Note that there are two exceptions to that implementation:

  1. For Criterion 3, add the single JAR file slf4j-jdk14-1.5.0.jar to the classpath. Obtain this JAR file from the SLF4J distribution.

  2. For criterion 4, specify a different system property, java.util.logging.config.file, as follows:

    <java-vm>
    ...
      <sysproperty name="java.util.logging.config.file" value="logging.properties"/>
    ...
    </java-vm>

Advanced: Replacing the SLF4J Front End

If your site has standardized on a different version of SLF4J that conflicts with the version bundled in both sbclient.jar and sbclient-no-logging-backend.jar, follow these steps:

  1. Specify sbclient-no-logging.jar on your classpath. This version of the Client API JAR file includes no logging front end or back end at all.

  2. Arrange for the JARs that implement a compatible version of SLF4J, along with a properly configured logging back end, to appear on the classpath.