Log Output Adapter

Overview

The Log output adapter is an embedded adapter that sends its input tuples to both its output port and to the currently configured StreamBase Java logging system, which is described in Using StreamBase Logging. Each input tuple that enters the Log adapter is sent to the logging system, and then passes unchanged to the adapter's output port.

The Log adapter participates automatically in the logging system currently in use by its containing StreamBase Server. This means that by default, the Log adapter uses:

  • Logback logging when run in Studio or when run from the command line with sbd and no -b switch.

  • Log4j logging when run as a Windows service or when run from the command line with sbd -b.

If you have configured StreamBase Server to use an alternative logging system (such as using Log4j under all circumstances), the Log adapter still participates in the configured logging system. You can configure logging for the Log adapter as appropriate for the logging system in use. See Using StreamBase Logging for more on the StreamBase logging system.

The Log Asynchronously Option

The Log Output adapter itself adds negligible overhead to each tuple when the configured log level is such that no logging actually occurs. When the log level is such that tuples are actually sent into the logging system, the overhead can become significant, depending on the type of appenders configured. Even logging appenders that are said to be asynchronous can introduce surprising delays.

To avoid all overhead delays introduced by logging system appenders, select the Log Asynchronously check box. With this option enabled, each incoming tuple is sent immediately to the adapter's output port, and a copy of the tuple is sent to a queue. The queue is managed by a separate thread that immediately retrieves the tuple from the queue and sends it to the currently configured logging system. Because each tuple's retrieval from the queue is immediate, this option's actions do not constitute buffering, just separately-threaded logging. When you enable the Log Asynchronously option, the profiling Size attribute for the Log adapter reports the size of this queue.

There can be unintended consequences of enabling the Log Asynchronously option:

  • In a large application with multiple instances of the Log Output adapter, if all Log adapter instances are configured to be synchronous, or all Log adapter instances are configured to be asynchronous (the default state), then logged events are written to log appender locations in the downstream order of the various Log adapter instances, as expected.

  • However, if you mix synchronous and asynchronous Log adapters in the same execution region, you are no longer guaranteed that all Log adapters write their log entries in sequence.

There is a Java system property, streambase.log-adapter-async, that can be used to set the synchronous state of all Log adapters in an application at the same time, without having to edit the Properties view for all Log adapters in the application. Set streambase.log-adapter-async=true to force all Log adapters to enable the Log Asynchronously option. Set the property =false to restore the default synchronous state of all Log adapters.

Formatting the Log Message

The Log adapter provides several ways to format the message portion of log entries emitted from the adapter.

Note

The overall format of emitted log messages is still controlled by the logging system in use and by its configuration file. Logback logging is configured with a file usually named logback.xml, while Log4j logging is configured with log4j.configuration or log4j.xml. In both cases, a PatternLayout string defines the format and structure of all log messages emitted from StreamBase.

The Log adapter provides ways to format only the %m portion of the log message.

The Logger Name property must be specified, and corresponds to the %c portion of the log message pattern string. Use different Logger names for Log adapters in different portions of your application.

By default, the Log adapter emits the fields of its input tuple preceded by the name of the Logger that you assign. For example, a Log adapter instance might be configured with the Logger1 Logger Name but no other formatting options. With the input tuple {IBM, 164.35, 1000}, the default-configured adapter emits the following message:

Logger1 - IBM,164.35,1000.0

You can optionally precede the emitted tuple with a string prefix that helps further identify log messages emitted from this instance of the adapter. For example:

Logger1 - Incoming tuple: IBM,164.35,1000.0

You can optionally precede each field value with its corresponding field name (with or without the message prefix). In this case, the listing of input fields is preceded by an internal counting field, tupleid. For example:

Logger1 - Incoming tuple: (tupleid=0,symbol="IBM",price=164.35,numshares=1000.0)

Lastly, instead of emitting the input tuple's fields in order, you can provide a string pattern in the format defined by java.text.MessageFormat, with zero-based positional parameters in the form {0}, {1}, and so on, filled in with input field values in input schema order. The message pattern option allows you to construct log message text in phrase or sentence form, with some or all of the log message text provided by the value of fields in the input tuple. For the same input tuple as above, with the message format string {2, number, integer} shares of {0} traded at {1, number, currency}., the adapter emits the following:

Logger1 - 1,000 shares of IBM traded at $164.35.

Note

Do not confuse the java.text.MessageFormat string you provide to the Log adapter with the PatternLayout string you provide in Logback and Log4J configuration files. The two message pattern standards are not the same and are not interchangeable.

Ports

An instance of the Log adapter has a single input port and a single output port. This is a pass-through adapter, so the schema of the output port is always the same as the schema of the input port.

Adapter Properties

General Tab

Name: Use this field to specify or change the component's name, which must be unique in the application. The name must contain only alphabetic characters, numbers, and underscores, and no hyphens or other special characters. The first character must be alphabetic or an underscore.

Adapter Name: A read-only field that shows the formal name of the adapter.

Class: A field that shows the fully qualified class name that implements the functionality of this adapter. Use this class name when loading the adapter in StreamSQL programs with the APPLY JAVA statement. You can right-click this field and select Copy from the context menu to place the full class name in the system clipboard.

Start with application: If this check box is selected, an instance of this adapter starts as part of the containing StreamBase Server. If cleared, the adapter is loaded with the server, but does not start until you send an sbadmin resume command, or start the component with StreamBase Manager. With this option cleared, the adapter does not start even if the application as a whole is suspended and later resumed. The default and recommended setting is selected.

Enable Error Output Port: Select this check box to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports and Error Streams to learn about Error Ports.

Description: Optionally enter text to briefly describe the component's purpose and function. In the EventFlow canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.

Adapter Properties Tab

The Log adapter has the following properties on the Adapter Settings tab in its Properties View in StreamBase Studio:

Property Data Type Description
Log level drop-down list Specify the logging message type you want to send to the logging system from this adapter. The choices are to send INFO, WARN, DEBUG, or TRACE log messages.
Display Tuple Field Names check box Enable to display tuple field names in logged messages.
Logger Name string Specify the name to use in creating the logger object for this adapter instance. The logger name appears in each logged message if the %c pattern is specified in the message pattern string in the <layout> element of the Logback or Log4j configuration file. (StreamBase's default Logback message configuration includes %c.)
Log Asynchronously check box Cleared by default, which means each incoming tuple passes through each configured appender for the current logging system, and then passes to the adapter's output port.

When selected, each incoming tuple is sent immediately to the output port, and a copy of the tuple is sent to a queue managed by a separate thread that immediately retrieves the tuple from the queue and sends it to the currently configured logging system. See The Log Asynchronously Option above for cautions and considerations when using this option.

Message Prefix string The specified prefix string is prepended to the log message text for each tuple emitted from this Log adapter instance. You can use this feature with several instances of the Log adapter to prefix different strings to the log messages emitted from different portions of your application to help identify the source of messages.
Custom MessageFormat string

This field cannot be used if you enabled the Display Tuple Field Names option, or used the Message Prefix option. If neither of those options are enabled, use Custom MessageFormat to specify a string that defines a pattern in the format defined in java.text.MessageFormat, where the ArgumentIndex is used as the zero-based index of the fields in the operator's incoming tuple. The specified pattern defines the format of output log messages in any arbitrary form, using MessageFormat's patterns.

For example, if the adapter's incoming tuple consists of fields symbol(string), price(double), and numshares(double) in that order, you could configure a log message with the following MessageFormat string:

{2, number, integer} shares of {0} traded at {1, number, currency}.

Concurrency Tab

Use the Concurrency tab to specify separate threading for this instance of this component, or multiplicity options, or both. The Concurrency tab settings are described in Concurrency Options, and dispatch styles are described in Dispatch Styles.

Caution

Concurrency settings are not suitable for every application, and using these settings requires a thorough analysis of your application. For details, see Execution Order and Concurrency, which includes important guidelines for using the concurrency options.

Typechecking and Error Handling

Typechecking detects a value for the output that is null or unspecified.