This TIBCO StreamBase® Adapter for Informatica Ultra Messaging Publishing Output was formerly known as part of the 29 West LBM adapter suite.
This adapter allows a StreamBase application to publish LBM messages to one or more LBM topics. The adapter is embedded in the StreamBase application and has an input port that receives tuples to be published as LBM messages and an output port that emits status tuples. Each tuple enqueued to the adapter's input port causes a single LBM message to be published.
The adapter supports all StreamBase data types, including tuples and lists, though lists cannot contain elements of type list. Tuple fields of type tuple can be used to send nested LBM messages and LBM DECIMAL fields. The schema of the adapter's input port is determined by the output port of the upstream operator.
The adapter is configured through several properties set in the adapter's Properties view in StreamBase Studio. The properties include an optional configuration file to configure the LBM library, the input port field containing the topic to publish to, and the default topic value to use if the topic field is null.
The 29West LBM Publishing Output adapter can consume significant system memory, depending on the data cache options you enable in the LBM configuration file. Memory allocated by the adapter is native memory that is allocated independent of (and competing for memory resources with) the JVM memory consumed by StreamBase Server. See Java VM Memory Settings for more on JVM versus native memory usage.
This adapter requires access to the JAR file that implements the 29West LBM Java API on your system, and any files referenced by that JAR file. You will need to add your site's LBM implementation version of that file to your maven repository. The 29 west sample includes a pom.xml file with comments on how to maven install the required jar files for V6.0
mvn install:install-file -DgroupId=Informatica -DartifactId=UMS -Dversion=6.0.0 -Dpackaging=jar -Dfile=path/to/Informatica/UMQ_6.0/UM_Java_API_6_0/UMS_6.0_jdk1.5.0_12.jar mvn install:install-file -DgroupId=Informatica -DartifactId=UMSPDM -Dversion=6.0.0 -Dpackaging=jar -Dfile=path/to/Informatica/UMQ_6.0/UM_Java_API_6_0/UMSPDM_6.0_jdk1.5.0_12.jar mvn install:install-file -DgroupId=Informatica -DartifactId=UMSSDM -Dversion=6.0.0 -Dpackaging=jar -Dfile=path/to/Informatica/UMQ_6.0/UM_Java_API_6_0/UMSSDM_6.0_jdk1.5.0_12.jar
Before launching any of the sample application you will also need to setup the launch config to include the
Path which includes the location of the Informatica install location. Also a
LBM_LICENSE_FILENAME variable should be defined with the path to the license file. *** Please note that these variables are ONLY SET AT INSTALL
OF A NODE, if you are using an existing node these will not take effect.
Informatica purchased 29West and has since released an update of the LBM API, renamed Ultra Messaging Streaming Edition, or UMS. This adapter works with the UMS 6.x JAR files, but you cannot mix LBM and UMS installations. You must thoroughly uninstall any LBM 4.x kit before reinstalling a UMS 6.x kit. Since UMS is a workalike superset of LBM, this page uses "LBM" to refer to both.
If you get an error message whose text refers to
can't load lbmj: java.lang.UnsatisfiedLinkError: no lbmj in java.library.path, make sure the directory containing the LBM native libraries is on the path.
The LBM API implementation described in this section is a product of a third party, and its specifications and file names are subject to change by 29West. See your LBM documentation for the latest information.
|LBM Configuration File||
The name of an optional LBM configuration file used to override the default settings used by the LBM library. Configuration
files can be shared between one or more LBM subscribing and publishing adapter instances. To support such sharing, configuration
files can have named sections, which are referenced by the
Lines that precede the first section name are considered part of the unnamed (default) section.
Lines that follow a section name are considered part of that section.
Section names must be unique within and across all referenced configuration files; a runtime error is flagged when a duplicate section name is encountered. Section names can include alphanumeric characters, digits, underscores, dots, and embedded spaces. They must start with an upper- or lower-case letter or an underscore.
|LBM Configuration File Section||The name of the LBM configuration file section from which to retrieve the parameters for configuring the LBMContext and LBMSource objects. If unspecified, parameters from the unnamed section are used. For enhanced performance, all LBM adapter instances that specify the same configuration file and section name share an LBMContext object.|
|Default Topic||Contains the topic value under which to publish a LBM message if no topic field exists in the input schema, or its value is null in a specific input tuple.|
|Topic Field Name||The name of the input port field containing the topic value under which to publish the LBM message. This field is optional if a Default Topic value is specified, which is used if no topic field exists, or its value is null in a specific input tuple.|
|Client Token Field Name||The name of the input port field that can be used to send an arbitrary value that is returned in various publish status tuples.|
|Idle LBM Source Timeout||The time, in seconds, to wait before an LBM source for an idle topic is discarded, or zero (the default) to disable the expiration of idle sources. An LBM source is created on demand when publishing to each new topic. When publishing to an unbounded number of topics, this property should be set to a non-zero value to limit the total number of LBM sources — each of which consumes finite resources — maintained by the adapter.|
|Capture Transform Strategy||The strategy to use when transforming capture fields for this operator: FLATTEN or NEST.|
|Log Level||Controls the level of verbosity the adapter uses to send notifications to the console. This setting can be higher than the containing application's log level. If set lower, the system log level is used. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE.|
|Enable Pre-defined Messages||
If disabled (the default), the LBM publish adapter sends self-describing messages, in which metadata describing the structure of the message is sent in each messsage. The LBM subscribe adapter uses this metadata to parse incoming messages and populate tuples.
If enabled, the LBM publish adapter sends pre-defined messages, in which the structure of the message is derived from the schema of the adapter's publish input port. The LBM subscribe adapter uses the schema of its message output port to parse incoming messages and populate tuples.
|Schema Matching Policy||
|Definition ID||The message definition ID of the top-level schema used when exchanging pre-defined messages in BestVersion mode.|
|Major Version||The major version of the message definition when using BestVersion schema matching.|
|Minor Version||The minor version of the message definition when using BestVersion schema matching.|
|Nested Message Definition IDs||The name and message definition ID of each tuple field in the output schema used when exchanging pre-defined messages in BestVersion mode.|
|Publish Type Overrides||A table used to override the default LBM type used to publish specific fields when using self-describing messages. Each table
row contains a StreamBase field name and the LBM type used to publish that field. StreamBase fields not present in the override
table are published using the default LBM type specified in the table below.
See the Publish Type Overrides Table below for the correspondence between StreamBase types, default LBM types, and valid overrides.
|Use Fully-Qualified StreamBase Field Names||When enabled, nested StreamBase fields are specified in the Publish Type Overrides table using fully-qualified names. Otherwise, simple field names relative to the enclosing tuple are used. Use fully-qualified names when a top-level and nested field with the same relative name need to be published using different LBM types.|
|Enable Persistence||If enabled (not the default), the adapter registers with a UME store and recovers from its previous position in the message stream after a failure.|
|UME Session ID||Session ID used by the Ultra Messaging library to manage recovery.|
|UME Store Name||Name of UME store as defined in the context-name option of the umestored's XML configuration file.|
|Sequence Number Retrieval Ports||If enabled (not the default), the adapter creates an additional input and output port to retrieve the retrieve the current sequence number for a topic.|
|UME Store Notifications||If enabled (not the default), the adapter emits status tuples in response to UME store registation events.|
|Reciver Connection Notifications||If enabled (not the default), the adapter emits status tuples when receives connect and disconnect to topics.|
|Message Sequence Number Information||If enabled (not the default), the adapter emits status tuples to convey the sequence numbers with which messages are sent.|
|Message Delivery Confirmations||If enabled (not the default), the adapter emits status tuples when messages are delivered to receivers.|
|Message Stability Notifications||If enabled (not the default), the adapter emits status tuples when messages become durable in the UME sture.|
|Message Reclaimed Notifications||If enabled (not the default), the adapter emits status tuples when messages are reclaimed.|
|Flight Size Notifications||If enabled (not the default), the adapter emits status tuples when the number of in-flight messages exeeds or falls below the configured threshold.|
The following table shows, for each StreamBase type, the default LBM type, and the valid override values that can be used to publish the StreamBase field using a non-default LBM type.
Publish Type Overrides Table
|StreamBase Type||Default LBM Type||Valid Overrides|
|int||INT32||INT8, INT16, UINT8, UINT16|
|list<double>||DOUBLE ARRAY||FLOAT ARRAY|
|list<int>||INT32 ARRAY||INT8 ARRAY, INT16 ARRAY, UINT8 ARRAY, UINT16 ARRAY|
|list<long>||INT64 ARRAY||UINT32 ARRAY, UINT64 ARRAY|
|list<string>||STRING ARRAY||UNICODE ARRAY|
|list<tuple>||MESSAGE ARRAY||DECIMAL ARRAY|
This section discusses how to use the 29West LBM Publishing Output Adapter in a StreamBase application. As shown in the diagram below depicting one of the adapter's sample applications, the publishing adapter has one primary input and one primary output port to communicate with the surrounding application. When configured for persistence, the publishing adapter provides an additional input port and output port to retrieve a topic's current message sequence number.
The 29West LBM Publishing Output Adapter's ports are used as follows:
Publish: Tuples enqueued on this input port cause the adapter to publish LBM messages, one per tuple. The port's schema is derived from the output port of the upstream operator and can contain all StreamBase data types, including lists and nested tuples to provide hierarchy in LBM messages. Tuple fields are mapped to LBM message fields by name. List fields can contain all StreamBase element types, except lists. Fields containing null in the tuple are discarded; no corresponding field is placed in the LBM message.
DECIMAL fields are published using a tuple field of type tuple containing two sub-fields: a long for the mantissa and an int for the exponent. The two sub-fields may appear in either order in the nested tuple field. If a field to be published as a DECIMAL contains null values in a sub-field, a warning is emitted, the tuple field is discarded, no DECIMAL field is placed in the LBM message. If both sub-fields are null, the tuple field is discarded silently.
ReadSequenceNumber: This optional port is used to retrieve a topic's current message sequence number and is enabled by setting the
Sequence Number Retrieval Portsproperty on the
Persistencetab. Tuples enqueued on this input port cause the adapter to publish LBM messages, one per tuple. Each tuple enqueued to this port results in a tuple emitted on the
SequenceNumberOutport. The ReadSequenceNumber port has the following schema:
Topic, string: Contains the name of the topic for which to retrieve the current sequence number.
ClientToken, string: An arbitrary value that is echoed in the response, allowing the StreamBase application to associate requests with responses.
PublishStatus: This output port emits status, information, and error tuples. The PublishStatus port has the following schema:
type, string: Contains one of the following values describing the type of event that occurred:
Process LBM Message
object, string: the name of the object associated with the event, such as the name of the tuple field that couldn't be converted to an LBM field or the name of the adapter that was suspended or resumed.
action, string: the action associated with the event, such as a conversion error or adapter suspended or resumed.
message, string: A human-readable message string.
SequenceNumberOut: This output port emits tuples containing the current sequence number for a topic in response to request tuples enqueued to the
ReadSequenceNumberport. The SequenceNumberOut port has the following schema:
Topic, string: Contains the name of the topic for which the current sequence number has been retrieved.
ClientToken, string: The value placed in the corresponding field of the ReadSequenceNumber request tuple.
SequenceNumber, long: The current sequence number for the topic, or
nullif no sequence number is available.
Add an instance of the adapter to a new StreamBase application as follows:
In StreamBase Studio, create a project, and create an EventFlow application file to host the adapter.
Drag the Output Adapter icon from the Operators & Adapters drawer in the Palette view to the canvas. Select the 29West LBM Publishing Output Adapter from the list and click.
Connect an input and output stream to the adapter's input and output ports, respectively. Configure the schema of input stream with the fields required in the LBM messages to be published. Include a Topic field of type string if the default value for the Topic Field Name property is to be used.
Select or double-click the adapter icon, and in the Properties view, click Adapter Settings and fill in any desired properties. Either a Topic Field Name or Default Topic must be specified.
The 29West LBM Publishing Output adapter uses typecheck messages to help you configure the adapter in your StreamBase application. In particular, the adapter generates typecheck messages when:
The schema of the input port contains one or more list fields with elements of type list.
An invalid LBM configuration file is specified.
An invalid topic field name within the input port schema is specified.
The topic name within the input port schema is not of type string.
Neither a topic field not a default topic is specified.
An invalid client token field name within the input port schema is specified.
The client token name within the input port schema is not of type string.
Persistence is enabled and no UME sesion ID is specified.
Persistence is enabled and an invalid UME sesion ID is specified.
Persistence is enabled and no UME store name is specified.
The datatypes of one or more fields in the input schema are incompatible with the specified LBM override types.
The adapter generates messages on the status port during runtime under various conditions, including:
The topic field of a publish tuple is null and no default topic has been configured.
An error occurs populating an LBM field from a tuple field.
An error is returned by the LBM library when the adapter attempts to publish an LBM message.
The adapter is suspended or resumed.
When suspended, the 29West LBM Publishing Output Adapter stops processing input tuples and emitting output tuples.
When resumed, the adapter begins processing input tuples and emitting output tuples once again.