Contents
The Spotfire Streaming Output Adapter for Apache Kafka Consumer Commit allows StreamBase applications to perform commit operations for a given consumer operator. The commit can be a commit of all topics and partitions or be a more granular approach and specify the topic, partition, and offset to commit.
The Apache Kafka adapter suite is implemented against the version of the Kafka libraries listed on the Supported Configurations page. Note however that this version of the Apache Kafka client API supports connections to older Apache Kafka brokers back to v2.8.
This section describes the properties you can set for this adapter, using the various tabs of the Properties view in StreamBase Studio.
Name: Use this required field to specify or change the name of this instance of this component. The name must be unique within the current EventFlow module. The name can contain alphanumeric characters, underscores, and escaped special characters. Special characters can be escaped as described in Identifier Naming Rules. The first character must be alphabetic or an underscore.
Adapter: A read-only field that shows the formal name of the adapter.
Class name: Shows the fully qualified class name that implements the functionality of this adapter. If you need to reference this class name elsewhere in your application, you can right-click this field and select Copy from the context menu to place the full class name in the system clipboard.
Start options: This field provides a link to the Cluster Aware tab, where you configure the conditions under which this adapter starts.
Enable Error Output Port: Select this checkbox to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports to learn about Error Ports.
Description: Optionally, enter text to briefly describe the purpose and function of the component. In the EventFlow Editor canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.
| Property | Type | Description |
|---|---|---|
| Linked Consumer | string | The name of the consumer adapter that this adapter will link to for performing the commit operation. |
| Linked Consumer Field | string | The field name of the input tuple that contains the fully qualified name of the consumer adapter that this adapter will link to for performing the commit operation. |
| Synchronous Commit | check box | If enabled the commit action will be done synchronously. |
| Enable Status Port | check box | When enabled, the adapter sends informational data on the status port about various adapter states. For more information see Status Port. |
| Log Level | INFO | Controls the level of verbosity the adapter uses to issue informational traces to the console. This setting is independent of the containing application's overall log level. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE. |
Use the settings in this tab to enable this operator or adapter for runtime start and stop conditions in a multi-node cluster. During initial development of the fragment that contains this operator or adapter, and for maximum compatibility with releases before 10.5.0, leave the Cluster start policy control in its default setting, Start with module.
Cluster awareness is an advanced topic that requires an understanding of StreamBase Runtime architecture features, including clusters, quorums, availability zones, and partitions. See Cluster Awareness Tab Settings on the Using Cluster Awareness page for instructions on configuring this tab.
Use the Concurrency tab to specify parallel regions for this instance of this component, or multiplicity options, or both. The Concurrency tab settings are described in Concurrency Options, and dispatch styles are described in Dispatch Styles.
Caution
Concurrency settings are not suitable for every application, and using these settings requires a thorough analysis of your application. For details, see Execution Order and Concurrency, which includes important guidelines for using the concurrency options.
Use the Commit input port to send commit operations to the adapter.
All of the commit fields are optional. If the topic is empty or null, or the partition is null or less than 0, or the offset is null or less than 0 then a default commit all at current offset operation will be done. All fields must be specific for a granular commit to be completed. All other fields passed into this adapter are ignored.
-
topic (optional), string. The topic to perform the commit on.
-
partition (optional), int. The partition to perform the commit on.
-
offset (optional), long. The offset to perform the commit on. The committed offset should always be the offset of the next message that your application will read. Thus, when calling commit you should add one to the offset of the last message processed.
-
Linked Consumer Field (optional), string. The fully qualified name of the consumer operator that this message is linked to. This field is optional.
You can optionally enable the status output port for this adapter instance by means of the Enable Status Port property in the Adapter Properties page of the Properties view. Use the status port to retrieve status output messages from the adapter.
-
type, string. The type of status information emitted. Status types are:
-
Error—This message relates to an error that occurred.
-
Warn—This message relates to a warning that the user should be aware of.
-
Info—This message relates to extra status information.
-
-
action, string. Valid values are:
-
Commit—This message is output for success or error of the commit operation.
-
-
object, string. This field will contain a string consisting of the topic-partition. If a synchronous commit is done without specifying a topic and partition, then the object field will contain an empty string.
-
message, string. This is a formatted human-readable message that explains the status message.
-
time, timestamp. The time the status message occurred.
-
inputTuple, tuple. The input tuple that was used to start the commit.