Contents
The Spotfire Streaming SBD-to-SBD Output (Upstream) Adapter runs within an upstream application, sending tuples to a downstream application. It is suitable for situations in which multiple data producers—upstream applications—wish to enqueue data to a single consumer. The producers can be stopped and started independently without affecting the consumer.
The adapter is configured with the host name and port of the downstream application as well as the name of the stream, or streams to send data to. The adapter is configured by default to reconnect after a restart of the downstream application.
This section describes the properties you can set for this adapter, using the various tabs of the Properties view in StreamBase Studio.
Name: Use this required field to specify or change the name of this instance of this component. The name must be unique within the current EventFlow module. The name can contain alphanumeric characters, underscores, and escaped special characters. Special characters can be escaped as described in Identifier Naming Rules. The first character must be alphabetic or an underscore.
Adapter: A read-only field that shows the formal name of the adapter.
Class name: Shows the fully qualified class name that implements the functionality of this adapter. If you need to reference this class name elsewhere in your application, you can right-click this field and select Copy from the context menu to place the full class name in the system clipboard.
Start options: This field provides a link to the Cluster Aware tab, where you configure the conditions under which this adapter starts.
Enable Error Output Port: Select this checkbox to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports to learn about Error Ports.
Description: Optionally, enter text to briefly describe the purpose and function of the component. In the EventFlow Editor canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.
The following table explains the adapter properties that you can set in the Adapter Settings tab of the Properties view for an SBD2SBDOutput icon in the EventFlow canvas.
| Property | Data Type | Description |
|---|---|---|
| Downstream host | string | The host name or IP address of the downstream application. |
| Downstream port | string | The TCP port number of the downstream application. |
| Downstream stream | string |
The name of the first stream to write to in the downstream application. If
no stream with this name is present in the downstream application, or the
stream's schema is incompatible with the schema configured in the adapter,
the upstream application behaves as though the downstream application is
inaccessible. The Reconnect Interval property
determines whether the upstream application attempts to reconnect
periodically until a compatible stream is present.
To specify the names of the second and subsequent streams to write to, enter the stream names in separate rows in the Additional Streams tab of the Properties view. |
| Downstream container | string |
The name of the container in the downstream application that holds the
Downstream stream designated in the previous
field. The default entry is default.
|
| Reconnect Interval (secs) | int |
The period, in seconds, the upstream application waits between reconnection
attempts after the downstream application fails or no compatible stream is
present. Set this property to zero to disable reconnection. The default
value is 10 seconds. When reconnection is disabled, the downstream
application must be started first and must have a compatible output stream
from which the upstream application can read.
NoteWhen specifying a reconnection interval as part of a container connection to a remote host, specify the interval value in milliseconds, not seconds. See Remote Container Connection Parameters. |
| Enqueue buffer size (tuples) | int | The numbers of tuples to enqueue before tuples are sent to the downstream application. For a low-latency application, set the buffer size to zero. |
| Max enqueue buffer size (tuples) | int |
The maximum size of the enqueue buffer before tuples are dropped. Enter
-1 to disable this setting.
|
| Enqueue flush interval (msecs) | integer | The flush interval of the enqueue buffer in milliseconds. For a low-latency application, set the flush interval to a small non-zero number. Do not set it to zero, which causes the adapter to perform a busy loop and consume excessive CPU cycles. |
| Use SSL | check box | Select this checkbox if the connection to the downstream host uses secure SSL-based communication to the downstream server. |
| Authentication type | drop-down list |
Specifies the type of authentication to use when connecting to the
downstream server. Options in the dropdown list are: No authentication, Basic
authentication, and Client certificate
authentication.
|
| User name | string | For secure connections, the login name on the downstream host to authenticate as. Only one of User name or Keystore pathname can be specified. |
| Keystore file pathname | string | The path on the local machine to the file containing keystore information used for Client certificate authentication. |
| Password | string | The password for the User name or the keystore passphrase. |
| Schema matching | two-option radio buttons |
Identical structure required means that, in
the schemas of the outgoing and incoming streams, each field must match in
data type and sequence, but the field names do not have to match. For
fields of type tuple, their fields must also match in type and sequence.
Match field names, the default setting, means that fields in the outgoing stream are matched by name against fields in the incoming stream. Fields whose names match must have the same data type in both streams. Any fields in the outgoing schema whose names do not match the incoming schema are not streamed. Any fields in the incoming schema whose names do not match anything in the incoming schema are set to null.
For example, consider an outgoing stream with schema |
| Capture Transform Strategy | radio button | The strategy to use when transforming capture fields for this operator: FLATTEN or NEST. |
| Explicit URI (Advanced) | string | This option is for advanced users who have a standard StreamBase URI they use in other parts of the StreamBase application. Use this option to paste in a full StreamBase URI to the downstream application, including security connection information. If this field is used, the following fields must be blank: Downstream host, Downstream port, Downstream stream, Downstream container, User name, Keystore file pathname, and password. |
| Enable event port | check box |
Select this checkbox to enable an output port that receives connection-up
and connection-down events. The schema of the event output port is as
follows:
|
| Connect on adapter init | check box | Selected by default. When selected, or no control port is enabled, the adapter attempts to connect to the other server during adapter start-up. When cleared, the adapter connects to the other server when a connect tuple is received on its control input port. |
| Enable control port | check box | Cleared by default. When selected, an additional input port is created, used to control and request status of the adapter. See below for more on this input port |
| Low Latency | check box | Cleared by default. When selected, the adapter configures itself to minimize data latency by disabling buffering, Nagle algorithms, and so on. |
| Log Level | INFO | Controls the level of verbosity the adapter uses to issue informational traces to the console. This setting is independent of the containing application's overall log level. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE. |
Low Latency, Enqueue buffer size and Enqueue flush interval are three properties that affect the latency and throughput of the Spotfire Streaming to Spotfire Streaming output adapter.
Enabling Low Latency configures the adapter in its lowest latency mode. Enqueue buffer size and Enqueue flush interval are ignored in this mode.
For maximum throughput, set the Enqueue buffer size property large enough to reduce the per-packet overhead to a minimum. Values of 10 to 1000 are common. The default is 100. Set the Enqueue flush interval property high enough so that, in general, the enqueue buffer size is reached, and the data is sent, before the flush interval expires. For high throughput applications, the flush interval is rarely hit.
For minimum latency, set the Enqueue buffer size property to 0. This forces each tuple to be sent immediately, in line. With a buffer size of zero, the Max enqueue buffer size and Enqueue flush interval properties are not used. This results in the lowest latency configuration, but comes at the cost of higher system overhead. Tuples are sent one per network packet, which increases network overhead and increases CPU usage on both the client and server. For low latency applications, see Java System Properties for more information on configuring clients and servers.
Setting the Enqueue buffer size property to 1 or higher causes tuples to be placed on a queue, and a network writer thread performs the actual network writes. The latency difference between an Enqueue buffer size of 0 and 1 is only the cost of adding the tuple to a queue and notifying the write thread. The disadvantage of an Enqueue buffer size of 0 is that if the downstream dequeuer is slow, the application must pause when network backup occurs, as the network write occurs in line.
Use this tab to specify additional downstream streams that the Spotfire Streaming to Spotfire Streaming Input adapter is configured to receive.
Use the settings in this tab to enable this operator or adapter for runtime start and stop conditions in a multi-node cluster. During initial development of the fragment that contains this operator or adapter, and for maximum compatibility with releases before 10.5.0, leave the Cluster start policy control in its default setting, Start with module.
Cluster awareness is an advanced topic that requires an understanding of StreamBase Runtime architecture features, including clusters, quorums, availability zones, and partitions. See Cluster Awareness Tab Settings on the Using Cluster Awareness page for instructions on configuring this tab.
Use the Concurrency tab to specify parallel regions for this instance of this component, or multiplicity options, or both. The Concurrency tab settings are described in Concurrency Options, and dispatch styles are described in Dispatch Styles.
Caution
Concurrency settings are not suitable for every application, and using these settings requires a thorough analysis of your application. For details, see Execution Order and Concurrency, which includes important guidelines for using the concurrency options.
The Spotfire Streaming to Spotfire Streaming output adapter has an optional event output port defined above.
For the default stream, and each additional configured stream, an input port is added to the adapter.
If you select the Enable control port option, an
input port is added to the adapter for use as a control port. The first field of
the control port schema must be a string field named command. This port is used to send the following commands to the
adapter:
-
connect—The adapter tries to connect to the configured downstream host. If configured to reconnect, it continues to try to connect at the configured reconnect interval until it succeeds. -
disconnect—The adapter disconnects from the downstream host. It does not try to reconnect, even if a reconnect interval is configured. -
status—The adapter emits a status tuple on the event port, if configured.
Typechecking fails if the Downstream Host or
Downstream Stream properties are empty or if the
Downstream Port property contains an illegal TCP port
number (unless the Explicit URI field is used instead).
The behavior in response to errors depends upon the setting of the Reconnect Interval property. When configured to reconnect, the upstream application polls periodically until the downstream application is accessible and contains an output stream with an identical name and schema. Otherwise, the upstream application emits an error message and makes no further attempts to connect to the downstream application.
The Spotfire Streaming to Spotfire Streaming output adapter throttles dropped-tuple warnings as follows:
-
The first dropped tuple generates an immediate warning.
-
Subsequent dropped tuples are counted, but not immediately reported.
-
When at least one tuple has been dropped, and at least 10 seconds have elapsed since the previous dropped-tuple warning, a new warning is generated the next time a tuple is processed, whether that tuple is dropped or not.
-
Dropped-tuple warnings indicate the number of tuples dropped and the number of seconds elapsed (normally 10) since the last warning, or in the case of the first warning, since the adapter was started.
-
No warnings are generated if no tuples are dropped since the last warning.
-
A warning can cover more than 10 seconds if the adapter drops a tuple and then becomes idle.
When suspended, the adapter stops sending tuples to the downstream application. Note that doing so may cause the application to drop the connection with the downstream application, resulting in lost tuples. Upon resuming, the adapter reconnects to the downstream application, if necessary and configured to do so, and begins sending tuples. If the adapter is configured not to reconnect with the downstream application and the connection is dropped during the suspension, the application will not enqueue tuples to the downstream application upon resume.
However, when the container enqueue (dequeue) control is set explicitly to ENABLED from any other state (DISABLED, DROP_TUPLES, or even ENABLED), the StreamBase to Spotfire Streaming adapter then starts (that is, enters the STARTED state), regardless of the adapter's setting.