Bidirectional CSV Socket Reader Adapter

Introduction

The Spotfire Streaming Bidirectional CSV Socket Reader Adapter was created to allow for two-way socket communication using a pair of adapters, one for reading, and one for writing. The two adapters work together by sharing the connections that are made. As the name implies the reader allows for data to be read in from one or more sockets and then converted and sent downstream as a tuple.

Adapter Properties

This section describes the properties you can set for this adapter, using the various tabs of the Properties view in StreamBase Studio.

General Tab

Name: Use this required field to specify or change the name of this instance of this component. The name must be unique within the current EventFlow module. The name can contain alphanumeric characters, underscores, and escaped special characters. Special characters can be escaped as described in Identifier Naming Rules. The first character must be alphabetic or an underscore.

Adapter: A read-only field that shows the formal name of the adapter.

Class name: Shows the fully qualified class name that implements the functionality of this adapter. If you need to reference this class name elsewhere in your application, you can right-click this field and select Copy from the context menu to place the full class name in the system clipboard.

Start options: This field provides a link to the Cluster Aware tab, where you configure the conditions under which this adapter starts.

Enable Error Output Port: Select this checkbox to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports to learn about Error Ports.

Description: Optionally, enter text to briefly describe the purpose and function of the component. In the EventFlow Editor canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.

Adapter Properties Tab

Property Description
Port (int) The port to use. When in server mode, this is the port to listen on. When in client mode, this is the default port to use when making new connections.
Server Mode (check box) This option determines whether this adapter runs as a client (check box cleared) or as a server (check box selected).
Enable Control Port (check box) This option determines whether this control port is available. The control port allows for commands to be sent to the adapter to start or stop in server mode, or to connect or disconnect in client mode.
Enable Status (check box) This option enables a status port with status information about the state of the adapter during run time. Some common status tuples include connection and disconnection information.
Socket Identifier Field Name (string) This required field determines the name of the field added to all outgoing data tuples. Each tuple emitted on the data output port will contain this field with a socket identifier for the connection in the format host:port:socketidentifier, or if no socket identifier is specified for the connection, then just host:port.
Log Level Controls the level of verbosity the adapter uses to send notifications to the console. This setting can be higher than the containing application's log level. If set lower, the system log level is used. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE.

Server Tab

Property Description
Max Connection Count (int) The maximum number of incoming connections that this adapter allows at any given time. If a client tries to connect when the maximum number of connections is already reached, the connection is rejected and a status tuple is emitted.
Connection Backlog (int) The backlog parameter is the maximum number of pending connections on the socket. Its exact semantics are implementation specific. In particular, an implementation may impose a maximum length or may choose to ignore the parameter altogether. If the backlog parameter has the value 0, or a negative value, then an implementation specific default is used.
Start Server At Startup (check box) Determines whether the adapter starts the server when the application starts. If disabled, the end-user must enable the control port and emit a command tuple to start and stop the server.

Client Tab

Property Description
Default Host Name (string) The default host name to connect to. This value is used when the Connect at startup value is selected or when a command tuple is received that has null for the host value.
Connect at startup (check box) Determines whether the adapter connects to the default host and port when the adapter starts.
Reconnect Count (int) Determines how many times this adapter will retry to connect to a host.
Timeout Period (int) The number of seconds to wait to try to connect after failing to connect.

CSV Tab

Property Description
Byte Order (option) The byte order to use when converting data.
Buffer Size (int)

The size of the read or write buffer used to hold data in transition from the socket to the data transformer. If this value is too small the application will emit error messages because of a full buffer.

Use Default Charset (check box) Specifies whether the Java platform default character set is to be used.
Character Set (string) The name of the character set encoding that the adapter is to use to read input or write output.
Capture Transform Strategy (option) The strategy to use when transforming capture fields for this operator.
Field Delimiter (character) The delimiter used to separate tokens, defaults to a comma. Control characters can be entered as &#ddd; where ddd is the character's ASCII value.
String Quote Character (character) Matching pairs of the quote character used to delimit string constants.
Timestamp Format (String) The format used to parse timestamp fields extracted from the input tuples. This should be in the form expected by java.text.SimpleDateFormat class described in Java Platform SE reference documentation. The default format string is yyyy-MM-dd HH:mm:ss.SSSZ.

If a timestamp value is read that does not match the specified format string, the entire record is discarded and a WARN message appears on the console that includes the text invalid timestamp value.

Lenient Parsing (check box) Set this to true if you would like to parse timestamp values that do not conform to the specified format using default formats.
Incomplete Records (option)

Specifies what should be done when the adapter receives a record with less than the required number of fields.

Discard

Discard records with less than the required number of fields.

PopulateWithNulls

When records with less than the required number of fields are encountered, process the records after populating the unspecified fields with nulls.

Discard Empty Records (check box)

This is a special case to handle empty lines. If rows with some fields must send output, but not empty lines, leave this true. Set to false to send empty tuples for empty lines.

Log Warning (check box)

Set this to true if warning messages should be logged when incomplete records are encountered. If false, no warning messages will be logged for records with less than required number of fields.

Edit Schema Tab

  • This schema is used to define how the data is translated from byte information into a tuple.

  • Use the dropdown list to select the name of a named schema previously defined in or imported into this module. The drop-down list is empty unless you have defined or imported at least one named schema for the current module.

    When you select a named schema, its fields are loaded into the schema grid, overriding any schema fields already present. Once you import a named schema, the schema grid is dimmed and can no longer be edited. To restore the ability to edit the schema grid, re-select Private Schema from the dropdown list.

Cluster Aware Tab

Use the settings in this tab to enable this operator or adapter for runtime start and stop conditions in a multi-node cluster. During initial development of the fragment that contains this operator or adapter, and for maximum compatibility with releases before 10.5.0, leave the Cluster start policy control in its default setting, Start with module.

Cluster awareness is an advanced topic that requires an understanding of StreamBase Runtime architecture features, including clusters, quorums, availability zones, and partitions. See Cluster Awareness Tab Settings on the Using Cluster Awareness page for instructions on configuring this tab.

Concurrency Tab

Use the Concurrency tab to specify parallel regions for this instance of this component, or multiplicity options, or both. The Concurrency tab settings are described in Concurrency Options, and dispatch styles are described in Dispatch Styles.

Caution

Concurrency settings are not suitable for every application, and using these settings requires a thorough analysis of your application. For details, see Execution Order and Concurrency, which includes important guidelines for using the concurrency options.

Control Port

Description

Use the control port to send action commands to the adapter. The adapter can be run either in server mode (where it listens for connections) or client mode (where it makes outbound connections), and each mode has its own set of commands. Tuples enqueued on this port cause the adapter to start or stop a server, or to connect or disconnect clients.

Control Port Schema

  • Command, string, the command to send to the adapter.

    If the adapter is in server mode, the values are:

    • START -- This tells the adapter to start the server and listen on the specified or default host and port. Any new connection established has a host:port key value which outputs on each data tuple.

    • STOP -- This tells the adapter to stop the server and close all client connections.

    • DISCONNECT - This tells the adapter to disconnect a client from the server. The socketidentifer must be included and match a current connection.

    If the adapter is in client mode, the values are:

    • CONNECT -- This tells the adapter to make a new connection to the specified or default host and port. The associated values host:port:socketidentifer are used to identify this connection for further actions; those values are output as the socket identifier on all emitted data tuples.

    • DISCONNECT - This tells the adapter to disconnect from the specified or default host and port. The host:port:socketidentifer combination must match the values used to initially make the connection.

  • Host, string. The host to connect to. This value is only valid when sending commands to adapters in client mode. If the value is null, the default value of the adapter is used.

  • Port, int. The port to use when making connections. If the value is null the default value of the adapter is used. If the adapter is in server mode, the port value determines which port to listen on. If the adapter is in client mode, the port is used for outgoing connections.

  • SocketIdentifier (Optional), string. Only used when the adapter is in client mode. If specified, this value is added to the socket identifier value emitted on the status port as the object field with the format Host:Port:SocketIdentifier.

Status Port

Description

The status port is used to send status information tuples downstream to inform the user of changes.

Status Port Schema

  • type, string. The type of status information emitted on this port. Status types are:

    • Connection -- Indicates this message is about a connection.

    • Server -- Indicates this message is about the listening server.

  • action, string.

    • Connected -- This type of action occurs when a new connection is made either as a client or server. The socket identifier will be outputted as the object of this message. For a server the socket identifier will be host:port and for clients it will be 'host:port:(optional) inputted socket identifier'

    • Disconnected -- This type of action occurs when a connection disconnects either as a client or server.

    • Failed -- A failure occurred and the failure message will be included.

    • Limit -- Used when in server mode, this message will be emitted when the max number of connections to the server has been reached.

    • Stopped -- Used when in server mode, sent when the server has been stopped.

    • Started -- Used when in server mode, sent when the server has been started.

  • object, string. This value may be null. If it is not null, it contains a value relevant to the status message. On new connections, this value contains the socket identifier of the new connection.

  • message, string. This is a formatted human readable message that explains the status message.

  • inputTuple. Not used for this adapter.