LiveView Query Input Adapter

Introduction

The Spotfire LiveView Query input adapter allows a StreamBase application to register ad hoc queries with LiveView servers, and receive snapshot and real time results. When used for real time results, the results are generated whenever a change to the table occurs which meets the requirements of the query. For snapshot queries, the result is a snapshot of the current state and no further results will be generated and the query is automatically unregistered.

The result will always have the name of the query included. For this reason, query names should be carefully chosen and must be unique among all the queries managed by a single adapter.

When results are no longer desired, the query should be unregistered.

If the table name and schema are known at design time, the table name and schema can be set in the adapter at that time. In this case, all queries will be done against that table and the results will exit from one of the output ports, depending on whether it is a regular or dynamic aggregation query. For dynamic aggregations queries, the results are output in a JSON format while other queries are output using the supplied schema. If the name and/or schema are not known at design time, the table name may be passed in via the optional TableName field of the input schema. Queries to tables named in the TableName field will always be sent out the JSON output port.

The operation of the adapter changes based on the existence of the TableName field in the input at design time. If the field is not present, the Target table name property must have a value that resolves to a valid table name and the Initial Queries and table Schema (schema0) are available to be set. In this case, the table schema is required.

At design time, if the TableName field is present, the Target table name property becomes optional. When the Target table name property has a value, the Initial Queries and table schema are both available to set and the table schema (schema0)is required. If the Target table name property is empty, those additional properties are disabled. If the TableName field is not present or there is not a Query Input Port, the Target table name property is required as is a valid table Schema (schema0).

At runtime, if the Query Input Port has a valid TableName, then the query will be done on that table regardless of the contents of Target table name and the output will be on the JSONQueryPort. If there is no Query Input Port, Query Input Port does not contain a TableName or the TableName is empty, the Target table name must contain a valid table name and all queries will be done against that table and the output will go to the appropriate output port. It is a non-fatal error if both the TableName field and the Target table name property are empty.

So if you know the table name and schema at design time and want tuple results, fill in the Target table name property, the Initial Queries and the schema0.

If you do not know these, wish to have one adapter register queries on multiple tables or have JSON output, leave the Target table name property empty, enable the Query Input Port and ensure that its schema contains the TableName field. Then at runtime send the queries to the requested tables using this port.

Properties

This section describes the properties you can set for this adapter, using the various tabs of the Properties view for the adapter's icon in StreamBase Studio.

General Tab

Name: Use this required field to specify or change the name of this instance of this component. The name must be unique within the current EventFlow module. The name can contain alphanumeric characters, underscores, and escaped special characters. Special characters can be escaped as described in Identifier Naming Rules. The first character must be alphabetic or an underscore.

Adapter: A read-only field that shows the formal name of the adapter.

Class name: Shows the fully qualified class name that implements the functionality of this adapter. If you need to reference this class name elsewhere in your application, you can right-click this field and select Copy from the context menu to place the full class name in the system clipboard.

Start options: This field provides a link to the Cluster Aware tab, where you configure the conditions under which this adapter starts.

Enable Error Output Port: Select this checkbox to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports to learn about Error Ports.

Description: Optionally, enter text to briefly describe the purpose and function of the component. In the EventFlow Editor canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.

Adapter Properties Tab

This section describes the properties on the Adapter Properties tab in the Properties view for the LiveView Query adapter.

Property Description
Set Server URI to encapsulating LiveView Select this checkbox when the adapter is part of a LiveView project. In this case, the adapter connects to the port of the running LiveView server. When authentication is enabled, the credentials used to connect to the LiveView server are retrieved from the liveview.internal.username and liveview.internal.password system properties. The corresponding LiveView user must be configured with the appropriate permissions to allow this adapter to perform its operations.
LiveView Server URI A LiveView server URI, or an expression or variable that resolves to a LiveView server URI. For example, the URI for the local machine and default port is lv://localhost:11080.
Use Runtime URIs Enables an additional input port to this adapter. The input port expects tuples of the schema (ControlAction: string, URI: string). The author time URI options are greyed out and the adapter will wait to connect until it receives a Connect tuple. If the connection fails, the adapters will continue to try and connect. If the adapter receives a Disconnect tuple, it will disconnect from its current connection, if it has one, or stop trying to connect if it’s currently in failure.
Connection Key Only enabled if Use Runtime URIs and Share LiveView Connection are enabled. If both are enabled, this is a required property which defines groups of adapters that will share connections. An author-defined string of alphanumeric characters is expected here like Pool1 or Pool2. All adapters set to share connections and use runtime URIs with the Connection Key Pool1 will connect and disconnect using a single shared connection as Connect/Disconnect tuples are received by any one of the adapters in the group.
Target Table Name The name of the LiveView table to issue queries against. This is a required field unless the Query Input Port contains a TableName field of type String.
Include old row value Include both new and old row values in updates. This feature uses more client memory to hold the old values.
Query Input Port When selected, this instance of the adapter has an input port that allows you to send ad hoc queries at runtime to be registered or unregistered. When this checkbox is cleared, this adapter instance has no input port, and therefore only runs the queries specified in the Initial Queries tab.
Share LiveView Connection When selected (the default), this instance of the adapter shares a single LiveView connection to the configured LiveView server with all other LiveView adapters in the same container that also have the Share LiveView Connection property selected.
Enable Compressed Connection If checked, compress this adapter's LiveView connection. Not enabled if Use Runtime URI is also selected. To compress a connection defined at runtime, see the "ConnectCompress" control action defined in the Runtime URI Schema
Log Level Controls the level of verbosity the adapter uses to send notifications to the console. This setting can be higher than the containing application's log level. If set lower, the system log level is used. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE.

Initial Queries Tab

This tab allows you to configure one or more ad hoc queries that are to be registered each time the adapter connects to the specified LiveView server. If no Query Input Port is specified for this adapter instance, then the queries listed in this tab are the only ones this adapter instance runs. Note that these are only applied to a table named in the Target table name property.

Each query line in this tab's grid is automatically assigned an ad hoc query name of the form InitialQuerynn, where nn is an integer starting with zero. The query in the first line is assigned InitialQuery0, the second line gets InitialQuery1, and so on. Use these query names to identify individual queries in the adapter's output streams. The query name also allows you to identify a query to be unregistered by means of a tuple sent to the adapter's input port.

Note

The auto-generated names for queries are numbered separately for each adapter instance, and start over with InitialQuery0 for each instance. If you have two or more instances of the LiveView Query adapter in the same EventFlow module, then this module has more than one InitialQuery0, InitialQuery1, and so on.

The tab's grid has two columns, each representing part of a LiveView query string. The Select column specifies the projection expression for the query. For simple, non-aggregate queries, this column typically contains a single asterisk, which specifies selecting all fields in the table. For dynamic aggregation queries, the expression in this column defines the aggregate functions and values to use for this query. For example, a Select column might contain: avg(lastSoldPrice) as AvgSoldPrice, category 

The Query column specifies the predicate expression for each query. This predicate can include a time window specified with between and, and can include order by, group by, and limit where appropriate.

Edit Schema Tab

Use the Edit Schema tab to specify the schema of the LiveView table specified in the Target Table Name field of Adapter Properties tab. This is the table that this adapter instance is to issue queries against. The schema defined here is not used for queries that come via the TableName field of the Query Input Port or that are dynamic aggregation queries, those use a generic JSON format for their output.

For general instructions on using the Edit Schema tab, see Defining Input Streams.

Cluster Aware Tab

Use the settings in this tab to enable this operator or adapter for runtime start and stop conditions in a multi-node cluster. During initial development of the fragment that contains this operator or adapter, and for maximum compatibility with releases before 10.5.0, leave the Cluster start policy control in its default setting, Start with module.

Cluster awareness is an advanced topic that requires an understanding of StreamBase Runtime architecture features, including clusters, quorums, availability zones, and partitions. See Cluster Awareness Tab Settings on the Using Cluster Awareness page for instructions on configuring this tab.

Concurrency Tab

Use the Concurrency tab to specify parallel regions for this instance of this component, or multiplicity options, or both. The Concurrency tab settings are described in Concurrency Options, and dispatch styles are described in Dispatch Styles.

Caution

Concurrency settings are not suitable for every application, and using these settings requires a thorough analysis of your application. For details, see Execution Order and Concurrency, which includes important guidelines for using the concurrency options.

Adapter Ports

The LiveView Query adapter provides up to two optional input ports, and up to three output ports:

  • The optional QueryIn input port can be used to register and unregister ad hoc queries at runtime.

  • The optional Use Runtime URI input port can be used to connect and disconnect to different LiveView servers at runtime.

  • The QueryStatus output port emits tuples that provide information about each query, and the status of each query's connection to the LiveView server.

  • The QueryOut output port reports the results of simple queries. If the Target table name property is empty, this port is disabled.

  • The JSONQueryOut output port reports the results of dynamic aggregation queries. This port is also used for queries that are made on a table named in the TableName field.

As with other Spotfire Streaming adapters, you can optionally enable an Error Output Port, as described in Using Error Ports and Error Streams.

LiveView Query Adapter Input Port

The optional QueryIn port can be used to register new ad hoc queries sent to the LiveView server at runtime, and to delete (unregister) queries. This port is added by selecting the Query Input Port check box on the Adapter Properties tab.

The port's schema is shown in the following table. When sending a new query to be registered, use all five fields. When sending a tuple to unregister an existing query, use only the first and last fields.

Field Data Type Description
QueryName string When sending a tuple that registers an ad hoc query, use this field to specify an arbitrary string name to identify this query. The name must be unique for all currently registered queries. When sending a tuple that unregisters an existing query, use this field to specify the target query name specified when the query was registered.
Select string The select, or projection, expression for this query. For simple queries this is a single asterisk; for dynamic aggregation queries, this is the set of aggregate functions to use and their associated field names.
Query string The predicate for the query. You can include order by, group by, limit, between and, and so on.
SnapshotOnly boolean If true, make this query a snapshot query. If false, or null, make this query a snapshot plus continuous query. SnapshotOnly queries do not have to be unregistered.
Unregister boolean Unregister the query that matches the provided QueryName. This deletes the query from running on the LiveView server. If this field is true, only the QueryName and TableName fields will be used.
TableName string When this optional field if present, if the field contains a valid table name, the current operation will be done on this table. Any query results generated by a query of this type will be presented on the JSONQueryOut port.

The adapter can import a proposed schema for the input port that you can use as a starting point.

To import an input schema, click the Import proposed schemas hyperlink in the top right corner above the Adapter Properties tab. From the dialog, click Browse to select an existing destination interface file, or click New File to save a Named schema file to a project folder.

Runtime URI Input Port

When Use Runtime URIs is enabled on this adapter, so is this port.

This port accepts tuples of the below schema.

Field Data Type Description
ControlAction string A value of either
Connect ConnectCompress Disconnect
If Connect or ConnectCompress, the adapter (and any adapters with whom it shares a connection) will connect to the LiveView server specified in the URI field. If currently connected (or attempting to connect) to a URL, that will be closed and the new URL attempted. If Disconnect, this adapter or adapter group will disconnect from its currently connected LiveView server. ConnectCompress will initiate a compressed connection, and Connect will initiate an uncompressed connection.
URI string The URI of a LiveView server such as lv://localhost:11080. The adapter will attempt to connect to the LiveView server at this address if Connect or ConnectCompress is supplied in this tuple's ControlAction. If Disconnect is supplied as the ControlAction, a value for URI need not be supplied and will be ignored.

LiveView Query Adapter Output Ports

The LiveView Query adapter has one output status port and up to two query output ports.

The status port has the fields shown in the following table:

Field Data Type Description
Type string The basic state reported, always one of the following:
CONNECTION ADMIN
QUERY ERROR
Object string The name on the EventFlow canvas of the adapter reporting the status.
Action string The action that occurred, one of the following:
CONNECTED FAILED
REGISTER_QUERY ERROR
UNREGISTER_QUERY  
Message string A human-readable string that provides additional context for the status message.
Time timestamp The time the action occurred.

The LiveView Query adapter has a QueryOut output port, on which tuples are emitted that describe the results of all registered non-aggregate queries. The schema of this output has six fields, two of which have the tuple data type. The schemas for the two tuple fields are configured in the Edit Schema tab.

Field Data Type Description
QueryName string The QueryID of the query whose results are reported in this tuple.
CQSInternalID long An internally-generated ID for the affected row.
CQSScopeTransition int One of three values to report the event that modified the affected row:
0 - row added 2 - row updated
1 - row deleted  
Snapshot boolean This tuple is part of the snapshot. In this case, CQSScopeTransition is 0.
New tuple The new value for the row. The schema of this tuple must match the LiveView table's schema, which must be specified in the Edit Schema tab.
Old tuple The old value for the row. The schema of this tuple must match the LiveView table's schema, which be specified in the Edit Schema tab. This tuple's value is null unless the Include old row value checkbox is selected in the Adapter Properties tab.

The LiveView Query adapter has a separate output port, JSONQueryOut, to report the results from queries for which the result schema is not known. This includes dynamic aggregation queries and queries that result from using the TableName input. In such queries, the result itself defines the schema. In response, the variable output schema from such queries are represented in a JSON string. You can use StreamBase expression language functions, such as parsejson() to extract the tuple values from these JSON strings.

Field Data Type Description
QueryName string The QueryID of the query whose results are reported in this tuple.
CQSInternalID long An internally-generated ID for the affected row.
CQSScopeTransition int One of three values to report the event that modified the affected row:
0 - row added 2 - row updated
1 - row deleted  
Snapshot boolean This tuple is part of the snapshot. In this case, CQSScopeTransition is 0.
NewJSON string The JSON string representation of the new value for the row.
OldJSON string The JSON string representation of the old value for the row. This row's value is null unless the Include old row value checkbox is selected in the Adapter Properties tab.

Using the Adapter in a StreamBase Application

Adding the Adapter to an EventFlow Application

Add an instance of the adapter to a new EventFlow application with the following steps:

  1. In StreamBase Studio, create a project, including an empty StreamBase EventFlow application file to host the adapter.

  2. From the Operators and Adapters drawer of the Palette view, drag the Adapters, Java Operators icon to the canvas. This opens the Insert an Operator or Adapter dialog.

  3. In the search field, type a string such as liveview to narrow the list of adapters. Select the icon for the LiveView Query adapter, then click OK.

  4. Double-click the adapter's icon and select the Adapter Properties tab in the Properties view.

  5. Next, either:

    1. If this adapter is part of an EventFlow module that is itself part of a LiveView project, select the Set Server URI to encapsulating LiveView check box. This check box tells the adapter to connect to the host and port for the current project's LiveView server.

    2. Or specify the URI for a local or remote LiveView server to connect to (or accept the default URI, lv:/localhost:11080).

  6. Next, If the table name and schema are known:

    1. Specify the Target table name in the Adapters Properties tab.

    2. Specify zero or more queries in the Initial Queries tab.

    3. Specify the schema for the target LiveView table in the Edit Schema tab.

  7. Next, if you do not know the table name and schema, wish to dynamically name the table, or wish the output to be in JSON

    1. Ensure that the Query Input Port is enabled

    2. Ensure that the schema for the input port contains a string value named TableName.

    3. At runtime, the TableName field can be used to direct the queries to the named table.

  8. Specify the other options in the Adapter Properties tab as required.

  9. If you opted to have an input port, add an Input Stream and connect an arc from it to the LiveView Query adapter's input port.

  10. Specify a schema for the input port, as described in LiveView Query Adapter Input Port.

  11. Add the required Output Streams and connect arcs to them from the LiveView Query adapter's output ports.

  12. At runtime, this LiveView Query adapter connects to the specified LiveView server and registers the initial queries against the target table.

  13. If you configured an input port, send a tuple to this module's input port to register a new query or unregister an existing one, either for the table named in the Target table name property or the one named in the TableName field.

Typechecking and Error Handling

The LiveView Query adapter uses typecheck messages to help you configure the adapter.

The adapter generates warning messages at runtime under various conditions, including:

  • The configured schema of this adapter does not match the LiveView table.

  • The configured LiveView server is not available.

  • The table named in the TableName field does not exist.

  • The TableName field is empty and the Target table name property is also empty.

Suspend and Resume Behavior

When suspended, the adapter continues to receive and process LiveView updates, but no longer emits tuples on its primary output ports.

When resumed, the adapter once again starts emitting tuples on its primary output ports.