LVconf Type: Data Table

< Previous		Next >

Overview

Use the LiveView Configuration File Editor to configure the Data Table lvconf file type. This lvconf type is created by both the Data Table wizard (which opens automatically after creating a new LiveView fragment project), and by the Data Source wizard, which creates a data table and an EventFlow application to feed into that table.

Specify the table's schema, either by reference to a named schema or by editing the schema grid directly.

Designate one or more primary keys, either by selecting an available field and clicking the arrow button, by dragging and dropping from left to right, or by double-clicking an available field's name.

Schema Tab

Specify a schema by importing a named schema, and/or adding fields directly to the table below

Schema Field

Property	Type	Description
Interface/Application	drop‑down list, chooser	Path to the sbapp or sbint file from which to import the schema. The path is relative to the project root (where the top-level lvconf must reside). If you specify a simple file name, LiveView searches for the file on the current project's module search path.
Schema	drop‑down list	The name of the schema within the specified sbapp or sbint file.

To define your own schema:

Click the Plus sign to add fields.
Select a field type from the dropdown list.
Enter an optional description.

Options Field

Define your data table options:

Property	Type	Description
Snapshot Concurrency	int	Defines the maximum number of snapshot queries (one per thread) within a partition that are executed at once.
Snapshot Parallelism	int	Defines the number of table partitions and the number of parallel regions used to scan a table during the snapshot portion of a query.
Publish Interval (ms)	int	This setting applies to all table types, and is used to specify in milliseconds the amount of delay before publishing data from the table. This condition is known as conflation of data. Set this attribute to a positive, non-zero integer ≥ 5 to specify conflation for this table. A value below 5, including zero and all negative numbers, is forced equal to 5. Static aggregation tables have an internal setting that is the equivalent of setting publish-interval-millis=1000 (1 second). For such tables, you cannot disable conflation, but you can change the conflation interval by setting publish-interval-millis to a different integer value. For all other table types, specify a value for publish-interval-millis to both enable conflation and to specify the conflation interval. To disable conflation for non-aggregation tables, remove the publish-interval-millis attribute entirely from your table’s lvconf entry. Without conflation, data is published from a table as soon as it is received. With conflation enabled, all downstream components see conflated data, including alerts, clients built with the Java or .NET APIs, or another table in a transformation sequence. If an alert is set for a conflated table, be aware that it is possible for conditions that would otherwise trigger an alert to occur briefly during a conflation period; in this case, the trigger conditions are conflated away and the alert does not trigger. See the Data Conflation topic in the LiveView Reference Guide for further information.
Table Space Reference	drop‑down list	A reference to a defined table-space.
Row Delete Rule	string	An optional rule specifying a predicate that is evaluated against the data in any incoming insert or update. If the incoming data satisfies the predicate and a row with a matching primary key exists, then no update occurs and the row is deleted. If the incoming data satisfies the predicate and no row with a matching primary key exists, then no insert occurs. If the incoming data does not satisfy the predicate, then an insert or update occurs as normal.
Group	string	Optional logical grouping for a table.

Description Field

Enter a short or long description (string) for the data table type.

Indexes Tab

Select an primary key from the Available Fields list and either double-click it or click arrow to move it to the Index Fields box.

Optionally, select a secondary index or click Add to create one.

Auto-increment Field

You can set auto-increment for LONG type primary key fields. Enable the auto-increment by selecting the relevant field and clicking the "++" button in the toolbar. Auto-incrementing Primary Indexes are given a "++" overlay in the Indexes tab and the Schema tab.

Data Sources Tab

Choose up to three data sources in any combination. For most applications, only one data source type is required.

Aggregation Field

Property	Type	Description
Table Reference	drop‑down list	A reference to a table from which aggregated data is received.
Projection	string	The projection expression that is used to populate the fields of this aggregate table. For example, max(price) as MaxPrice, avg(price) as AvgPrice.
Predicate	string	The predicate used to filter rows from the base table. That is, area=='US' AND price>100 when transactionTime between now()-minutes(1) and now().
Group By	string	The group names or aliases must be a primary key field of this table. By default, the base table has column names that match this table's primary keys and will be used as group by keys.
Define Pivot Table	check box	Pivot: The pivot aggregate expression that is used to populate the fields of this pivot table. For example, max(price) as MaxPrice, avg(price) as AvgPrice. For: Used to provide a pivot column if any literals are used in the values clause. Each literal in the values clause is interpreted as the value for the pivot column. Values: The values clause should have comma-separated values where each value can either be a literal or an expression resulting in the Boolean value. If even a single value is literal, the for clause becomes mandatory. If selected, Group By becomes disabled.

Application Field

Property	Type	Description
Application Reference	drop‑down list	Reference to a defined application.
Output Stream	string	A valid output stream of the application to receive data from. Application streams that are intended to send data to LiveView or receive data from LiveView must be marked public; that is, the stream property "Always expose Stream for Dequeue" or "Always expose Stream for Enqueue" must be set.
Disable validation typecheck	check box	There are certain unusual situations where a start-up artifact from a preprocessing app is not available early in LiveView server start-up. In these rare cases, in order to allow LiveView to run, you can skip typechecking of the referenced preprocessing app by setting this attribute to false. The downside is that you will no longer receive compile-time typecheck warnings, and may get runtime exceptions if your configuration is wrong. If typechecking is enabled, the system attempts to coerce fields to be the correct type, such as converting an integer to a long, or filling in empty fields with null.

Transform Field

Property	Type	Description
Application	drop‑down list, chooser	Used by the StreamBase interface generator to identify the transform.
Table Reference	drop‑down list	A reference to the source table.
Parameters set by this configuration	selector	Parameters for the StreamBase application. Select one from the available list.
Available Module Parameters	string	Select the application parameters to be set by this configuration. The Available module parameters table shows all parameters defined in the currently selected application. Default values are shown in brackets, when defined.
Disable validation typecheck	check box	There are certain unusual situations where a start-up artifact from a preprocessing app is not available early in LiveView server start-up. In these rare cases, in order to allow LiveView to run, you can skip typechecking of the referenced preprocessing app by setting this attribute to false. The downside is that you will no longer receive compile-time typecheck warnings, and may get runtime exceptions if your configuration is wrong. If typechecking is enabled, the system attempts to coerce fields to be the correct type, such as converting an integer to a long, or filling in empty fields with null.
Allow Cycle	check box	Optional. Under normal circumstances, cycles in the configuration graph are not allowed. Under certain advanced cases it might be necessary to have a cycle in the graph. A simple example would be table A publishing to table B and table B publishing back to table A. If a scenario like this is required, a custom transform should be introduced and the “allow-cycle” attribute should be set to true. This setting should be made with great care.
Use Snapshot Parallelism	check box	Optional. When a table is configured with snapshot-parallelism, setting this attribute to true pushes the processing of this transform into the same parallel region as snapshot processing. In general, performance should be better by running inside the parallel region.

Preprocessor Tab

None = default. If you select Use preprocessor application, configure the following as needed.

Preprocessor Field

Property	Type	Description
Application	drop‑down list, chooser	Filename of the StreamBase application. This file is located using the project's module search path.
Disable validation typecheck	check box	There are certain unusual situations where a start-up artifact from a preprocessing app is not available early in LiveView server start-up. In these rare cases, in order to allow LiveView to run, you can skip typechecking of the referenced preprocessing app by setting this attribute to false. The downside is that you will no longer receive compile-time typecheck warnings, and may get runtime exceptions if your configuration is wrong.
Parameters set by this configuration	click Add	Name-value pair property definitions.
Available Module Parameters	N/A	This table shows all parameters defined in the currently selected application. Default values are shown in brackets, when defined. To set a parameter value (or override its default), select a parameter and click the Add button or double click a parameter, then enter the desired value.
Tables this preprocessor depends on	click Add	Since StreamBase applications can have container connections defined within them, it is possible to have a container connection from this application to another LiveView table. Explicitly stating these dependencies here allows LiveView to know about them such that LiveView will load tables in the proper order on startup.

Optionally:

Add all available module parameters without defaults
Add all available module parameters
Add a parameter not yet defined by the application

Retention Policy

Use the Retention Policy tab to define how long you want table data retained. Retention policies become especially relevant when dealing with large tables of streaming data and server memory is a consideration. When setting a retention policy, the current LiveView server time is compared with the timestamp field value, and that becomes the basis for whether a row is deleted.

Set one of the following Table Delete Rules:

Elapsed Time Rule

Define how long to store table data before the table is trimmed. Optionally, generate a Timestamp field. The chosen timestamp is added as a secondary index for the table. Valid timestamp fields in the data table's schema populate the dropdown list.

You can set a timestamp for event arrival. This action creates a field rule.

Predicate Rule

Build a rule to register a query against the table where rows are added to the result set are deleted through the publish data path. For this reason, the predicate usually has a time component (see WHEN and FOR clauses for more information).

No Data Retention Policy

Table rows will not be trimmed — based on retention policy. See Managing the Size of Data Tables for table trimming options that you can configure independently from retention policies.

Aliases Tab

Property	Type	Description
Table Aliases	string	This can be simply the name of the aliased table, or a formatter string as would be passed to the java.lang.String.format method, where %s is to be replaced with the name of the base table.
Filter	string	Part of a predicate that is to be appended (using AND or its equivalent in the query language of the table being used) to all queries coming to the aliased table. This predicate should be in the query language of the base table.

Field Rules Tab

Field Rules: Click the Plus sign to add a new table-level rule.

Field Rule Details: Add a description for the rule.

Variables: Click the Plus sign to define a variable for the rules. Typed variables can be defined that will be used in the table-level rules. Use these to save intermediate calculations which otherwise would have to be re-calculated.

You can optionally automatically set the timestamp on arrival, for a configured retention policy.

Source Tab

The Source tab provides a text-based XML configuration file editor, which is a validating XML editor that is aware of the schema that defines the XML syntax of LiveView configuration files. See Text-Based XML Configuration File Editor for more information.

Semantic Interpretation

LiveView supports inclusion of metadata hints in data tables. You can add semantic interpretations using visual editor or by directly editing a data table lvconf file's XML code.

To add semantic interpretations, do the following:

Right-click the table field.

The following options appear.

/>

Select Semantic Interpretations.

The Edit Semantic Interpretation screen appears.

For example, If semantic interpretation for TotalPassengers is added as 40. You can see the same in code of the xml file like below:

<?xml version="1.0" encoding="ASCII"?>
<liveview-configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
  xsi:noNamespaceSchemaLocation="http://www.streambase.com/schemas/lvconf/">
  <data-table id="RunningAggregates">
    <fields>
      <field name="TotalPassengers" type="int">
        <semantic-interpretation>
          <interpretation>40</interpretation>
        </semantic-interpretation>
      </field>
    <primary-key>
      <field ref="Type"/>
    </primary-key>
  </data-table>
</liveview-configuration>