< Previous		Next >

Creating, Starting, and Dropping Data Tables

Contents

Overview
Managing Add and Drop Table Transitions
Monitor System Table Behavior
Legal State Transitions Shown in LVTables
Steps to Add a Table While Monitoring LVTables
Using TableListListener

Spotfire LiveView provides support for:

Adding a new Data Table to a project while the Server is running
Dropping a running Data Table
Starting a new table
Restarting a dropped table

This page describes the state transitions you can expect for Data Tables transitioning between these states.

Note

When using lv-client or the Java API to add or remove a Data Table from the Server, it might create new files and directories in the project's workspace. For example, you might see a new folder lv-dynamicLVConfs; consider this folder as internal use only, and do not modify the folder or any of its contents. If you need to make changes to a newly created or recently dropped Data Table, use supported client commands and API operations.

Overview

The add and remove feature addresses the case where remote clients need to create simple tables that do not have field rules, Data Sources, Transforms, and so on. There are two ways to invoke these features:

With the lv-client command, which provides createtable, droptable,and starttable subcommands, as described on the lv-client reference page.
With the Java client API Interface TableAdminManager in com.streambase.liveview.client that includes createTable(), startTables(), and dropTables() methods.

Notice that lv-client's subcommands droptable and starttable are all lowercase and singular, whereas the Java Client API's equivalent methods are dropTables() and startTables() are mixed case and plural.

This primary goal of these features is to allow remote clients using the Java API to dynamically create LiveView tables and to publish data to these tables. You can consume the published data using LiveQL, LiveView Web, Spotfire, or your own custom clients. Only the Java Client API supports these features.

A secondary goal is to allow LiveView developers to add or drop tables from a LiveView Server running in StreamBase Studio. The LiveView Project Viewer provides ways to create lvconf files that can then be added to a LiveView Server instance running in the context of Studio. You cannot add or remove tables from a remote system from Studio.

Managing Add and Drop Table Transitions

The Java Client API provides the methods listed above to add, drop, and start tables. However, client applications must also confirm various state transitions of the tables they are manipulating, both to know when state transitions have completed and to monitor for error conditions.

There are two ways to track table transitions:

Monitor the LiveView system tables LVTables, LVTableColumns, and LiveViewStatistics.
Use the Java client API TableListListener.

The following sections describe these two approaches.

Monitor System Table Behavior

This section describes the changes to look for in the LiveView system tables when the create, drop, and start table API calls are made.

createTable()

Use the createTable() method to define a table's name and schema. When run, each createTable() call creates a new row in the LVTables table. The row's group is DYNAMIC, its StatusENUM is DEFINED and the create time is set to the current time on the server.

The equivalent lv-client command like the following:

lv-client createtable --name Foo --schema "(x int, s string)" --keys x

produces the following LVTables results:

Name,Group,ShortDescription,Description,TableSpace,Capabilities,QueryLanguages,
  Indices,StatusENUM,StatusMessage,IsSystemTable,CreateTime
Foo,DYNAMIC,null,null,null,"Snapshot, Continuous, Alert Rules, Delete, 
  Top-N Queries, Time-windowed Queries, Time-delayed Queries, Aggregate Queries, 
  Publish, Can be externally hosted, Calculated Columns",LIVEVIEW,x,DEFINED,
  "Defined, but not loaded",false,2015-09-30 11:33:53.060-0400

New rows are added to the LVTableColumns table for each field in the newly added table:

Table,Name,Type,ShortName,Description
Foo,s,string,s,
Foo,x,int,x,

There are no changes to the LiveViewStatistics table from a createTable() call.

startTables()

Use the startTables() method to compile and load a newly created table. From Studio, you can use the LiveView Project Viewer to start lvconf files you have added.

For a call to startTables(), an LVTables row for the specified table is updated for Java API users and added for Studio users. The row's StatusENUM shows as LOADING, which tells you that the table is being compiled. The row updates to ENABLED when the table is ready.

For example, use an equivalent lv-client command like the following:

lv-client starttable Foo

Use an option to the startTables() method to take a snapshot of the transition state of started tables, based on a configurable time duration. For example, use this option before performing table operations synchronously, such as creating aggregate tables from a base table only after a base table is created and enabled. See the Java API documentation for details on using it.

When you use the method with a time duration of 10 seconds:

(10, TimeUnit.Seconds, Table 1, Table 2, Table n...)

If 10 seconds are given as the duration:

If all tables have reached their end state before 10 seconds elapse (before the max time), the call returns with a status map containing all end states.
Even if the tables have not reached their end state within 10 seconds, the status map is returned at the 10th second.

Time is based on the standard Java TimeUnit enum. Entering a time value of 0 or a negative value returns every table transition state to completion.

Note

Currently there is no lv-client command that returns a snapshot at the end of a time duration you provide, to start a table. However, you can use the starttable command with the --wait argument to show the live transition state for one specified table at a time.

dropTables()

When using the dropTables() method, the LVTables row for a table that was formerly ENABLED updates its StatusENUM field to DISABLED. When the table is fully dropped, its LVTables row is deleted. The LVTableColumns rows for the dropped table are also deleted.

For example, running an equivalent lv-client command like the following:

lv-client droptable Foo

Produces the following results:

Name,Group,ShortDescription,Description,TableSpace,Capabilities,QueryLanguages,
  Indices,StatusENUM,StatusMessage,IsSystemTable,CreateTime
Foo,DYNAMIC,null,null,null,"Snapshot, Continuous, Alert Rules, Delete, 
  Top-N Queries, Time-windowed Queries, Time-delayed Queries, Aggregate Queries, 
  Publish, Can be externally hosted, Calculated Columns",LIVEVIEW,x,ENABLED,
  Table is being removed.,false,2015-09-30 11:33:52.060-0400
Name,Group,ShortDescription,Description,TableSpace,Capabilities,QueryLanguages,
  Indices,StatusENUM,StatusMessage,IsSystemTable,CreateTime
Foo,DYNAMIC,null,null,null,"Snapshot, Continuous, Alert Rules, Delete, 
  Top-N Queries, Time-windowed Queries, Time-delayed Queries, Aggregate Queries, 
  Publish, Can be externally hosted, Calculated Columns",LIVEVIEW,x,DISABLED,
  Table is being removed.,false,2015-09-30 11:33:53.060-0400

Also, when you drop a table that was created but not started, its LVTables row and associated LVTableColumns rows are removed.

Use an option to the dropTables() method take a snapshot of the transition of dropped tables, for the time duration you set. For example, use this method to take a snapshot before performing table operations synchronously, such as recreating a base table after an existing table drops successfully.

When you use the method with a time duration of ten seconds:

(10, TimeUnit.Seconds, Table 1, Table 2, Table n...)

If 10 seconds are given as the duration:

If all tables have reached their end state before 10 seconds elapse (before the max time), the call returns with a status map containing all end states.
Even if the tables have not reached their end state within 10 seconds, the status map is returned at the 10th second.

Time is based on the standard Java TimeUnit enum. Entering a time value of 0 or a negative value returns every table transition state to completion.

Note

Currently there is no lv-client command that returns a snapshot at the end of a time duration you provide, to drop a table. However, you can use the droptable command with the --wait argument to show the live transition state for one specified table at a time.

Error Conditions

Error conditions can occur when creating or starting tables. There are some createTable() error conditions that throw an exception from the call with no corresponding system table entries created or changed. However both createTable() and startTables() can return successfully, and it is not until sometime later in the create and start process that errors are detected. If an error occurs with a table after it has had an LVTables row added, that row is updated with a StatusENUM of ERROR, and the StatusMessage provides some helpful text on the nature of the problem. For example, creating a table with an index field that is not present in the table schema, using a command like the following:

lv-client createtable --name Bar --schema "(x int, s string)" --keys y
lv-client starttable Bar

produces LVTables entries like these:

Name,Group,ShortDescription,Description,TableSpace,Capabilities,QueryLanguages,
  Indices,StatusENUM,StatusMessage,IsSystemTable,CreateTime
Bar,DYNAMIC,null,null,null,"Snapshot, Continuous, Alert Rules, Delete, 
  Top-N Queries, Time-windowed Queries, Time-delayed Queries, Aggregate Queries, 
  Publish, Can be externally hosted, Calculated Columns",LIVEVIEW,x,DEFINED,
  "Defined, but not loaded",false,2015-09-30 11:48:30.625-0400
Bar,DYNAMIC,null,null,null,"Snapshot, Continuous, Alert Rules, Delete, 
  Top-N Queries, Time-windowed Queries, Time-delayed Queries, Aggregate Queries, 
  Publish, Can be externally hosted, Calculated Columns",LIVEVIEW,y,ERROR,
  Invalid field reference 'y',false,2015-09-30 11:48:30.625-0400

The ERROR LVTables row remains until the client issues a dropTable() call.

Legal State Transitions Shown in LVTables

The following table shows the legal states that can be reached for each current state of the StatusENUM field of the LVTables system table:

Current State	Next States	API Call and Description
DEFINED	LOADING	`startTables()` — Table is being compiled.
	ERROR	`startTables()` — Table was not successfully compiled.
	NA	`dropTables()` — Table was dropped and is gone.
DISABLED	NA	NA — Table is gone.
ENABLED	DISABLED	`dropTables()` — Table is disabled and being removed.
ERROR	LOADING	`startTables()` — Table was started again and is trying to be compiled.
	NA	`dropTables()` — Table was dropped and is gone.
LOADING	ENABLED	NA — Table is ready for use.
	ERROR	NA — Table was not loaded successfully.
NA	DEFINED	`createTable()` — Table was created successfully.
	ERROR	`createTable()` — Table was not created successfully.

Steps to Add a Table While Monitoring LVTables

Follow these steps:

Create a table using from the command line or a with a Client API call.
Register a live query against the LVTables table. You can restrict the results to the table of interest by using a predicate in your query such as where Name=="YourTableName".
Confirm that the row associated with your newly added table has a StatusENUM of DEFINED. If it shows ERROR, you must drop the table, fix the problem, then try again.
Start the table.
Confirm the row associated with the newly added table has a StatusENUM of LOADING, followed by an update to ENABLED.

Once a LiveView table has transitioned to the ENABLED state, it does not change its state again unless there is a subsequent dropTables() call. However, if the state changes to ERROR, you must drop the table, fix the problem, then try again.

Using TableListListener

The TableListListener interface is an API that provides client callbacks when a table's state changes. LiveView tables issue transition events as they are dynamically created, started, and dropped. See the Javadoc for this API for details on using it.