Contents
Live Datamart release 2.1.0 added support for adding a new Data Table to a project while the Server is running, for dropping a running Data Table, and for starting a new table or restarting a dropped table. This page describes the state transitions you can expect for Data Tables transitioning between these states.
Note
When using lv-client or the Java API to add or remove a Data Table from the Server, it might create new files and directories
in the project's workspace. For example, you might see a new folder lv-dynamicLVConfs; consider this folder as internal use only, and do not modify the folder or any of its contents. If you need to make changes
to a newly created or recently dropped Data Table, use supported client commands and API operations.
The add and remove feature addresses the use case where remote clients need to create simple tables that do not have field
rules, Data Sources, Transforms, and so on. There is a Java client API Interface TableAdminManager in com.streambase.liveview.client that includes createTable(), startTables(), and dropTables() methods. The lv-client command provides createtable, droptable,and starttable subcommand, as described on the lv-client reference page.
This primary goal of these features is to allow remote clients using the Java API to dynamically create LiveView tables and to publish data to these tables. You can consume the published data using LiveQL, LiveView Desktop, or your own custom clients. Only the Java Client API supports these features.
A secondary goal is to allow LiveView developers to add or drop tables from a LiveView Server running in StreamBase Studio. The LiveView Project Viewer provides ways to create lvconf files that can then be added to a LiveView Server instance running in the context of Studio. You cannot add or remove tables from a remote system from Studio.
The Java Client API provides the methods listed above to add, drop, and start tables. However, client applications must also confirm various state transitions of the tables they are manipulating, both to know when state transitions have completed and to monitor for error conditions.
There are two ways to track table transitions:
-
Monitor the LiveView system tables LVTables, LVTableColumns, and LiveViewStatistics.
-
Use the Java client API TableListListener.
The following sections describe these two approaches.
This section describes the changes to look for in the LiveView system tables when the create, drop, and start table API calls are made.
Use the createTable() method to define a table's name and schema. When run, each createTable() call creates a new row in the LVTables table. The row's group is DYNAMIC, its StatusENUM is DEFINED and the create time is set to the current time on the server.
An lv-client command like the following:
lv-client createtable --name Foo --schema "\"(x int, s string)\"" --keys x
produces the following LVTables results:
Name,Group,ShortDescription,Description,TableSpace,Capabilities,QueryLanguages, Indices,StatusENUM,StatusMessage,IsSystemTable,CreateTime Foo,DYNAMIC,null,null,null,"Snapshot, Continuous, Alert Rules, Delete, Top-N Queries, Time-windowed Queries, Time-delayed Queries, Aggregate Queries, Publish, Can be externally hosted, Calculated Columns",LIVEVIEW,x,DEFINED, "Defined, but not loaded",false,2015-09-30 11:33:53.060-0400
New rows are added to the LVTableColumns table for each field in the newly added table:
Table,Name,Type,ShortName,Description Foo,s,string,s, Foo,x,int,x,
There are no changes to the LiveViewStatistics table from a createTable() call.
Use the startTable() method to compile and load a newly created table. From Studio, you can use the LiveView Project Viewer to start lvconf files you have added.
For a call to startTable(), an LVTables row for the specified table is updated for Java API users and added for Studio users. The row's StatusENUM to LOADING, which tells you that the table is being compiled. The row updates to ENABLED when the table is ready.
For example, running an lv-client command like the following:
lv-client starttable --name Foo
Produces the following results:
Name,Group,ShortDescription,Description,TableSpace,Capabilities,QueryLanguages, Indices,StatusENUM,StatusMessage,IsSystemTable,CreateTime Foo,DYNAMIC,null,null,null,"Snapshot, Continuous, Alert Rules, Delete, Top-N Queries, Time-windowed Queries, Time-delayed Queries, Aggregate Queries, Publish, Can be externally hosted, Calculated Columns",LIVEVIEW,x,LOADING, Loading.,false,2015-09-30 11:33:53.060-0400 Foo,DYNAMIC,null,null,null,"Snapshot, Continuous, Alert Rules, Delete, Top-N Queries, Time-windowed Queries, Time-delayed Queries, Aggregate Queries, Publish, Can be externally hosted, Calculated Columns",LIVEVIEW,x,ENABLED,, false,2015-09-30 11:33:53.060-0400
Use an option to the startTable() method to take a snapshot of the transition state of started tables, based on a configurable time duration. For example, use this option before performing table operations synchronously, such as creating aggregate tables from a base table only after a base table is created and enabled. See the Java API documentation for details on using it.
When you use the method with a time duration of 10 seconds:
(10, TimeUnit.Seconds, Table 1, Table 2, Table n...)
If 10 seconds are given as the duration:
-
If all tables have reached their end state before 10 seconds elapse (before the max time), the call returns with a status map containing all end states.
-
Even if the tables have not reached their end state within 10 seconds, the status map is returned at the 10th second.
Time is based on the standard Java TimeUnit enum. Entering a time value of 0 or a negative value returns every table transition state to completion.
Note
Currently there is no lv-client command that returns a snapshot at the end of a time duration you provide, to start a table. However, you can use the starttable command with the --wait argument to show the live transition state for one specified table at a time.
When using the dropTable() method, the LVTables row for a table that was formerly ENABLED updates its StatusENUM field to DISABLED. When the table is fully dropped, its LVTables row is deleted. The LVTableColumns rows for the dropped table are also deleted.
For example, running an lv-client command like the following:
lv-client droptable --name Foo
Produces the following results:
Name,Group,ShortDescription,Description,TableSpace,Capabilities,QueryLanguages, Indices,StatusENUM,StatusMessage,IsSystemTable,CreateTime Foo,DYNAMIC,null,null,null,"Snapshot, Continuous, Alert Rules, Delete, Top-N Queries, Time-windowed Queries, Time-delayed Queries, Aggregate Queries, Publish, Can be externally hosted, Calculated Columns",LIVEVIEW,x,ENABLED, Table is being removed.,false,2015-09-30 11:33:52.060-0400 Name,Group,ShortDescription,Description,TableSpace,Capabilities,QueryLanguages, Indices,StatusENUM,StatusMessage,IsSystemTable,CreateTime Foo,DYNAMIC,null,null,null,"Snapshot, Continuous, Alert Rules, Delete, Top-N Queries, Time-windowed Queries, Time-delayed Queries, Aggregate Queries, Publish, Can be externally hosted, Calculated Columns",LIVEVIEW,x,DISABLED, Table is being removed.,false,2015-09-30 11:33:53.060-0400
Also, when you drop a table that was created but not started, its LVTables row and associated LVTableColumns rows are removed.
Use an option to the dropTable() method take a snapshot of the transition of dropped tables, for the time duration you set. For example, use this method to take a snapshot before performing table operations synchronously, such as recreating a base table after an existing table drops successfully.
When you use the method with a time duration of ten seconds:
(10, TimeUnit.Seconds, Table 1, Table 2, Table n...)
If 10 seconds are given as the duration:
-
If all tables have reached their end state before 10 seconds elapse (before the max time), the call returns with a status map containing all end states.
-
Even if the tables have not reached their end state within 10 seconds, the status map is returned at the 10th second.
Time is based on the standard Java TimeUnit enum. Entering a time value of 0 or a negative value returns every table transition state to completion.
Note
Currently there is no lv-client command that returns a snapshot at the end of a time duration you provide, to drop a table. However, you can use the droptable command with the --wait argument to show the live transition state for one specified table at a time.
Error conditions can occur when creating or starting tables. There are some createTable() error conditions that throw an exception from the call with no corresponding system table entries created or changed. However both createTable() and startTable() can return successfully, and it is not until sometime later in the create and start process that errors are detected. If an error occurs with a table after it has had an LVTables row added, that row is updated with a StatusENUM of ERROR, and the StatusMessage provides some helpful text on the nature of the problem. For example, creating a table with an index field that is not present in the table schema, using a command like the following:
lv-client createtable --name Bar --schema "\"(x int, s string)\"" --keys y lv-client starttable Bar
produces LVTables entries like these:
Name,Group,ShortDescription,Description,TableSpace,Capabilities,QueryLanguages, Indices,StatusENUM,StatusMessage,IsSystemTable,CreateTime Bar,DYNAMIC,null,null,null,"Snapshot, Continuous, Alert Rules, Delete, Top-N Queries, Time-windowed Queries, Time-delayed Queries, Aggregate Queries, Publish, Can be externally hosted, Calculated Columns",LIVEVIEW,x,DEFINED, "Defined, but not loaded",false,2015-09-30 11:48:30.625-0400 Bar,DYNAMIC,null,null,null,"Snapshot, Continuous, Alert Rules, Delete, Top-N Queries, Time-windowed Queries, Time-delayed Queries, Aggregate Queries, Publish, Can be externally hosted, Calculated Columns",LIVEVIEW,y,ERROR, Invalid field reference 'y',false,2015-09-30 11:48:30.625-0400
The ERROR LVTables row remains until the client issues a dropTable() call.
The following table shows the legal states that can be reached for each current state of the StatusENUM field of the LVTables system table:
| Current State | Next States | API Call and Description |
|---|---|---|
| DEFINED | LOADING | startTable() — Table is being compiled. |
| ERROR | startTable() — Table was not successfully compiled. | |
| NA | dropTable — Table was dropped and is gone. | |
| DISABLED | NA | NA — Table is gone. |
| ENABLED | DISABLED | dropTable — Table is disabled and being removed. |
| ERROR | LOADING | startTable — Table was started again and is trying to be compiled. |
| NA | dropTable — Table was dropped and is gone. | |
| LOADING | ENABLED | NA — Table is ready for use. |
| ERROR | NA — Table was not loaded successfully. | |
| NA | DEFINED | createTable — Table was created successfully. |
| ERROR | createTable — Table was not created successfully. |
Follow these steps:
-
Create a table using from the command line or a Client API call.
-
Register a live query against the LVTables table. You can restrict the the results to the table of interest by using a predicate in your query such as
where Name==".YourTableName" -
Confirm that the row associated with your newly added table has a StatusENUM of DEFINED. If it has ERROR, you must drop the table, fix the problem, then try again.
-
Start the table.
-
Confirm the row associated with the newly added table has a StatusENUM of LOADING, followed by an update to ENABLED.
Once a LiveView table has transitioned to the ENABLED state, it does not change its state again unless there is a subsequent dropTable() call. However, if the state changes to ERROR, you must drop the table, fix the problem, then try again.