CREATE TABLE Statement

CREATE TABLE defines a private or shared Query Table.

Syntax

CREATE [INPUT | OUTPUT] [MEMORY | DISK] TABLE table_identifier (
  named_schema_identifier | anonymous_schema
  PRIMARY KEY(field_identifier[, ...]) [USING {HASH | BTREE}]
  [WITH PARAMETERS (parameter_name = "parameter_value" [, ...] )]
);

The older syntax, which is still supported, uses an anonymous schema:

CREATE [INPUT | OUTPUT] [MEMORY | DISK] TABLE table_identifier (
  column_identifier column_type PRIMARY KEY [USING {HASH | BTREE}]
  [, column_identifier column_type[, ...]]
);

Or

CREATE [INPUT | OUTPUT] [MEMORY | DISK] TABLE table_identifier (
  column_identifier column_type[, ...],
  PRIMARY KEY(column_identifier[, ...]) [USING {HASH | BTREE}]
);

The following statement is used in an outer StreamSQL module to declare a reference to a shared Query Table contained in an inner StreamSQL or EventFlow module. This statement is not used when an inner StreamSQL module accesses a shared Query Table in an outer module.

CREATE TABLE table_identifier;

Substitutable Fields

table_identifier

A unique identifier (name) for the table.

named_schema_identifier

The identifier of a previously-defined named schema. Imports all of the named schema's fields. No parentheses are required, and no other fields are permitted. Named schemas must be defined in dependency order. If a schema is used before it is defined, an error results.

anonymous_schema

A schema definition, delimited by parentheses, in the following format:

(field_definition [, ...])
field_definition

A field definition takes the form:

field_identifier field_type

field_identifier

A unique name for a field in the schema associated with the stream. For table indexes, if the field references a named schema, the entire schema is used as the key.

field_type

One of the supported StreamBase data types.

column_identifier

A unique identifier (name) for a column in the table. Since each row in the table corresponds to a stored tuple, a column corresponds to a field_definition.

column_type

The data type contained in a specified column. Analogous to the field_type.

parameter

A parameter takes the form:

parameter_name parameter_value

parameter_name

The name of a StreamBase parameter appropriate to the statement

parameter_value

A quoted value for the parameter, which if a string must include escaped quotes

Examples

The following statements demonstrate how to use a previously defined, named schema as the table schema:

CREATE SCHEMA SymbolSchema (ID int, Symbol string, Price double);
CREATE MEMORY TABLE Symbols1 
  SymbolSchema
  PRIMARY KEY(ID);

Note that in this format, the primary key can reference an individual field in the named schema.

Here is a table with an anonymous schema. Again, we designate a specific field as the primary key:

CREATE MEMORY TABLE Symbols2 
  (ID int, Symbol string, Price double)
  PRIMARY KEY(ID) ;

The next example includes a named schema as a field within an anonymous schema:

CREATE MEMORY TABLE Symbols3 
  (ss SymbolSchema, Quantity int)
  PRIMARY KEY(ss) ;

Notice that in this example, the ss field that we specify as the primary key is a tuple field that references a named schema. This means that all of the named schema's fields are used, in sort order. We cannot designate an individual field in the named schema as the primary key, as we can with the second, non-hierarchical field (Quantity).

Discussion

Private or shared Query Tables are defined in a CREATE TABLE statement. A private table is declared without either the INPUT or OUTPUT keyword. The OUTPUT keyword is used to indicate that the table can share its content across modules. The INPUT keyword indicates that a shared Query Table in another module can be the target for statements run against this table.

Each Query Table can be declared to reside in-memory or on-disk. In the case of memory-resident tables, any data stored in the table at runtime is not saved when the StreamBase Server shuts down. In the case of disk-based tables, the data stored in the tables can persist between StreamBase Server sessions, provided you enabled this behavior through configuration.

Configuration file entries specify where a disk-based table is stored and how transactional semantics should be applied to the table.

Each table must have a primary index (key). Secondary indexes are optional and are defined in CREATE INDEX Statement statements.

How keys are indexed for table read operations can also be specified.

  • Unordered, no ranges (hash): Keys are unsorted, and they are evenly distributed (hashed) across the index. A hash index is used for accessing keys based on equality.

  • Ordered, with ranges (btree): Keys are sorted. A btree index is used when output ordering and range queries are desired. Note that the sort depends on the order of the fields in the index keys.

There are two statements that can be used to create a private or shared Query Table. The first syntax is used when the table's primary key (or index) is composed of only one field. In this case, identify the primary key field as part of the field's declaration.

The second syntax can be used when the table has a composite primary key. In this case, define the primary key in a separate clause. This syntax can also be used if the primary key is based on a single table column.

Optionally, whether the table resides in memory (the default) or on disk, and whether a BTREE (the default) or HASH is used for ordering key/index values, can be specified.

Secondary indexes are defined in separate CREATE INDEX statements.

A StreamSQL application/module that accesses a shared Query Table implemented in an inner module must include a CREATE TABLE statement that specifies a local table identifier but does not define the schema or indexing behavior for the table. The structure and indexing of the table is inferred from syntax of the APPLY Statement statement used to integrate the module into the outer StreamSQL application/module. You can not redefine the table's schema or indexing from the outer application/module.

A StreamSQL module that accesses a shared Query Table implemented in an outer application/module contains a CREATE TABLE statement that completely specifies the schema and indexing behavior. The outer application/module also includes an identical table description, although the table identifier can be different. The fact that the table is implemented within the outer application/module is inferred from syntax of the APPLY statement used to integrate the module into the StreamSQL application/module.

StreamSQL tables are analogous to relational database tables and applications will use stream queries to manipulate the content of these tables. Stream queries issued against a table alter the contents of the table and can return a result set of tuples that documents the changes made to the table. List, in the stream query's optional RETURNING clause, the tuple fields and table columns included in the result set.

Stream queries that do not include a RETURNING clause modify the content of StreamSQL tables but additional query operations will be needed to confirm the changes.

A SELECT statement against a table is equivalent to a table read operation. A table read operation also returns a result set of tuples but does not alter the content of the table.

The tuples returned by a table read operation or stream query against a table must be "captured" into a stream by either defining the stream query as part of a stream definition or by explicitly applying the query's result set to an existing stream.

Sharing Query Tables

This section illustrates in detail how to use StreamSQL to share Query Tables across modules. The overall procedure is to use CREATE TABLE to create the shared and placeholder tables, and to use APPLY MODULE to integrate the modules and specify the table associations.

In general, when defining shared Query Tables across modules in StreamSQL you use the same kinds of access level concepts as in EventFlow; there is one difference, which is related to the placeholder access level. See the following list, which shows the access levels for tables specified in EventFlow and the corresponding StreamSQL definitions:

EventFlow Access Level StreamSQL StreamSQL Sample Code
Private Specify the name of the table and the schema using the CREATE TABLE statement.
CREATE TABLE 
    QueryTable;
Placeholder in the inner module Specify the name of the table and the schema using CREATE INPUT TABLE.
CREATE INPUT TABLE 
    Placeholder_QueryTable_Inner(
..)
;
Placeholder in the outer module

Specify the name of the table using CREATE INPUT TABLE.

NOTE: You can specify just the name of the table without the schema only when defining a placeholder in the outer module.

CREATE INPUT TABLE 
    Placeholder_QueryTable_Outer;
Shared Specify the name of the table and the schema using CREATE OUTPUT TABLE.
CREATE OUTPUT TABLE 
    Shared_QueryTable(
..)
;

Shared Query Table in an Inner Module

In this StreamSQL example, the actual, or shared, Query Table exists in an inner module. The outer module has access to the data in the actual Query Table by means of a placeholder table.

The placeholder table is specified in the outer module using the CREATE INPUT TABLE statement without specifying any schema. (This kind of table, as defined in StreamSQL, is not precisely the same as the kind of placeholder table defined in EventFlow.) The actual Query Table, that is, the shared table, is specified in the inner module using the CREATE OUTPUT TABLE statement. The outer module holds a reference to the inner module. The module reference is declared using the APPLY MODULE statement, which also specifies that the table in the outer module is a placeholder for the table in the inner module.

In this scenario, you must develop the inner module first. Then, to provide direct access to the shared Query Table from the outer module, you specify a reference to the inner module from the outer module.

This sample application performs the following operations:

  • Inner module writes data from the input stream to a shared Query Table.

  • Inner module emits output to OS1, the output stream.

  • Outer module gets input from OS1 and assigns it into the placeholder table.

  • Outer module reads all rows in the placeholder Query Table every 10 seconds and writes the data to the output stream, OS2.

Declaring the Shared Query Table in the Inner Module

The following StreamSQL code for the inner module creates the input and output streams, creates the shared Query Table, and emits to the output stream the data from both the input stream and the shared Query Table.

-- Inner Module Example 1

CREATE INPUT STREAM InputStream1 (
    stock string,
    volume int,
    price int
);

CREATE OUTPUT STREAM OS1;

CREATE OUTPUT MEMORY TABLE QueryTable1 (
    sym string,
    number int,
    cost int,
    PRIMARY KEY(sym) USING hash
);

REPLACE INTO QueryTable1 (sym, number, cost)
  SELECT *
    FROM InputStream1
    RETURNING InputStream1.stock AS input_stock,
              InputStream1.volume AS input_volume,
              InputStream1.price AS input_price,
              QueryTable1.sym AS table_sym,
              QueryTable1.number AS table_number,
              QueryTable1.cost AS table_cost
    INTO OS1;

Placeholder Table in Outer Module Referencing Inner

The following StreamSQL code specifies the outer module.

-- Outer Module Example 1

CREATE INPUT STREAM InputStream2 (
    stock string,
    volume int,
    price int
);

CREATE OUTPUT STREAM OS2;
CREATE OUTPUT STREAM OS3;

CREATE INPUT TABLE InnerModule_QueryTable1;

CREATE METRONOME Metronome1 (time, 10.0);

APPLY MODULE "innerModule.ssql"  -- or "innerModule.sbapp"
     FROM InputStream1 = InputStream2 INTO OS1 = OS3,
         QueryTable1 = InnerModule_QueryTable1;  

SELECT InnerModule_QueryTable1.sym
         AS table_current_sym,
       InnerModule_QueryTable1.number
         AS table_current_number,
       InnerModule_QueryTable1.cost
         AS table_current_cost
  FROM Metronome1 OUTER JOIN 
       InnerModule_QueryTable1
  INTO OS2;  

The following CREATE INPUT TABLE statement in the outer module creates the placeholder table:

CREATE INPUT TABLE InnerModule_QueryTable1;

The table declaration for the placeholder does not include a description of the table's schema, indexing, or persistence characteristics. Because the shared table has been fully specified in the inner module, its definition cannot be changed from within the outer module.

In the following APPLY MODULE statement, the INTO clause specifies the Query Table association:

APPLY MODULE "innerModule.ssql"
   FROM InputStream1 = InputStream2 INTO OS1 = OS3,
   QueryTable1 = InnerModule_QueryTable1;

Note the order of the entries in the association: the name of a Query Table in the inner module is on the left and the name of the corresponding Query Table in the outer module is on the right. This is the same ordering paradigm as the paradigm for the output stream associations—you can think of the shared Query Table defined in the inner module as equivalent to an output stream from the inner module.

Defining Two or More Shared Query Tables

If your design pattern includes an inner module that holds two or more shared Query Tables, you must specify both of the following in the StreamSQL for the outer module:

  • A separate CREATE INPUT TABLE statement for each placeholder table representing a shared table in the inner module.

  • An INTO clause in the APPLY MODULE statement specifying each Query Table assignment.

See the following example of the code for an outer module holding two placeholder tables, SharedTables_QueryTable1 and SharedTables_QueryTable2:

CREATE INPUT STREAM IS1 (
    stock string,
    cost int,
    volume int
);
CREATE OUTPUT STREAM OS1;
CREATE OUTPUT STREAM OS2;
CREATE OUTPUT STREAM OS3;

CREATE INPUT TABLE SharedTables_QueryTable1;
CREATE INPUT TABLE SharedTables_QueryTable2;

CREATE METRONOME out__Metronome1_1 (time, 10.0);

APPLY MODULE "innerModule.ssql" -- or "innerModule.sbapp"
   FROM InputStream1 = IS1
   INTO OutputStream1 = OS1,
      QueryTable1 = SharedTables_QueryTable1,
      QueryTable2 = SharedTables_QueryTable2;

Query Table in an Outer Module

In this scenario, the outer module holds the actual Query Tables to be shared. The outer module as well as the inner module must each be developed as complete, running applications. Then the inner module is referenced in the outer module, making it possible for the inner module to use the shared Query Table by means of query operators or other StreamSQL statements.

Developing the Outer Module

The outer module specifies a metronome operator, which, at 10 second intervals, initiates a read all rows query against two shared Query Tables, QT1 and QT2. A query against either of the tables emits a single tuple with null values in the fields derived from the table. The module writes data into QT1 and QT2 as output streams, OS2 and OS3.

--Outer Module Example 2

CREATE OUTPUT STREAM OS2;
CREATE OUTPUT STREAM OS3;

CREATE OUTPUT MEMORY TABLE QT1 (
    sym string,
    shares int,
    price int,
    PRIMARY KEY(sym) USING hash
);

CREATE OUTPUT MEMORY TABLE QT2 (
    sym string,
    shares int,
    price int,
    PRIMARY KEY(sym) USING hash
);

CREATE METRONOME Metronome1 (time, 10.0);

SELECT *
  FROM Metronome1 OUTER JOIN QT2
  INTO OS2;

SELECT Metronome1.time AS input_time,
       QT1.sym AS table_sym,
       QT1.shares AS table_shares,
       QT1.price AS table_price
  FROM Metronome1 OUTER JOIN QT1
  INTO OS3;

Note that the table declarations include the keyword OUTPUT, which specifies that the table being declared can share its data with other modules; that is, the table data is output from this module.

Even though the inner module has not yet been developed and referenced in the application, this outer module can run as a standalone application.

Developing the Inner Module

The inner module uses the update operation to write the same data into two Query Tables, QueryTable1 and QueryTable2, which are placeholders for the shared tables, QT1 and QT2, respectively. Each of two query operators is configured to pass fields from the input stream to its emitted tuple. After writing to the tables, the input data is emitted on the output stream OS1.

In the StreamSQL representation of this module, the table declarations include the keyword INPUT. This indicates that the table data come from a shared Query Table implemented in another module. In other words, data in the shared table in the outer module is input into this inner module.

-- Inner Module Example 2

CREATE INPUT STREAM InputStream1 (
    stock string,
    cost int,
    volume int
);
CREATE OUTPUT STREAM OS1;

CREATE INPUT MEMORY TABLE QueryTable1 (
    sym string,
    shares int,
    price int,
    PRIMARY KEY(sym) USING hash
);

CREATE INPUT MEMORY TABLE QueryTable2 (
    sym string,
    shares int,
    price int,
    PRIMARY KEY(sym) USING hash
);

CREATE STREAM Query1;

REPLACE INTO QueryTable1 (sym, shares, price)
  SELECT stock AS sym, volume AS shares, cost AS price
    FROM InputStream1
    RETURNING InputStream1.stock AS stock,
              InputStream1.cost AS cost,
              InputStream1.volume AS volume
    INTO Query1;

REPLACE INTO QueryTable2 (sym, shares, price)
  SELECT stock AS sym, volume AS shares, cost AS price
    FROM Query1
    RETURNING Query1.stock AS stock,
              Query1.cost AS cost,
              Query1.volume AS volume
    INTO OS1;

Even though this inner module has not been referenced in the outer module, it can run as a standalone application.

Referencing the Inner Module in the Outer Module

After developing the inner and outer modules independently, you can add to the outer module a reference to the inner module.

The following StreamSQL shows the finished application, which includes code for the outer module, the inner module, and code required to integrate the shared Query Table.

--Final Application Example 2

CREATE INPUT STREAM InputStream2 (
 stock string, cost int, volume int
 );
CREATE OUTPUT STREAM OS2;
CREATE OUTPUT STREAM OS3;
CREATE OUTPUT STREAM OS4;

CREATE OUTPUT MEMORY TABLE QT1 (
    sym string,
    shares int,
    price int,
    PRIMARY KEY(sym) USING hash
);

CREATE OUTPUT MEMORY TABLE QT2 (
    sym string,
    shares int,
    price int,
    PRIMARY KEY(sym) USING hash
);

CREATE METRONOME Metronome1 (time, 10.0);

APPLY MODULE "innerModule.ssql"  -- or "innerModule.sbapp"
FROM InputStream1 = InputStream2,
   QueryTable1 = QT1, QueryTable2 = QT2
INTO OS1 = OS4;


SELECT Metronome1.time AS input_time,
       QT2.sym AS table_sym,
       QT2.shares AS table_shares,
       QT2.price AS table_price
  FROM Metronome1 OUTER JOIN QT2
  INTO OS2;

SELECT Metronome1.time AS input_time,
       QT1.sym AS table_sym,
       QT1.shares AS table_shares,
       QT1.price AS table_price
  FROM Metronome1 OUTER JOIN QT1
  INTO OS3;

Note that, in the finished application, the APPLY MODULE statement specifies the inner module, as shown:

APPLY MODULE "innerModule.ssql"  -- or "innerModule.sbapp"

The FROM clause includes the following line, which specifies the Query Table associations:

QueryTable1 = QT1, QueryTable2 = QT2