HOCON Syntax Reference

Overview

This article describes the HOCON configuration syntax.

Configuration Syntax

The configuration syntax for StreamBase is Human-Optimized Configuration Object Notation (HOCON). HOCON is a superset of JavaScript Object Notation (JSON), enhanced for readability. The specification for HOCON syntax is found here. Features in HOCON that are unchanged from JSON are described here.

TIBCO StreamBase extends standard HOCON in several ways, including:

  • The first three lines of each configuration file must contain the name, version, and type keywords as a header, which together uniquely identify the configuration realm and version of an individual configuration file.

  • StreamBase HOCON does not support the include keyword defined in the HOCON specification. Instead, StreamBase HOCON recognizes a multi-line array syntax within square brackets as a way to embed a subordinate configuration specification within another. Multi-line array syntax is bounded by three double-quote symbols; the brackets define an array:

    [
    """
    embedded configuration
    """
    ]

Further differences between standard HOCON format and configuration file syntax are described below.

File Names and Encoding

DTM and StreamBase HOCON configuration files must have a suffix of .conf. The basename of the file can be any valid name supported by the underlying operating system. There is no special significance associated with the file name; that is, configuration files can be named anything that reminds you of its contents.

Certain configuration file types are generated by StreamBase Studio with conventional names, such as engine.conf and node.conf. These file names are not required, but are suggested starting points.

A configuration file's character encoding can be UTF-8, UTF-16, or UTF-32 as defined by the JSON specification.

Headers and Envelope

Every configuration file must start with a header that contains the configuration type, name, and version identifiers. These identifiers are all string values. Following the header, a configuration envelope must be specified in which the configuration data is contained. The configuration data allowed within the configuration envelope is defined by the configuration type specified in the header.

The supported configuration types are:

DTM Configuration types, to specify and manage DTM applications, nodes, engines, and the node-to-node security realm.

com.tibco.ep.dtm.configuration.application

See Reference for DTM Application Configuration

com.tibco.ep.dtm.configuration.node

See Reference for DTM Node Configuration

com.tibco.ep.dtm.configuration.javaengine

See Reference for DTM JavaEngine Configuration

com.tibco.ep.dtm.configuration.security

See Reference for DTM Security Configuration

StreamBase Configuration types to specify and manage features of StreamBase EventFlow Fragments running within a DTM node.

com.tibco.ep.streambase.configuration.sbengine

See Reference for StreamBase Engine Configuration

com.tibco.ep.streambase.configuration.sbclientapilistener

See Reference for StreamBase ClientAPIListener Configuration

com.tibco.ep.streambase.configuration.jdbcdatasource

See Reference for StreamBase JDBC DataSource Configuration

com.tibco.ep.streambase.configuration.ldapauthrealm

See Reference for LDAP Authentication Realm Configuration

com.tibco.ep.streambase.configuration.commsecurity

See Reference for StreamBase CommSecurity Configuration

com.tibco.ep.streambase.configuration.defaultjmscontainerconnectionsettings

See Reference for StreamBase DefaultJMSContainerConnectionSettings Configuration

Live Datamart Configuration types to specify and manage features of Live Datamart EventFlow fragments running in a DTM node.

com.tibco.ep.ldm.configuration.ldmclientapilistener

See Reference for Live Datamart Client API Listener Configuration

com.tibco.ep.ldm.configuration.ldmengine

See Reference for Live Datamart Engine Configuration

com.tibco.ep.ldm.configuration.ldminternalcredentials

See Reference for Live Datamart Internal Credentials Configuration

Configuration Header and Envelope Example

Header values:

name = "integration-test-application"
version = "10.1.0"
type = "com.tibco.ep.dtm.configuration.application"

Configuration envelope:

configuration =
{
//your configuration data specific to configuration type
}

Comment Characters

As in standard JSON and standard HOCON, any text between // or # and the next newline is considered a comment and is ignored (unless the // or # is inside a quoted string).

Substitution Variables

Configuration files provide substitution variable support, where the substitution variable values come from (in priority order):

  1. The substitutions= parameter of the epadmin install node or epadmin load configuration commands.

  2. A substitution file specified using the substitutionfile= parameter of the same two epadmin commands.

  3. Java system properties in the JVM in which the configuration is being processed.

For example, if the same substitution variable value is specified as both a Java system property and using install node's substitutions parameter, the value specified in the parameter would be used.

There is no support for using an environment variable value as a substitution variable value. This is different behavior than standard HOCON, which allows environment variable values to be used as a substitution variable value.

Substitution variables must follow these rules:

  • Defined using ${variable-name[:-default-value]}. This defines a substitution variable named variable-name with an optional default value of default-value.

  • variable-name can only contain characters specified by this regular expression [a-zA-Z0-9\\.-_]+.

  • variable-name cannot contain the HOCON escape character (\).

  • Substitution variable values, and default-value expressions, cannot contain unescaped newlines. Escaped newlines (\\n) are permitted, and will result in a newline character appearing in the substituted value.

  • Nested substitution variables are not allowed (these are allowed by standard HOCON). For example:

    foo = bar 
    bar = biz 
    bar = biz ${${foo}} // returns an error instead of the value biz
  • Substitution variables are supported anywhere in a configuration file, including the left-hand side of an assignment. This is an extension to standard HOCON.

  • Substitution variables on the left-hand side of an assignment that contain the "." character must be surrounded with double quotes to prevent the value from being interpreted as a HOCON path. This is an extension to standard HOCON.

  • Substitution variables are supported in quoted strings. This is an extension to standard HOCON.

Substitution Variable Example

The following is an example of a substitution variable in the configuration file header as well as in the body of the file.

Note

When using the HOCON Editor in StreamBase Studio, substitution variables MUST be specified with default values in order to pass Studio typechecking. Nevertheless, configuration files like the sample below (without default values) can be run as shown from the epadmin command line.

//
// Setting substitution variables on command line
//
epadmin load configuration source=my.conf substitutions=
  "NODE_NAME=A.X,PROJECT_VERSION=1.0"

//
// Use substitution variable in configuration file header
//
name     = "integration-test-application"
version  = ${PROJECT_VERSION}
type     = "com.tibco.ep.dtm.configuration.node"
configuration =
{
  NodeDeploy =
  {
    nodes =
      {
        //
        //  Use substitution variable on left-hand side
        //  of an assignment in a quoted string
        //
        "${NODE_NAME}" = { ... }
      }
  }
}

Default Substitution Variable Value Example

The following example shows a configuration file that specifies a default for its substitution variable using the colon-hyphen (:-) syntax.

Note

When using the HOCON Editor in StreamBase Studio, substitution variables MUST be specified with default values in order to pass Studio typechecking. Nevertheless, configuration files without default values can be successfully run from the epadmin command line.

configuration =
{
  ApplicationDefinition =
  {
    execution
    {
      dataTransport =
        {
          //
          //    Default value of 10 if not specified
          // 
          nodeActiveTimeoutSeconds = ${TIME_OUT_VALUE:-10}
        }
     }
  }
}

Escaping Characters

Per the standard HOCON specification, the \ escape character is used to escape the next character in the file. In all cases, except one, the escape character is significant and must follow the standard HOCON parsing rules. The one exception is escape characters in front of substitution variable definitions are ignored. For example:

configuration = 
{
    name = \${value}
}

//
//  Processed as (escape character stripped)
//
configuration = 
{
    name = ${value}
}

To escape the $ character that starts a substitution variable definition use:

configuration = 
{
    name = \\\${value}
}

//
//  Processed as (substitution variable escaped)
//
configuration = 
{
    name = \${value}
}

Data Unit Syntax

The values in configuration objects follow the standard HOCON syntax, including support for unit formatting (for example: ms, bytes, MB, and so on). The complete list of the standard supported unit formatting can be found here.

Date Syntax

The supported date formats for date fields are:

  • ISO8601 combined date and time — yyyy-MM-dd'T'HH:mm:ss

  • ISO8601 with milliseconds — yyyy-MM-dd'T'HH:mm:ss.SSS

  • Time only — HH:mm:ss.

Example:

//   Date value with timezone
//
dateValue = "1970-01-30T12:34:00"

//
//    Date value with milliseconds
//
dateValue = "1970-01-30T12:34:00.100"

//
//   Time-only date value
//
dateValue = "12:34:56"

Configuration File Includes

HOCON configuration files can be embedded in another configuration file. The following example shows a complete node configuration file defined by its header lines at the top, that embeds an entire sbclientapilistener configuration file starting just below the fragmentIdentifier keyword. The embedded configuration file is bounded by a pair of triple quotes (""") within a pair of square brackets.

(The embedded type= line is broken artificially into two lines for clarity, but that must be a single long line in practice.)

name = "NodeDeployment"
version = "1.0"
type = "com.tibco.ep.dtm.configuration.node"
configuration = {
  NodeDeploy = {
    nodes = {
      "B.sbuser" = {
        engines = {
          engine1 = {
            fragmentIdentifier = "com.example.sample_operator"
            configuration = 
            [
              """
              name = "myapilistenersettings"
              version = "1.0.0"
              type = "com.tibco.ep.streambase.configuration.
                      sbclientapilistener"
              configuration = {
                ClientAPIListener = {
                  apiListenerAddress = {
                    portNumber = 10000
                  }
                }
              }
              """
            ]
          }
        }
      }
    }
  }
}

HOCON Configuration File Editor

The HOCON Configuration File Editor is a validating HOCON editor that is aware of the structure that defines the HOCON syntax of StreamBase configuration files.

See HOCON Configuration File Editor for instructions on using the Editor.