epadmin-container

< Previous		Next >

epadmin-container

epadmin container target — Manages StreamBase containers in an EventFlow module.

SYNOPSIS

epadmin [globalparameters] command container [commandparameters]

DESCRIPTION

Use the container target to manage StreamBase containers, which are structures that encapsulate and name a single EventFlow module, complete with Input Stream or adapter entry points, and Output Stream or adapter exit points. There is one EventFlow module per container, and each container's name serves as a handle for that module's resources. Multiple containers and modules can be run in a single EventFlow or LiveView JVM engine process, and you can share resources between containers. Further information is in the StreamBase Container Overview in the StreamBase Administration Guide.

The StreamBase container is a legacy structure that is not cluster aware; you must address containers with StreamBase path notation. Changes to a container in one node are not propagated to other nodes.

Containers and optional container connections are defined with the EventFlowDeployment root object in a configuration file of type sbengine. The defined container arrangement starts when the node containing the EventFlow fragment starts and continues as long as the node runs. Under normal circumstances, StreamBase containers are part of the architecture of the StreamBase application, and do not need maintenance. Thus, the commands in the epadmin container target (and in the legacy sbadmin command) are usually needed only for administrative, debugging, troubleshooting, or development purposes.

For those purposes, containers can be suspended and resumed, restarted, or removed entirely, independent of other containers in the same engine process. You can interactively add or modify containers, or connect and disconnect container connections. However, changes made at runtime are not persisted in the module's configuration.

If your application architecture uses a StreamBase container to isolate the operation of an adapter's communication with an external data source, consider instead using the cluster awareness feature of operators and adapters to let them start and stop based on cluster conditions.

The container target has nine commands:

The designation [sn|ad] in syntax examples is a reminder that this command requires either the global parameter ‑‑servicename or the ‑‑adminport and ‑‑hostname combination to identify the node or cluster of interest.

Suspending and Resuming

You can use epadmin suspend container to request suspension of a container within a running EventFlow module. Streams in the container ignore the suspend state and instead respond to enqueue and dequeue STREAMSTATUS as set by epadmin modify container --enqueue and --dequeue commands, described below. The container suspend state is a flag to which custom operators and adapters within the container may respond, depending on how they are coded. Suspending a container applies the suspend state to all operators and adapters in that container, but if there are none in the container that have a specific suspend state defined, then the application behavior does not change when receiving the suspend command. Restore suspended containers with the epadmin resume container command.

Shutting Down and Restarting Containers

The epadmin remove container commands are equivalent, and you can use either one.

Removing a container stops the execution of all operators in the container as soon as the current tuple and any pending JDBC operations complete processing. The current tuple only needs to enter an output queue to be considered complete. The shutdown process then unloads all operators, queues, and connections from memory. Once shut down, a container is no longer present in the engine and cannot be restarted or reloaded with an application module. You can, however, add a new container with the same name.

If you are shutting down a container that has a container connection, remove the containers using the reverse order in which they were added to the server.

Using the epadmin restart container command, you can restart a container without recompiling or reloading its EventFlow module. Doing so eliminates any in-memory state and re-initializes any container connections. Restarting a container is more efficient than removing the container and re-adding it, which makes this command useful for testing.

Container Connections

You can set up connections between containers in initial configuration or when you interactively add a container with the epadmin add container command. Container connections can be one of the following types:

Stream-to-stream connections, where an output stream in one container connects to an input stream in another container. These can be asynchronous (default) or synchronous connections. The two streams being connected must have exactly matching schemas, except that numeric fields can be coerced to match.

CSV file connections, where an input or output stream in a container is connected to a URI specifying the path to a CSV file.

The syntax for expressing both forms of container connections is always destination=source, like an assignment statement in some programming languages. See Container Connections in the StreamBase Administration Guide for more on this feature.

Runtime Tuple Tracing

When enabled, runtime tracing writes tuple trace information to standard output or to a trace file, one file per container. The default trace information is a single timestamped line per tuple. Each tuple is shown as received at each operator and stream in the application. Trace files can grow quite large very quickly, so runtime tracing is best used only for short bursts, for debugging only, to follow the progress of a single tuple or group of related tuples through an application.

You can limit the traced operators and streams by using a regular expression to name the components of interest.

You must set up for runtime tracing by specifying the system property streambase.codegen.trace-tuples=true in a configuration file. Once set, you enable and disable tracing using epadmin add container or epadmin modify container, or using the configuration elements in a HOCON file of type sbengine. See Interpreting Trace Files for more on setting up for and using the runtime tuple tracing feature.

epadmin add container

Adds or re-adds a StreamBase container to a running engine, optionally specifying container connections and/or stream status, and optionally enabling or disabling runtime tuple tracing. Containers can be added in initially suspended state.

epadmin [sn|ad] add container --module=mymodule.sbapp --name=myContainer \
  --connections="myContainer.anotherModule.inputStream=default.myModule.outputStream"
epadmin [sn|ad] add container --engine=JVMengine1 --module=mymodule.sbapp \
  --name=myContainer --dequeue=disabled --enqueue=droptuples --suspend
epadmin [sn|ad] add container --module=mymodule.sbapp --name=myContainer \
  --moduleparameters="a=b,c=d" --modulesearchpaths="module-directory" \
  --resourcesearchpaths="resources-directory"
epadmin [sn|ad] add container --module=mymodule.sbapp --name=myContainer \
  --trace --tracebuffered=false --tracecompress --tracefilebase="trace_" \
  --traceoverwrite --tracestreampattern="value=APPL"

Command Parameter	Description	Required
`connections`	Comma-separated list of cross-container connections to establish. Cross-container connections are specified as `input-stream-path=output-stream-path`.	No. No default.
`dequeue`	Dequeue status for the container being added, one of: `enabled`, `disabled`, or `droptuples`.	No. Default value is `enabled`.
`engine`	Engine name filter.	No. The default is all engines.
`enqueue`	Enqueue status for the container being added, one of: `enabled`, `disabled`, or `droptuples`.	No. Default value is `enabled`.
`module`	Operating system path to the file containing the EventFlow module to run in the container.	Yes.
`moduleparameters`	Comma-separated list of container parameters, specified as `name`=`value` pairs.	No. No default.
`modulesearchpaths`	Comma-separated list of module search paths to be evaluated on the node machine. These paths are determined by Maven at development time and as part of the node directory structure at runtime. Specify either a full, absolute path or one relative to the node directory.	No. No default.
`name`	Container name.	Yes.
`resourcesearchpaths`	Comma-separated list of resource search paths to be evaluated on the node machine. At development time, resources are found in `src/main/resources` (and/or `src/test/resources`), and at runtime in the node directory. Specify either a full, absolute path or one relative to the node directory.	No. No default.
`suspend`	Boolean value to specify whether the container starts suspended or not.	No. Default value is `false`.
`trace`	Boolean value to enable runtime tuple tracing. A value of `true` enables runtime tracing.	No. Default value is `false`.
`tracebuffered`	Boolean value controlling whether trace output is buffered. A value of `true` buffers the trace output for improved performance, with the trade-off that trace output lags module execution and may be lost on a hard failure.	No. Default value is `true`.
`tracecompress`	Boolean value controlling trace file compression. A value of `true` causes the trace files to be compressed.	No. Default value is `false`.
`tracefilebase`	String value specifying the base name for trace files. Setting this parameter to a value causes trace output to be redirected to files. There is one trace file generated per module, including one trace file for each parallel region in a module. Trace file names are generated as `tracefilebase+containername.sbtrace`. For modules running with parallel execution enabled, the module's path becomes part of the trace file name, such as `tracefilebase+containername.modulepath.sbtrace`. Tracing also creates a metadata file with extension `.sbtrace.meta`.	No. The default is that trace output is written to standard output.
`traceoverwrite`	Boolean value controlling whether trace files are overwritten on engine restart. A value of `true` causes the trace files to be overwritten.	No. Default value is `false`.
`tracestreampattern`	Enable stream tracing using the specified pattern as a filter, using `java.util.regex.Pattern` syntax for the pattern.	No. No default.

epadmin connect container

Establishes cross-container connections, which must be between containers in the same engine. The two streams being connected must have exactly matching schemas, except that numeric fields can be coerced to match. The syntax for container connections is always inputStream=outputStream.

epadmin [sn|ad] connect container 
  --connections="myContainer.anotherModule.inputStream=default.myModule.outputStream" 
  --where="value=5"
epadmin [sn|ad] connect container --engine=JVMengine1 
  --connections="myContainer.anotherModule.inputStream=default.myModule.outputStream"

Command Parameter	Description	Required
`connections`	Comma-separated list of container connections to establish. Container connections are specified `inputStreamPath=outputStreamPath`, using StreamBase path notation.	Yes.
`engine`	Engine name filter.	No. Default all containers on all engines.
`where`	Specifies a predicate expression that filters tuples sent over the connections being established. The predicate expression must resolve to true or false, and should match against one or more fields in the schema of the connection's stream. Tuples are only sent over the connection if they match the predicate with a value of true.	No. No default.

epadmin disconnect container

Disconnects one or more container connections.

epadmin [sn|ad] disconnect container 
 --connections="myContainer.anotherModule.inputStream=default.myModule.outputStream"
epadmin [sn|ad] disconnect container --engine=JVMengine1 
 --connections="myContainer.anotherModule.inputStream=default.myModule.outputStream"

Command Parameter	Description	Required
`connections`	Comma-separated list of container connections to establish. Container connections are specified `inputStreamPath=outputStreamPath`, using StreamBase path notation.	Yes.
`engine`	Engine name filter.	No. Default all containers on all engines.

epadmin display container

Displays container information for the specified node, one line for each setting.

epadmin [sn|ad] display container
epadmin [sn|ad] display container --detailed
epadmin [sn|ad] display container --engine=JVMengine1
epadmin [sn|ad] display container --name=myContainer

Example output:

Engine = com_tibco_streaming_example_til_Main1
Name = default
Type = NORMAL
Enqueue = ENABLED
Dequeue = ENABLED
State = RUNNING
Main Module = com.tibco.streaming.example.til.Main
Data Distribution Policy = 
Availability Zone = 
Partition = 
Backup Nodes = 
Input Streams = MainIn
Output Streams = MainIn
Trace Status = Disabled

Command Parameter	Description	Required
`detailed`	A boolean value controlling whether to display details about a container. A value of `true` adds lines about runtime tracing settings.	No. The default is `false`.
`engine`	Engine name filter.	No. The default is to display all containers on all engines.
`name`	Container name filter.	No. The default is to display all containers.

epadmin modify container

Modifies settings for a running container. This can be useful to enable or disable queuing for a stream, or to start, modify, or stop runtime tuple tracing.

epadmin [sn|ad] modify container --engine=JVMengine1 --name=myContainer \
 --dequeue=disabled --enqueue=droptuples
epadmin [sn|ad] modify container --name=myContainer --trace --tracebuffered=false \
 --tracecompress --tracefilebase=trace-directory --traceoverwrite \
 --tracestreampattern="value=APPL"

Command Parameter	Description	Required
`dequeue`	Specifies dequeue status for the container, one of: `enabled`, `disabled`, or `droptuples`.	No. Default value is `enabled`.
`engine`	Engine name filter.	No. The default is all engines.
`enqueue`	Specifies enqueue status for the container, one of: `enabled`, `disabled`, or `droptuples`.	No. Default value is `enabled`.
`name`	Container name filter.	No. The default is to is all containers.
`trace`	Boolean value to enable runtime tracing. A value of `true` enables runtime tracing.	No. Default value is `false`.
`tracebuffered`	Boolean value controlling whether trace output is buffered. A value of `true` buffers the trace output for improved performance with the trade-off that trace output may be lost on a hard failure.	No. Default value is `true`.
`tracecompress`	Boolean value controlling trace file compression. A value of `true` causes the trace files to be compressed.	No. Default value is `false`.
`tracefilebase`	String value specifying the base name for trace files. Setting this parameter to a value causes trace output to be redirected to files. There is one trace file generated per module, including one trace file for each parallel region in a module. Trace file names are generated as `tracefilebase+containername.sbtrace`. For modules running with parallel execution enabled, the module's path becomes part of the trace file name, such as `tracefilebase+containername.modulepath.sbtrace`. Tracing also creates a metadata file with extension `.sbtrace.meta`.	No. The default is that trace output is written to standard output.
`traceoverwrite`	Boolean value controlling whether trace files are overwritten on engine restart. A value of `true` causes the trace files to be overwritten.	No. Default value is `false`.
`tracestreampattern`	Enables stream tracing using the specified pattern as a filter, using `java.util.regex.Pattern` syntax.	No. No default.

epadmin remove container

Removes a container from a running engine. Containers so removed cannot be restarted or restored while the containing engine is running.

epadmin [sn|ad] remove container
epadmin [sn|ad] remove container --engine=JVMengine1
epadmin [sn|ad] remove container --name=myContainer

Command Parameter	Description	Required
`engine`	Engine name filter.	No. The default is to remove all containers on all engines.
`name`	Container name filter.	No. The default is to remove all containers.

epadmin restart container

Restarts the specified containers. Restarting eliminates any in-memory state and re-initializes any container connections. Restarting a container is more efficient than removing the container and re-adding it, which makes this command useful for testing.

epadmin [sn|ad] restart container
epadmin [sn|ad] restart container --engine=JVMengine1
epadmin [sn|ad] restart container --name=myContainer

Command Parameter	Description	Required
`engine`	Engine name filter.	No. The default is to restart all containers on all engines.
`name`	Container name filter.	No. The default is to restart all containers.

epadmin resume container

Resumes a suspended container.

epadmin [sn|ad] resume container
epadmin [sn|ad] resume container --engine=JVMengine1
epadmin [sn|ad] resume container --name=myContainer

Command Parameter	Description	Required
`engine`	Engine name filter.	No. The default is to resume all containers on all engines.
`name`	Container name filter.	No. The default is to resume all containers.

epadmin suspend container

Suspends a running container. Suspending a container applies the suspend state to all operators and adapters in that container, but if there are none in the container that have a specific suspend state defined, then the application behavior does not change when receiving the suspend command.

epadmin [sn|ad] suspend container
epadmin [sn|ad] suspend container --engine=JVMengine1
epadmin [sn|ad] suspend container --name=myContainer

Command Parameter	Description	Required
`engine`	Engine name filter.	No. The default is to suspend all containers on all engines.
`name`	Container name filter.	No. The default is to suspend all containers.