StreamBase New and Noteworthy

Date: 22 May 2017

This topic summarizes the history of significant changes in TIBCO StreamBase® releases. For current and resolved product limitations, see the StreamBase Release Notes.

New in StreamBase 10.1.1

TIBCO StreamBase® 10.1.1 fixes several product limitations, as described in the Resolved Limitations and Known Issues sections of the Release Notes.

New Features

TIBCO StreamBase® 10.1.1 adds the following updates and new features:

Studio Menu Option to Create Additional Package Names Added

Studio now includes an option in the FileNew menu to create additional package names in the currently selected Maven-structured StreamBase or Live Datamart fragment project. The selected project must contain a src/main/eventflow folder, or the invoking the menu option which you can use to organize similar file types, for example. New package names must reside in your project folder's src/main/eventflow directory.

Deployment Parameters Support Added to Configurations

Deployment command line argument options were added to Studio's launch configurations for EventFlow, LiveView, and Java. The enhancements include changes to the Run Configuration, Debug Configuration, and Trace Configuration dialogs.

Upgrade Wizard Enhanced to Prevent Name Collision

The Upgrade Wizard now warns about fully qualified name collisions and renames clashing files. For more information, see Upgrading Legacy Projects to StreamBase 10.

Pure Maven Project Import Supported

If you saved a Maven EventFlow or LiveView project in Maven-only format without Eclipse artifacts such as .project or .classpath files, you can now import such projects using FileImportMavenExisting Maven Projects. Studio configures such projects in standard StreamBase EventFlow or LiveView fragment project format.

Projects exported with FileExportGeneralArchive File should still be imported with FileImportGeneralExisting Projects into Workspace.

New OSI PI Adapter

The OSI PI adapter suite now includes an Event Frames adapter which allows applications to access Event Frames on Asset Framework servers. The OSI PI adapters are described in OSIsoft PI Adapters and their use is demonstrated in OSI PI Adapter Sample.

New TERR Operators

The general purpose TERR operator was replaced with a new version that has upgraded functionality. The previous TERR operator is still present for compatibility with your existing modules, but its use is now discouraged. The previous TERR operator is now labeled TERR Legacy in the Insert Operator or Adapter dialog. TIBCO recommends upgrading your modules to use the new TERR operator.

This release also provides a new TERR Predict operator, which implements a locked down predict-only instance of TERR functionality. This operator is used to load RDS files and perform predict operations on them.

The TERR operators are described in Using the TERR Operator and Using the TERR Predict Operator. The TERR Sample provides modules that illustrate both new operators.

New WITSML Adapters

Four new adapters were added to the WITSML adapter suite. These are:

  • Three WITSML Trajectory adapters, which read, write, and delete trajectory records, respectively.

  • WITSML Log Data Convert adapter, which takes in data read from the Read Log adapter and converts the comma-separated strings to data tuples.

For information about their use, see Using the WITSML Operators.

Deprecations

Container Connections Over JMS Deprecated

Support for making EventFlow module container connections with the intermediary of a JMS or EMS bus is now deprecated, and will be removed in a future release.

New in StreamBase 10.1.0

TIBCO StreamBase® 10.1.0 fixes several product limitations, as described in the Resolved Limitations and Known Issues sections of the Release Notes.

New Features

TIBCO StreamBase® 10.1.0 adds the following updates and new features:

Project Upgrade Wizard Added

Every time StreamBase Studio starts, the current workspace is scanned for projects in the non-Maven format of a previous release. If any older format projects are found, an upgrade wizard prompts with a dialog like the following that offers to upgrade the project. For more information about its use, see Upgrading Legacy Projects to StreamBase 10.

Run Configuration Includes Option to Select Random Port

EventFlow and LiveView run configuration settings now include the option to select a random port. The Main tab of the Run Configurations dialog adds a third radio button, labeled Any available, in addition to the existing Configuration or default and Specified radio buttons.

HOCON Elements Relocated

The StreamBaseEngine HOCON file type's modules and containerConnections properties are now deprecated. A new configuration class, EventFlowDeployments, contains those properties, with identical documentation. See Reference for StreamBase Engine Configuration for more information.

Configuration File Support for Live Datamart

As part of StreamBase 10's re-introduction of Live Datamart in 10.1.0, the StreamBase HOCON editor now supports three Live Datamart-specific HOCON file types, as follows:

Eclipse Upgraded

Eclipse 4.6.2 is now the basis for TIBCO StreamBase Studio™. If you install the StreamBase Studio plug-ins into your own installation of Eclipse, version 4.6.2 is now the recommended starting point.

Typecheck Errors View Removed

The Typecheck Errors view was removed from Studio. Instead, use the Problems view or problem-specific dialogs when errors appear in your application or configuration.

HDFS File Delete Adapter Removed

The HDFS Delete adapter, deprecated in a previous release, was removed from Studio.

New in StreamBase 10.0.1

TIBCO StreamBase® 10.0.1 fixes several product limitations, as described in the Resolved Limitations and Known Issues sections of the Release Notes.

New Features

TIBCO StreamBase® 10.0.1 adds the following updates and new features:

Syncing Project POM files to Current Release

When opening a project created in a previous release, StreamBase Studio offers to update the project's POM file to refer to the currently installed product artifacts. These updates are available as Quick Fixes. By default, Studio runs all relevant Quick Fixes for you, or you can choose to run them at a later time from the Problems view.

JVM Options for Fragment Launching Supported

A VM arguments text box was added to the Advanced tab of the EventFlow Fragment launch configuration type's view, so that you can specify JVM options to use when deploying such fragments. Quoting can be used to include spaces in arguments, and backslashes can be used to escape quotes. The text box's tooltip shows how your text is parsed into the sequence of arguments.

Ability to Specify Default Debug Port Added

A new user preference allows you to select the number of the port used when an EventFlow fragment is launched in debug mode. In debug mode, an additional socket connection is required. The default port number used for that socket connection is 8000.

Specify the default debug port for all fragment launches using WindowPreferencesStreamBase StudioTest/Debug. You can also specify the debug port number for an individual fragment launch using the Main tab of an EventFlow Fragment launch configuration.

Log Level Setting in Run Configurations Added

The Run Configurations dialog, Advanced tab, now includes a configurable log level setting for a particular fragment launch. The higher the number you specify, the more messages and message types are sent to the Console view or to an output log file.

New Studio Demo Added

A new StreamBase Studio demo was added to the SB Demos perspective. The Financial - Transactions demo is a simple credit-debit application that makes use of a new feature available only in StreamBase 10, the transactionality and rollback available to StreamBase Query Tables.

Clusters View Remove Node Buttons Behavior Change

When you select the Forceful Remove button, if the node is not in the started state, the button deletes the node. If the node IS in the started state, then a dialog more explicitly explains that using the button does NOT gracefully shut down any running fragments on the node.

When you select the Remove button and your node is not in the started state, nothing occurs visibly. That is, StreamBase Studio executes the corresponding epadmin command while a dialog is displayed. However, when the node IS in the started state, the action invoked by the button depends on whether Studio deployed the current fragment (as opposed to an epadmin command line deployment). The resulting dialog more explicitly explains the action Studio takes, depending on your deployment scenario.

Duplicate File Basenames Prevented at Runtime

To prevent a potential runtime error, two resources with the same file basename with different extensions are no longer allowed, and this is enforced at runtime. For example, abcde.sbapp and abcde.sbint in the same project are not allowed.

Configurations Folder Relocated

In the Project Explorer view, the src/test/configurations folder is now displayed as a top-level item. Previously, you had to navigate through the src entry to locate this folder.

EventFlow Editor's Metadata Tab Labeling Change

The top-level panel of the Metadata tab was previously labeled Application Description. Now it is labeled Module Description, to more accurately describe its intended contents.

Studio File Menu Labeling Change

The FileNewStreamBase EventFlow™ Application menu changed to StreamBase EventFlow™ Module to more accurately describe the intended file creation type.

Jetty Upgraded

In StreamBase Studio, the Jetty web server version was upgraded to 9.3.14.v20161028. The upgrade applies to, for example, the TIBCO StreamBase® Web Server Request adapter and the TIBCO StreamBase® Web Server Response adapter, as well as for developing your own applications that require Jetty.

ActiveSpaces Reference Version for ActiveSpaces Operators

TIBCO ActiveSpaces release 2.2.0.017 is now the default version of TIBCO ActiveSpaces used by the TIBCO ActiveSpaces operators.

Adapters Deprecated

The following adapters are deprecated and removed from this release:

  • The Legacy EMS and JMS Input and Output Adapters and their samples. For EMS or JMS connectivity, use the current versions as described in TIBCO EMS Adapter and JMS Adapter, respectively.

  • The Legacy ActiveSpaces operator and its sample. For the current ActiveSpaces operator, see Using the TIBCO ActiveSpaces Operators.

  • The Drools operator.

sbconfutil Command Deprecated

As of 10.0.1, the sbconfutil command is deprecated and removed.

sbunit Command Deprecated

As of 10.0.1, the sbunit command is deprecated and removed.

New in StreamBase 10.0.0

TIBCO StreamBase® 10 is a new product milestone with new features. Like previous releases, StreamBase 10 preserves the same easy-to-use programming interface that allows developers to quickly assemble the business logic of a stream analysis application with drag and drop efficiency. The StreamBase editing environment allows you to develop and test event processing applications much faster than when using a traditional programming language.

At the same time, StreamBase 10 now runs on a new StreamBase Deployment Runtime, which is based on TIBCO's Distributed Transactional Memory platform (DTM) platform. The StreamBase Runtime provides:

  • Transactions — high performance, distributed all-or-none, ACID-compliant processing of events as an option for StreamBase Query Tables.

  • In-Memory Durable Object Store — ultra low-latency transactional persistence.

  • Transactional High Availability — transparent memory-to-memory replication with instant failover and fallback.

  • Distributed Computing — location-transparent objects that allow transparent horizontal scaling.

  • Integrated Hotspot JVM — tightly integrated Java execution environment that allows transparent low latency execution.

New Features

TIBCO StreamBase® 10.0 has the following major features:

StreamBase Studio Now Uses Maven

StreamBase Studio projects are now in Maven project format, and all projects are built and run under Maven control. The Run As context menu now includes several Maven commands and goals.

The change to using Maven gives you a well-known, thoroughly documented, and standard way to specify inter-project and within-project dependencies. Maven automatically configures search paths for code in specified dependencies, replacing StreamBase 7's proprietary module search path feature. Maven also manages resource search paths automatically.

Execution With sbd Is Replaced with DTM-Based StreamBase Runtime

Once your StreamBase 7 project is ported to StreamBase 10 format, you can run or debug your project in Studio as before. Select the top-level EventFlow module in your project, right-click, and select Run AsEventFlow Fragment. This starts a single-cluster, single-engine node in which your EventFlow module runs.

The Manual Input view works as in previous StreamBase releases to send single-tuple events to your module. The Feed Simulation view works as before, if your project has one or more feed simulations ready to run.

To test your EventFlow Fragment running in the context of a distributed application, you must create a separate Deployment Application project in Studio, add your EventFlow Fragment's archive to that project, and then use the new epadmin command at the command prompt. This process is described in more detail in Deploying EventFlow Fragments. Documentation for the epadmin command is available from usage text such as epadmin help node, or from the full command reference page in the DTM documentation at this location.

Studio Now Provides a Clusters View

The Clusters View shows the nodes present in the current DTM cluster. By default, the Clusters view shows you only the nodes in your personal cluster, which is your system username by default. You can toggle on or off a view of all clusters in the DTM fabric, which is configured to be the local IP subnet by default. In this alternate view, you can see nodes that you started as well as nodes from other local users.

A DTM node is a container for a fragment, which runs on an engine. When you run your EventFlow Fragment in Studio, Studio configures and starts a node named by default A.username, which means Studio's default node name is A, and its default cluster name is your current logged-in user name. You can change these defaults in Studio Preferences, StreamBase StudioCluster.

See StreamBase Runtime Introduction for more on DTM terminology.

New Configuration File Format

StreamBase 10 introduces a new file format, HOCON, for specifying configuration information. With one exception, the XML-based StreamBase configuration file format of previous releases is no longer supported. StreamBase Studio provides a new HOCON Configuration Editor that offers syntax coloring, command completion, and other convenience features.

HOCON stands for Human-Optimized Configuration Object Notation, and has a formal specification. HOCON is a superset of the JSON array syntax.

Each HOCON configuration file is completely defined by three lines that must be at the top of the file. Each HOCON file is uniquely identifiable from others by these lines, which allows two or more configuration files to reside in the same folder.

HOCON configuration files must have the .conf extension. Place HOCON configuration files in the src/main/configurations folder of your Studio project. Use the HOCON Configuration Guide to learn the syntax of each file type.

The exception to using HOCON format files is that certain adapters rely on adapter configuration files in XML syntax. Support for configuration files whose top level is <adapter-configurations> are still supported. Place such files in the src/main/resources folder of your Studio project.

Development Environment Changes
  • Your command-line development environment must include the path to a system-installed JDK, in addition to the JDK bundled with StreamBase Studio. The system JDK is for running the epadmin command and its subcommands, as described in Installing a Command Line JDK.

  • Your shell environment must include the environment variable TIBCO_EP_HOME set to the same location as STREAMBASE_HOME.

  • In addition to STREAMBASE_HOME/bin, your PATH must include STREAMBASE_HOME/distrib/tibco/bin.

  • On Windows, always use a StreamBase Command Prompt, which sets the above environment and PATH settings for you.

  • On macOS and Linux, set up your shell environment to call the sb-config --env command inside of back quotes as described in the Post-Installation Setup sections of the Installation Guide pages for those operating systems. Use the full path to the sb-config command, such as in the following example for macOS:

    `/Users/sbuser/Application/TIBCO StreamBase 10.0.0/bin/sb-config --env`

    And the following example for Linux:

    `/opt/tibco/sb-cep/10.0/bin/sb-config --env`

Documentation Available

The documentation for StreamBase 10 is extensive, but incomplete. The following books in the documentation set are completely updated for StreamBase 10:

The other books in the StreamBase documentation have partial updates, but should be presumed to primarily describe StreamBase 7 features. In some cases, such as the Authoring Guide, StreamBase 7 and StreamBase 10 features are largely the same. Each book has a note on its home page describing the update level of that book.

Documentation for the StreamBase Runtime is provided separately at this URL.

If You Are Familiar With StreamBase 7

If you have developed with StreamBase 7, pay attention to the following differences.

Start With a New, Unused Studio Workspace

Do not try to open a StreamBase 7 Studio workspace in Studio 10. There is no conversion process provided to convert your projects into the newly required Maven project format. Instead, open Studio 10 with a new, unused workspace and port your projects one by one, following the guidance in Project Upgrade Reference.

First Run Takes Several Minutes

The first time you open and run a new or ported project in StreamBase 10, Studio follows the instructions in the Maven pom.xml file and downloads all the necessary JAR files from various external resources. This can take several minutes.

First Open of a Large Project Takes Time to Build

When you first open a large project, such as the Operator Sample Group taken from the Load StreamBase Sample dialog, the project opens with red marks, indicating typecheck errors. Notice, however, that the bottom right corner of Studio shows the message Building Workspace.

Let this build operation complete and the typecheck error marks will resolve.

Known Issues

The following issues are present at the time of the initial StreamBase 10 release.

Live Datamart Not Yet Supported

StreamBase 10 does not yet support running Live Datamart programs as fragments under the new StreamBase Runtime management.

Many Samples Not Yet Converted

Many of the samples loadable from the FileLoad StreamBase Sample dialog are not yet converted to the Maven project format. When you load an unconverted sample, Upgrade Wizard offers to convert the sample for you. To determine whether a sample has been converted, look for "SB10" at the beginning of its description field. To reduce the dialog to the list of only converted samples, type "SB10" in the search field.

Component Exchange Projects Not Converted

All StreamBase and Live Datamart projects on the StreamBase Component Exchange remain in StreamBase 7 project format. When you download and install these components, convert them to StreamBase 10 format using the Upgrade Wizard.

Windows 10 Start Menu Issues

StreamBase has long supported the installation of more than one major StreamBase release on the same Windows machine, and still does. However, a change in the way Windows 10 supports Start Menu folders makes all StreamBase releases appear together in a single TIBCO folder, without distinction between releases. Future releases of StreamBase are expected to correct for this Windows 10 change, but cannot do so retroactively for currently installed releases.

EventFlow Debugger Limitations

The EventFlow Debugger works as expected for a single module EventFlow fragment, but fails to navigate to breakpoints set in submodules.

StreamBase Unit Test Feature Not Working

The Run AsStreamBase Unit Test feature is not yet implemented. You can instead run Java Junit tests using the Maven Test feature, which reports its results in the Console view.

Runtime Performance Issues on Windows

The start time when running EventFlow fragment launches on supported Windows platforms is decreased from previous StreamBase releases, and from the same fragment on other supported operating systems.

Limited Adapter Support

Not all adapters have been updated or tested with StreamBase 10.

Deprecated and Removed Features

The following features present in previous StreamBase releases are either deprecated or removed from StreamBase 10.

Trace Debugging

The trace debugging feature of StreamBase Studio is not supported in StreamBase 10, and the SB-Trace Debugger perspective was removed from Studio. Trace debugging is still supported at the command line using new HOCON file settings to enable trace files.

Zing ZDK

StreamBase 10 does not support using a Zing ZDK as an alternative to the provided Oracle JDK. Zing ZDK support may return in a future edition of StreamBase 10.

Running StreamBase Runtime as a Windows Service

StreamBase 10 does not support running the StreamBase Runtime as a Windows service. The StreamBase 10 Windows installer no longer places initial entries in the Windows registry to prepare for service running.

Running StreamBase Runtime at Linux Start-up

StreamBase 10 does not support being automatically started as part of the boot process of a Linux-based server. The StreamBase 10 Linux installer no longer provides /etc/init.d commands for this purpose.

Remote Attachment for Debugging

StreamBase Studio still has support for debugging your EventFlow modules while running them in Studio. However, the StreamBase Runtime no longer supports the remote attachment feature that lets you debug a running EventFlow module from another machine on your network.

Security Tagging

StreamBase 10 does not support the security tagging feature of previous releases.

Certain Command-Line Tools Not Included

StreamBase 10 removes the sbd, sbargen and sbbundle commands. The sbconfutil and sbfeedsim-old commands are still present, but do not run. (The functionality of sbd --eval is expected to be replaced with a new command in a later release.)

Troubleshooting

Run Maven Clean

If you encounter any issues with starting and running EventFlow Fragments, try selecting the project name in the Project Explorer, right-clicking, and selecting Run AsMaven Clean.

Run Maven > Update Project

For projects that show persistent Maven build issues, select your EventFlow project's name in the Project Explorer view, right-click, and select MavenUpdate Project.

Remove ~/.m2 to Start Completely Over

To start from scratch with a new installation of StreamBase 10, remove the com/tibco portion of your local Maven repository before reinstalling StreamBase 10. By default, your local Maven repository is in the .m2 folder of your home directory. For macOS and Linux users, you can run:

rm -rf ~/.m2/repositories/com/tibco

On Windows, use a file manager window to locate the .m2 folder, which is written by default at the top of your /Users/loginname folder.