Streaming 11.1.0 Release Notes

These were the release notes for the earlier versions of Spotfire Streaming 11.1.0.

New Features

Spotfire® Streaming 11.1.0 adds the following updates and new features:

TIBCO Streaming is Now Spotfire Streaming

This rebrand has no impact on your licenses, support, or existing functionality (including TIBCO integrations); it reflects our commitment to building rich, real-time capable analytical experiences with Spotfire. Please reach out to your account representative for more information.

Spotfire Streaming Introduces Data Streams for Cloud

Spotfire Streaming introduces a new feature called Data Streams for Cloud, a separately installable web service that allows you to create LiveView Tables (referred to as Data Streams) in private, public, or hybrid cloud environments from your web browser. Data Streams are managed windows of event data from a streaming data source backed by Liveview, that directly becomes available for analysis in Spotfire, or other clients using new REST interfaces.

Install Format and Name Changes

Streaming products from Cloud Software Group are now part of the Spotfire product line, and to reflect the change, the installer files on eDelivery are now prefixed with SPOT_ instead of TIB_.

The archive files from the unpack action of the archive-bin installers for Streaming products are now in XZ (LZMA) format, and so have the file extension tar.xz. The tar xf (extract) command determines the compression type from the file name, so switches like J (LZMA) and z (gzip) are no longer needed when extracting an installation from them.

Program names and identifiers for Streaming are unchanged, and the default install locations follow the same pattern of previous releases.

Spotfire Streaming Bundles Maven 3.9.4

Spotfire Streaming 11.1.0 includes a standalone installation of Maven 3.9.4 for customer command-line use as described in the Maven entry of the Supported Configurations page.

Added a New Field in sbprofile Queue Report

A new field named Region Load has been added in the sbprofile queue report. The Region Load represents the percentage of the time a region was processing tuples since the last profiler report. For more information, see Queue Statistics().

Added Two New Overloads to the parse_time() Function

The parse_time() function now has two new overloads that introduce a lenient argument. For more information, see parse_time().

Windows Server 2022 Support for Third-Party Applications

Spotfire® Streaming has added support of Windows Server 2022 for third-party applications.

Extension Points with No Modules Configured

If you do not configure any modules to an extension point, StreamBase generates a warning message to remind you to utilize the StreamBase’s extensibility by configuring the necessary modules

Spotfire Streaming Supports Debian Linux 11

Debian Linux is now a supported operating system for deployment.

Export to CSV Button Added to the Decision Table Editor

Added an Export to CSV... button to the Decision Table editor which exports the current Decision Table to a CSV file.

Text Encoding for the Code and Studio is Now UTF-8 Instead of US-ASCII

By default, the text encoding for the code and Studio is now UTF-8 instead of US-ASCII. As a result, a large number of character sets are now available to be utilized. Also, new lossless methods for the API's bytesToString and stringToBytes methods are available.

LDAP (Lightweight Directory Access Protocol) Read Adapter Includes All Read Results into a Single Output Tuple

LDAP read adapter now includes all the results into a single output tuple. This eliminates the need to perform aggregation outside of the adapter in the application to wait for all LDAP query results. For more information, see LDAP Read Adapter.

Spotfire Streaming Uses Eclipse 4.28

For this release, Spotfire Streaming uses Eclipse 4.28 (2023-06) instead of 4.21 (2021-09) as in the previous version.

Added SB Trace Debugger Perspective to StreamBase Studio

A new perspective was added to Studio to allow visual analysis of StreamBase trace files. For more information, see SB Trace Debugger Perspective.

StreamBase Studio Supports Semantic Interpretation in lvconf

You can now configure semantic interpretations in the lvconf editor in Studio instead of editing the lvconf XML directly like before. For more information, see Data Table.

Added Support for OAuth2 Client Credentials Authentication in the HTTP Client Adapter

HTTP Client Adapter now supports OAuth2 Client Credentials Authentication.

Manual Input View's Text Fields Handle Multi-line Text

You can now can have multiple lines in the Manual Input View's text fields instead of single line of text.

To insert a newline into a manual input field, press the Shift + Enter key combination. This inserts a newline without sending a tuple. Pressing Enter by itself sends a tuple as before.

Added an "Empty EventFlow Parent Packages" Filter

A new Empty EventFlow parent packages filter is added to the Project Explorer, which hides empty parent packages (packages that do not have anything in them besides other packages). Users can clear this filter to show empty parent packages.

StreamBase Studio has TIBCO Accelerator Discovery Wizard

The TIBCO Accelerator Discovery wizard is a new dialog similar to the StreamBase Component Exchange dialog, but pre-filtered to only show TIBCO Accelerator projects.

EventFlow Palette Search Includes Adapters and Operators

The EventFlow palette search bar in Spotfire Streaming 11.1.0 now searches for global operators and adapters. The Adapters, Java Operators palette item can still be used for searching as before.

Added Support for src/test/eventflow in Fragment Projects

Spotfire Streaming adds better Studio support for src/test/eventflow modules. A special folder now shows up in the project explorer where new packages and modules can be created. Unit tests containing modules in this location can be run from Spotfire StreamBase Studio.

Added Feature to Manage Fragment Dependencies

You may manage fragment dependencies either by editing the project's POM (pom.xml) as a standard Maven project, or, for a simpler interface that filters and narrows dependencies just to fragment dependencies, then use the new Manage Fragment Dependencies project property dialog. For more information, see Add a Dependency.

Information about the Primary and Backup Nodes

The display availabilityzone now provides Primary Nodes and Backup Nodes columns instead of a single Nodes column. This information is useful for dynamic data distribution policies.

The eFTL Adapter Converts String to Boolean

From this release, the eFTL adapter can now convert String to Boolean.

Added TCP_NODELAY Support to Bidirectional Socket Family Adapters

The Bidirectional Socket Writer Adapter now supports TCP_NODELAY for better performance.

Added Dialog to Install Non-Maven JARs into Local Maven Repository

There is a dialog that you can open to install non-Maven JARs into the local Maven repository instead of invoking Maven on the command line. For more information, see Using External Jars.

Added JAR Manifest Information to the Properties View

For all Java operators and adapters, a hyperlink is added to the General tab to view the JAR file's manifest headers. This can include the adapter version, Maven coordinates, and company name.

UMDF Adapter Uses New Version of FIX Antenna Java

From this release, UMDF adapter uses version 2.24.1 for FIX messages and and version 2.14.26 for FAST messages.

Added Discovery Mechanism to Work Cross-Namespace

The EventFlow pod installation now runs smoothly as the discovery mechanism is now available in member clusters and works cross-namespace.

Added Support to Search Java Files from Open StreamBase Resource Dialog

The Open StreamBase Resource dialog can now be used to search the workspace for Java files in addition to EventFlow files.

StreamBase Studio Supports epadmin Encryption

Studio now has a StreamBase Secrets Helper dialog to generate a master secret and substitution file for sensitive data encryption. The HOCON configuration editors have an option to encrypt sensitive data values using the default quick-fix key (Ctrl+1).

Added Support for Binary EventFlow Dependencies

Spotfire Streaming has added support for binary eventflow dependencies.

Added New Decision Table Operator Property

A new Decision Table operator property is added to override the Single Row Execution property that was previously only settable in the Decision Table file.

Added Support for OpenAPI 3.1 Schema

Spotfire Streaming adds OpenAPI 3.1 to validate OpenAPI schemas. This new feature introduces validation profiles and adds OpenAPI schema extensions. This feature can also be used to support any additional keywords required to represent EventFlow tuple schemas in JSON Schema format.

Support for Microsoft Visual Studio 2019 C++ Clients

Spotfire Streaming .NET APIs and native programs now require the Microsoft Visual Studio 2019 C++ runtime environment.

Eventflow and LiveView Fragments Use Random API Listener Port Numbers

By default, StreamBase's API listener port is any available port. The system picks a random, unused port number between 1025 and 65535. Similarly, LiveView's default API listener port is also set to any available port. But for LiveView, it means it starts looking for an unused port beginning from port 10080 and keep checking one port at a time until it finds an available one.

JDK 17 Requirement Update for Spotfire Streaming

Spotfire Streaming now requires Java 17 for development and deployment. If you attempt to deploy Streaming applications with Java 16 or earlier, applications fail to open. Container builds for Streaming applications must include JDK 17 instead of Java 11.

On all supported operating systems, the Spotfire Streaming installation kit provides the runtime components of Eclipse Adoptium JDK 17.0.8 (formerly known as AdoptOpenJDK) in a subdirectory of the Streaming installation directory, for a private use to run Streaming program fragments and applications from within StreamBase Studio. As before, Streaming detects and prefers a system-installed JDK, and uses the bundled JDK as a fallback. See further details about the JDK requirements of this release in the JDK entry of the Supported Configurations page.

Added Support for Shell Command Completion

An epadmin generate completion command is added in Spotfire Streaming 11.1.0. It generates a bash completion script for epadmin commands and parameters. See the command output for usage instructions.

Support for Shorter Global Parameter Names When Using epadmin Command

The epadmin now supports shorter global parameter names with single dash (e.g., -sc is a short parameter name for --schema). Check epadmin-globals for more information.

Spotfire Streaming 11.1 is an LTS Release

Spotfire Streaming 11.1 is designated as an LTS release.

The LVNodeInfo Shows Information about Multiple Table Groups

The LVNodeInfo table now shows which TableGroup a table belongs to.

Changes in Functionality

This release includes the following changes in functionality:

Single HOCON (Human-Optimized Configuration Object Notation ) Data Source Group File with All Data Sources

Previously, any JDBC data sources in a legacy SB7 project's sbd.sbconf file were made into a separate HOCON file for each data source. Now, each data source is added to a single JDBCDataSourceGroup type that permits multiple data sources in a single HOCON file.

Dropped Support For Red Hat 7 Compatibles on Linux

Support for CentOS Linux 7 and 8 is dropped as of Streaming 11.1.0. Since Red Hat earlier declared that CentOS Linux has reached end of life, there are no security updates. Also, support for Red Hat Enterprise Linux 7 and Oracle Linux 7 is dropped from this release onwards.

Content of streambase-client RPM moved to streambase-devel RPM

The rpm-bin installer no longer includes a streambase-client installer; its content has been moved to the streambase-devel RPM.

HTML Files Open in the External Browser

You can now open HTML files in your default external browser because of the changes in Eclipse 4.22. Sample HTML READMEs continues to open in StreamBase Studio when the sample is imported using the Open README when the sample is imported option. The sample READMEs written in Markdown and plain text open in StreamBase Studio as before.

Path of the OIDC Issuer URL is Used to Construct URL for the OIDC Provider's Configuration Document

The path of the OIDC Issuer URL is now used to construct URL for the OIDC provider's configuration document. For an OIDC issuer URL of https://example.com/tenant-id the URL downloads the following OIDC provider's configuration document https://example.com/tenant-id/.well-known/openid-configuration.

Change in Deleting a Module Package Name in StreamBase Studio

You can now delete a package in the same way as deleting a java package.

EventFlow Packages can be Renamed Properly

You can now rename the EventFlow packages. When a package is renamed, all references to it in workspace modules are updated as well.

Project Creation Wizard Supports Name Templates

When creating a new project, you can either enter a simple project name (current behavior) or select a Maven template.

You can either select a template from the predefined ones or type in their own with the keys [groupId], [artifactId], and [version] being replaced with their respective Maven parts. With this template, you can create the project name with the maven coordinates entered on the next page. This setting is saved on future wizard runs for quick creation of template projects.

Separate StreamBase Manager Installers

The StreamBase Manager application has been moved to an add-on package,SPOT_str-manager. This is available as .msi installer for Windows, .archive-bin installer for macOS and, .rpm-bin installer for Linux. StreamBase Manager functionality is still available from the SB Manager perspective in StreamBase Studio. This package is not available in Spotfire Data Streams.

UTF-8 by Default

Instead of the file.encoding system property, now, Studio and Manager launch explicitly with -Dfile.encoding=UTF-8

Substitution Variables No Longer Require Default Value in StreamBase Studio

Substitution variables no longer require default value when editing in Studio. Any variables that are missing a default value shows a warning. The previous quick-fix to insert a default null value is still available.

Client API No Longer Uses TIMESTAMP and INTERVAL long values

Client API no longer uses the existing TIMESTAMP and INTERVAL long values. The Timestamp.Type is used to specify TIMESTAMP or INTERVAL.

Metric Registration Restriction Added

Exponentially decaying reservoir metrics now correctly fail registration if the metric has multiple properties. Previous releases allowed the registration, but reported an error when the metric was read.

Consolidated the Functions setTupleFromJSON and setTupleFromJSONLoose

A new enum/value is added to the setTupleFromJSON function that allows you to choose whether you want it to be strict or loose.

LiveView Metadata is Stored in an SQLite Database by Default

LiveView metadata is now stored with SQLite instead of H2 by default. LiveView metadata automatically migrates from H2 to SQLite at run time. You can also right-click on LiveView H2 database files in Studio and migrate them to SQLite through a context menu.

Deprecated and Removed Features

The following features are deprecated as of Spotfire Streaming 11.1.0:

Operator Parameters are Deprecated

StreamBase Studio has deprecated the use of Operator Parameters.

Eclipse Docker Tooling Plug-in is now Removed

The Eclipse Docker plug-in is no longer included in Studio because of the 3rd party library incompatibilities. It is a best practice to use Docker Desktop.

Deprecated Support for older versions of Microsoft Windows Operating System

To enable UTF-8 support in Windows, starting from this release, the support for the following versions of Windows operating systems has been deprecated:

  • Microsoft Windows Server 2019

  • Microsoft Windows 10 (builds before 1903)

Check System Requirements for supported Microsoft Windows Operating System versions.

C++ Client APIs are now Deprecated

There is no replacement for the C++ API and the client code must be migrated to one of the other supported language bindings such as Java, .NET, and Python.

sbc and sbadmin Command Line Tools are now Deprecated

The functionality of sbc and sbadmin command line tools have been replaced with epadmin and any use of them must be migrated to epadmin.

TERR Bundle is Deprecated for macOS

macOS is longer supported on new version of TERR. Check Using the TERR Operator (Deprecated) for using existing versions.

StreamBase Client API for Python 2.7 is now Deprecated

The StreamBase Client API for Python 2.7 is now deprecated on all operating systems. It was previously removed on Windows, and now is deprecated on Linux. This deprecation does not affect the LiveView Client Python wheel, the Python Operator, and the Python Instance Operator, which continue to be supported.

Subversion Support from Studio is Deprecated

Subversion support is deprecated in Studio.

Creation of Artifact Service Discovery Record is Removed

Artifact Distribution Service (ADS) no longer publishes an artifact service record. ADS service discovery records were required for backward compatibility with pre-10.6 Artifact Management Servers (AMS). Upgrade to AMS 10.6 for compatibility with current releases. The ADS now uses the Last Model Request Time property in the node service record to track deployed model state in Streaming nodes.

FIX-based adapters' support for QuickFIX/J engine is Deprecated

Currently all FIX-based adapters offer a choice between using QuickFIX/J engine or Spotfire's own High-Performance FIX (HPFIX) engine. Going forward, HPFIX should be used as support for QuickFIX/J will be removed in an upcoming version of the product.

Discovery Settings in Cluster Service Discovery Input Adapter is Deprecated

StreamBase Studio has deprecated the discovery settings in Cluster Service Discovery Input Adapter.

Artifact Management Server is now Removed

AMS is removed in this release and all future release. AMS remains supported in the past releases where it has been delivered for as long as those releases are supported.

get_conf_param() is Deprecated

The get_conf_param() has been deprecated, which return the value portion of a name-value pair specified with a <param> element in a project's sbconf file.

LiveView Static Alerts are Removed

The alert-table element of the lvconf XML specification that supports author-time alert configuration (also referred to as static alerts), which was previously announced in StreamBase 10.4 as deprecated, is now removed. Instead, use dynamic alert configure.

HA Heartbeat Adapter is Deprecated

The HA Heartbeat Adapter has been deprecated in StreamBase Studio.

H2 Database is Deprecated for LiveView Metadata

LiveView metadata is now stored with SQLite instead of H2 by default. Metadata automatically migrates from H2 to SQLite at runtime. You can also right-click LiveView H2 database files in Studio and migrate them to SQLite through a context menu.

Support for Ant-based Task is now Removed

Streaming no longer supports Ant-based tasks.

Migration and Compatibility

This release contains the following migration and compatibility notes.

macOS Minimum Version Updated

macOS minimum version now updated to Monterey (12) (dropped Big Sur).

Operator Statistics are Disabled by Default

Operator statistics are now disabled by default. They were previously enabled. This was done to avoid significant CPU utilization to capture statistics by default. Operator statistics can be re-enabled using engine configuration.

name = "enable-operator-statistics"
type = "com.tibco.ep.streambase.configuration.sbengine"
version = "1.0.0"
configuration = {
 
  StreamBaseEngine = {
    streamBase = {
      engineMonitor = {
        enabled = true
      }
    }
  }
}

Streaming Resolved Issues

This section provides a list of issues resolved in release 11.1.0 of Spotfire Streaming.

Fixed in 11.1.0
Number Resolution
Spotfire Streaming 11.1.0 incorporates all fixes resolved in the release 10.0.0 series through 11.0.1.
SB-53170

The CSV File Reader adapter did not parse Unicode BOMs.

From this release, Unicode BOMs will now be ignored if present.

SB-53079 JQuery has been updated to 3.7.0 from 3.3.1 in the Web Server Response Adapter's test pages.
SB-52951

Earlier, sysinfo.txt use to report Windows version 6.2(9200) for all Windows.

From this release, sysinfo.txt shows Windows versions that are more accurate and readable.

SB-52618 When connecting to URIs, the WebSocket Client Read Adapter configures the SSL accordingly for ws or wss protocols.
SB-52509

The EventFlow package operations Copy, Copy Qualified Name, Paste, Move, and Rename did not work properly and only affected the last segment of the package.

SB-52460

Deleting an EventFlow package only deleted the last segment.

SB-52421

Linux users are unable to open StreamBase Command Prompt in Studio because the desktop environment can not be determined.

From this release onwards, the system property streambase.ide.openshell-forcecommandstring can be set to the command which will open a terminal.

SB-52331 Maven deployment using the epdev command failed when a library's name contained an uppercase letter.
SB-52252

The Maven index for tracking workspace projects did not always get initialized on Studio startup. This prevented the Add Dependency from offering workspace choices when updating a project pom.

SB-51994 The dependency of FastJSON is removed. It is still being used by the Webserver Adapter.
SB-51953 EventFlow Fragments were unable to launch in Debug mode if any other module had a breakpoint set
SB-51835

Generating a unit test from the Output Streams view in Studio generated a test with JSON that was not properly formatted, causing the tests to fail.

SB-51825 CSV Writer flush interval was previously ignored. The only flush() that used to happen was when the file was closed either by command or shutdown.
SB-51819 Previously, when the Include node snapshots option was checked to create a node snapshot in the Spotfire StreamBase Studio Support Wizard, the resulting archive zip did not have the node snapshot included.
SB-51813 The source Distributed Router now receives the notification that the target node has become available and delivers the queued tuples for that node as well as reporting the change from the Status output port. No queued tuples are dropped.
SB-51672

The parameters pages in the documentation contains outdated SB7-style configuration settings.

SB-51668

The tojson() SB function only operated on tuple type and generated a typecheck error when used on captures.

SB-51607

The Database Change Data Capture (CDC) adapter was not using the JDBC Data Source HOCON configuration to enable pooling options. Because of this, CDC Adapter could not monitor multiple CDC-enabled tables efficiently.

SB-50998 No transaction read locks are taken during the transaction, allowing object modifications while reading an object. Therefore, the administration command does not take read transactions. This allows the administration command to run without preventing other transactions from modifying the data read by the administration command.
SB-49469 EventFlow typecheck errors are not raised until application deploy time. To catch invalid code sooner, typecheck errors are now raised at Maven project build time.
SB-49285 The Aggregate operator emits a window, but does not close the window when the output tuple hits an Error Output Stream within the region. This behavior differs from StreamBase 7.7, which closes and clears the window regardless of whether an error is reached later.
CQS-5438 Exposed input streams with parameterized types like "list(string)" no longer cause startup failures when used in EventFlow modules inside LiveView projects.
CQS-5385

Cannot set AlertGroup in LV Alert Configuring adapter.

From this release, the adapter accepts AlertGroup field in its AlertGroup schema.

CQS-5371

LiveView authentication fails when unrelated Kerberos system properties are present.

Kerberos can be selectively configured for Adapters and not LiveView.

CQS-5201 Java clients correctly return null values changing into non-null values in calculated queries.