Deploying Applications

This page describes several approaches and techniques for deploying StreamBase applications into QA and production environments.

Startup Scripts

On Linux and Solaris hosts, the startup script is likely to be a Bash script. On Windows hosts, the script might be a cmd.exe script, or a Windows PowerShell script.

TIBCO recommends using simple startup scripts that contain just enough to:

  • Set any necessary environment variables.

  • Change to the directory that will serve as the base directory of your application. This directory is analogous for deployment to the primary StreamBase Studio project during development.

  • Start StreamBase Server with a bundle file as its argument, using a simple file name in the base directory and not a path name to the bundle file. This sets the server's working directory to the base startup directory and allows you to evaluate other directory paths relative to the base directory.

Set the ownership and permissions for the startup script (and specify any paths in the script), such that it can be used interactively by developers or testers on a QA server as well as started by a cron job on UNIX, or started at system startup with the /etc/init.d mechanism.

If you are deploying a highly available cluster of StreamBase Servers, you can prepare individual scripts for each machine in the cluster, plus a single master script that starts all members of the cluster, possibly using SSH to invoke the individual startup scripts on each host.

Of course, it is possible to come up with a complex script that covers several runtime environments, and swaps different configuration files in and out as needed. However, with careful planning, such scripts are not necessary and are often difficult to maintain.

Install Only What Is Needed

When selecting a machine to host StreamBase Server, make sure it complies with the directives on the Supported Platforms page.

On UNIX hosts, use the StreamBase installation option that allows you to install individual RPM or tar.gz files, as described in Installing StreamBase on Linux.

When installing StreamBase on UNIX, you have the option of omitting the following packages:

  • streambase-docs-*

  • streambase-development-*

  • streambase-studio-*

You must always install the following packages:

  • The streambase-server-* package, which installs StreamBase Server and its support files.

  • The streambase-client-* package, which installs the StreamBase client programs on which the server depends and the embedded JDK.

Be sure to install any external adapter packages that you are licensed to use, and that your application depends on.

To allow StreamBase Studio on another machine to access and start StreamBase Server on a UNIX host, be sure to follow the post-installation steps in Configuring UNIX Servers for Remote Studio Access.

Don't forget to install a valid StreamBase license file for each QA and production host, as described on Configuring a StreamBase License.

Use StreamBase Deployment Files

TIBCO strongly recommends using a StreamBase deployment file to specify the application, container, and container connection information for your application. Using deployment files allows you to separate the runtime startup information from the server configuration elements, which are still specified in the server configuration file.

See Deployment File Overview to learn about deployment files.

Deploy a StreamBase Application Bundle

TIBCO recommends bundling your application into a StreamBase application bundle file with .sbbundle extension. Bundle files are archive files that include:

  • A deployment file to start your application modules and containers.

  • A server configuration file.

  • All EventFlow and StreamSQL modules on the Studio project's module search path, including any referenced modules in referenced projects.

  • All resource files such as CSV files that are needed by an operator or adapter in your application and are located on the Studio project's resource search path.

  • All custom Java operators and adapters.

Bundle files generated by the command-line bundler, sbbundle, can have the same functionality as bundle files exported from StreamBase Studio, especially when used with the -p or -P options. This allows you to make bundling your application with sbbundle the last stage of your application's build script, automatically generated with your application.

The bundler generates a bundle-specific server configuration file that resolves all the files and paths to relative paths. Thus, a bundle file should run without change on a Windows development machine and a UNIX deployment server.

The complications for bundle files are:

  • Absolute paths to third-party JAR files or other resources.

  • Different JDBC connection strings to different database servers for development, QA, and production environments.

To handle these cases, StreamBase provides commands to extract the bundle-specific configuration file from the bundle (sbbundle -x), edit the extracted configuration file to provide different paths or JDBC connection strings for each environment, and replace the edited file back into the bundle (sbbundle -r). See sbbundle for details.

These extraction, edit, and replace steps can be automated by your application's build process, and can result in different automatically generated bundle files for your different target environments. For example, your build system can generate three bundle file with different JAR paths and JDBC connection strings for your development, QA, and production environments.

Keep Resource Files in the Studio Project

Resource files are non-StreamBase file types referenced by operators or adapters in your application. These can include CSV files that load a Query Table for lookup, adapter-specific configuration files, or other data files.

If you develop and control a resource file, then be sure to keep it in a folder on your Studio project's resource search path. This allows the StreamBase bundler to locate and include the file in the bundle.

If the resource file is provided by a third-party, such as a configuration file for a market data feed, copy the file to a folder in your resource search path, or make a symbolic link there.

Relative Paths and Configuration Files

Paths to modules and resource files are ideally located by Studio on its search paths. This allows the bundler to locate and copy the files.

If you do specify module and resource file paths in configuration files, whenever possible, use relative paths in your application's server configuration file, relative to the top-level Studio project folder. If your startup script includes a change directory command to a deployment base directory, as advised in Startup Scripts above, the relative paths will be valid both for development and deployment.

If your configuration file must provide paths to third-party files that are installed by third parties in different locations (such as JAR, DLL, or shared object files that support an adapter or market data feed), then consider copying those files to a folder in your Studio project's resource search path so that relative path locations can be maintained. In some cases, a symbolic link from the third-party file to a Studio project folder may also work.

Using Native Code Library Files

Native code provided by you or a third party might be needed by an operator or adapter in your application, and is usually provided in the form of a DLL on Windows or a shared object file on UNIX. The StreamBase bundler never bundles native code library files into the bundle, but always preserves the path to such files in the <plugin> and <library> elements of the server configuration file. For such files, use an absolute, well-known file system path in the server configuration file. This is one of the cases where you must extract the bundle-specific configuration file from a bundle, adjust the path to a native library file for your deployment environment, and replace the edited configuration file back in the bundle.

If your application includes custom operators or adapters that you develop in C++, there is a simple method of making the code available to your application in a platform-independent way.

  1. Develop a DLL for Windows and a shared object file for UNIX, giving both files the same basename. For example, develop:

    fastalgo.dll
    fastalgo.so
  2. Copy both files to a subfolder of your Studio project folder. For example, copy both files to a folder named plugins.

  3. In your server configuration file, specify a <plugin> element, using the file attribute (not the directory attribute), and specify only the basename:

    ...
    <global>
      <plugin file="./plugins/fastalgo" />
    </global>
    ...
  4. StreamBase automatically appends .dll on Windows and .so on Unix when attempting to load the file. This allows you to use the same configuration file without change on Windows development machines and on a UNIX server machine.

If your application and its server configuration file must reference native-code library files from third-parties, consider copying the third-party DLL and shared object files to a Studio project subfolder and renaming them in a way that emulates the configuration described above.

Use Modular Configuration Files

StreamBase Server configuration files provide an internal <sb-include> element (and also support the XInclude standard) which allows you to include elements in separate fragment files into a master configuration file, as described in Using Modular Configuration Files.

You can use this feature to enable certain features only for QA and deployment and disable them during development. For example, you may want to deploy with various security settings from the <authentication> element, or may want to deploy with certain <high-availability> settings.

Configure your master configuration file to XInclude files named, for example, security.xml and cluster.xml. Then create files with no actual settings like the following for use in the development environment:

<?xml version="1.0"?>
<streambase-configuration>
  <security />
</streambase-configuration>

and

<?xml version="1.0"?>
<streambase-configuration>
  <high-availability />
</streambase-configuration>

For deployment, create files with the same names that include a full complement of security and HA settings:

<?xml version="1.0"?>
<streambase-configuration>
  <security>
    <ssl-authentication>
      <param name="keystore" value="../test/proxy/security/signed.keystore"/>
      <param name="keystore-password" value="secret"/></ssl-authentication>
      <param name="key-password" value="secret1"/></ssl-authentication>
    <param name="perform-authentication" value="true"/>
...
</streambase-configuration>

and

<?xml version="1.0"?>
<streambase-configuration>
  <high-availability>
    <param name="leadership-status" value="LEADER"/>
    <param name="enabled" value="auto"/>
  ...
</streambase-configuration>

Then, using the same configuration file in both environments, you only need to maintain separate security.xml and cluster.xml files for the development and deployment environments. You can use an environment variable with <sb-include> to specify the location of each set of fragment files at runtime. See Using Modular Configuration Files for details.

Environment Variables and Configuration Files

You can use environment variables for most values in the server configuration file, using the ${VARNAME} syntax. StreamBase resolves the variable when it loads the application and configuration file.

You can also provide default values so that application loading does not fail if the variable is not set. For this, use the := (colon-equals) syntax: ${VARNAME:=false}

This allows you to specify certain configuration values generically, and then use environment variables in your startup script to provide host-specific values.

See Using Environment Variables in Configuration Files for further information.

Configuration Files Without Bundling

If you must deploy an application without the benefit of bundling, then you must ensure that the configuration file emulates the typecheck environment of your primary Studio project as closely as possible. Use the following as a reminder checklist:

  • Emulate the Studio project's module search path with the <module-search-directory> element.

  • Emulate the Studio project's resource search path with the <operator-resource-search> element.

  • Emulate the Studio project's Java Build Path with the <java-vm> element:

    • Use as many <jar> child elements as necessary to identify JAR files needed by your application.

    • As an alternative to several <jar> elements, use the <dir> child element to add a directory to the classpath of the JVM that hosts the server.

    • Use the <library> child element to specify the path to native code library files called by Java code.

  • Use the <plugin> element to specify the path to native code library files that implement custom operators or adapters.