Feed Simulation Custom Reader Sample

Contents

Sample Overview
Importing This Sample into StreamBase Studio
Running This Sample in StreamBase Studio
This Sample's Files
Sample Location

This sample demonstrates how to implement a custom file reader to read non-standard, proprietary, or binary files as the source of a stream of input tuples for feed simulations. Your Java file-reading code must extend one of the classes in the com.streambase.sb.feedsim package in the StreamBase Client Library. StreamBase provides a way to use your custom class instead of its internal CSV-reading code in conjunction with the Feed Simulation Editor's Data File option.

For more information about configuring Feed Simulations to use custom readers, see Feed Simulation with Custom File Reader.

Sample Overview

This sample demonstrates and provides Java source code for two implementations of custom file readers for feed simulations:

  • A custom CSV file reader

  • A custom tuple reader

The custom CSV file reader implementation forces all characters to lowercase if the first character in the file is a pound sign (#), forcing all characters to uppercase otherwise. This sample's implementation class, MyFeedSimCSVPlugin, extends the required FeedSimCSVInputStream interface. Because only the read() method needs to be overridden, this implementation works for any schema CSV file.

The custom Tuple reader implementation parses lines in a sample file that contains web server log data, stored in the NCSA Common Log Format. The getSchema() method returns a schema that assigns appropriate field names and types to the log data, and each readTuple() call returns the Tuple for each successive line of log data.

See Feed Simulation with Custom File Reader for a discussion of the classes in the com.streambase.sb.feedsim package in the StreamBase Client Library.

Importing This Sample into StreamBase Studio

In StreamBase Studio, import this sample with the following steps:

  • From the top-level menu, select File>Import Samples and Community Content.

  • Enter custom reader in the Search field to narrow the list of options.

  • Select the Feed Simulation Custom Reader sample from the Extending StreamBase category.

  • Click Import Now.

StreamBase Studio creates a project for the sample in your current Studio workspace.

Running This Sample in StreamBase Studio

The steps to run this sample in Studio are as follows:

  1. In the Project Explorer view, open this sample's folder.

    Keep an eye on the bottom right status bar of the Studio window. Make sure any Updating, Downloading, Building, or Rebuild project messages finish before you proceed.

  2. Open the src/main/eventflow/packageName folder.

  3. Double-click to open the MyFeedSimCSVPlugin.sbapp module. Make sure the module is the currently active tab in the EventFlow Editor.

  4. Click the Run button. This opens the SB Test/Debug perspective and starts the module.

  5. Wait for the Waiting for fragment to initialize message to clear.

  6. The Feed Simulation adapter in this module is configured to be self-running. Thus, soon after the module starts, look in the Output Streams view for data emitted on the CsvPluginOutputStream stream.

    Notice that the stock symbols are all uppercase as sent to the StreamBase application. In the originating CSV file, the symbols are mixed case.

  7. To reread the same data file, in the Manual Input view, select the FeedSimControl stream, enter the string start in the command field, and click Send Data.

  8. When done, press F9 or click the Terminate EventFlow Fragment button.

  9. Next, open the second module in this sample, MyFeedSimTuplePlugin.sbapp.

  10. Make sure the new module is the currently active tab in the EventFlow Editor, then click the Run button to start the module.

  11. The second sample is also self-running. In the Output Streams view, look for data emitted on the TuplePluginOutputStream.

  12. To re-read the same data file, in the Manual Input view, select the FeedSimControl stream, enter the string start in the command field, and click Send Data.

  13. When done, press F9 or click the Terminate EventFlow Fragment button.

This Sample's Files

This sample includes the following files that demonstrate a custom CSV file reader plug-in.

File Description
MyFeedSimCSVPlugin.sbapp A simple, pass-through EventFlow application that demonstrates the custom CSV file reader plug-in.
feedSimForCsvPlugin.sbfs A feed simulation file set up to read timestamp-simple.csv as its input data source, using the custom file reader MyFeedSimCSVPlugin.
timestamp-simple.csv A simple three-field CSV file of stock symbol, price, and timestamp data.
MyFeedSimCSVPlugin.java In the src/main/java/package-name directory, the Java source file for the MyFeedSimCSVPlugin class.

This sample also includes the following files that demonstrate a custom tuple reader plug-in.

File Description
MyFeedSimTuplePlugin.sbapp A simple, pass-through EventFlow application that demonstrates a custom tuple reader plug-in.
feedSimForTuplePlugin.sbfs A feed simulation file set up to read log-data.txt as its input data source, using the custom file reader MyFeedSimTuplePlugin.
log-data.txt A text file containing a few lines of sample web server log data.
MyFeedSimTuplePlugin.java In the src/main/java/package-name directory, the Java source file for the MyFeedSimTuplePlugin class.

Sample Location

When you load the sample into StreamBase Studio, Studio copies the sample project's files to your Studio workspace, which is normally part of your home directory, with full access rights.

Important

Load this sample in StreamBase Studio, and thereafter use the Studio workspace copy of the sample to run and test it, even when running from the command prompt.

Using the workspace copy of the sample avoids permission problems. The default workspace location for this sample is: 

studio-workspace/sample_feedsim-plugin

See Default Installation Directories for the default location of studio-workspace on your system.