Structured Streaming Data

This and the next few pages introduce concepts that provide the foundation for using TIBCO StreamBase® and TIBCO® Live Datamart components of TIBCO® Streaming. If you are familiar with streaming data concepts, skip ahead to About StreamBase Studio.

TIBCO StreamBase® is a development environment for creating, testing, and deploying programs that process and analyze structured, streaming data.

The word structured in this definition is important. You may be familiar with a streaming video service such as Netflix, or a streaming audio service. The data streamed by those services is in the form of video or audio files, which are unstructured by nature. However, if a streaming media service also provides metadata about each movie or song, chances are good that the metadata would be provided in structured form.

A structured stream of data is in a format that is known or discoverable, where that format is repeated for each unit of data. A structured stream might be a sequence of name-value pairs, with a defined start and stop keyword at the beginning and end. The name-value pairs between each start and stop keyword define one unit of data in that stream. Another stream might be a complex mix of hierarchical data that include list subunits with no fixed length, or where one unit of data can contain subunits of data with a different structure than the containing unit.

The twenty-first century has many examples of streams of structured data:

  • Stock market transactions are reported in structured streams from one brokerage to its peers, or reported from markets such as NASDAQ or the New York Stock Exchange to the public.

  • Sports statistics are streamed while each game is played to news sites and to the public.

  • In the petroleum industry, each oil or gas well reports a stream of operating statistics to a central administration site.

  • Sensors stream reports of seismic activity at their location to a monitoring site.

  • Commercial web sites stream purchase information from customers to back-end servers that process each order and return real-time results back to those customers.

  • Medical offices exchange information with insurance companies in a well-defined XML format.

A TIBCO StreamBase application can be developed to ingest each of these stream types and many others in order to analyze the incoming data in many ways. StreamBase applications then emit the same stream, or a modified stream, or any number of alternative streams as outputs.