Web Server Request Adapter

Introduction

The TIBCO StreamBase® Web Server Request adapter allows StreamBase applications to handle HTTP and WebSocket requests. It is powered by an embedded Jetty web server. The Web Server Request input adapter handles the parsing of HTTP data into an outgoing tuple which your code can handle and then feed the response into an associated Web Server Response output adapter. The adapter uses property values in the adapter configuration file to set up the Jetty engine. For details, see the Configuration section below.

Multiple web server request adapters can share a single instance of the Jetty web server engine by selecting the same web server configuration setting. A new Jetty web server instance is created for each port that is set in the configuration file.

Adapter Properties

This section describes the properties you can set for this adapter, using the various tabs of the Properties view in StreamBase Studio.

General Tab

Name: Use this required field to specify or change the name of this instance of this component, which must be unique in the current EventFlow module. The name must contain only alphabetic characters, numbers, and underscores, and no hyphens or other special characters. The first character must be alphabetic or an underscore.

Adapter: A read-only field that shows the formal name of the adapter.

Class name: Shows the fully qualified class name that implements the functionality of this adapter. If you need to reference this class name elsewhere in your application, you can right-click this field and select Copy from the context menu to place the full class name in the system clipboard.

Start options: This field provides a link to the Cluster Aware tab, where you configure the conditions under which this adapter starts.

Enable Error Output Port: Select this check box to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports to learn about Error Ports.

Description: Optionally enter text to briefly describe the component's purpose and function. In the EventFlow Editor canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.

Adapter Properties Tab

Property Type Description
WebServer Configuration Edit Button Shortcut to the StreamBase Configuration File Editor, used for adapter configuration or converting an existing application's adapter-configurations.xml file to HOCON format.
Web Server Config drop-down list This required field will select a value from the adapter-configurations.xml file that contains the configuration for the web server that this adapter will be associated with. This value is selected from the adapter-configuration section with name equal to webservers, section name equal to webserver; the value is pulled from the id parameter.
Request Id Field Name string The name used for the field in the outgoing data tuple that contains the request ID for each HTTP request. The request ID is used to associate this request connection with an outgoing response to send a response back.
Context Path string The path that this adapter will handle for the web server. The path should be anything after the root web address. For example, http://localhost/test would have a target /test. A path can also be / which means handle all requests not handled by another adapter. The context path can also contain path variables in the form of {varname} for example http://localhost/test/{var1}/{var2} contains to path variables. If a variable is located on the context path, a new tuple field will appear on the adapters output tuple called PathVariables and will contain the fields present in the path. When a request is made to the URL http://localhost/test/first/second the request tuple will contain var1 with a value of first and var2 with a value of second. Note that if path variables are used then the context path is effectively ended at the first path parameter to compare against other adapters' context path usage. For example, if one adapter tries to set http://localhost/test/{var1} and another set http://localhost/test/{var1}/{var2} a runtime error will occur because both adapters now have a base context path of http://localhost/test.
Enable Control Port check box When enabled, the adapter allows incoming tuples to control the actions of this adapter.
Command Field Name string The name of the field in the incoming control tuple that contains the command.
Enable Status Port check box When enabled, the adapter sends out informational data on the status port about various adapter states.
Log Level INFO Controls the level of verbosity the adapter uses to issue informational traces to the console. This setting is independent of the containing application's overall log level. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE.

Request Tab

Property Type Description
Web Socket check box Determines whether this context path should be treated as a WebSocket connection.
Web Socket Read Data Type radio Determines the type of data the will be read from the WebSocket connection.
Multipart/Form-Data check box Determines if this endpoint should process multipart/form-data (file uploads). This is normally via a POST request with form values. If POST data contains inputs with of type file, then the files are saved to the file system and the full path to the file is given in the output tuple files list. If there are other input fields, they are added to the parameters list of the outgoing data tuple.
File Storage Path string The path (relative or full) to the location of where to store files sent via HTTP multipart/form-data POST requests.
Request Data Outgoing FieldName string The name given to the field in the outgoing data tuple which will contain the web request data. This value is optional, and if left blank request data will not be included in the outgoing data tuple. If a request data schema is supplied this field will contain that schema, if not this field will be a string.
Request Data Parameter Name string The name given to the field in the outgoing data tuple which will contain the web request parameter data. This value is optional, and if left blank request parameter data will not be included in the outgoing data tuple.
Request Callback Parameter Name string The name given to the field in the outgoing data tuple which will contain the web request callback name, commonly used for JSONP requests. This value is optional, and if left blank the request callback name is not be included in the outgoing data.
Request Data Schema schema The schema of POST requests made to this operator. This schema is used with the Request Data Transformers to convert the inbound POST data into a tuple.
Request Data Transformer Content Type Map string, string The key/value map which has a regex key that matches to the request's content type, and the value is a class that implements com.streambase.sb.adapter.webserver.data.IRequestDataTransformer. By default JSON and XML data transformers are set up to handle any JSON or XML standard requests.
Request Data Transformer Settings string, string The key/value map of settings to pass to the Request Data Transformers. The key is the classname.setting of each setting to send to individual data transformers. The value is the setting value to set. For example, to set the JSONRequestDataTransformer timestamp format, you must have a key of JSONRequestDataTransformer.TimestampFormat and some string format for the value. (The format MUST be transformerClassName.settingField). Another example: to set the buffer size, you must have a key of BinaryRequestDataTransformer.BufferSize and some value (default is 1024 bytes).

Parameters Schema Tab

Property Type Description
Parameters Timestamp Format string The timestamp format to use when converting parameters to timestamp fields.
Parameters Schema schema The schema used to map the incoming parameters of a request to a well-defined schema. NOTE: Name mapping to parameters is case-sensitive.

Cluster Aware Tab

Use the settings in this tab to allow this operator or adapter to start and stop based on conditions that occur at runtime in a cluster with more than one node. During initial development of the fragment that contains this operator or adapter, and for maximum compatibility with TIBCO Streaming releases before 10.5.0, leave the Cluster start policy control in its default setting, Start with module.

Cluster awareness is an advanced topic that requires an understanding of StreamBase Runtime architecture features, including clusters, quorums, availability zones, and partitions. See Cluster Awareness Tab Settings on the Using Cluster Awareness page for instructions on configuring this tab.

Concurrency Tab

Use the Concurrency tab to specify parallel regions for this instance of this component, or multiplicity options, or both. The Concurrency tab settings are described in Concurrency Options, and dispatch styles are described in Dispatch Styles.

Caution

Concurrency settings are not suitable for every application, and using these settings requires a thorough analysis of your application. For details, see Execution Order and Concurrency, which includes important guidelines for using the concurrency options.

Control Port

Description

Use the control port to send action commands to the adapter. Tuples enqueued on this port cause the adapter to start or stop a server.

Control Port Schema

  • Command, string. The command to send to the adapter.

    The values are:

    • START — Tells the adapter to start the server and listen on the port specified in the configuration.

    • STOP — Tells the adapter to stop the server and close all client connections.

Status Port

Description

The status port is used to send status information tuples downstream to inform the user of changes.

Status Port Schema

  • type, string. The type of status information emitted on this port. Status types are:

    • WebServer — Indicates this message is about a web server.

    • WebSocket — Indicates this message is about a WebSocket connection.

  • action, string.

    • Error — A error has occurred.

    • Command — A command status information is being reported such as start and stop of server.

    • Closed — A WebSocket was closed.

    • Connected — A web socket connected.

  • object, string. This value may be null. If it is not null, it contains a value relevant to the status message.

  • message, string. This is a formatted human readable message that explains the status message.

Configuration

Description

The web server adapter configuration section starts with an <WebServer.conf> element containing one <WebServer.conf name="webservers"> element that contains one or more <section name="webserver"> elements.

Each <section name="webserver"> configuration must contain a "id" and "Port" settings fields. All other setting elements are optional.

This example configuration shows two different web servers. The first contains all values available and the second shows the required values. You can have as many configurations as your application requires, but each must have a unique port number and a unique ID.

Example

name = "WebServer.conf"
          type = "com.tibco.ep.streambase.configuration.adapter"
          version = "1.0.0"
          configuration = {
          
          // An adapter group type defines a collection of EventFlow adapter configurations, indexed by adapter type.
            AdapterGroup = {
          
          // A collection of EventFlow adapter configurations, indexed by adapter type. This key is required and must contain at
          // least one configuration.
                adapters = {
          
          // The root section for an EventFlow adapter configuration.
                    webservers = {
          
          // Section list. This key is optional and has no default value.
                        sections = [
          
          // A configuration for an EventFlow adapter named section.
                            {
          
          // Section name. The value does not have to be unique; that is, you can have multiple sections with the same name
          // in the same array of sections. This key is required.
                                name = "webserver"
          
          // Section property bag. All values must be strings. This key is optional and has no default value.
                                settings = {
                                  GZipMimeTypes = "text/html,text/plain,text/xml,application/xhtml+xml,text/css,application/javascript,text/javascript,image/svg+xml"
                                  IdleTimeoutMS = "30000"
                                  OutputBufferSize = "32768"
                                  Port = "8080"
                                  RequestHeaderSize = "8192"
                                  ResponseHeaderSize = "8192"
                                  SendDateHeader = "false"
                                  SendServerVersion = "true"
                                  StartOnStartup = "true"
                                  WebSocketAsyncWriteTimeoutMS = "5000"
                                  WebSocketIdleTimeoutMS = "30000"
                                  id = "Web Server Port 8080"
                                }
                            }
          
          // A configuration for an EventFlow adapter named section.
                            {
          
          // Section name. The value does not have to be unique; that is, you can have multiple sections with the same name
          // in the same array of sections. This key is required.
                                name = "webserver"
          
          // Section property bag. All values must be strings. This key is optional and has no default value.
                                settings = {
                                  Port = "9090"
                                  id = "Web Server Port 9090"
                                }
                            }
                        ]
                    }
                }
            }
          }

Settings

Setting Type Description
id string The value to display in the drop-down list and is used to key to this section of the configuration file.
Port integer The port the web server will listen on for connections.
StartOnStartup true/false Start the web server on adapter startup.
IdleTimeoutMS integer The amount of time in milliseconds to wait for connections to send data.
OutputBufferSize integer Set the size of the buffer into which response content is aggregated before being sent to the client.
RequestHeaderSize integer Set the maximum size of a request header.
ResponseHeaderSize integer Set the maximum size of a response header.
SendServerVersion true/false If true, send the Server header in responses.
SendDateHeader true/false If true, include the date in HTTP headers.
SelectorThreads int The number of selector threads, or less than or equal to 0 for a default value. Selectors notice and schedule established connection that can make IO progress.
AcceptorThreads int The number of acceptor threads to use, or -1 for a default value. Acceptors accept new TCP/IP connections. If 0, then the selector threads are used to accept connections.
ReuseAddress true/false Whether the server socket reuses addresses.
LingerTime integer The linger time. Use -1 to disable.
GZipMimeTypes string Set the mime types associated with GZip requests, blank or missing means do not use GZip.
WebSocketAsyncWriteTimeoutMS integer The time in milliseconds before a write command times out.
WebSocketIdleTimeoutMS integer The time in milliseconds before a WebSocket times out from idle behavior.
SecurePort integer Set the TCP/IP port used for CONFIDENTIAL and INTEGRAL redirections.
* SSLKeyStorePath string The file or URL of the SSL keystore.
* SSLKeyStorePassword string The password for the keystore
* SSLKeyStoreManagerPassword string The password (if any) for the specific key within the keystore.
* SSLTrustStorePath string The file name or URL of the trust store location.
* SSLTrustStorePassword string The password for the trust store.

* NOTE: Please refer to http://wiki.eclipse.org/Jetty/Howto/Configure_SSL when setting up SSL.

Request Data Transformer

The request data transformer interface com.streambase.sb.adapter.webserver.data.IRequestDataTransformer is used to convert inbound POST requests into a tuple. The interface also uses the com.streambase.sb.adapter.webserver.data.IRequestDataTupleSender to send one or more tuples per POST request. This is useful for instances when you send multiple chunks of data or file parts per request. Be careful to send a single tuple into the web server response adapter per request. A good strategy is to output a tuple with all null values to signal the end of a request.

com.streambase.sb.adapter.webserver.data.IRequestDataTransformer

package com.streambase.sb.adapter.webserver.data;

import java.util.Map;

import javax.servlet.http.HttpServletRequest;

import org.eclipse.jetty.server.Request;
import org.slf4j.Logger;

import com.streambase.sb.Schema;
import com.streambase.sb.StreamBaseException;
import com.streambase.sb.operator.TypecheckException;

public interface IRequestDataTransformer {
    /***
     * This method will be called during typecheck time to allow a request data transformer 
     * to validate a key value map of settings against the transformer 
     * @param settings A key value map of settings the end user specified for this adapters 
     * custom data transformation
     * @param schema The schema provided by the end user for which the data should be converted to. 
     * @throws TypecheckException
     */
    public void validateSettings(Map<String, String> settings, Schema schema) throws TypecheckException;
    
    /***
     * This method allows the HTTP request data to be transformed into a tuple with any custom logic required
     * 
     * NOTE: only one of the two readers can be used (httpServletRequest or request) once one is read the 
     * other will become invalid
     * 
     * @param sender The senders methods are used to send completed tuples downstream.  It is expected 
     * that for each call to this method at least 1 tuple is create and a sendTuple or sendTupleAsync is called.
     * @param logger A logger than can be used to log any extra information which may be required
     * @param settings A key value map of settings the end user specified for this adapters custom data 
     * transformation 
     * @param schema The schema provided by the end user for which the data should be converted to.
     * @param httpServletRequest The HTTP request to use for transform into a tuple
     * @param request The base HTTP request to use for transforming into a tuple
     * @throws StreamBaseException
     */
    public void transformRequestData(IRequestDataTupleSender sender, Logger logger, Map<String, String> settings, 
    Schema schema, HttpServletRequest httpServletRequest, Request request) throws StreamBaseException;
}         

com.streambase.sb.adapter.webserver.data.IRequestDataTupleSender

package com.streambase.sb.adapter.webserver.data;

import com.streambase.sb.Tuple;

/***
 * The request data tuple sender is used to send a tuple downstream after parsing the HTTP request into a tuple.
 * 
 * It is recommended that if you expect to send more than one tuple per http request that you mark a single tuple 
 * to send as the response.  
 * Sending more than one tuple for the same request into the response web server adapter will fail.
 */
public interface IRequestDataTupleSender {
    /***
     * Send a tuple downstream and wait for its execution to fully complete 
     * @param tuple The tuple to set as the request data tuple
     */
    public void sendTuple(Tuple tuple);
    
    /***
     * Send a tuple downstream asynchronously 
     * @param tuple The tuple to set as the request data tuple
     */
    public void sendTupleAsync(Tuple tuple);
}          

com.streambase.sb.adapter.webserver.data.JSONRequestDataTransformer

The JSON request data transform converts a JSON request to a tuple.

Settings

Setting Type Default Description
TimestampFormat string yyyy-MM-dd HH:mm:ss.SSSZ The timestamp format used when converting JSON data to a timestamp field.
NumericTimestampsAsInterval true/false false If true any numeric values in a timestamp field will be converted to interval types, if false they are considered timestamp.

com.streambase.sb.adapter.webserver.data.XMLRequestDataTransformer

The XML request data transform converts an XML request to a tuple.

Settings

Setting Type Default Description
TimestampFormat string yyyy-MM-dd HH:mm:ss.SSSZ The timestamp format used when converting XML data to a timestamp field.

com.streambase.sb.adapter.webserver.data.BinaryRequestDataTransformer

The Binary request data transform converts a request to a series of tuples containing the request POST data.

This transformer requires the request schema and contains three fields: Data of type Blob, Name of type String, and Value of type String.

The data field contains any data from the body. The Value field will contain any values from form fields.

This adapter will output 1-N number of tuples containing the request body data. After reading all the body data, a tuple with all null fields is emitted to represent the completion.

Settings

Setting Type Default Description
BufferSize int 1024 The max number of bytes to output per tuple of the POST requests body data.