Connecting Container Streams To a CSV File

< Previous		Next >

CSV File Connection Overview

You can optionally connect an output stream in a container to a csvfile URI that specifies the path to a CSV file, using either the full absolute path or a relative path on the same system running sbd.

A CSV container connection has an advantage over dequeuing from an output stream, which is that a CSV container connection starts working immediately on sbd startup (or on startup of the container). This means that even the first few tuples are captured in the specified CSV file. By contrast, it can take some time to start sbd, then start an sbc dequeue command, during which time you can miss the initial tuples.

Note on Backslashes in Examples

In the examples on this page, long lines are artificially broken into shorter pieces for publication clarity. Lines in these examples are shown ending with a backslash to escape the final end-of-line character. If you are testing these commands on UNIX, you can copy and paste the commands as shown, including the trailing backslash. On Windows, you must collapse the pieces of the example into one long command, removing the trailing backslashes.

Specifying CSV URIs

To identify the path to a CSV file, use a URI in RFC 2396 format. The following is an example CSV container connection to an output stream, specified as a URI relative to the current working directory of the sbd process:

sbadmin addContainer A appA.sbapp \
    csvfile://sbproject/logs/appA-output.csv=A.Outgoing

which specifies that the file is in sbproject/logs/ under the current working directory.

To specify an absolute path to the file, follow csvfile:// with one slash:

sbadmin addContainer A appA.sbapp \
    csvfile:///usr/local/logs/appA-output.csv=A.Outgoing

which specifies that the file is in /usr/local/logs/.

Note

Remember that the syntax for expressing all forms of container connections is always destination=source, like an assignment statement in some programming languages.

For paths on Windows machines, you must use the same URI syntax with forward slashes. Here an absolute path is indicated:

sbadmin addContainer A appA.sbapp \
    csvfile:///c/logs/appA-output.csv=A.Outgoing

Use URI encoding for any spaces in directory names:

sbadmin addContainer A appA.sbapp \
    csvfile:///c/Users/sbuser/My%20Documents/logs/appA-output.csv=A.Outgoing

where c indicates your C: drive. If you were in your /sbuser home directory, you could also specify the container file using a relative path:

sbadmin addContainer A appA.sbapp \
    csvfile://My%20Documents/logs/appA-output.csv=A.Outgoing

CSV container connections also work with input streams. In this case, the stream is fed from the contents of the incoming CSV file, whose fields, field order, and field data types must exactly match the schema of the input stream. The following is an example of a CSV container connection with an input stream:

sbadmin addContainer B appB.sbapp \
    B.Incoming=csvfile:///c/sbapps/appB-input.csv

Specifying CSV Adapter Options

The default settings for CSV files written or read in container connections are the defaults for the CSV File Writer and Reader adapters. For example, the default string quote character is the double-quote, and the default field delimiter is the comma.

When connecting an output stream to a CSV URI, your container connection command can change the defaults by specifying any property of the CSV File Writer adapter, as listed on CSV File Writer Output Adapter. Similarly, when connecting an input stream to a CSV URI, your command can specify any property of the CSV File Reader adapter, as listed on CSV File Reader Input Adapter.

To specify custom adapter options, enclose the CSV URI in parentheses, and use standard HTML-like URI syntax to separate adapter options in the same URI. That is, separate the first option with a question mark, and subsequent options with ampersands. Use the StreamSQL version of the adapter property names, which are usually the field names seen in the Properties view for an instance of the adapter, with spaces removed. If a property's value is a string in a drop-down list in the Properties view, use URI encoding of any spaces in the property name to provide the same string in your CSV URI. For example:

sbadmin modifyContainer default addConnection \
    (csvfile://home/sbuser/logs/out.csv?MaxFileSize=30\
&HeaderType=Ignore%20header)=default.OutputStream1