< Previous		Next >

Using the FTP Operator

Contents

Introduction
Properties View Settings
Ports
Related Topics

This topic explains how to use the FTP operator and describes the configuration settings you can make in the operator's Properties view.

Introduction

The Spotfire Streaming FTP operator is a global Java operator that provides a way to run FTP commands through a Java-based FTP client. This is especially useful for the execution of programmatic file transfers.

Configure the operator to send either a single StreamBase FTP client command, or a single protocol-level FTP command, to a single target FTP server. The operator's command is sent on receipt of a tuple with any schema on the operator's input port. To send more than one FTP command, use more than one instance of the operator in your application, or use a tuple expression as the Command property.

Specify the command to run using the Command field of the Operator Properties tab. The command you select may require parameters or arguments. If so, enter those on other tabs of the Properties view.

The FTP operator creates a single FTP connection for each command it sends; therefore a command such as CWD does not do anything, since the user is logged out before any other commands can be executed. When the next command is executed, the user is logged back in and his CWD will be his home directory.

Placing a FTP Operator on the Canvas

Select the FTP from the Insert an Operator or Adapter dialog, which you invoke with one of the following methods:

Drag the Adapters, Java Operators token from the Operators and Adapters drawer of the Palette view to the canvas.
Click in the canvas where you want to place the operator, and invoke the keyboard shortcut O V
From the top-level menu, invoke Insert>Operator>Java.

From the Insert an Operator or Adapter dialog that opens, select FTP and double-click or press OK.

Properties View Settings

This section describes the properties you can set for an FTP operator, using the various tabs of the Properties view in StreamBase Studio.

General Tab

This section describes the properties on the General tab in the Properties view for the FTP operator.

Name: Use this required field to specify or change the name of this instance of this component. The name must be unique within the current EventFlow module. The name can contain alphanumeric characters, underscores, and escaped special characters. Special characters can be escaped as described in Identifier Naming Rules. The first character must be alphabetic or an underscore.

Operator: A read-only field that shows the formal name of the operator.

Class name: Shows the fully qualified class name that implements the functionality of this operator. If you need to reference this class name elsewhere in your application, you can right-click this field and select Copy from the context menu to place the full class name in the system clipboard.

Start options: This field provides a link to the Cluster Aware tab, where you configure the conditions under which this operator starts.

Enable Error Output Port: Select this checkbox to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports to learn about Error Ports.

Description: Optionally, enter text to briefly describe the purpose and function of the component. In the EventFlow Editor canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.

Operator Properties Tab

This section describes the properties on the Operator Properties tab in the Properties view for the FTP operator.

Command

This field is central to the operation of the operator. Enter a StreamBase expression in the Command field to determine which FTP client command to send with each tuple. The available commands are:

APPEND
PUT
GET
FREE_FORM

All commands except FREE_FORM correspond to a traditional FTP client command. The FREE_FORM command lets you type a protocol-level FTP command, such as DELE or MKD, with any necessary arguments.

Once you select an FTP client command with the Command field, you need to provide further information on one of the other tabs of the Properties view.

Asynchronous Operation

This determines if the file transfer operations should be performed asynchronously. NOTE: If operations are performed asynchronously the output tuple of the operation will not be sent until the operation is complete but the system will continue processing the next command tuple.

Async worker threads

This is the number of worker threads to allocate for command execution. The default value of 1 ensures that this operator's commands are executed serially. A positive integer will allocate a fixed number of threads. Setting this property to 0 will allocate an unlimited number of worker threads.

Enable Progress Updates

If enabled the operator will emit progress update tuples on the progress port.

Server Hostname

Enter a hostname or IP address in this field in the form of an expression. This should resolve to the IP address of the FTP server to which you wish to connect.

Server Port

The default value for this property is 21, the default FTP control port. Enter the number of the FTP server's control port in the form of an expression.

Username

Enter a user name in this field in the form of an expression. If the server you are connecting to supports anonymous FTP, be sure to enter the appropriate anonymous username for that server.

Password

Enter a password in this field in the form of an expression. If the server you are connecting to supports anonymous FTP, be sure to enter an appropriate anonymous password for that server.

Log Level

Controls the level of verbosity the adapter uses to issue informational traces to the console. This setting is independent of the containing application's overall log level. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE.

Connection Options Tab

This section describes the properties on the Connection Options tab in the Properties view for the FTP operator. This tab allows you to configure certain parameters regarding the FTP connection.

Local Passive Mode: Check this box to send a PASV command to the server before every data transfer. If left unchecked data transfers will occur in Local Active Mode. Local Passive Mode causes the server to open a port to which the client will connect for data transfer; in Active mode the opposite is ture. Passive mode is generally more likely to be compatible with most firewall configurations.
Transfer Timeout: The default value of 0 indicates an unlimited timeout. Any positive integer will cause all GET commands to timeout after being idle for that number of seconds. PUT commands will not timeout, regardless of this value.
Retry Count: The number of times to retry should there be a connection issue or an error with the file transfer. 0 is considered unlimited.
Poll Period: The time, in seconds, to wait between retry attempts.
Buffer Size: The internal data size of the buffer used to for buffered data streams.

Transfer Options Tab

This section describes the properties on the Transfer Options tab in the Properties view for the FTP operator. If the expression you entered in the Command field of the Operator Properties tab evaluates to a StreamBase FTP client command, then you need to fill in the fields in this tab.

Local File: Enter an expression that evaluates to the path name of the local file you wish to use in the transfer (for a GET this need not be an existing file; however all directories listed in the filename must exist). If you enter a relative path name, it will be relative to the project directory of the StreamBase application in which the operator is running.
Local File Is Resource: Enter an expression that evaluates to true or false which determines if the Local File should be loaded as a resource file (true) or as a fully qualified path (false).
Remote File: Enter an expression that evaluates to the name of the remote file you wish to use in the transfer. This can either be an absolute path name or relative to the home directory of the user you specified in the Username field in the Operator Properties tab.
File Type: Choose an FTP File Type to be used in the file transfer from the dropdown menu; be sure that the file type you choose is supported by the FTP server you are connecting to (both ASCII and BINARY should be supported by all FTP servers). The supported file types are:

ASCII
BINARY
EBCDIC
LOCAL

Protocol Command Options Tab

This section describes the properties on the Protocol Command Options tab in the Properties view for the FTP operator. The fields in this tab will be used when the Command field on the Operator Properties tab evaluates to either the empty string or the value FREE_FORM. For any given tuple, if the Command field does not evaluate to either of these values, then the fields on this tab will not be evaluated.

FTP Command

Enter the name of the FTP command you wish to execute, as if you were typing a protocol-level FTP command (such as DELE or MKD).

Note

Any FTP commands that do not return replies on the control channel are not guaranteed to be supported by this function. If you wish to transfer data, use a StreamBase FTP client command, such as PUT or GET.

FTP Params

As a single String, enter the argument to the FTP command. If no argument is required, this may be left empty. If the command requires or supports multiple arguments, separate them with whitespace. For example:

"TYPE A T"

Cluster Aware Tab

Use the settings in this tab to enable this operator or adapter for runtime start and stop conditions in a multi-node cluster. During initial development of the fragment that contains this operator or adapter, and for maximum compatibility with releases before 10.5.0, leave the Cluster start policy control in its default setting, Start with module.

Cluster awareness is an advanced topic that requires an understanding of StreamBase Runtime architecture features, including clusters, quorums, availability zones, and partitions. See Cluster Awareness Tab Settings on the Using Cluster Awareness page for instructions on configuring this tab.

Concurrency Tab

Use the Concurrency tab to specify parallel regions for this instance of this component, or multiplicity options, or both. The Concurrency tab settings are described in Concurrency Options, and dispatch styles are described in Dispatch Styles.

Caution

Concurrency settings are not suitable for every application, and using these settings requires a thorough analysis of your application. For details, see Execution Order and Concurrency, which includes important guidelines for using the concurrency options.

Ports

Input Port

The FTP operator has one input port that accepts a tuple with any schema. The operator runs the appropriate command on the receipt of a tuple, based on its contents.

Error Port

You can also add an optional Error Output port, which outputs a StreamBase error tuple for any error thrown by the operator, as described in General Tab.

Status Output Port

The FTP operator has one mandatory output port, which serves as its status port. For each tuple that comes in on the input port, exactly 1 tuple will exit this port. The schema for this port is pre-set by the operator and is described in the following table:

Field Name	Field Type	Field Description
`type`	string	Describes the type of status being emitted. Currently the only value supported is `COMMAND`.
`action`	string	The action that triggered this status tuple. One of: `SUCCESS` - The FTP command has been successfully issued. `FAILED` - The FTP command has failed; either the server sent a negative response, or a connection could not be established with the server. `ERROR` - Some property field evaluated to an invalid value; for example, the application did not have read access to the given `Local File` for a `PUT` command.
`object`	string	The Server response to the FTP command. This will include the three digit status code as the first three non-whitespace characters. If this is `null`, that means either the server did not respond or a connection could not be established. For a protocol command such as `SIZE`, which causes the server to issue a response on the control connection, this will contain the response with the relevant information.
`message`	string	A message either confirming success or with some information as to why a command failed. This has no well-defined format and is for manual debugging use only.
`inputTuple`	tuple	The input tuple which caused this command to be issued.

Progress Output Port

The FTP operator has an optional progress port. If the progress port is enabled and a file transfer is performed, one or more tuples will exit from the progress port with transfer information. The schema for this port is pre-set by the operator and is described in the following table:

Field Name	Field Type	Field Description
`totalBytesTransferred`	long	The total bytes that have been transferred so far for this operation.
`bytesTransferred`	int	The number of bytes that have been transferred since the last progress update (or since the start if this is the first progress tuple).
`inputTuple`	tuple	The input tuple which caused this command to be issued.