File Monitor Input Adapter

Contents

Introduction
File Monitor Properties
Description of This Adapter's Ports
Typechecking and Error Handling
Related Topics

Introduction

The Spotfire Streaming File Monitor adapter monitors the file system for creation, modification, and deletion of directories and files. For each such event, the adapter emits tuples that specify the path name and the type of change that occurred.

Note

Operating systems do not all use the same reporting and monitoring protocols. As a result, certain events may be reported by one operating system but not by another. For example, when you modify a file, your system may issue you a modified event for both the file and the folder containing that file, but another system would inform you only of the file modified event.

Caution

This adapter has the ability to monitor folder structures for changes, but it is not recommended to monitor large folder structures, such as trying to register the root folder with Watch Subdirectories enabled. Trying to do this causes large slowdowns on most, if not all, systems.

Tip

In some cases, depending on the operating system, the adapter may not receive notification of events (such as deletion) performed against a watched folder. If you require such notifications, consider watching its parent folder instead.

The adapter has a sample, described in File Monitor Input Adapter Sample.

File Monitor Properties

This section describes the properties you can set for this adapter, using the various tabs in its Properties view.

General Tab

Name: Use this required field to specify or change the name of this instance of this component. The name must be unique within the current EventFlow module. The name can contain alphanumeric characters, underscores, and escaped special characters. Special characters can be escaped as described in Identifier Naming Rules. The first character must be alphabetic or an underscore.

Adapter: A read-only field that shows the formal name of the adapter.

Class name: Shows the fully qualified class name that implements the functionality of this adapter. If you need to reference this class name elsewhere in your application, you can right-click this field and select Copy from the context menu to place the full class name in the system clipboard.

Start options: This field provides a link to the Cluster Aware tab, where you configure the conditions under which this adapter starts.

Enable Error Output Port: Select this checkbox to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports to learn about Error Ports.

Description: Optionally, enter text to briefly describe the purpose and function of the component. In the EventFlow Editor canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.

Adapter Properties Tab

Property Data Type Default Description
Enable Control Port check box Selected If enabled, a control input port is present for enqueuing tuples with the path (file or directory) to monitor.
Enable Status Port check box Selected If enabled, a status output port is present on which tuples are emitted to convey important events within the adapter.
Path String None The default path to monitor when no path is provided on the control input port. If this field has a value, the adapter watches this path when it starts.
Notify On Create Boolean true If enabled, the file monitor watches for create events on the specified path. This value is used as the default for incoming control tuples if Notify On Create Field Name is not supplied or is null.
Notify On Delete Boolean true If enabled, the file monitor watches for delete events on the specified path. This value is used as the default for incoming control tuples if Notify On Delete Field Name is not supplied or is null.
Notify On Modified Boolean true If enabled, the file monitor watches for modify events on the specified path. This value is used as the default for incoming control tuples if Notify On Modified Field Name is not supplied or is null.
Watch Subdirectories Boolean false If enabled, the file monitor recursively watches all subdirectories of the given path. Only valid for paths that are directories. This value is used as the default for incoming control tuples if Watch Subdirectories Field Name is not supplied or is null.
Output File Names On Monitor Boolean false If enabled, the system outputs all file names and directories for the path being monitored when the initial monitoring begins.

WARNING: If given a root path, and watching subdirectories, this setting can cause long delays traversing the file system.

Ignore Duplicate Updates Boolean true If enabled, the file monitor ignores duplicate updates for a set amount of time. Some file systems generate multiple updates for a single file update or directory name change. This option enables basic filtering of such duplicates, defined as any update with the same event type and path within the period specified by Timespan In Milliseconds. Two different update events within that period, such as a create event followed directly by a modify event, would not be ignored as duplicates.
Timespan In Milliseconds int 5000 The time in milliseconds within which updates on the same path can be regarded as duplicates.
Path Field Name String Path The field in the incoming control port tuple that represents the path to monitor.
Notify on Create Field Name String NotifyOnCreate The field in the incoming control port tuple that represents the boolean flag determining whether the path is to be monitored for create events. If this value is empty or evaluates to null, the default value is used.
Notify on Delete Field Name String NotifyOnDelete The field in the incoming control port tuple that represents the boolean flag determining whether the path is to be monitored for delete events. If this value is empty or evaluates to null, the default value is used.
Notify on Modified Field Name String NotifyOnModified The field in the incoming control port tuple that represents the boolean flag determining whether the path is to be monitored for modify events. If this value is empty or evaluates to null, the default value is used.
Watch Sub Directories Field Name String WatchSubDirectories The field in the incoming control port tuple that represents the boolean flag determining whether the file monitor should recurse all subdirectories. If this value is empty or evaluates to null, the default value is used.
Output File Names On Monitor Field Name String   The field in the incoming control port tuple that represents the flag to output monitored field names. If this field is not assigned or is null, the default value is used.
Log Level drop-down list INFO Controls the level of verbosity the adapter uses to send notifications to the console. This setting can be higher than the containing application's log level. If set lower, the system log level will be used. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE.

Cluster Aware Tab

Use the settings in this tab to enable this operator or adapter for runtime start and stop conditions in a multi-node cluster. During initial development of the fragment that contains this operator or adapter, and for maximum compatibility with releases before 10.5.0, leave the Cluster start policy control in its default setting, Start with module.

Cluster awareness is an advanced topic that requires an understanding of StreamBase Runtime architecture features, including clusters, quorums, availability zones, and partitions. See Cluster Awareness Tab Settings on the Using Cluster Awareness page for instructions on configuring this tab.

Concurrency Tab

Use the Concurrency tab to specify parallel regions for this instance of this component, or multiplicity options, or both. The Concurrency tab settings are described in Concurrency Options, and dispatch styles are described in Dispatch Styles.

Caution

Concurrency settings are not suitable for every application, and using these settings requires a thorough analysis of your application. For details, see Execution Order and Concurrency, which includes important guidelines for using the concurrency options.

Description of This Adapter's Ports

The File Monitor adapter's ports are used as follows:

  • Control (input): Tuples enqueued on this port cause the adapter to monitor a directory on the file system. The schema for this port has the following fields:

    • Path, string. The path on the file system to monitor. If null, the path is taken from the adapter's Path property.

    • NotifyOnCreate, boolean. The flag that determines whether create events are to be monitored on the file system. If null, the value is taken from the adapter's Notify On Create property.

    • NotifyOnDelete, boolean. The flag that determines whether delete events are to be monitored on the file system. If null, the value is taken from the adapter's Notify On Delete property.

    • NotifyOnModified, boolean. The flag that determines whether modify events are to be monitored on the file system. If null, the value is taken from the adapter's Notify On Modified property.

    • WatchSubDirectories, boolean. The flag that determines whether file monitor should also watch all subdirectories of the given path. If null, the value is taken from the adapter's Watch Subdirectories property.

  • Status (output): The adapter emits tuples from this port when significant events occur, such as when an attempt to monitor the file system starts, stops, or fails. The schema for this port has the following fields:

    • Type, string: Always returns Monitor.

    • Action, string: Returns an action associated with the event Type:

      • Error

      • Register

      • Unregister

    • Object, string: Returns the path associated with this event.

    • Message, string: Returns a human-readable description of the event.

  • Data (output): Tuples are emitted on this port when path events are available. The schema for this port has the following fields:

    Note

    If Enable Control Port is selected, then this schema adds an additional tuple field PassThroughFields, which contains all the values from the control port.

    • Path, string, The full path associated with the event.

    • Context, string, The context path associated with the event, usually just the file name or folder name.

    • Event, string, One of the following event types:

      • Created

      • Deleted

      • Modified

    • PassThroughFields, tuple, If the control port is enabled, this field contains all the values from the last incoming control port tuple.

Typechecking and Error Handling

The File Monitor adapter issues typecheck messages to help you configure the adapter within your StreamBase application, principally for the following reasons:

  • The Control Input Port does not contain the field name specified in the Path Field Name property, or that field is not of type string.

  • The Control Input Port does not contain the field name specified in the Notify On Create field Name property, or that field is not of type boolean.

  • The Control Input Port does not contain the field name specified in the Notify On Delete field Name property, or that field is not of type boolean.

  • The Control Input Port does not contain the field name specified in the Notify On Modified field Name property, or that field is not of type boolean.

  • The Control Input Port does not contain the field name specified in the Watch Subdirectories field Name property, or that field is not of type boolean.

  • The Path property contains the name of an invalid path.

The adapter generates warning messages at runtime under various conditions, including:

  • A control tuple is received with a null value in its Path field and a value for the adapter's Default Path property has not been specified.

  • An error occurs attempting to monitor the path.

Related Topics