Using the HBase Scan Operator

The <HBase.conf> element of a project's HOCON file, despite its name, is used to specify configuration value groups for either operators or adapters.

The HBase configuration section of the HOCON file starts with an <HBase.conf> element that contains one <HBase.conf name="hbase"> element. This element, in turn, contains one or more <setting> elements.

Each <section name="hbase"> element must contain one element in the form <setting id="HBaseConfigName"/>, where HBaseConfigName is the name you assign to a group of settings that uniquely define an individual HBase database connection. All other <setting> elements are optional.

The example configuration below shows a basic configuration to connect to an HBase server. You can have as many configurations as your application requires, but each configuration must have a unique id.

name = "HBase.conf"
type = "com.tibco.ep.streambase.configuration.adapter"
version = "1.0.0"
configuration = {
          
// An adapter group type defines a collection of EventFlow adapter configurations,
// indexed by adapter type.
  AdapterGroup = {
          
// A collection of EventFlow adapter configurations, indexed by adapter type. 
// This object is required and must contain at least one configuration.
    adapters = {
          
// The root section for an EventFlow adapter configuration.
      hbase = {
          
// Section list. This array is optional and has no default value.
        sections = [
          
// A configuration for an EventFlow adapter named section.
            {
          
// Section name. The value does not have to be unique; that is, you can have multiple sections
// with the same name in the same array of sections. This property is required.
              name = "hbase"
          
// Section for setting adapter properties. All values must be strings. This object
// is optional and has no default value.
                settings = {
                  connectAtStartup = "true"
                  "hbase.client.retries.number" = "5"
                  "hbase.master" = "127.0.0.1:60000"
                  "hbase.zookeeper.property.clientPort" = "2181"
                  "hbase.zookeeper.quorum" = "127.0.0.1"
                  id = "HBase Demo"
                  "zookeeper.recovery.retry" = "5"
                  "zookeeper.session.timeout" = "5000"
            }
          }
        ]
      }
    }
  }
}

Name: Use this required field to specify or change the name of this instance of this component. The name must be unique within the current EventFlow module. The name can contain alphanumeric characters, underscores, and escaped special characters. Special characters can be escaped as described in Identifier Naming Rules. The first character must be alphabetic or an underscore.

Operator: A read-only field that shows the formal name of the operator.

Class name: Shows the fully qualified class name that implements the functionality of this operator. If you need to reference this class name elsewhere in your application, you can right-click this field and select Copy from the context menu to place the full class name in the system clipboard.

Start options: This field provides a link to the Cluster Aware tab, where you configure the conditions under which this operator starts.

Enable Error Output Port: Select this checkbox to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports to learn about Error Ports.

Description: Optionally, enter text to briefly describe the purpose and function of the component. In the EventFlow Editor canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.

Use the Concurrency tab to specify parallel regions for this instance of this component, or multiplicity options, or both. The Concurrency tab settings are described in Concurrency Options, and dispatch styles are described in Dispatch Styles.

The filter factory interface (com.streambase.sb.operator.hbase.IHBaseFilterFactory) is used to create a FilterList for an HBase Scan or Get operation. The Filter list allows HBase to return only those rows that match the criteria provided by the specified filters. The interface allows an interaction between the inbound tuple and the Scan or Get operations. See the HBase API documentation for further information about the org.apache.hadoop.hbase.filter.FilterBase class and its sub-classes, and for how-to information on filtering data.

When you have compiled a Java class to serve as a filter, provide the fully qualified name of your class in the Filter Factor Class field in the Operator Properties tab in your HBase Scan or Get operator's Properties view.

The next section provides an example class file that takes the inbound tuples and uses some of the fields of each tuple to create a filter to use for the read operation.

package com.streambase.sb.hbase.filterfactory;

import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp;
import org.apache.hadoop.hbase.filter.FilterList;
import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
import org.apache.hadoop.hbase.filter.SubstringComparator;
import org.apache.hadoop.hbase.util.Bytes;

import com.streambase.sb.Tuple;
import com.streambase.sb.operator.hbase.IHBaseFilterFactory;

public class DemoFilterFactory implements IHBaseFilterFactory {

  @Override
  public FilterList createFilterList(Tuple tuple) throws Exception {
    FilterList filterList = 
      new FilterList(FilterList.Operator.MUST_PASS_ALL);
    filterList.addFilter(new 
      SingleColumnValueFilter(Bytes.toBytes(tuple.getString("family")), 
      Bytes.toBytes(tuple.getString("column")), CompareOp.EQUAL, 
      new SubstringComparator(tuple.getString("matchSubString"))));    
    return filterList;
  }
}