Custom Functions with Complex Data Types

When integrating a custom Java function with StreamBase, the server tries to automatically map between StreamBase data types and their Java equivalent. For example, StreamBase int is mapped to Java int, while a StreamBase string is mapped to a to Java String.

When it comes to complex data types, the automatic mapping cannot occur. Even though the natural representation for a list(int) or list(string) in Java is a List<Integer> or List<String>, respectively, at the JVM level, these types don't exist. Both types are reduced to an untyped List after Java type erasure. For this reason, the server cannot automatically infer the type of an argument or return for parameterized data types.

The solution is to provide a custom function resolver method associated with the custom Java function. The custom function resolver method has the following purposes:

Custom function resolvers are called at build time for an application that refers to custom functions with associated resolvers.

A custom function's resolver method is specified using the Java CustomFunctionResolver annotation. For example:

@CustomFunctionResolver("myCustomFnResolver")
public static List myCustomFn(int i, Tuple t) {
...
}

public static CompleteDataType myCustomFnResolver(CompleteDataType arg1, CompleteDataType arg2) {
...
}

Both the annotation and the stub of a resolver class are generated for you when you use the New StreamBase Java Functions wizard in Studio to generate function code.

A resolver takes as many arguments as the method that refers to it. The types of all of these arguments must be CompleteDataType. The return type of the resolver must also be CompleteDataType.

For custom simple functions, a resolver can return either:

For custom simple functions that return simple data types (that is, the Java equivalents of the simple StreamBase types blob, bool, double, int, long, string, and timestamp), the returned CompleteDataType must be the appropriate corresponding simple type. For example, the return for an int must be as returned by CompleteDataType.forInt()).

For custom simple functions that return complex data types (that is, the Java equivalents of the complex StreamBase types tuple and list), the returned CompleteDataType must fully describe the appropriate complex type (as returned by CompleteDataType.forTuple(Schema) or CompleteDataType.forList(CompleteDataType)).

For custom aggregate functions, only the accumulate() methods can have associated resolvers. Even though the accumulate() methods return no value, the return type of their resolvers (if present) must be a CompleteDataType. If a resolver returns a non-null value, the value is taken to represent the return type of the custom aggregate function as a whole, when used with arguments of the type specified by the chosen accumulate() method.

See the Javadoc for CustomFunctionResolver for more information.

Consider the following example:

class ExprUtil {
  @CustomFunctionResolver("concatenateResolver")
  public static List<?> concatenate(List<?> v1, List<?> v2) {
    List<?> res = new ArrayList<?>();
    if (v1 != null) { res.addAll(v1); }
    if (v2 != null) { res.addAll(v2); }
    return res;
  }

  public static CompleteDataType concatenateResolver(
        CompleteDataType arg1, CompleteDataType arg2) {
    if (arg1.getElementType() == null) {
      return null; // not a list
    }
    if (arg1.equals(arg2)) { return arg1; }
    return null;
  }
}

The concatenate() function defined in this example takes two list arguments, as long as both lists have the same element type. The function returns a list of the same element type.

We use the @CustomFunctionResolver to annotate the concatenate() method with a function resolver, which is the concatenateResolver() method. At typecheck time, the resolver is called as necessary to determine whether the function's arguments are consistent with the definition of the function. Then at runtime, the concatenate() function is called appropriately.