Contents
Note
Starting with StreamBase 7.7.8, this operator is deprecated and is expected to be removed in a future release.
This sample shows basic usage of the Spark MLlib model evaluator. The scenario uses an audit data dataset from the PMML examples at http://dmg.org/pmml/pmml_examples/index.html.
The EventFlow module in this sample shows an example of runtime binomial classification. The input events contain some demographics data about bank customers. The events leave the flow enriched by the Target information computed by the model.
In StreamBase Studio, import this sample with the following steps:
-
From the top-level menu, select
→ . -
Type
to narrow the list of options.Spark
-
Select Spark from the StreamBase Model Operators category.
-
Click
.
StreamBase Studio creates a single project for the Spark operator samples in your current Studio workspace.
Run this sample in Studio as follows:
-
If you have not yet loaded the Spark MLlib Model sample into Studio, follow the steps in Importing This Sample.
-
Download Spark MLlib 2.2.0 binary distribution from http://spark.apache.org and save the assembly file to the project's
lib
directory. -
Add the extracted assembly to the project's build path.
-
In the Package Explorer view, open the
models-spark-sample
folder. -
Double-click to open the
audit.sbapp
application. -
Make sure the application is the currently active tab in the EventFlow Editor, then click the Run button. This opens the SB Test/Debug perspective and starts the application.
-
In the Feed Simulations view, select the
audit.sbfs
and click Run. -
In the Application Output view, select All Output Streams and view the data emitted by the StreamBase application. It starts with tuples similar to the following:
Target=0, Age=38.0, Employment=Private, Education=College, Marital=Unmarried, Occupation=Service, Income=81838.0, Gender=Female, Deductions=0.0, Hours=72.0 Target=0, Age=35.0, Employment=Private, Education=Associate, Marital=Absent, Occupation=Transport, Income=72099.0, Gender=Male, Deductions=0.0, Hours=30.0 Target=0, Age=32.0, Employment=Private, Education=HSgrad, Marital=Divorced, Occupation=Clerical, Income=154676.74, Gender=Male, Deductions=0.0, Hours=40.0 Target=0, Age=45.0, Employment=Private, Education=Bachelor, Marital=Married, Occupation=Repair, Income=27743.82, Gender=Male, Deductions=0.0, Hours=55.0 Target=0, Age=60.0, Employment=Private, Education=College, Marital=Married, Occupation=Executive, Income=7568.23, Gender=Male, Deductions=0.0, Hours=40.0 Target=0, Age=74.0, Employment=Private, Education=HSgrad, Marital=Married, Occupation=Service, Income=33144.4, Gender=Male, Deductions=0.0, Hours=30.0 Target=1, Age=43.0, Employment=Private, Education=Bachelor, Marital=Married, Occupation=Executive, Income=43391.17, Gender=Male, Deductions=0.0, Hours=50.0 Target=0, Age=35.0, Employment=Private, Education=Yr12, Marital=Married, Occupation=Machinist, Income=59906.65, Gender=Male, Deductions=0.0, Hours=40.0 Target=0, Age=25.0, Employment=Private, Education=Associate, Marital=Divorced, Occupation=Clerical, Income=126888.91, Gender=Female, Deductions=0.0, Hours=40.0 Target=0, Age=22.0, Employment=Private, Education=HSgrad, Marital=Absent, Occupation=Sales, Income=52466.49, Gender=Female, Deductions=0.0, Hours=37.0 Target=0, Age=48.0, Employment=Private, Education=College, Marital=Divorced, Occupation=Service, Income=291416.11, Gender=Female, Deductions=0.0, Hours=35.0
-
When done, press F9 or click the Stop Running Application button.
The artifact management sample demonstrates how you can use a TIBCO Artifact Management Server to connect to the TibcoAMS operator and feed data into the SparkML operator to load new version of models at runtime.
The TIBCO Artifact Management Server is a separately installed product. This sample assumes that you have a running Artifact Management Server (AMS) that is accessible by StreamBase. Refer to the product documentation for more information regarding AMS.
This sample also requires that you do the following:
-
Create a project folder in the Artifact Management Server called
sample
. -
Navigate to
sample/models-spark
from your StreamBase installation directory and upload theaudit.zip
file to thesample
project folder. This file is a compressed version of the audit folder. Note that thesample/models-spark
directory in StreamBase is a different folder from the AMSsample
project folder.
Run this sample in Studio as follows:
-
If you have not yet loaded the Spark MLlib Model sample into Studio, follow the steps in Importing This Sample.
-
Download Spark MLlib 1.6.0 binary distribution from http://spark.apache.org and save the assembly file to the lib directory.
-
Add the extracted assembly to project's build path.
-
In the Package Explorer view, open the
models-spark-sample
folder. -
Double-click to open the
artifactmanagement.sbapp
application. -
Double-click the TibcoAMS operator to view the operator properties and verify the information is correct. The operator is configured to work with a default, fresh installation of the Artifact Management Server.
-
Make sure the application is the currently active tab in the EventFlow Editor, then click the Run button. This opens the SB Test/Debug perspective and starts the application.
-
The application will try to pull the audit SparkML model from the Artifact Management Server.
-
In the Feed Simulations view, select the
audit.sbfs
and click Run. -
In the Application Output view, select All Output Streams and view the data emitted by the StreamBase application. It starts with tuples similar to the following:
Target=0, Age=38.0, Employment=Private, Education=College, Marital=Unmarried, Occupation=Service, Income=81838.0, Gender=Female, Deductions=0.0, Hours=72.0 Target=0, Age=35.0, Employment=Private, Education=Associate, Marital=Absent, Occupation=Transport, Income=72099.0, Gender=Male, Deductions=0.0, Hours=30.0 Target=0, Age=32.0, Employment=Private, Education=HSgrad, Marital=Divorced, Occupation=Clerical, Income=154676.74, Gender=Male, Deductions=0.0, Hours=40.0 Target=0, Age=45.0, Employment=Private, Education=Bachelor, Marital=Married, Occupation=Repair, Income=27743.82, Gender=Male, Deductions=0.0, Hours=55.0 Target=0, Age=60.0, Employment=Private, Education=College, Marital=Married, Occupation=Executive, Income=7568.23, Gender=Male, Deductions=0.0, Hours=40.0 Target=0, Age=74.0, Employment=Private, Education=HSgrad, Marital=Married, Occupation=Service, Income=33144.4, Gender=Male, Deductions=0.0, Hours=30.0 Target=1, Age=43.0, Employment=Private, Education=Bachelor, Marital=Married, Occupation=Executive, Income=43391.17, Gender=Male, Deductions=0.0, Hours=50.0 Target=0, Age=35.0, Employment=Private, Education=Yr12, Marital=Married, Occupation=Machinist, Income=59906.65, Gender=Male, Deductions=0.0, Hours=40.0 Target=0, Age=25.0, Employment=Private, Education=Associate, Marital=Divorced, Occupation=Clerical, Income=126888.91, Gender=Female, Deductions=0.0, Hours=40.0 Target=0, Age=22.0, Employment=Private, Education=HSgrad, Marital=Absent, Occupation=Sales, Income=52466.49, Gender=Female, Deductions=0.0, Hours=37.0 Target=0, Age=48.0, Employment=Private, Education=College, Marital=Divorced, Occupation=Service, Income=291416.11, Gender=Female, Deductions=0.0, Hours=35.0
-
From the Artifact Management Server, deploy the model again to the running
artifactmanagement.sbapp
application using the following properties:-
Target Type:
-
StreamBase Server URI:
sb://localhost
If you are running the Artifact Management Server and StreamBase Studio on the same machine, you may need to change
sb://localhost
tosb://
.your machine name
-
Target StreamBase Operator:
default.TibcoAMS
-
-
In the Application Output view, notice the ModelLoadStatus output tuple showing a new model was loaded.
-
When done, press F9 or click the Stop Running Application button.
When you load the sample into StreamBase Studio, Studio copies the sample project's files to your Studio workspace, which is normally part of your home directory, with full access rights.
Important
Load this sample in StreamBase Studio, and thereafter use the Studio workspace copy of the sample to run and test it, even when running from the command prompt.
Using the workspace copy of the sample avoids permission problems. The default workspace location for this sample is:
studio-workspace
/sample_operator_models_Spark
See Default Installation Directories for the default location of studio-workspace
on your system.