Contents
The TIBCO StreamBase® Twitter Input adapter connects a StreamBase application to the web service provided by Twitter, Inc. It can receive Twitter status messages and direct messages, commonly called tweets.
The input adapter can be initially set up to respond to control messages for changing at runtime the stream being watched. It can watch the user, sample, link, and filter twitter streams, and can perform searches. The adapter can also process direct messages on the user stream, if so enabled.
By default, the input adapter is configured to not connect to any twitter streams on startup, awaiting a control message to perform an operation.
The adapter must be configured with the correct OAuth tokens for it to perform any operations. The OAuth can be set up in the security tab, as described in Obtaining Twitter OAuth Tokens below.
The adapter has samples, described in Twitter Adapter Samples.
Fields in schemas used by the Twitter adapters correspond to attributes in JSON-encoded tweets accessed via Twitter4J library calls. The library is provided with StreamBase. To help you understand what fields in schemas represent, the following table summarizes Twitter objects and attributes, and provides links to their documentation at dev.twitter.com.
Object | Example Attributes | Reference |
---|---|---|
Tweet | id, following, followers_count, geo_enabled, status | Platform Objects:Tweets |
User | id, following, followers_count, geo_enabled, status | Platform Objects:Users |
Geo Place | country_code, locality, region, street_address, postal_code | Platform Objects:Places |
Entity | hashtags, media, urls, user_mentions | Platform Objects:Entities |
Briefly,
-
Tweets are the basic atomic building block of all things Twitter. Users tweet Tweets, also known more generically as status updates. Tweets can be embedded, replied to, favorited, unfavorited and deleted.
-
Users can be anyone or anything. They tweet, follow, create lists, have a home_timeline, can be mentioned, and can be looked up in bulk.
-
Places are specific, named locations with corresponding geo coordinates. They can be attached to Tweets by specifying a place_id when tweeting.
-
Entities provide metadata and contextual information about content posted on Twitter and are not autonomous objects.
Schemas for the StreamBase Twitter adapters encode a large subset of attributes for these objects. You can modify the schemas to include others.
The input adapter has multiple twitter stream types it can handle. Each separate input adapter can handle one of the following at a time:
-
Filter — Start consuming public statuses that match one or more keywords or user IDs. At least one keyword or user ID must be specified. Multiple parameters may be specified, allowing most clients to use a single connection to the Streaming API. The default access level allows tracking up to 200 keywords, 400 follow user IDs, and ten 1-degree location boxes. Increased access levels allow 80,000 follow user IDs ("shadow" role), 400,000 follow user IDs ("birddog" role), 10,000 track keywords ("restricted track" role), 200,000 track keywords ("partner track" role), and 200 10-degree location boxes ("locRestricted" role). Increased track access levels also pass a higher proportion of statuses before limiting the stream.
Note
To increase your access level (to gardenhose or firehose levels), you must make arrangements with Twitter, Inc. and possibly with one of its resellers. Contact the Twitter Certified Products Program, and also consult its Developer Rules of the Road page.
-
Sample — Starts listening to a random sample of all public statuses. The default access level provides a small proportion (1% or less) of the Firehose. The gardenhose access level provides a proportion more suitable for data mining and research applications that desire a larger proportion to make up a statistically significant sample.
-
Link — Starts listening on the link feed, which publishes all status updates that contain Web links.
-
**(Deprecated)User — User Streams provide a stream of data and events specific to the authenticated user and provide real-time updates of all data needed to update a desktop application display of the current user's feed, with almost no rate limits or latency. This feed also produces direct messages.
This section describes the properties you can set for this adapter, using the various tabs of the Properties view in StreamBase Studio.
Name: Use this required field to specify or change the name of this instance of this component, which must be unique in the current EventFlow module. The name must contain only alphabetic characters, numbers, and underscores, and no hyphens or other special characters. The first character must be alphabetic or an underscore.
Adapter: A read-only field that shows the formal name of the adapter.
Class name: Shows the fully qualified class name that implements the functionality of this adapter. If you need to reference this class name elsewhere in your application, you can right-click this field and select Copy from the context menu to place the full class name in the system clipboard.
Start with application: If this field is set to Yes (default) or to a module parameter that evaluates to true
, this instance of this adapter starts as part of the JVM engine that runs this EventFlow module. If this field is set to
No or to a module parameter that evaluates to false
, the adapter instance is loaded with the engine, but does not start until you send an sbadmin resume command, or until you start the component with StreamBase Manager.
Enable Error Output Port: Select this check box to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports to learn about Error Ports.
Description: Optionally enter text to briefly describe the component's purpose and function. In the EventFlow canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.
Property | Description |
---|---|
Enable Direct Messages Port | Enabling this port allows your application to receive tweets directly from users. Your API key must be have appropriate security to allow this feature. |
Enable Users Port | The users output port is used for the SearchUsers, Followers, and Following commands. |
Enable Places Port | The places output port is used for SearchPlaces commands. |
Enable Friendships Port | The friendships output port is used for LookupFriendship commands. |
Enable Trends Port | The trends output port is used for SearchTrends commands. |
Enable Rate Limits Status Port | The rate limits status output port is used for RateLimitsStatus commands. |
Enable Status Port | The status output port emits status messages for your application's consumption. |
Startup Twitter Stream | The initial stream to connect to when the application is started. |
Keywords | The keywords to use when the start up type is set up to filter. |
User Ids | The user IDs to use when the start up type is set up to filter. |
Log Level | Controls the level of verbosity the adapter uses to send notifications to the console. This setting can be higher than the containing application's log level. If set lower, the system log level is used. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE. |
Property | Type | Description |
---|---|---|
Twitter Consumer Key | string | Obtain Consumer Key from app @ developer.twitter.com |
Twitter Consumer Secret | string | Obtain Consumer Secret from app @ developer.twitter.com |
Twitter Access Token Secret | string | Obtain Access Token Secret from app @ developer.twitter.com |
Twitter Access Token | string | Obtain Access Token from app @ developer.twitter.com |
Use the Concurrency tab to specify parallel regions for this instance of this component, or multiplicity options, or both. The Concurrency tab settings are described in Concurrency Options, and dispatch styles are described in Dispatch Styles.
Caution
Concurrency settings are not suitable for every application, and using these settings requires a thorough analysis of your application. For details, see Execution Order and Concurrency, which includes important guidelines for using the concurrency options.
-
Command, string. The command to send to the adapter. Specify one of the following:
-
Filter — This command tells the adapter to listen to a filter stream based on the criteria given in the Filter tuple.
-
Sample — This command tells the adapter to start listening to the sample twitter stream.
-
Link — This command tells the adapter to start listening to the link twitter stream. This stream needs special permission from Twitter, Inc. to use.
-
**(Deprecated)User — This command tells the adapter to start listening to the current users twitter stream and to direct messages, if enabled.
-
SearchTweets — This command performs a search of all tweets based on the criteria given in the SearchTweet tuple.
-
SearchUsers — This command performs a search of users based on the criteria given in the SearchUsers tuple.
-
SearchPlaces — This command performs a search of place based on the criteria given in the SearchPlaces tuple.
-
SearchTrends — This command performs a search for trends in a geographic location using the SearchTrends tuple.
-
Timeline — This command tries to receive as many historical status updates from the current user's own timeline as it can.
-
LookupFriendship — This command looks up the friendship relationship between the current user and the specified ScreenName.
-
Follow — Start following a particular user.
-
Unfollow — Stop following a particular user.
-
Followers — List the current user's followers.
-
Following — List the users that the current user is following.
-
RateLimitsStatus — Outputs all the current user's rate limits to determine whether we can perform more operations.
-
Disconnect — tells the adapter to disconnect from the current twitter stream.
-
-
Filter, Tuple. The tuple used to create the filter stream. This field is only used when the Command is filter, and is set to one of:
-
Count, int. The number of tweets to go back to in history and fetch, before starting to stream.
-
UserIDs, list(long), (optional). A list of user IDs to add to this stream.
-
Keywords, list(string), (optional). A list of keywords to use when creating this stream.
-
-
SearchTweets, Tuple, the tuple used to search. This field is only used when the Command is SearchTweets.
-
SearchTerm, string. The text to search tweets for.
-
Count, int. The maximum number of results to return. Please note that Twitter has its own maximum that may be less than this number.
-
Since, timestamp. The starting date to look for tweets.
-
GeoLocation, Tuple. The geo location to search for tweets.
-
Longitude, double. The longitude point to search.
-
Latitude, double. The latitude point to search.
-
Radius, double. The radius search starting from the position specified by the longitude and latitude.
-
Units, string. The units to use for the radius:
km
for kilometer,mi
for miles.
-
-
-
SearchUsers, Tuple. The tuple used to search for users. This field is only used when Command is SearchUsers.
-
SearchTerm, string. The text to search tweets for.
-
-
SearchPlaces, Tuple, the tuple used to search for locations. This field is only used when Command is SearchPlaces.
Either IP or Longitude and Latitude can be specified but only one will be used.
-
Longitude, double. The latitude to search near. This parameter is ignored unless it is inside the range -90.0 to +90.0 (North is positive), inclusive. It is also ignored if there is no corresponding
Latitude
parameter. -
Latitude, double, The longitude to search near. The valid range for longitude is -180.0 to +180.0 (East is positive), inclusive. This parameter is ignored if outside that range, if it is not a number, or if there is no corresponding
Longitude
parameter. -
IP, string. An IP address used when attempting to assign geolocation based on the user's IP address. Example Values:
74.125.19.104
-
Granularity, string. This is the smallest region to return, and must be one of:
-
poi
-
neighborhood
-
city
-
admin
-
country
If no granularity is specified for the request,
neighborhood
is assumed. Specifying a granularity implies accepting all larger geographical units. For example, if set tocity
, results include places of typecity
,admin
, andcountry
. -
-
Accuracy, string. A hint on the region in which to search. If a number, then this specifies a radius in meters. It can also be a string that must be a number suffixed with
ft
to specify feet, such as100ft
. If Accuracy is not specified, it defaults to0m
. In practice, accuracy is whatever accuracy the user's device is capable of measuring at its current location.
-
-
ScreenName, String. The screen name to use with the Command Follow or Unfollow.
-
SearchTrends, Tuple. The tuple used to search for trends. This field is only used when Command is SearchTrends.
-
GeoLocation (Both fields must have valid values.)
-
Longitude, double. The latitude to search near. The valid range is -90.0 to +90.0 (North is positive), inclusive.
-
Latitude, double. The longitude to search near. The valid range is -180.0 to +180.0 (East is positive), inclusive.
-
-
The Twitter adapter must be configured with valid Twitter OAuth consumer and access tokens, obtained without charge from Twitter's developer web site. Twitter OAuth tokens encapsulate the login credentials for one Twitter account. The tokens encapsulate your Twitter account's authentication rights without storing or requiring your account's password, and thereby remain effective even when you change your Twitter password. For additional details, see the Twitter OAuth documentation.
Follow the steps below to obtain OAuth tokens from Twitter.
-
Decide in advance which Twitter account you will use for communication with your StreamBase application. You probably do not want to use a personal account for business communication, so set up a new business Twitter account if one does not already exist.
-
Proceed to http://dev.twitter.com and use the Sign in link to log in with the designated Twitter account.
-
Your Twitter account name is now displayed where the sign-in link was. Hover over it to display a context menu and select My Applications.
-
Click the Create a new application link. This opens a form titled Create an Application.
-
Fill in a Name. This name must be unique for the logged-in Twitter account.
-
Fill in the Description field. Twitter requires at least 10 characters in this field.
-
Provide a URL in the Website field.
-
For this example, leave the Callback URL blank.
-
Accept the Develop Rules Of The Road by checking the Yes, I agree box.
-
Provide the CAPTCHA response.
-
Click the Create your Twitter application button.
-
-
The application has been created. You will see the screen shown below. Click the Settings tab.
-
Under the Application Type section select the Read and Write radio button (Note: if you plan to send and receive direct messages with your application, you must instead select Read, Write and Access direct messages). This allows your application to receive and post tweets. Click Update this Twitter application's settings to complete the update.
-
Click the Details tab.
-
At the bottom of the page click Create my access token tab.
-
Copy, label, and save the Consumer Key , Consumer Secret , Access Token and Access Token Secret strings provided on this page. You will need to enter these tokens in your application's
sbd.sbconf
file.
**: User Streams and legacy Direct Message endpoints have been turned off by Twitter https://twittercommunity.com/t/details-and-what-to-expect-from-the-api-deprecations-this-week-on-august-16-2018/110746.