org.openimaj.stream.provider.twitter

Class AbstractTwitterSearchDataset<T>

java.lang.Object

org.openimaj.util.stream.AbstractStream<T>

org.openimaj.util.stream.BlockingDroppingBufferedStream<T>

org.openimaj.stream.provider.twitter.AbstractTwitterSearchDataset<T>

Type Parameters:

T - Type of items in the stream

All Implemented Interfaces:

Iterable<T>, Iterator<T>, Dataset<T>, StreamingDataset<T>, Stream<T>

Direct Known Subclasses:

ContextRoundRobinTwitterSearchAPIDataset, RoundRobinTwitterSearchAPIDataset, TwitterSearchDataset

public abstract class AbstractTwitterSearchDataset<T>
extends BlockingDroppingBufferedStream<T>
implements StreamingDataset<T>

Calls the SearchResource.search(Query) function periodically and offers all discovered Status instances with the underlying BlockingDroppingBufferedStream. This stream initially takes all search results available and then attempts to retrieve results which occurred after the latest Tweet seen. Apart form this, this Stream makes no special efforts to:

• stop duplicate items
• stop when a search returns nothing
• remove retweets or spam.

These must all be handled externally!

This class is abstract in terms of the object the stream consumes. Instances of this class can construct the specific object they require for each twitter status using:

• the query that created it
• the status object
• the raw json of the status

Author:

Sina Samangooei (ss@ecs.soton.ac.uk)

Field Summary

Fields

Modifier and Type	Field and Description
protected twitter4j.conf.Configuration	config
protected org.apache.log4j.Logger	logger
protected twitter4j.Query	query
protected twitter4j.Twitter	twitter

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractTwitterSearchDataset(TwitterAPIToken token, BlockingDroppingQueue<T> buffer, twitter4j.Query query)

Method Summary

Modifier and Type	Method and Description
abstract twitter4j.Query	getQuery() Get the current query
T	getRandomInstance()
protected twitter4j.conf.Configuration	makeConfiguration(TwitterAPIToken token)
int	numInstances() Returns the number of instances in this dataset.
protected abstract void	registerStatus(twitter4j.Query query, twitter4j.Status status, String rawjson) Handle the given incoming status and optionally BlockingDroppingBufferedStream.register(Object) it with the stream.

Methods inherited from class org.openimaj.util.stream.BlockingDroppingBufferedStream

close, getBuffer, hasNext, next, register

Methods inherited from class org.openimaj.util.stream.AbstractStream

filter, forEach, forEach, forEach, iterator, map, map, parallelForEach, parallelForEach, remove, transform

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.openimaj.util.stream.Stream

filter, forEach, forEach, forEach, map, map, parallelForEach, parallelForEach, transform

Methods inherited from interface java.util.Iterator

forEachRemaining, hasNext, next, remove

Methods inherited from interface java.lang.Iterable

forEach, iterator, spliterator

Field Detail

query

protected twitter4j.Query query

logger

protected org.apache.log4j.Logger logger

config

protected twitter4j.conf.Configuration config

twitter

protected twitter4j.Twitter twitter

Constructor Detail

AbstractTwitterSearchDataset

protected AbstractTwitterSearchDataset(TwitterAPIToken token,
                                       BlockingDroppingQueue<T> buffer,
                                       twitter4j.Query query)

Parameters:

token -

buffer -

Method Detail

registerStatus

protected abstract void registerStatus(twitter4j.Query query,
                                       twitter4j.Status status,
                                       String rawjson)
                                throws InterruptedException

Handle the given incoming status and optionally BlockingDroppingBufferedStream.register(Object) it with the stream.

Parameters:

query - the query

status - the parsed Status

rawjson - the raw json string

Throws:

InterruptedException

getQuery

public abstract twitter4j.Query getQuery()

Get the current query

Returns:

the query

makeConfiguration

protected twitter4j.conf.Configuration makeConfiguration(TwitterAPIToken token)

getRandomInstance

public T getRandomInstance()

Specified by:

getRandomInstance in interface Dataset<T>

Returns:

a randomly selected instance from the dataset.

numInstances

public int numInstances()

Description copied from interface: Dataset

Returns the number of instances in this dataset.

Specified by:

numInstances in interface Dataset<T>

Returns:

the number of instances in this list