org.openimaj.picslurper

Class StatusConsumer

java.lang.Object

org.openimaj.picslurper.StatusConsumer

public class StatusConsumer
extends Object

A status consumer knows how to consume a StreamJSONStatusList.ReadableWritableJSON and output image files. Currently this StatusConsumer only understands Twitter JSON, perhaps making it abstract and turning consume(Status) into an abstract function that can deal with other types of status would be sensible

Author:

Sina Samangooei (ss@ecs.soton.ac.uk)

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	StatusConsumer.StatusConsumerRedirectStrategy An extention of the HttpUtils.MetaRefreshRedirectStrategy which disallows all redirects and instead remembers a redirect for use later on.

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.log4j.Logger	logger The logger
static List<SiteSpecificConsumer>	siteSpecific the site specific consumers

Constructor Summary

Constructors

Constructor and Description
StatusConsumer() for convenience
StatusConsumer(boolean outputStats, File globalStats, File outputLocation, List<OutputListener> outputModes)

Method Summary

Modifier and Type	Method and Description
void	add(String newURL) Add a URL to process without allowing already seen URLs to be added
StatusConsumption	consume(twitter4j.Status status)
StatusConsumption	processAll(twitter4j.Status status) Process all added URLs
File	resolveURL(URL url, StatusConsumption cons) Given a URL, use urlToImage(URL) to turn the url into a list of images and write the images into the output location using the names "image_N.png"
static String	sanitizeFilename(String name) Replaces illegal characters in a filename with "_" illegal characters : : \ / * ? | < >
List<IndependentPair<URL,MBFImage>>	urlToImage(URL url) First, try all the SiteSpecificConsumer instances loaded into siteSpecific.
static File	urlToOutput(URL url, File outputLocation) Construct a file in the output location for a given url

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

logger

public static org.apache.log4j.Logger logger

The logger

siteSpecific

public static final List<SiteSpecificConsumer> siteSpecific

the site specific consumers

Constructor Detail

StatusConsumer

public StatusConsumer(boolean outputStats,
                      File globalStats,
                      File outputLocation,
                      List<OutputListener> outputModes)

Parameters:

outputStats - whether statistics should be outputted

globalStats - the global statistics file

outputLocation - the output location for this status

outputModes - the output modes informed on image downloads

StatusConsumer

public StatusConsumer()

for convenience

Method Detail

consume

public StatusConsumption consume(twitter4j.Status status)
                          throws Exception

Parameters:

status -

Returns:

the statistics of the consumption

Throws:

Exception

processAll

public StatusConsumption processAll(twitter4j.Status status)
                             throws IOException

Process all added URLs

Parameters:

status -

Returns:

the StatusConsumption statistics

Throws:

IOException

add

public void add(String newURL)

Add a URL to process without allowing already seen URLs to be added

Parameters:

newURL -

resolveURL

public File resolveURL(URL url,
                       StatusConsumption cons)

Given a URL, use urlToImage(URL) to turn the url into a list of images and write the images into the output location using the names "image_N.png"

Parameters:

url -

cons - the consumption stats

Returns:

the root output location

urlToImage

public List<IndependentPair<URL,MBFImage>> urlToImage(URL url)

First, try all the SiteSpecificConsumer instances loaded into siteSpecific. If any consumer takes control of a link the consumer's output is used if this fails use HttpUtils.readURLAsByteArrayInputStream(URL, org.apache.http.client.RedirectStrategy) with a StatusConsumer.StatusConsumerRedirectStrategy which specifically disallows redirects to be dealt with automatically and forces this function to be called for each redirect.

Parameters:

url -

Returns:

a list of images or null

urlToOutput

public static File urlToOutput(URL url,
                               File outputLocation)
                        throws IOException

Construct a file in the output location for a given url

Parameters:

url -

outputLocation -

Returns:

a file that looks like: outputLocation/protocol/path/query/...

Throws:

IOException

sanitizeFilename

public static String sanitizeFilename(String name)

Replaces illegal characters in a filename with "_" illegal characters : : \ / * ? | < >

Parameters:

name -

Returns:

Sanitised filename