org.openimaj.stream.functions

Class SiteSpecificURLExtractor

java.lang.Object

org.openimaj.stream.functions.SiteSpecificURLExtractor

All Implemented Interfaces:

MultiFunction<URL,URL>

Direct Known Subclasses:

ImageSiteURLExtractor

public class SiteSpecificURLExtractor
extends Object
implements MultiFunction<URL,URL>

This class implements a function that will given an input URL outputs a list of URLs based on applying a list of SiteSpecificConsumers to the input.

Author:

Jonathon Hare (jsh2@ecs.soton.ac.uk)

Field Summary

Fields

Modifier and Type	Field and Description
protected List<SiteSpecificConsumer>	siteSpecific the site specific consumers

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	SiteSpecificURLExtractor() Construct with empty list of consumers.
	SiteSpecificURLExtractor(List<SiteSpecificConsumer> consumers) Construct with the given list of consumers.
	SiteSpecificURLExtractor(SiteSpecificConsumer... consumers) Construct with the given consumers.

Method Summary

Modifier and Type	Method and Description
List<URL>	apply(URL in) Apply the function to the input argument and return the result(s).
protected List<URL>	processURLs(URL url) First, try all the SiteSpecificConsumer instances loaded into siteSpecific.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

siteSpecific

protected List<SiteSpecificConsumer> siteSpecific

the site specific consumers

Constructor Detail

SiteSpecificURLExtractor

public SiteSpecificURLExtractor(List<SiteSpecificConsumer> consumers)

Construct with the given list of consumers.

Parameters:

consumers - the consumers

SiteSpecificURLExtractor

public SiteSpecificURLExtractor(SiteSpecificConsumer... consumers)

Construct with the given consumers.

Parameters:

consumers - the consumers

SiteSpecificURLExtractor

protected SiteSpecificURLExtractor()

Construct with empty list of consumers.

Method Detail

apply

public List<URL> apply(URL in)

Description copied from interface: MultiFunction

Apply the function to the input argument and return the result(s).

Specified by:

apply in interface MultiFunction<URL,URL>

Parameters:

in - the input object

Returns:

the result(s) of applying the function.

processURLs

protected List<URL> processURLs(URL url)

First, try all the SiteSpecificConsumer instances loaded into siteSpecific. If any consumer takes control of a link the consumer's output is used if this fails use HttpUtils#readURLAsByteArrayInputStream(URL, org.apache.http.client.RedirectStrategy) with a StatusConsumerRedirectStrategy which specifically disallows redirects to be dealt with automatically and forces this function to be called for each redirect.

Parameters:

url -

Returns:

a list of images or null