org.openimaj.hadoop.tools.twitter

Class HadoopTwitterTokenToolOptions

java.lang.Object

org.openimaj.tools.InOutToolOptions

org.openimaj.hadoop.tools.twitter.HadoopTwitterTokenToolOptions

public class HadoopTwitterTokenToolOptions
extends InOutToolOptions

Hadoop specific options for twitter preprocessing

Author:

Sina Samangooei (ss@ecs.soton.ac.uk)

Field Summary

Fields

Modifier and Type	Field and Description
static String	ARGS_KEY The key in which command line arguments are held for each mapper to read the options instance

Fields inherited from class org.openimaj.tools.InOutToolOptions

output

Constructor Summary

Constructors

Constructor and Description
HadoopTwitterTokenToolOptions(String[] args)
HadoopTwitterTokenToolOptions(String[] args, String[] originalArgs, boolean beforeMaps) Initialise the options

Method Summary

Modifier and Type	Method and Description
String[]	getArgs()
JsonPathFilterSet	getFilters()
String	getJsonPath()
String[]	getNonHadoopArgs()
StatusType	getStatusType()
boolean	noOutput()
void	output(TwitterTokenMode mode)
void	performPreprocessing() If there were any preprocessing arguments, perform the preprocessing and use the preprocessing output as the input to the rest of the process.
void	prepare()
void	prepareCL() prepare the tool for running (command line version)
USMFStatus	readStatus(String svalue)
<T> T	readStatusPart(String svalue) Read json from text and try to extract the part to the type required

Methods inherited from class org.openimaj.tools.InOutToolOptions

getAllInputs, getInput, getInputFile, getOutput, isContinue, isForce, overwriteOutput, prepareMultivaluedArgument, setInput, setInputFile, setOutput

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ARGS_KEY

public static final String ARGS_KEY

The key in which command line arguments are held for each mapper to read the options instance

See Also:

Constant Field Values

Constructor Detail

HadoopTwitterTokenToolOptions

public HadoopTwitterTokenToolOptions(String[] args,
                                     String[] originalArgs,
                                     boolean beforeMaps)
                              throws org.kohsuke.args4j.CmdLineException

Initialise the options

Parameters:

args - the arguments after going through the hadoop tool (i.e. minus the -D hadoop arguments)

originalArgs - the original arguments as typed into the command line (useful for subhadoop tasks launched)

beforeMaps - whether this job is occuring before the maps

Throws:

org.kohsuke.args4j.CmdLineException

HadoopTwitterTokenToolOptions

public HadoopTwitterTokenToolOptions(String[] args)
                              throws org.kohsuke.args4j.CmdLineException

Parameters:

args - Just the arguments (hadoop arguments assumed to be the same)

Throws:

org.kohsuke.args4j.CmdLineException

Method Detail

prepareCL

public void prepareCL()

prepare the tool for running (command line version)

prepare

public void prepare()
             throws org.kohsuke.args4j.CmdLineException

Throws:

org.kohsuke.args4j.CmdLineException

noOutput

public boolean noOutput()

Returns:

is there any actual output this phase

getJsonPath

public String getJsonPath()

Returns:

the JSONPath query used to extract tokens

getArgs

public String[] getArgs()

Returns:

the original arguments including the hadoop arguments

getNonHadoopArgs

public String[] getNonHadoopArgs()

Returns:

the arguments minus the hadoop arguments

output

public void output(TwitterTokenMode mode)
            throws Exception

Parameters:

mode - output a completed token mode

Throws:

Exception

performPreprocessing

public void performPreprocessing()
                          throws Exception

If there were any preprocessing arguments, perform the preprocessing and use the preprocessing output as the input to the rest of the process.

Throws:

Exception

getFilters

public JsonPathFilterSet getFilters()

readStatus

public USMFStatus readStatus(String svalue)
                      throws IOException

Throws:

IOException

readStatusPart

public <T> T readStatusPart(String svalue)
                     throws IOException

Read json from text and try to extract the part to the type required

Type Parameters:

T -

Parameters:

svalue -

Returns:

a part of type T

Throws:

IOException

getStatusType

public StatusType getStatusType()