org.openimaj.text.nlp.namedentity

Class IgnoreTokenStripper

java.lang.Object

org.openimaj.text.nlp.namedentity.IgnoreTokenStripper

public class IgnoreTokenStripper
extends Object

Class to remove stopwords from a list of tokens, or to check if a word is a stopword.

Author:

Laurence Willmore (lgw1e10@ecs.soton.ac.uk), Sina Samangooei (ss@ecs.soton.ac.uk)

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	IgnoreTokenStripper.Language Language to build stripper from.

Constructor Summary

Constructors

Constructor and Description
IgnoreTokenStripper(IgnoreTokenStripper.Language language) Constructor for specified language.

Method Summary

Modifier and Type	Method and Description
ArrayList<String>	getNonStopWords(List<String> intokens) Strips given list of tokens of all ignore words.
boolean	isIgnoreToken(String token) Checks if given token is an ignore word

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

IgnoreTokenStripper

public IgnoreTokenStripper(IgnoreTokenStripper.Language language)

Constructor for specified language.

Parameters:

language -

Method Detail

getNonStopWords

public ArrayList<String> getNonStopWords(List<String> intokens)

Strips given list of tokens of all ignore words.

Parameters:

intokens -

Returns:

list of clean tokens.

isIgnoreToken

public boolean isIgnoreToken(String token)

Checks if given token is an ignore word

Parameters:

token -

Returns:

true if ignore Token