org.openimaj.text.nlp.language

Class LanguageDetector

java.lang.Object

org.openimaj.text.nlp.language.LanguageDetector

@Reference(type=Article,
           author={"Lui, Marco","Baldwin, Timothy"},
           title="Cross-domain Feature Selection for Language Identification",
           year="2011",
           booktitle="in Proceedings of 5th International Joint Conference on Natural Language Processing")
public class LanguageDetector
extends Object

Short text language detection ported from langid: https://github.com/saffsd/langid.py

Author:

Sina Samangooei (ss@ecs.soton.ac.uk)

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	LanguageDetector.WeightedLocale A langauge with an associated confidence

Field Summary

Fields

Modifier and Type	Field and Description
static String	LANGUAGE_MODEL_BINARY default location of the compressed binary version of the language model
static String	LANGUAGE_MODEL_JSON default location of the compressed json version language model

Constructor Summary

Constructors

Constructor and Description
LanguageDetector() Load a language model from "/org/openimaj/text/language/language.model.binary.gz"
LanguageDetector(LanguageModel model) Create a language detector with a provided language model

Method Summary

Modifier and Type	Method and Description
LanguageDetector.WeightedLocale	classify(String text) Classify the language using a naive-bayes model
LanguageModel	getLanguageModel()
static void	main(String[] args) prints available languages

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LANGUAGE_MODEL_JSON

public static final String LANGUAGE_MODEL_JSON

default location of the compressed json version language model

See Also:

Constant Field Values

LANGUAGE_MODEL_BINARY

public static final String LANGUAGE_MODEL_BINARY

default location of the compressed binary version of the language model

See Also:

Constant Field Values

Constructor Detail

LanguageDetector

public LanguageDetector()
                 throws IOException

Load a language model from "/org/openimaj/text/language/language.model.binary.gz"

Throws:

IOException

LanguageDetector

public LanguageDetector(LanguageModel model)

Create a language detector with a provided language model

Parameters:

model -

Method Detail

classify

public LanguageDetector.WeightedLocale classify(String text)

Classify the language using a naive-bayes model

Parameters:

text -

Returns:

the detected language

getLanguageModel

public LanguageModel getLanguageModel()

Returns:

the underlying LanguageModel

main

public static void main(String[] args)
                 throws IOException

prints available languages

Parameters:

args -

Throws:

IOException