org.openimaj.experiment.evaluation.retrieval

Class RetrievalEvaluator<RESULT extends AnalysisResult,DOCUMENT extends Identifiable,QUERY>

java.lang.Object

org.openimaj.experiment.evaluation.retrieval.RetrievalEvaluator<RESULT,DOCUMENT,QUERY>

Type Parameters:

RESULT - Type of analysed data

DOCUMENT - Type of documents

QUERY - Type of query

All Implemented Interfaces:

Evaluator<Map<QUERY,List<DOCUMENT>>,RESULT>

public class RetrievalEvaluator<RESULT extends AnalysisResult,DOCUMENT extends Identifiable,QUERY>
extends Object
implements Evaluator<Map<QUERY,List<DOCUMENT>>,RESULT>

An implementation of an Evaluator for the evaluation of retrieval experiments using the Cranfield methodology.

Author:

Jonathon Hare (jsh2@ecs.soton.ac.uk)

Field Summary

Fields

Modifier and Type	Field and Description
protected RetrievalAnalyser<RESULT,QUERY,DOCUMENT>	analyser
protected RetrievalEngine<DOCUMENT,QUERY>	engine
protected Collection<QUERY>	queries
protected Map<QUERY,Set<DOCUMENT>>	relevant

Constructor Summary

Constructors

Constructor and Description
RetrievalEvaluator(Map<QUERY,List<DOCUMENT>> results, Map<QUERY,Set<DOCUMENT>> relevant, RetrievalAnalyser<RESULT,QUERY,DOCUMENT> analyser) Construct a new RetrievalEvaluator with the given ranked results lists and sets of relevant documents for each query, and a RetrievalAnalyser to analyse the results.
RetrievalEvaluator(RetrievalEngine<DOCUMENT,QUERY> engine, Collection<QUERY> queries, Map<QUERY,Set<DOCUMENT>> relevant, RetrievalAnalyser<RESULT,QUERY,DOCUMENT> analyser) Construct a new RetrievalEvaluator with a search engine, a set of queries to perform, relevant documents for each query, and a RetrievalAnalyser to analyse the results.
RetrievalEvaluator(RetrievalEngine<DOCUMENT,QUERY> engine, Map<QUERY,Set<DOCUMENT>> relevant, RetrievalAnalyser<RESULT,QUERY,DOCUMENT> analyser) Construct a new RetrievalEvaluator with a search engine, relevant documents for each query, and a RetrievalAnalyser to analyse the results.

Method Summary

Modifier and Type	Method and Description
RESULT	analyse(Map<QUERY,List<DOCUMENT>> results) Analyse the raw data produced by a call to Evaluator.evaluate() and return the analysed data.
Map<QUERY,List<DOCUMENT>>	evaluate() Run, or evaluate the system, returning the raw data.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

engine

protected RetrievalEngine<DOCUMENT extends Identifiable,QUERY> engine

queries

protected Collection<QUERY> queries

relevant

protected Map<QUERY,Set<DOCUMENT extends Identifiable>> relevant

analyser

protected RetrievalAnalyser<RESULT extends AnalysisResult,QUERY,DOCUMENT extends Identifiable> analyser

Constructor Detail

RetrievalEvaluator

public RetrievalEvaluator(RetrievalEngine<DOCUMENT,QUERY> engine,
                          Collection<QUERY> queries,
                          Map<QUERY,Set<DOCUMENT>> relevant,
                          RetrievalAnalyser<RESULT,QUERY,DOCUMENT> analyser)

Construct a new RetrievalEvaluator with a search engine, a set of queries to perform, relevant documents for each query, and a RetrievalAnalyser to analyse the results.

Parameters:

engine - the query engine

queries - the queries

relevant - the relevant documents for each query

analyser - the analyser

RetrievalEvaluator

public RetrievalEvaluator(RetrievalEngine<DOCUMENT,QUERY> engine,
                          Map<QUERY,Set<DOCUMENT>> relevant,
                          RetrievalAnalyser<RESULT,QUERY,DOCUMENT> analyser)

Construct a new RetrievalEvaluator with a search engine, relevant documents for each query, and a RetrievalAnalyser to analyse the results. The queries are determined automatically from the keys of the map of relevant documents.

Parameters:

engine - the query engine

relevant - the relevant documents for each query

analyser - the analyser

RetrievalEvaluator

public RetrievalEvaluator(Map<QUERY,List<DOCUMENT>> results,
                          Map<QUERY,Set<DOCUMENT>> relevant,
                          RetrievalAnalyser<RESULT,QUERY,DOCUMENT> analyser)

Construct a new RetrievalEvaluator with the given ranked results lists and sets of relevant documents for each query, and a RetrievalAnalyser to analyse the results.

Internally, this constructor wraps a simple RetrievalEngine implementation around the results, and determines the set of queries from the keys of the relevant document map.

Parameters:

results - the ranked results per query

relevant - the relevant results per query

analyser - the analyser

Method Detail

evaluate

public Map<QUERY,List<DOCUMENT>> evaluate()

Description copied from interface: Evaluator

Run, or evaluate the system, returning the raw data.

Specified by:

evaluate in interface Evaluator<Map<QUERY,List<DOCUMENT extends Identifiable>>,RESULT extends AnalysisResult>

Returns:

the raw data

analyse

public RESULT analyse(Map<QUERY,List<DOCUMENT>> results)

Description copied from interface: Evaluator

Analyse the raw data produced by a call to Evaluator.evaluate() and return the analysed data.

Specified by:

analyse in interface Evaluator<Map<QUERY,List<DOCUMENT extends Identifiable>>,RESULT extends AnalysisResult>

Parameters:

results - the raw data.

Returns:

the analysed data