org.openimaj.experiment.evaluation.retrieval.analysers

Class TRECEvalAnalyser<QUERY,DOCUMENT extends Identifiable>

java.lang.Object

org.openimaj.experiment.evaluation.retrieval.analysers.TRECEvalAnalyser<QUERY,DOCUMENT>

Type Parameters:

QUERY - Type of query

DOCUMENT - Type of document

All Implemented Interfaces:

RetrievalAnalyser<TRECResult,QUERY,DOCUMENT>

public class TRECEvalAnalyser<QUERY,DOCUMENT extends Identifiable>
extends Object
implements RetrievalAnalyser<TRECResult,QUERY,DOCUMENT>

A RetrievalAnalyser that uses the trec_eval commandline tool to perform the analysis.

Author:

Jonathon Hare (jsh2@ecs.soton.ac.uk)

Constructor Summary

Constructors

Constructor and Description
TRECEvalAnalyser() Default constructor

Method Summary

Modifier and Type	Method and Description
TRECResult	analyse(Map<QUERY,List<DOCUMENT>> results, Map<QUERY,Set<DOCUMENT>> relevant) Analyse ranked results from a RetrievalEngine against a ground-truth set of relevant results and produce an AnalysisResult which can be read by a human.
static <Q,D extends Identifiable> void	writeQRELS(Map<Q,Set<D>> relevant, PrintStream os) Write the ground-truth data in TREC QRELS format.
static <Q,D extends Identifiable> void	writeTop(Map<Q,List<D>> results, PrintStream os) Write retrieval results in TREC TOP format.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TRECEvalAnalyser

public TRECEvalAnalyser()

Default constructor

Method Detail

analyse

public TRECResult analyse(Map<QUERY,List<DOCUMENT>> results,
                          Map<QUERY,Set<DOCUMENT>> relevant)

Description copied from interface: RetrievalAnalyser

Analyse ranked results from a RetrievalEngine against a ground-truth set of relevant results and produce an AnalysisResult which can be read by a human.

Specified by:

analyse in interface RetrievalAnalyser<TRECResult,QUERY,DOCUMENT extends Identifiable>

Parameters:

results - the results per query from the RetrievalEngine.

relevant - the ground-truth relevant documents per query.

Returns:

the results of the analysis

writeTop

public static <Q,D extends Identifiable> void writeTop(Map<Q,List<D>> results,
                                                       PrintStream os)

Write retrieval results in TREC TOP format.

Type Parameters:

Q - Type of query

D - Type of Document

Parameters:

results - the ranked results.

os - stream to write to

writeQRELS

public static <Q,D extends Identifiable> void writeQRELS(Map<Q,Set<D>> relevant,
                                                         PrintStream os)

Write the ground-truth data in TREC QRELS format.

Type Parameters:

Q - Type of query

D - Type of Document

Parameters:

relevant - the relevant docs for each query

os - stream to write to