org.openimaj.hadoop.tools.twitter.token.mode.pointwisemi.count

Class TokenPairUnaryCount

java.lang.Object

org.openimaj.util.pair.IndependentPair<T,T>

org.openimaj.util.pair.Pair<String>

org.openimaj.hadoop.tools.twitter.token.mode.pointwisemi.count.TokenPairCount

org.openimaj.hadoop.tools.twitter.token.mode.pointwisemi.count.TokenPairUnaryCount

All Implemented Interfaces:

Readable, ReadableASCII, ReadableBinary, ReadWriteable, Writeable, WriteableASCII, WriteableBinary

public class TokenPairUnaryCount
extends TokenPairCount

A Pair count with a unary count for each item of the pair. The values here are the 3 counting functions: c(x,y) = Number of times the pair x and y were seen together c(x) = Number of times x was seen with ANY other token c(y) = Number of times y was seen with ANY other token

Author:

Sina Samangooei (ss@ecs.soton.ac.uk)

Field Summary

Fields

Modifier and Type	Field and Description
long	tok1count count of token 1 pairs
long	tok2count count of token 2 pairs

Fields inherited from class org.openimaj.hadoop.tools.twitter.token.mode.pointwisemi.count.TokenPairCount

isSingle, paircount

Constructor Summary

Constructors

Constructor and Description
TokenPairUnaryCount() Convenience
TokenPairUnaryCount(String tok1, String tok2, long paircount, long tok1count, long tok2count)
TokenPairUnaryCount(TokenPairCount tpc, long tok1count, long tok2count) same as TokenPairUnaryCount(String, String, long, long, long) using the values from the TokenPairCount instance

Method Summary

Modifier and Type	Method and Description
double	pmi(double n) Calculate the Pointwise mutual information score such that: PMI(x,y) = log( p(x,y) / ( p(x) p(y) ) ) where we can estimate the probabilities as: p(x,y) = c(x,y) / n p(x) = c(x) / n p(y) = c(y) / n where n is the total number of pairs observed
void	readASCII(Scanner in) Read internal state from in.
void	readBinary(DataInput in) Read internal state from in.
void	writeASCII(PrintWriter out) Write the content of this as ascii to out.
void	writeBinary(DataOutput out) Write the content of this as binary to out.

Methods inherited from class org.openimaj.hadoop.tools.twitter.token.mode.pointwisemi.count.TokenPairCount

add, asciiHeader, binaryHeader, identifier, identifier, identifierBinary, parseTimeTokenID, timeFromBinaryIdentity, timeFromBinaryIdentity, toString

Methods inherited from class org.openimaj.util.pair.IndependentPair

equals, firstObject, getFirst, getFirstFunction, getFirstObject, getSecond, getSecondFunction, getSecondObject, pair, pairList, secondObject, setFirstObject, setSecondObject, swap, swapList

Methods inherited from class java.lang.Object

clone, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

tok1count

public long tok1count

count of token 1 pairs

tok2count

public long tok2count

count of token 2 pairs

Constructor Detail

TokenPairUnaryCount

public TokenPairUnaryCount()

Convenience

TokenPairUnaryCount

public TokenPairUnaryCount(String tok1,
                           String tok2,
                           long paircount,
                           long tok1count,
                           long tok2count)

Parameters:

tok1 - the first token (x)

tok2 - the second token (y)

paircount - the count of the tokens together (c(x,y))

tok1count - the count of the first token with any other token (c(x))

tok2count - the count of the second token with any other token (c(y))

TokenPairUnaryCount

public TokenPairUnaryCount(TokenPairCount tpc,
                           long tok1count,
                           long tok2count)

same as TokenPairUnaryCount(String, String, long, long, long) using the values from the TokenPairCount instance

Parameters:

tpc -

tok1count -

tok2count -

Method Detail

writeBinary

public void writeBinary(DataOutput out)
                 throws IOException

Description copied from interface: WriteableBinary

Write the content of this as binary to out.

Specified by:

writeBinary in interface WriteableBinary

Overrides:

writeBinary in class TokenPairCount

Parameters:

out - sink to write to

Throws:

IOException - an error writing to out

readBinary

public void readBinary(DataInput in)
                throws IOException

Description copied from interface: ReadableBinary

Read internal state from in.

Specified by:

readBinary in interface ReadableBinary

Overrides:

readBinary in class TokenPairCount

Parameters:

in - source to read from.

Throws:

IOException - an error reading input

writeASCII

public void writeASCII(PrintWriter out)
                throws IOException

Description copied from interface: WriteableASCII

Write the content of this as ascii to out.

Specified by:

writeASCII in interface WriteableASCII

Overrides:

writeASCII in class TokenPairCount

Parameters:

out - sink to write to

Throws:

IOException - an error writing to out

readASCII

public void readASCII(Scanner in)
               throws IOException

Description copied from interface: ReadableASCII

Read internal state from in.

Specified by:

readASCII in interface ReadableASCII

Overrides:

readASCII in class TokenPairCount

Parameters:

in - source to read from.

Throws:

IOException - an error reading input

pmi

public double pmi(double n)

Calculate the Pointwise mutual information score such that: PMI(x,y) = log( p(x,y) / ( p(x) p(y) ) ) where we can estimate the probabilities as: p(x,y) = c(x,y) / n p(x) = c(x) / n p(y) = c(y) / n where n is the total number of pairs observed

Parameters:

n - the total number of pairs observed

Returns:

the PMI estimate