org.openimaj.ml.linear.learner

Class BilinearSparseOnlineLearner

java.lang.Object

org.openimaj.ml.linear.learner.BilinearSparseOnlineLearner

All Implemented Interfaces:

ReadableBinary, ReadWriteableBinary, WriteableBinary, OnlineLearner<gov.sandia.cognition.math.matrix.Matrix,gov.sandia.cognition.math.matrix.Matrix>

Direct Known Subclasses:

BilinearSparseFISTALearner, BilinearUnmixedSparseOnlineLearner

public class BilinearSparseOnlineLearner
extends Object
implements OnlineLearner<gov.sandia.cognition.math.matrix.Matrix,gov.sandia.cognition.math.matrix.Matrix>, ReadWriteableBinary

An implementation of a stochastic gradient decent with proximal perameter adjustment (for regularised parameters). Data is dealt with sequentially using a one pass implementation of the online proximal algorithm described in chapter 9 and 10 of: The Geometry of Constrained Structured Prediction: Applications to Inference and Learning of Natural Language Syntax, PhD, Andre T. Martins The implementation does the following: - When an X,Y is recieved: - Update currently held batch - If the batch is full: - While There is a great deal of change in U and W: - Calculate the gradient of W holding U fixed - Proximal update of W - Calculate the gradient of U holding W fixed - Proximal update of U - Calculate the gradient of Bias holding U and W fixed - flush the batch - return current U and W (same as last time is batch isn't filled yet)

Author:

Sina Samangooei (ss@ecs.soton.ac.uk)

Field Summary

Fields

Modifier and Type	Field and Description
protected gov.sandia.cognition.math.matrix.Matrix	bias
protected Boolean	biasMode
protected gov.sandia.cognition.math.matrix.Matrix	diagX
protected Double	eta0_u
protected Double	eta0_w
protected Double	lambda_u
protected Double	lambda_w
protected LossFunction	loss
protected BilinearLearnerParameters	params
protected Regulariser	regul
protected gov.sandia.cognition.math.matrix.mtj.SparseMatrixFactoryMTJ	smf
protected gov.sandia.cognition.math.matrix.Matrix	u
protected gov.sandia.cognition.math.matrix.Matrix	w

Constructor Summary

Constructors

Constructor and Description
BilinearSparseOnlineLearner() The default parameters.
BilinearSparseOnlineLearner(BilinearLearnerParameters params)

Method Summary

Modifier and Type	Method and Description
void	addU(int newUsers) Expand the U parameters matrix by added a set of rows.
void	addW(int newWords) Expand the W parameters matrix by added a set of rows.
byte[]	binaryHeader() Header for binary input.
BilinearSparseOnlineLearner	clone()
protected double	etat(int iter, double eta0)
static gov.sandia.cognition.math.matrix.mtj.SparseMatrix	expandY(gov.sandia.cognition.math.matrix.Matrix Y) Given a flat value matrix, makes a diagonal sparse matrix containing the values as the diagonal
gov.sandia.cognition.math.matrix.Matrix	getBias()
BilinearLearnerParameters	getParams()
gov.sandia.cognition.math.matrix.Matrix	getU()
gov.sandia.cognition.math.matrix.Matrix	getW()
gov.sandia.cognition.math.matrix.Matrix	predict(gov.sandia.cognition.math.matrix.Matrix x)
void	process(gov.sandia.cognition.math.matrix.Matrix X, gov.sandia.cognition.math.matrix.Matrix Y)
void	readBinary(DataInput in) Read internal state from in.
void	reinitParams() must be called if any parameters are changed
void	setU(gov.sandia.cognition.math.matrix.Matrix newu)
void	setW(gov.sandia.cognition.math.matrix.Matrix neww)
protected gov.sandia.cognition.math.matrix.Matrix	updateBias(gov.sandia.cognition.math.matrix.Matrix xt, gov.sandia.cognition.math.matrix.Matrix nu, gov.sandia.cognition.math.matrix.Matrix nw, double biasLossWeight)
protected gov.sandia.cognition.math.matrix.Matrix	updateU(gov.sandia.cognition.math.matrix.Matrix xtrows, gov.sandia.cognition.math.matrix.Matrix neww, double uLossWeight, double uWeightedLambda)
protected gov.sandia.cognition.math.matrix.Matrix	updateW(gov.sandia.cognition.math.matrix.Matrix xt, double wLossWeighted, double weightedLambda)
void	writeBinary(DataOutput out) Write the content of this as binary to out.

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

params

protected BilinearLearnerParameters params

w

protected gov.sandia.cognition.math.matrix.Matrix w

u

protected gov.sandia.cognition.math.matrix.Matrix u

smf

protected gov.sandia.cognition.math.matrix.mtj.SparseMatrixFactoryMTJ smf

loss

protected LossFunction loss

regul

protected Regulariser regul

lambda_w

protected Double lambda_w

lambda_u

protected Double lambda_u

biasMode

protected Boolean biasMode

bias

protected gov.sandia.cognition.math.matrix.Matrix bias

diagX

protected gov.sandia.cognition.math.matrix.Matrix diagX

eta0_u

protected Double eta0_u

eta0_w

protected Double eta0_w

Constructor Detail

BilinearSparseOnlineLearner

public BilinearSparseOnlineLearner()

The default parameters. These won't work with your dataset, i promise.

BilinearSparseOnlineLearner

public BilinearSparseOnlineLearner(BilinearLearnerParameters params)

Parameters:

params - the parameters used by this learner

Method Detail

reinitParams

public void reinitParams()

must be called if any parameters are changed

process

public void process(gov.sandia.cognition.math.matrix.Matrix X,
                    gov.sandia.cognition.math.matrix.Matrix Y)

Specified by:

process in interface OnlineLearner<gov.sandia.cognition.math.matrix.Matrix,gov.sandia.cognition.math.matrix.Matrix>

updateBias

protected gov.sandia.cognition.math.matrix.Matrix updateBias(gov.sandia.cognition.math.matrix.Matrix xt,
                                                             gov.sandia.cognition.math.matrix.Matrix nu,
                                                             gov.sandia.cognition.math.matrix.Matrix nw,
                                                             double biasLossWeight)

updateW

protected gov.sandia.cognition.math.matrix.Matrix updateW(gov.sandia.cognition.math.matrix.Matrix xt,
                                                          double wLossWeighted,
                                                          double weightedLambda)

updateU

protected gov.sandia.cognition.math.matrix.Matrix updateU(gov.sandia.cognition.math.matrix.Matrix xtrows,
                                                          gov.sandia.cognition.math.matrix.Matrix neww,
                                                          double uLossWeight,
                                                          double uWeightedLambda)

expandY

public static gov.sandia.cognition.math.matrix.mtj.SparseMatrix expandY(gov.sandia.cognition.math.matrix.Matrix Y)

Given a flat value matrix, makes a diagonal sparse matrix containing the values as the diagonal

Parameters:

Y -

Returns:

the diagonalised Y

etat

protected double etat(int iter,
                      double eta0)

getParams

public BilinearLearnerParameters getParams()

Returns:

the current apramters

getU

public gov.sandia.cognition.math.matrix.Matrix getU()

Returns:

the current user matrix

getW

public gov.sandia.cognition.math.matrix.Matrix getW()

Returns:

the current word matrix

getBias

public gov.sandia.cognition.math.matrix.Matrix getBias()

Returns:

the current bias (null if BilinearLearnerParameters.BIAS is false

addU

public void addU(int newUsers)

Expand the U parameters matrix by added a set of rows. If currently unset, this function does nothing (assuming U will be initialised in the first round) The new U parameters are initialised used BilinearLearnerParameters.EXPANDEDUINITSTRAT

Parameters:

newUsers - the number of new users to add

addW

public void addW(int newWords)

Expand the W parameters matrix by added a set of rows. If currently unset, this function does nothing (assuming W will be initialised in the first round) The new W parameters are initialised used BilinearLearnerParameters.EXPANDEDWINITSTRAT

Parameters:

newWords - the number of new words to add

clone

public BilinearSparseOnlineLearner clone()

Overrides:

clone in class Object

setU

public void setU(gov.sandia.cognition.math.matrix.Matrix newu)

Parameters:

newu - set the model's U

setW

public void setW(gov.sandia.cognition.math.matrix.Matrix neww)

Parameters:

neww - set the model's W

readBinary

public void readBinary(DataInput in)
                throws IOException

Description copied from interface: ReadableBinary

Read internal state from in.

Specified by:

readBinary in interface ReadableBinary

Parameters:

in - source to read from.

Throws:

IOException - an error reading input

binaryHeader

public byte[] binaryHeader()

Description copied from interface: ReadableBinary

Header for binary input. Will be automatically read by IOUtils when using readBinary().

Specified by:

binaryHeader in interface ReadableBinary

Specified by:

binaryHeader in interface WriteableBinary

Returns:

header

writeBinary

public void writeBinary(DataOutput out)
                 throws IOException

Description copied from interface: WriteableBinary

Write the content of this as binary to out.

Specified by:

writeBinary in interface WriteableBinary

Parameters:

out - sink to write to

Throws:

IOException - an error writing to out

predict

public gov.sandia.cognition.math.matrix.Matrix predict(gov.sandia.cognition.math.matrix.Matrix x)

Specified by:

predict in interface OnlineLearner<gov.sandia.cognition.math.matrix.Matrix,gov.sandia.cognition.math.matrix.Matrix>