org.openimaj.ml.linear.learner

Class BilinearUnmixedSparseOnlineLearner

java.lang.Object

org.openimaj.ml.linear.learner.BilinearSparseOnlineLearner

org.openimaj.ml.linear.learner.BilinearUnmixedSparseOnlineLearner

All Implemented Interfaces:

ReadableBinary, ReadWriteableBinary, WriteableBinary, OnlineLearner<gov.sandia.cognition.math.matrix.Matrix,gov.sandia.cognition.math.matrix.Matrix>

@Reference(author="Andre F. T. Martins",
           title="The Geometry of Constrained Structured Prediction: Applications to Inference and Learning of Natural Language Syntax",
           type=Phdthesis,
           year="2012")
public class BilinearUnmixedSparseOnlineLearner
extends BilinearSparseOnlineLearner

An implementation of a stochastic gradient decent with proximal parameter adjustment (for regularised parameters).

Data is dealt with sequentially using a one pass implementation of the online proximal algorithm described in chapter 9 and 10 of: The Geometry of Constrained Structured Prediction: Applications to Inference and Learning of Natural Language Syntax, PhD, Andre T. Martins

This is a direct extension of the BilinearSparseOnlineLearner but instead of a mixed update scheme (i.e. for a number of iterations W and U are updated synchronously) we have an unmixed scheme where W is updated for a number of iterations, followed by U for a number of iterations continuing as a whole for a number of iterations

The implementation does the following:

• When an X,Y is received:
  • Update currently held batch
  • If the batch is full:
    • While There is a great deal of change in U and W:
      • While There is a great deal of change in W:
        • Calculate the gradient of W holding U fixed
        • Proximal update of W
        • Calculate the gradient of Bias holding U and W fixed
      • While There is a great deal of change in U:
        • Calculate the gradient of U holding W fixed
        • Proximal update of U
        • Calculate the gradient of Bias holding U and W fixed
    • flush the batch
  • return current U and W (same as last time is batch isn't filled yet)

Author:

Sina Samangooei (ss@ecs.soton.ac.uk)

Field Summary

Fields inherited from class org.openimaj.ml.linear.learner.BilinearSparseOnlineLearner

bias, biasMode, diagX, eta0_u, eta0_w, lambda_u, lambda_w, loss, params, regul, smf, u, w

Constructor Summary

Constructors

Constructor and Description
BilinearUnmixedSparseOnlineLearner()

Method Summary

Modifier and Type	Method and Description
protected gov.sandia.cognition.math.matrix.Matrix	updateU(gov.sandia.cognition.math.matrix.Matrix currentU, gov.sandia.cognition.math.matrix.Matrix neww, double uLossWeighted, double weightedLambda)
protected gov.sandia.cognition.math.matrix.Matrix	updateW(gov.sandia.cognition.math.matrix.Matrix currentW, double wLossWeighted, double weightedLambda)

Methods inherited from class org.openimaj.ml.linear.learner.BilinearSparseOnlineLearner

addU, addW, binaryHeader, clone, etat, expandY, getBias, getParams, getU, getW, predict, process, readBinary, reinitParams, setU, setW, updateBias, writeBinary

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BilinearUnmixedSparseOnlineLearner

public BilinearUnmixedSparseOnlineLearner()

Method Detail

updateW

protected gov.sandia.cognition.math.matrix.Matrix updateW(gov.sandia.cognition.math.matrix.Matrix currentW,
                                                          double wLossWeighted,
                                                          double weightedLambda)

Overrides:

updateW in class BilinearSparseOnlineLearner

updateU

protected gov.sandia.cognition.math.matrix.Matrix updateU(gov.sandia.cognition.math.matrix.Matrix currentU,
                                                          gov.sandia.cognition.math.matrix.Matrix neww,
                                                          double uLossWeighted,
                                                          double weightedLambda)

Overrides:

updateU in class BilinearSparseOnlineLearner