org.openimaj.ml.regression

Class LinearRegression

java.lang.Object

org.openimaj.ml.regression.LinearRegression

All Implemented Interfaces:

Cloneable, EstimatableModel<double[],double[]>, Model<double[],double[]>

public class LinearRegression
extends Object
implements EstimatableModel<double[],double[]>

Given a set of independant variables a linear regressions finds the optimal vector B such that: (Y - Xb)^2 = 0 (Y - Xb)^{T}(Y-Xb) = 0 calculated by assuming a convex shape of (Y - Xb) with varying values of b (reasonable as the function is linear) and then calculating the point at which the first derivative of this function is 0. i.e.: d/db (y - Xb)^{T} (y - Xb) = -X^{T}(y - Xb) - X^{T}(y - Xb) = - 2 * X^{T}(y - Xb) which at the 0 is: - 2 * X^{T}(y - Xb) = 0 X^{T}(y - Xb) X^{T}y - X^{T}Xb = 0 X^{T}y = X^{T}Xb b = (X^{T} X)^{-1} X^{T} y Calculating this function directly behaves numerically badly when X is extremely skinny and tall (i.e. lots of data, fewer dimentions) so we calculate this using the SVD, using the SVD we can decompose X as: X = UDV^{T} s.t. U and V are orthonormal from this we can calculate: b = V D^{-1} U^{T} y which is equivilant but more numerically stable. Note that upon input any vector of independent variables x_n are automatically to turned into an n + 1 vector {1,x0,x1,...,xn} which handles the constant values added to y

Author:

Sina Samangooei (ss@ecs.soton.ac.uk)

Constructor Summary

Constructors

Constructor and Description
LinearRegression() linear regression model

Method Summary

Modifier and Type	Method and Description
LinearRegression	clone() Clone the model
boolean	equals(Object obj)
void	estimate(double[][] yd, double[][] xd) As in estimate(List) but using double arrays for efficiency.
boolean	estimate(List<? extends IndependentPair<double[],double[]>> data) Estimates the model from the observations in the list of data.
void	estimate(Jama.Matrix y, Jama.Matrix x) As in estimate(List) but using double arrays for efficiency.
int	numItemsToEstimate()
double[]	predict(double[] data) Uses the model to predict dependent data from an independent value.
Jama.Matrix	predict(Jama.Matrix x) Helper function which adds the constant component to x and returns predicted values for y, one per row
String	toString()

Methods inherited from class java.lang.Object

finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

LinearRegression

public LinearRegression()

linear regression model

Method Detail

estimate

public boolean estimate(List<? extends IndependentPair<double[],double[]>> data)

Description copied from interface: EstimatableModel

Estimates the model from the observations in the list of data. The data must contain at least EstimatableModel.numItemsToEstimate() pairs of dependent and independent data. It may contain more, in which case the estimate method may choose to make use of this data for validation, or obtaining a better model by a least squares method for example.

Specified by:

estimate in interface EstimatableModel<double[],double[]>

Parameters:

data - Data with which to estimate the model

Returns:

true if a successful estimate was made, false otherwise

See Also:

EstimatableModel.numItemsToEstimate()

estimate

public void estimate(double[][] yd,
                     double[][] xd)

As in estimate(List) but using double arrays for efficiency.

Parameters:

yd -

xd -

estimate

public void estimate(Jama.Matrix y,
                     Jama.Matrix x)

As in estimate(List) but using double arrays for efficiency. Estimates: b = V D^{-1} U^{T} y s.t. X = UDV^{T}

Parameters:

y -

x -

predict

public double[] predict(double[] data)

Description copied from interface: Model

Uses the model to predict dependent data from an independent value.

Specified by:

predict in interface Model<double[],double[]>

Parameters:

data - the data (independent variable)

Returns:

Dependent variable(s) predicted from the independent ones.

predict

public Jama.Matrix predict(Jama.Matrix x)

Helper function which adds the constant component to x and returns predicted values for y, one per row

Parameters:

x -

Returns:

predicted y

numItemsToEstimate

public int numItemsToEstimate()

Specified by:

numItemsToEstimate in interface EstimatableModel<double[],double[]>

Returns:

The minimum number of observations required to estimate the model.

clone

public LinearRegression clone()

Description copied from interface: EstimatableModel

Clone the model

Specified by:

clone in interface EstimatableModel<double[],double[]>

Overrides:

clone in class Object

Returns:

a cloned copy

equals

public boolean equals(Object obj)

Overrides:

equals in class Object

toString

public String toString()

Overrides:

toString in class Object