DMelt:Numeric/7 PCA Analysis

Table of contents

PCA analysis

Principal Component Analysis (PCA) is an important for many applications. Read Principal_component_analysis.

Below we will show examples of Principal Component Analysis (PCA) data transformation using matrices as input. We will consider a situation when some of the columns in the data matrix are linearly dependent or when there are more columns than rows in the data matrix i.e. there are more dimensions than samples in the data set. In the above example we train the data and then apply to a test data.

from jhplot.math.pca import *
from Jama import Matrix

trainingData = Matrix([[1, 2, 3, 4, 5, 6],[6, 5, 4, 3, 2, 1],[2, 2, 2, 2, 2, 2]])
pca = PCA(trainingData)
testData = Matrix([[1, 2, 3, 4, 5, 6],[1, 2, 1, 2, 1, 2]]);

transformedData = pca.transform(testData, PCA.TransformationType.WHITENING);
print transformedData.toString()

More details on the corresponding Java code is in pca_transform.

The output of this code is shown below:

-0.9999999999999998, -0.5773502691896268
-0.08571428571428596, 1.732050807568878

Principal_component_analysis is used for Dimensionality_reduction, a method of transforming complex data in large dimensions into data with lesser dimensions ensuring that it conveys similar information.

It was described in [DMelt:Statistics/6_Dimensionality_reduction]. Let us consider IRIS dataset . The IRIS data set has 4 numerical attributes. Therefore, it is difficult for humans to visualize such data. Therefore, one can reduce the dimensionality of this dataset down to two. We will use Principal component analysis (PCA) which convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components. PCA needs the data samples to have a mean of ZERO, so we need a transform to ensue this property as well.

Here is the code that uses the Java package  jsat.datatransform.PCA to perform this transformation:

from java.io import File
from jsat.classifiers import DataPoint,ClassificationDataSet
from jsat.datatransform import PCA,DataTransform,ZeroMeanTransform
from jsat import ARFFLoader,DataSet

print "Download iris_org.arff"
from jhplot import *
print Web.get("https://datamelt.org/examples/data/iris_org.arff")
fi=File("iris_org.arff")
dataSet = ARFFLoader.loadArffFile(fi)
# We specify '0' as the class we would like to make the target class. 
cData = ClassificationDataSet(dataSet, 0)

# The IRIS data set has 4 numerical attributes, unfortunately humans are not good at visualizing 4 dimensional things.
# Instead, we can reduce the dimensionality down to two. 
# PCA needs the data samples to have a mean of ZERO, so we need a transform to ensue this property as well
zeroMean = ZeroMeanTransform(cData);
cData.applyTransform(zeroMean);

# PCA is a transform that attempts to reduce the dimensionality while maintaining all the variance in the data. 
# PCA also allows us to specify the exact number of dimensions we would like 
pca = PCA(cData, 2, 1e-9);
        
# We can now apply the transformations to our data set
cData.applyTransform(pca);

c1 = SPlot()
c1.visible()
c1.setAutoRange()
c1.setMarksStyle('various')
#c1.setConnected(1, 0)
c1.setNameX('X')
c1.setNameY('Y')
# output
for i in range(cData.getSampleSize()):
  dataPoint = cData.getDataPoint(i)  
  category = cData.getDataPointCategory(i) # get category 
  vec=dataPoint.getNumericalValues();
  c1.addPoint(category,vec.get(0),vec.get(1),1)
  if (i%10==0): 
                c1.update()
                print i,dataPoint,category

The output image is shown here: