# DMelt:Statistics/6 Dimensionality reduction

# Dimensionality reduction

Dimensionality reduction is a method of transforming complex data in large dimensions into data with lesser dimensions ensuring that it conveys similar information.

Let us consider IRIS dataset ^{[1]}.
The IRIS data set has 4 numerical attributes. Therefore, it is difficult for humans to visualize such data.
Therefore, one can reduce the dimensionality of this dataset down to two.
We will use Principal component analysis (PCA) which
convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components.
PCA needs the data samples to have a mean of ZERO, so we need a transform to ensue this property as well.

Here is the code that uses the Java package jsat.datatransform.PCA to perform this transformation:

from java.io import File from jsat.classifiers import DataPoint,ClassificationDataSet from jsat.datatransform import PCA,DataTransform,ZeroMeanTransform from jsat import ARFFLoader,DataSet print "Download iris_org.arff" from jhplot import * print Web.get("https://datamelt.org/examples/data/iris_org.arff") fi=File("iris_org.arff") dataSet = ARFFLoader.loadArffFile(fi) # We specify '0' as the class we would like to make the target class. cData = ClassificationDataSet(dataSet, 0) # The IRIS data set has 4 numerical attributes, unfortunately humans are not good at visualizing 4 dimensional things. # Instead, we can reduce the dimensionality down to two. # PCA needs the data samples to have a mean of ZERO, so we need a transform to ensue this property as well zeroMean = ZeroMeanTransform(cData); cData.applyTransform(zeroMean); # PCA is a transform that attempts to reduce the dimensionality while maintaining all the variance in the data. # PCA also allows us to specify the exact number of dimensions we would like pca = PCA(cData, 2, 1e-9); # We can now apply the transformations to our data set cData.applyTransform(pca); c1 = SPlot() c1.visible() c1.setAutoRange() c1.setMarksStyle('various') #c1.setConnected(1, 0) c1.setNameX('X') c1.setNameY('Y') # output for i in range(cData.getSampleSize()): dataPoint = cData.getDataPoint(i) category = cData.getDataPointCategory(i) # get category vec=dataPoint.getNumericalValues(); c1.addPoint(category,vec.get(0),vec.get(1),1) if (i%10==0): c1.update() print i,dataPoint,category

The output image is shown here:

- ↑ Fisher,R.A. "The use of multiple measurements in taxonomic problems", Annual Eugenics, 7, Part II, 179-188 (1936); also in "Contributions to Mathematical Statistics" (John Wiley, NY, 1950).