DMelt:IO/3 Java Serialization
Using Java serialization
Serialization is the process of converting data or object state into a format that can be stored or transmitted and reconstructed later.
DataMelt Serialized class
Data can be serialized into a file using the jhplot.IO class. The only limitation is the computer memory, since the object has to be created first. In this example we create a list with two 2D arrays and write it to the file. Then we restore it.
from jhplot import * from jhplot.io import * p1=P1D("test") # create 2D array and fill it p1.add(10,20) p1.add(12,40) print p1.toString() p2=P0D("test2") # create 2D arrays and fill it for i in range(10): p2.add(i*i) a=[p1,p2] # make a list Serialized.write(a,'file.ser') # write the list to the file fas=Serialized.read('file.ser') # read the data from the file p1=fas; p2=fas # read 2D arryas from the list print "After serialization:n" print p1.toString(); print p2.toString()
One can also use SerializedXML class, which writes/reads data to/from XML files. Let us take a look how to write histograms into XML serialized file and then read it back
The class jhplot.io.HFile is designed to store any Java object (including containers) in a compact serialized form. It is designed to store large data volumes without memory limitations (unlike the class Serialized considered above). It is well suited for sequential input and outputs.
For example one can store data containers and functions described Data structures,
The class is based on the standard Java serialization mechanism (compression is by default, but can be switched off). A typical extension for such files is ".jser". Essentially, almost any Java object supported by DMelt can be stored and retrieved from the HFile file.
Data stored in a file created by the class jhplot.io.HFile can be viewed in the browser based on the class jhplot.io.HFileBrowser One can also store data in the XML form using the class jhplot.io.HFileXML.
Let us give an example: we create 3 objects (array and two histograms) and write then into compressed serialized file (use HFileXML to write into XML format). The we read this file and create these objects:
from jhplot.io import * from jhplot import * x=P0D('X'); y=P0D('Y') x.randomUniform(10,-3,3) y.randomUniform(10,0,30) p1=P1D("XY",x,y) h1 = H1D("fixed bins",10, -2, 2.0) h2 = H1D("variable",[-1,2,4,7]) r = Random() for i in range(100): h1.fill(r.nextGaussian()) h2.fill(5*r.nextGaussian()) f=HFile('tmp.jser','w') f.write(p1) f.write(h1) f.write(h2) f.close() # reading objects f=HFile('tmp.jser','r') p=f.read() h1=f.read() h2=f.read() print "Created the file=" print h1.toString() print h1.toString() f.close()
You can append any any number of objects to this file. You can even make a Jython map or list from different objects and write such containers in one go.
There is no any restriction on which Java object is written. One can write arbitrary complicated data in form of arrays, strings, lists, tuples, maps, dictionaries, DataMelt functions, histograms etc. Any Java, Jython, DataMelt or any third-party Java container which can hold data can be written into a file. If you want to write objects which can be retrieved using keys, use maps or dictionaries.
Using the keys
Before we considered data records organized sequentially. One can also store objects using keys in form of string. The keys should be unique.
from jhplot import * from jhplot.io import * p1=P0D() p1.randomNormal(1000,0,2) # 1000 random numbers p2=P0D() p2.randomNormal(1000,1,2) # 1000 random numbers pp=P1D("test",p1,p2) g=HFile("output.jser","w") g.write("key1",p1) g.write("key2",p2) g.write("key3",pp) g.write("directory/key4",p1) # store the objects in the directory g.write("directory/key5",p2) g.close() # now reading the objects back g=HFile("output.jser") print g.get("directory/key4") # prints p1
The notion of "directory" is important. Now one can organize data using some meaningful logic and open data in a browser as will be discussed below.
Be careful mixing data inside HFile without keys and with keys. If you have inserted a lot of records without using keys, you will pay a penalty in retrieval of objects with keys, since objects with keys will always be extracted last, after scanning through all other objects without the keys.
DataBrowser to open ".jser" files
All DataMelt objects stored in compressed Java-serialized files can be viewed using a browser. For example, if a serialized file contains P1D, P0D, H1D objects, one can view them and plot them using a mouse-click approach.
If you have a file with the extension ".jser", you can view it using the DataBrowser. Go to the toolbar, select [Plot}->[HPlot canvas]-> [File]-> [Open data file].
One can also open the browser fin a macro as:
from jhplot.io import * from jhplot import * c1=HPlot("Browser") c1.visible() f=HFile("test.ser") HFileBrowser(c1,f,1)