DMelt:IO/3 Java Serialization

From HandWiki
Limitted access. First login to DataMelt if you are a full DataMelt member. Then login to HandWiki as a user.

Using Java serialization

Serialization is the process of converting data or object state into a format that can be stored or transmitted and reconstructed later.

DataMelt Serialized class

Data can be serialized into a file using the jhplot.IO jhplot.IO class. The only limitation is the computer memory, since the object has to be created first. In this example we create a list with two 2D arrays and write it to the file. Then we restore it.

from jhplot  import *
from import *

p1=P1D("test") # create 2D array and fill it
print p1.toString()

p2=P0D("test2")  # create 2D arrays and fill it
for i in range(10):

a=[p1,p2]                      # make a list
Serialized.write(a,'file.ser') # write the list to the file'file.ser') # read the data from the file

p1=fas[0];  p2=fas[1]           # read 2D arryas from the list
print "After serialization:n"
print p1.toString(); print p2.toString()

One can also use SerializedXML class, which writes/reads data to/from XML files. Let us take a look how to write histograms into XML serialized file and then read it back

HFile class

The class is designed to store any Java object (including containers) in a compact serialized form. It is designed to store large data volumes without memory limitations (unlike the class Serialized considered above). It is well suited for sequential input and outputs.

For example one can store data containers and functions described Data structures,

The class is based on the standard Java serialization mechanism (compression is by default, but can be switched off). A typical extension for such files is ".jser". Essentially, almost any Java object supported by DMelt can be stored and retrieved from the HFile file.

Data stored in a file created by the class can be viewed in the browser based on the class One can also store data in the XML form using the class

Let us give an example: we create 3 objects (array and two histograms) and write then into compressed serialized file (use HFileXML to write into XML format). The we read this file and create these objects:

from import *
from jhplot import *

x=P0D('X'); y=P0D('Y') 

h1 = H1D("fixed bins",10, -2, 2.0)
h2 = H1D("variable",[-1,2,4,7])
r = Random()
for i in range(100):


# reading objects
print "Created the file="
print h1.toString()
print h1.toString()

You can append any any number of objects to this file. You can even make a Jython map or list from different objects and write such containers in one go.

There is no any restriction on which Java object is written. One can write arbitrary complicated data in form of arrays, strings, lists, tuples, maps, dictionaries, DataMelt functions, histograms etc. Any Java, Jython, DataMelt or any third-party Java container which can hold data can be written into a file. If you want to write objects which can be retrieved using keys, use maps or dictionaries.

Using the keys

Before we considered data records organized sequentially. One can also store objects using keys in form of string. The keys should be unique.

from jhplot import *
from import *

p1.randomNormal(1000,0,2) # 1000 random numbers
p2.randomNormal(1000,1,2) # 1000 random numbers
g.write("directory/key4",p1) # store the objects in the directory

# now reading the objects back
print g.get("directory/key4")  # prints p1

The notion of "directory" is important. Now one can organize data using some meaningful logic and open data in a browser as will be discussed below.

DataBrowser to open ".jser" files

All DataMelt objects stored in compressed Java-serialized files can be viewed using a browser. For example, if a serialized file contains P1D, P0D, H1D objects, one can view them and plot them using a mouse-click approach.

If you have a file with the extension ".jser", you can view it using the DataBrowser. Go to the toolbar, select [Plot}->[HPlot canvas]-> [File]-> [Open data file].

One can also open the browser fin a macro as:

from  import *
from jhplot  import *