DMelt:IO/8 Cross Paltform IO
Cross platform I/O
Many DMelt object (data arrays, histograms, functions) can be saved and restored using XML files without using the Java serialization. The Java class which writes data in XML is called jhplot.io.HBook or jhplot.io.CFBook (which has slightly different XML tags). The class jhplot.io.CFBook can read XML files with histograms created in C++ or Fortran programs.
These classes, unlike the standard Java serialization (including XML) discussed in Section Java Serialization, is very similar to the "AIDA" implementation. Notably:
- jhplot.io.HBook keeps only information content of objects (values, titles, labels), without graphical attributes (Color, fonts)
- jhplot.io.HBook does not use XML tags to keep separate values. This means that this class generates substantially smaller outputs since the standard tags for data values are not used. This is especially important for large data volumes, 2D matrices and histograms.
- jhplot.io.HBook is better suited for cross platform. In particular, C++ and fortran packages are available to read and write data in this format. See the package CFBook.
Here is a small example of how to write a few DataMelt objects:
from jhplot import * from jhplot.io import * hb = HBook("output.jdat","w") # HBook object p1=P1D("test") p1.add(1.0,2.0) p1.add(2.0,3.0) hb.write("data",p1) # we add data using "data" key f1=F1D("x*cos(x*10)") hb.write(10,f1) # we can also add a function using numeric key (integer) hb.close() # write and close the file.
The output of the JDAT file look as:
<dmelt> <created-by>JDAT file. Work.ORG: @S.Chekanov</created-by> <created-on>Sat Jan 26 20:43:42 CST 2013</created-on> <description>writer</description> <version>2</version> <f1d> <id>10</id> <title>x*cos(x*10)</title> <name>x*cos(x*10)</name> </f1d> <p1d> <id>data</id> <title>test</title> <size>2</size> <dimen>2</dimen> <labelx>X</labelx> <labely>Y</labely> <data> 1.0,2.0 2.0,3.0 </data> </p1d> </dmelt>
All XML tags of JDAT are self-explanatory. Now we will read both objects using their keys:
from jhplot import * from jhplot.io import * hb = HBook("output.jdat") # read HBook object print hb.getKeys() # print all the keys p1=hb.get("data") f1=hb.get('10') print p1.toString() print f1.getName()
The expected number is:
array(java.lang.String, [u'10', u'data']) # title=test dimension=2 # X Y 1.0 2.0 2.0 3.0 x*cos(x*10)
Here is a more detailed example. We write data arrays, histogram and a function to 3 types of files with the extensions .jdat (HBook), .jser (HFile), .jpbu (PFile, protocol buffer). Then we open the DataBrowser for each file automatically and then plot the objects using the mouse clicks.
The data can be organized in "directories" inside the HBook XML files and can be viewed in the browser as a trees. Here is an example showing this:
Parse HBook files in CPython
HBook files with the extension ".jdat" can be read in CPython (Python implemented in C) using "xml.dom" Python module. In this case, one can use the output in a platform-specific languages (for example, CPython can be interfaced with C++/C libraries). Here is a small example how to parse jdat files in the CPython:
In this example ".jdat" file includes several P1D objects. We read them all objects with the keys and create a map where the ID of the object is the key. The array of this map has data.
jhplot.io.CFBook can visualize histograms created by Fortran or C++ code. For this, use a light stand-alone library called CFBook (See https://datamelt.org/?id=cfbook-library). It can be linked to a C++ or Fortran code. You can compile it using gcc (for C++ programs) or gfortran (for Fortran program). This library creates XML file with 1D and 2D histograms, that can be read by Jas4pp. Here is an example of reading 1D histogram from fortran.xml file:
from jhplot import * from jhplot.io import * hb = CFBook() hb.read("fortran.xml") print hb.listAll() print hb.getKeysH1D() # list keys h1=hb.getH1D(1) # use the key 1 to retrive H1D c1=HPlot("Test") c1.setGTitle("Histograms from a file"); c1.visible(1) c1.setAutoRange() c1.draw(h1)
This class is an attempt to build a multi-platform (and multi-language) I/O format based the Google's protocol buffer package. This package can be used to write files using a C++ program (or any other language) and read using Java, or write data in Java and read by a C/C++ programs.
The DataMelt Java class which implements the Google's protocol buffer format is called jhplot.io.PFile. The class jhplot.io.PFile is designed to store mainly DataMelt containers (arrays) described in Data Structures and data projected to 1,2 and 3 dimensions (histograms) described in Histograms section.
Once data are written in C/C++ with the help of the CBook package, one can read such data with the jhplot.io.PFile class or even to open data in a interactive Browser using jhplot.io.PFileBrowser class.
Here is a more complicated example: We write a 2D histogram and 1D array into a file. Then we read the objects back and plot them. One can use the class jhplot.io.HFile as well (just replace PFile by HFile). But, in case of PFile, you can write histograms and arrays using a C++ code and read data back using Java/Jython!
Unlike the HFile class, only pre-defined data containers can be stored in PFile files (all jhplot arrays, functions, histograms). PFile class is optimized for write/read speed and small output. In addition, one can read/write such files in C++.
from jhplot.io import * from jhplot import * from java.util import Random c1 = HPlot3D("Canvas",600,400,2,1) c1.visible(1) c1.setAutoRange() h1 = H2D("input2D",5,-2, 2.0, 5, -3,3) r = Random(33) for i in range(100): h1.fill(r.nextGaussian(), r.nextGaussian()) c1.draw(h1) f=PFile('tmp.jpbu','w') f.write(h1) f.write(P0D("sss")) f.close() c1.cd(2,1) c1.setAutoRange() f=PFile('tmp.jpbu','r') print f.size() # how many objects are stored print f.listEntries() # list all entries, the size of objects and the compression level c1.draw( f.read("input2D") ) f.close()
In this example, instead of sequential reading the objects one by one, we retrieve the objects using its key "input2D" (which is the title of the histogram). Open the created file in a browser for easy plotting:
from jhplot.io import * from jhplot import * c1=HPlot("Browser") c1.visible() f=PFile("tmp.jpbu") PFileBrowser(c1,f,1)
You will see a pop-up windows with all data entries. Click on the entry - the object will be plotted on the canvas.
Using DataBrowser to open ".jpbu" files
All DataMelt objects stored in compressed Java-serialized files can be viewed using a browser. For example, if a serialized file contains P1D, P0D, H1D, etc. objects, one can view them and plot them using a mouse-click approach.
If you have a file with the extension ".jpbu", you can view it using the DataBrowser. Go to the toolbar, select [Plot}->[HPlot canvas]-> [File]-> [Open data file].
Working with C++ external programs
Here we discuss how to create files with DataMelt data containers (arrays, histograms, functions) using an C++ code and then how to read them using 100% Java code of the DataMelt.