DMelt:IO/6 EDN File Format

Table of contents

EDN file format

DataMelt includes library to handle extensible data notation format. The EDN format is described in EDN web page. The included implementation is based on the edn-java gitlab. The Java API can be found in  us.bpsm.edn.parser.package-summary.

The EDN files can hold any Java and Python object in a form of simple text files. The files are typically smaller then similar XML files since no tags is used. This feature make EDM files attractive to keep numeric data.

For example, Java ArrayList or Python list will be written to the text file exactly it should look:

[1,2,3,4,5]

You can also write lists, maps (dictionaries) to external files in arbitrary order. Then you can read such text files back, creating Java lists (Python lists) on the fly.

This is an example of parsing EDN string messages:

This is an example:

from us.bpsm.edn.parser import Parsers 
from us.bpsm.edn.parser import Parser
from us.bpsm.edn.parser import Parseable
from us.bpsm.edn.parser.Parsers import defaultConfiguration;
from java.util import *


pbr = Parsers.newParseable("{:x 1, :y 2}");
p = Parsers.newParser(defaultConfiguration());
a=p.nextValue(pbr)
print type(a), ":", a


pbr = Parsers.newParseable("{1 0 2 9 3 8 4 7 5 6}")
a=p.nextValue(pbr)
print type(a), ":", a

Now let us consider a complete example: We create Java list, Python list, Java map, Python dictionary, fill them and write all these objects sequentially into a file. Then we read the encoded objects back, creating on the fly lists, maps and dictionaries:

# This example shows how to write different objects in EDN files
 
from us.bpsm.edn.parser import Parsers 
from us.bpsm.edn.parser import Parser
from us.bpsm.edn.parser import Parseable
from us.bpsm.edn.printer import Printers
from us.bpsm.edn.parser.Parsers import defaultConfiguration;
from java.util import *
from java.io import *

sw = StringWriter()
p=Printers.newPrinter(sw)

print "Write a Java list"
al = ArrayList()
al.add(1)
al.add(2)
p.printValue(al)


print "Write Python dowble list"
s=[[22],[33,33]]
p.printValue(s)

print "Write a map"
map = HashMap()
map.put("name", 10000)
p.printValue(map)

print "Write a Python dictionary"
map={}
map["key1"]="VALUE1"
map["key2"]="VALUE2"
p.printValue(map)
data=sw.toString()
print "Data so far = ",data


print "Write this to text file"
fw = FileWriter("file.edn")
fw.write(data)
fw.close()


print "reading data back.."
pbr = Parsers.newParseable(data);
p = Parsers.newParser(defaultConfiguration());

a=p.nextValue(pbr)
print type(a), ":", a

a=p.nextValue(pbr)
print type(a), ":", a

a=p.nextValue(pbr)
print type(a), ":", a

a=p.nextValue(pbr)
print type(a), ":", a

You can use this class to help with parsing the file:

pbr = Parsers.newParseable(IOUtil.stringFromResource("us/bpsm/edn/edn-sample.txt"))

Read this using  us.bpsm.edn.parser.IOUtil API.