NeoJSON

Sven Van Caekenberghe with Damien Cassou and Stéphane Ducasse

JSON (JavaScript Object Notation) is a popular data-interchange format. NeoJSON is an elegant and efficient standalone Smalltalk library to read and write JSON converting to and from Smalltalk objects. The library is developed and actively maintained by Sven Van Caekenberghe.

1. An Introduction to JSON

JSON is a lightweight text-based open standard designed for human-readable data interchange. It was derived from the JavaScript scripting language for representing simple data structures and associative arrays, called objects. Despite its relationship to JavaScript, it is language independent, with parsers available for many languages.

References: http://www.json.org/, http://en.wikipedia.org/wiki/Json and http://www.ietf.org/rfc/rfc4627.txt?number=4627.

There are only a couple of primitive types in JSON:

numbers (integer or floating point)
strings
the boolean constants true and false
null

Only two composite types exist:

lists (an ordered sequenece of values)
maps (an unordered associative array, mapping string property names to values)

That is really all there is to it. No options or additions are defined in the standard.

2. NeoJSON

To load NeoJSON, evaluate the following:

Gofer it
   smalltalkhubUser: 'SvenVanCaekenberghe' project: 'Neo';
   configurationOf: 'NeoJSON';
   loadStable.

The NeoJSON library contains a reader (the class NeoJSONReader) and a writer (the class NeoJSONWriter) to parse, respectively generate, JSON to and from Pharo objects. The goals of NeoJSON are:

to be standalone (have no dependencies and little requirements);
to be small, elegant and understandable;
to be efficient (both in time and space);
to be flexible and non-intrusive.

Compared to other Smalltalk JSON libraries, NeoJSON

has less dependencies and little requirements;
can be more efficient (be faster and use less memory);
allows for the use of schemas and mappings.

3. Primitives

Obviously, the primitive types are mapped to corresponding Pharo classes. While reading:

JSON numbers become instances of Integer or Float
JSON strings become instances of String
JSON booleans become instances of Boolean
JSON null becomes nil

While writing:

Pharo numbers are converted to floats, except for instances of Integer that become JSON integers
Pharo strings become JSON strings
Pharo booleans become JSON booleans
Pharo nil becomes JSON null

4. Generic Mode

NeoJSON can operate in a generic mode that requires no further configuration.

4.1. Reading from JSON

While reading:

JSON maps become instances of mapClass, Dictionary by default;
JSON lists become instances of listClass, Array by default.

The following example creates a Pharo array from a JSON expression:

NeoJSONReader fromString: ' [ 1,2,3 ] '.

This expression can be decomposed to better control the reading process:

(NeoJSONReader on: ' [ 1,2,3 ] ' readStream)
   listClass: OrderedCollection;
   next.

The above expression is equivalent to the previous one except that a Pharo ordered collection will be used in place of an array.

The next example creates a Pharo dictionary (with 'x' and 'y' keys):

NeoJSONReader fromString: ' { "x" : 1, "y" : 2 } '.

To automatically convert keys to symbols, pass true to propertyNamesAsSymbols: like this:

(NeoJSONReader on: ' { "x" : 1, "y" : 2 } ' readStream)
   propertyNamesAsSymbols: true;
   next

The result of this expression is a dictionary with #x and #y as keys.

4.2. Writing to JSON

While writing:

instances of Dictionary and SmallDictionary become maps;
all other collections become lists;
all other non-primitive objects are rejected.

Here are some examples writing in generic mode:

NeoJSONWriter toString: #(1 2 3).
NeoJSONWriter toString: { Float pi. true. false. 'string' }.
NeoJSONWriter toString: { #a -> '1' . #b -> '2' } asDictionary.

Above expressions return a compact string (i.e., with neither indentation nor new lines). To get a nicely formatted output, use toStringPretty: like this:

NeoJSONWriter toStringPretty: #(1 2 3).

In order to use the generic mode, you have to convert your domain objects to and from Dictionary and SequenceableCollection. This is relatively easy but not very efficient, depending on the use case.

5. Schemas and Mappings

NeoJSON allows for the optional specification of schemas and mappings to be used when writing or reading.

When writing, mappings are used when arbitrary objects are seen. For example, in order to write an array of points, you could do as follows:

String streamContents: [ :stream |
   (NeoJSONWriter on: stream)
      prettyPrint: true;
      mapInstVarsFor: Point;
      nextPut: (Array with: 1@3 with: -1@3) ].

Collections are handled automatically, like in the generic case. As a result, the above expression returns a string containing:

[
   {
      "x" : 1,
      "y" : 3
   },
   {
      "x" : -1,
      "y" : 3
   }
]

When reading, a mapping is used to specify what Pharo object to instantiate and how to instantiate it. Here is a very simple case, reading a map as a point:

(NeoJSONReader on: ' { "x" : 1, "y" : 2 } ' readStream)
   mapInstVarsFor: Point;
   nextAs: Point.

Since JSON lacks a universal way to specify the class of an object, we have to specify the target schema that we want to use as an argument to nextAs:.

To define the schema of the elements in a list, write something like the following:

(NeoJSONReader
      on: ' [{ "x" : 1, "y" : 2 },
             { "x" : 3, "y" : 4 }] ' readStream)
   mapInstVarsFor: Point;
   for: #ArrayOfPoints
      customDo: [ :mapping | mapping listOfElementSchema: Point ];
   nextAs: #ArrayOfPoints.

The above expression returns an array of 2 points. As you can see, the argument to nextAs: can be a class (as seen previously) or any symbol, provided the mapper knows about it.

To get an OrderedCollection instead of an array as output, you should use the listOfType: message:

(NeoJSONReader on: ' [ 1, 2 ] ' readStream)
   for: #Collection
      customDo: [ :mapping | mapping listOfType: OrderedCollection ];
   nextAs: #Collection.

To specify how values in a map should be instantiated, use the mapWithValueSchema::

(NeoJSONReader on: ' { "point1" : {"x" : 1, "y" : 2 } }' readStream)
   mapInstVarsFor: Point;
   for: #DictionaryOfPoints
      customDo: [ :mapping | mapping mapWithValueSchema: Point ];
   nextAs: #DictionaryOfPoints.

The above expression returns a Dictionary with 1 key-value pair 'point1' -> (1@2).

You can go beyond pre-defined messages and specify a decoding block:

(NeoJSONReader on: ' "2015/06/19" ' readStream)
   for: DateAndTime
      customDo: [ :mapping |
         mapping decoder: [ :string |
            DateAndTime fromString: string ] ];
   nextAs: DateAndTime.

The above expression returns an instance of DateAndTime. The message encoder: is used to do the opposite, i.e. convert from a Smalltalk object to JSON:

String streamContents: [ :stream |
   (NeoJSONWriter on: stream)
      for: DateAndTime
         customDo: [ :mapping | mapping encoder: #printString ];
      nextPut: DateAndTime now ].

The above expression returns a string representing the current date and time.

NeoJSON deals efficiently with mappings: the minimal amount of intermediary structures are created. On modern hardware, NeoJSON can write or read tens of thousands of small objects per second. Several benchmarks are included in the unit tests package.

6. Emitting null Values

For efficiency reasons, by default, NeoJSONWriter does not write nil values:

String streamContents: [ :stream |
   (NeoJSONWriter on: stream)
      mapAllInstVarsFor: Point;
      nextPut: Point new ].

The above expression returns the '{}' string. If you want to see the uninitialized instance properties, pass true to the writeNil: message:

String streamContents: [ :stream |
   (NeoJSONWriter on: stream)
      mapAllInstVarsFor: Point;
      writeNil: true;
      nextPut: Point new ].

The above expression returns the '{"x":null,"y":null}' string.

7. Conclusion

NeoJSON is a powerful library to convert objects. Sven, the author of NeoJSON, also developed STON (Smalltalk object notation) which is closer to Pharo syntax and handles cycles and references between serialized objects.