Structured Binary Data

This page will document my thoughts and design ideas for the structured binary data project. The project aims to address #317; a description of my overall approach can be found on the ​GSoC project page.

Existing Tools

I am researching existing tools related to my project, so they can be used for inspiration.

​Construct

TODO: looks promising. Also look at issues and forks.

​BinData

TODO: looks promising.

​Wireshark Generic Dissector

The length and real value of a field can depend on all previous fields and use complex expressions. Structures can contain if/while/continue/break/… statements.

Less interesting tools

​Wireshark and ​tcpdump

As the Construct wiki notes, you would expect these developers to have some sort of DSL, but they just use C for everything. Wireshark does use ASN.1, Diameter, and MIDL for protocols developed with them.

​Okteta

Has an XML format for simple structures, where the length of a field can depend on a previous value. Also has an on‐line database of structures, but it isn’t very popular—there are only nine submissions!

Other simple formats

​ffe can only handle trivial structures. ​Python’s struct and ​VStruct use concise string formats to describe simple structures.

Other hex editors

​Beye, ​Bless, and ​GHex lack interesting features.

Protocol definition formats

​ASN.1, ​MIDL, and other IPC implementations go in the other direction: they generate a binary format from a text description of a structure. ASN.1 in particular has many features.