DEV Community

John Matthias Wabwire
John Matthias Wabwire

Posted on

Show Dev: A Data Interchange Format

#showdev #programming #api #data

Link: https://github.com/HyaenaTechnologies/configuration/blob/main/documentation/data-interchange.md

Dynamically Typed Syntax 

// Data Interchange Format
// File Extension: .data, .schema
// Schema Definition Header File: .schema
// Data Interchange Source File: .data
// .data Files May be Dynamically Typed
// .data Files May be Statically Typed
// .data Files are Required
// .schema Files Must be Statically Typed
// .schema Files Cannot be Dynamically Typed
// .schema Files are Optional

// This is a Comment

// Data Structure Initialization
// Syntax - Name = {fields}
DataStructure = {
    integer = 1;    
    float = 1.0;
    boolean_type = true;
    character = 'd';
    string = "string";
    array = [];
    hash_map = [()];
    DataStructure = {}; 
}

// Boolean Initialization
// Syntax - name = value;
// OR
// Null Syntax - name = null;
boolean_type = true;

// Character Initialization
// Syntax - name = value;
// OR
// Null Syntax - name = null;
character = 's';

// Float Initialization
// Syntax - name = value;
// OR
// Null Syntax - name = null;
float = 1.0;

// Integer Initialization
// Syntax - name = value;
// OR
// Null Syntax - name = null;
integer = 1;

// String Initialization
// Syntax - name = value;
// OR
// Null Syntax - name = null;
string = "string";

// Array Initialization
// Syntax - name = [values]
// OR
// Null Syntax - name = [null]
array = [
    1,
    2,
    3,
    4,
    5
]

// Hash Map Initialization
// Syntax - name = [(key|value)]
// OR
// Null Syntax - name = [(null|null)]
hash_map = [
    (1|"string"),
    (2|"string"),
    (3|"string"),
    (4|"string"),
    (5|"string")
]
Enter fullscreen mode Exit fullscreen mode

Statically Typed Syntax 

// Data Interchange Format
// File Extension: .data, .schema
// Schema Definition Header File: .schema
// Data Interchange Source File: .data
// .data Files May be Dynamically Typed
// .data Files May be Statically Typed
// .data Files are Required
// .schema Files Must be Statically Typed
// .schema Files Cannot be Dynamically Typed
// .schema Files are Optional

// This is a Comment

// Option Type Syntax - option<type|type> name = value;

// Data Structure Declaration
// May be Declared with Initialized Fields
// Syntax - type Name = {fields}
// Optional Field Syntax - optional<type> name;
struct DataStructure = {
    option<uint8|null> unsigned_integer_8bit;
    uint16 unsigned_integer_16bit;
    uint32 unsigned_integer_32bit;
    uint64 unsigned_integer_64bit;
    optional<uint128> unsigned_integer_128bit;
    option<int8|null> signed_integer_8bit;
    int16 signed_integer_16bit;
    int32 signed_integer_32bit;
    int64 signed_integer_64bit;
    optional<int128> signed_integer_128bit;
    option<float32|null> float_32bit;
    float64 float_64bit;
    optional<float128> float_128bit;
    bool boolean_type;
    option<char8|null> character_8bit;
    char16 character_16bit;
    optional<char32> character_32bit;
    option<str8|null> string_8bit;
    str16 string_16bit;
    optional<str32> string_32bit;
    array(5)<uint8> one_dimensional_array;
    array(5:2)<uint8> two_dimensional_array;
    map<uint8|str8> hash_map;
    DataStructure data_structure;
    EnumatorType enumerator_type;
}

// Enumerator Declaration
// May be Declared with Initialized Fields
// Syntax - type Name = {fields}
enum EnumeratorType = {
    FIRST_VALUE, // value = 0;
    SECOND_VALUE, // value = 1;
    THIRD_VALUE, // value = 2;
    FOURTH_VALUE, // value = 3;
    FIFTH_VALUE // value = 4;
}

// Data Structure Initialization
// Syntax - type name = {fields}
DataStructure data_structure = {
    uint8 unsigned_integer_8bit = null;
    uint16 unsigned_integer_16bit = 2;
    uint32 unsigned_integer_32bit = 3;
    uint64 unsigned_integer_64bit = 4;
    uint128 unsigned_integer_128bit = 5;
    int8 signed_integer_8bit = null;
    int16 signed_integer_16bit = -2;
    int32 signed_integer_32bit = -3;
    int64 signed_integer_64bit = -4;
    int128 signed_integer_128bit = -5;
    float32 float_32bit = null;
    float64 float_64bit = 2.0;
    float128 float_128bit = 3.0;
    bool boolean_type = true;
    char8 character_8bit = null;
    char16 character_16bit = 'c';
    char32 character_32bit = 'd';
    str8 string_8bit = null;
    str16 string_16bit = "string";
    str32 string_32bit = "string";
    array(5)<uint8> one_dimensional_array = [];
    array(5:2)<uint8> two_dimensional_array = [];
    map<uint8|str8> hash_map = [()];
    DataStructure data_structure = {};
    EnumratorType enumerator_type = FIRST_VALUE;
}

// Enumerator Initialization
// Syntax - type name = value;
// OR
// Null Syntax - type name = null;
EnumeratorType enumerator_type = FIRST_VALUE;

// Boolean Initialization
// Syntax - type name = value;
// OR
// Null Syntax - type name = null;
bool boolean_type = true;

// Character Initialization
// Syntax - type name = value;
// OR
// Null Syntax - type name = null;
char8 character_8bit = 's';

// Float Initialization
// Syntax - type name = value;
// OR
// Null Syntax - type name = null;
float32 float_32bit = 1.0;

// Integer Initialization
// Syntax - type name = value;
// OR
// Null Syntax - type name = null;
uint8 unsigned_integer_8bit = 1;

// String Initialization
// Syntax - type name = value;
// OR
// Null Syntax - type name = null;
str8 string_8bit = "string";

// One Dimensional Array Initialization
// Syntax - type(length)<type> name = [values]
// OR
// Unknown Size Syntax - type(?)<type> name = [values]
// OR
// Null Syntax - type(length)<type> name = [null]
array(5)<uint8> one_dimensional_array = [
    1,
    2,
    3,
    4,
    5
]

// Two Dimensional Array Initialization
// Syntax - type(length:width)<type> name = [values]
// OR
// Unknown Size Syntax - type(?:?)<type> name = [values]
// OR
// Null Syntax - type(length:width)<type> name = [null]
array(3:2)<uint8> two_dimensional_array = [
    1,
    2,
    3,
    4,
    5,
    6
]

// Hash Map Initialization
// Syntax - type<type|type> name = [(key|value)]
// OR
// Null Syntax - type<type|type> name = [(null|null)]
map<uint8|str8> hash_map = [
    (1|"string"),
    (2|"string"),
    (3|"string"),
    (4|"string"),
    (5|"string")
]
Enter fullscreen mode Exit fullscreen mode

Top comments (2)

Collapse
 
peacebinflow profile image
PEACEBINFLOW •

This is actually a really interesting middle ground between JSON, protobuf, and a proper type system. What stands out to me is that you’re not just defining a data format — you’re defining intent about data. The dynamic version feels like a cleaner, more explicit JSON, but the static side is where it gets spicy: fixed-width ints, option vs optional, array dimensions, and enum semantics all baked into the format itself.

I especially like the split between .schema and .data. That’s a smart move. It makes the format usable both as “just shove some data around” and as a contract between systems, which is where most interchange formats fall apart or get bolted-on validation later.

The option/optional distinction is subtle but powerful too — that’s the kind of thing that saves you from silent bugs in distributed systems. Same with array shape declarations; that’s very “stop pretending everything is a blob and start respecting structure.”

Big question for me (in a good way): what’s the target audience — config files, wire format, or human-authored schemas for APIs? Because this feels like it could be a legit alternative to JSON+Schema if tooling (parser, validator, serializer) comes along.

Overall: this doesn’t feel like a toy syntax experiment. It feels like someone actually thought about how data breaks in the real world and tried to design around it. That alone makes it worth paying attention to.

Collapse
 
johnmatthiaswabwire profile image
John Matthias Wabwire • • Edited

Thank you so much for providing feedback. I have been looking for feedback since I completed the Specification. The Format is designed to replace JSON, YAML, TOML, XML, Protocol Buffers, SQL and HTTP. This was achieved by splitting the design between a Dynamically Typed Syntax and a Statically Typed Syntax. The Dynamically Typed Syntax is for Configuration, Dynamic Data Storage and Low-Performance Network Communication (HTTP). The Statically Typed Syntax is for Inter-Process Communication, Remote Procedure Calls, Static Data Storage and High-Performance Network Communication with Binary Serialization (Protobuf). The Syntax of the design introduces Delimiters, Scoping, Optional Schemas and Optional Static Typing. Data Interchange Formats are critical and the ones that we have been using are poorly designed because they fail to understand Dennis Ritchie, Ken Thompson and Rob Pikes principles at Bell Labs, in cooperation with Niklaus Wirth. A header file for Definition, a Source File for Implementation, or Wirth's Module Systems in Modula. My Data Interchange Format embraces the C-Style Syntax, Delimiters, Scopes and C-Style Header Files, while allowing extensions if Modules, Functions and Control Flow are needed. No ambiguous Indentation-Based Parsing from YAML or TOML. It has Proper Data Structures instead of Object Literals from JSON, YAML, TOML and XML. A Better Type System than Protocol Buffers and SQL. It is easier to implement than HTTP because a Message is a Data Structure with Initialized Fields. This prevents the usage of Domain Specific Language. Allowing teams to require either a Configuration File or a Library for a General Purpose Programming Language, no more "JSON with If Statements".