The secure data format for a modern world

Release Status Draft (Please help by reviewing the specs and submitting issues!)

Solving today's problems

Times are different from the carefree days that brought us XML and JSON.

Security

Protecting your data and infrastructure

State actors, criminal organizations and mercenaries are now actively hacking governments, companies and individuals to steal secrets, plant malware, and hold your data hostage.

The existing ad-hoc data formats are too loosely defined to be secure, and can't be fixed because they're not versioned.

Concise Encoding is designed for security, and is versioned so that it can be updated to handle new threats.

Efficiency

But not at the cost of convenience

We send so much data these days that efficiency is critical, but switching to binary means giving up the ease of text formats.

... or does it?

Concise Encoding gives you ease and efficiency with its 1:1 compatible text and binary formats.

Types

We shouldn't have to stringify anymore

Lack of types forces everyone to add extra encoding steps to send their data, which is buggy, reduces compatibility, and opens even more security holes.

We live in the 21st century - base64 should be a footnote in history by now!

Concise Encoding supports all of the common types natively. No more encoding things into strings.



Typical Usage

99% of the time, the data remains in binary form. With a twin binary/text format, communication and storage can happen in the more efficient binary format, and humans still have easy access with seamless conversion to/from the text format whenever needed.


Specifications

Structure
Concise Encoding (Structual Specification)
Both formats follow this structure.
Binary
Concise Binary Encoding
Use this for data storage and transmission.
Text
Concise Text Encoding
Use this where humans are involved.

Examples

Concise Text Encoding documents that can be transparently 1:1 converted to/from Concise Binary Encoding.

Numeric

                
c1
{
    "boolean"       = true
    "binary int"    = -0b10001011
    "octal int"     = 0o644
    "decimal int"   = -10000000
    "hex int"       = 0xfffe0001
    "very long int" = 100000000000000000000000000000000000009
    "decimal float" = -14.125
    "hex float"     = 0x5.1ec4p+20
    "very long flt" = 4.957234990634579394723460546348e+100000
    "not-a-number"  = nan
    "infinity"      = inf
    "neg infinity"  = -inf
}
                
            

String-Like

                
c1
{
    "string" = "Strings support escape sequences: \n \t \+1f415."
    "url"    = @"https://example.com/"
    "email"  = @"mailto:me@somewhere.com"
}
                
            

Other Basic Types

                
c1
{
    "uuid"      = f1ce4567-e89b-12d3-a456-426655440000
    "date"      = 2019-07-01
    "time"      = 18:04:00.948/Europe/Prague
    "timestamp" = 2010-07-15/13:28:15.415942344
    "null"      = null
    "media"     = |m application/x-sh 23 21 2f 62 69 6e 2f 73 68 0a 0a
                    65 63 68 6f 20 68 65 6c 6c 6f 20 77 6f 72 6c 64 0a|
}
                
            

Containers

                
c1
{
    "list"          = [1 2.5 "a string"]
    "map"           = {"one"=1 2="two" "today"=2020-09-10}
    "bytes"         = |u8x 01 ff de ad be ef|
    "int16 array"   = |i16 7374 17466 -9957|
    "uint16 hex"    = |u16x 91fe 443a 9c15|
    "float32 array" = |f32 1.5e10 -8.31e-12|
}
                
            

Structs


Structs are composed of templates and instances, and are particularly useful for tabular data because you don't have to repeat the keys for every entry.
                
c1
[
    @vehicle<"make"   "model"    "drive" "sunroof">
    @vehicle("Ford"   "Explorer" "4wd"   true     )
    @vehicle("Toyota" "Corolla"  "fwd"   false    )
    @vehicle("Honda"  "Civic"    "fwd"   false    )
]
                
            
The above document is equivalent to:
                
c1
[
    {
        "make" = "Ford"
        "model" = "Explorer"
        "drive" = "4wd"
        "sunroof" = true
    }
    {
        "make" = "Toyota"
        "model" = "Corolla"
        "drive" = "fwd"
        "sunroof" = false
    }
    {
        "make" = "Honda"
        "model" = "Civic"
        "drive" = "fwd"
        "sunroof" = false
    }
]
                
            

Trees

                
c1
//
// The tree:
//
//       2
//      / \
//     5   7
//    /   /|\
//   9   6 1 2
//  /   / \
// 4   8   5
//
(2
    (7
        2
        1
        (6
            5
            8
        )
    )
    (5
        (9
            4
        )
    )
)
                
            
Notice how the text looks like the tree it represents when rotated:

Graphs

                
c1
//
// The weighted graph:
//
//     b
//    /|\
//   4 1 1
//  /  |  \
// a-3-c-4-d
//
{
    "vertices" = [
        &a:{}
        &b:{}
        &c:{}
        &d:{}
    ]
    "edges" = [
        @($a {"weight"=4 "direction"="both"} $b)
        @($a {"weight"=3 "direction"="both"} $c)
        @($b {"weight"=1 "direction"="both"} $c)
        @($b {"weight"=1 "direction"="both"} $d)
        @($c {"weight"=4 "direction"="both"} $d)
    ]
}
                
            

References

                
c1
{
    // Entire map will be referenced later as $id1
    "marked object" = &id1:{
        "recursive" = $id1
    }
    "ref1" = $id1
    "ref2" = $id1

    // Reference pointing to part of another document.
    "outside ref" = $"https://xyz.com/document.cte#some_id"
}
                
            

Custom Types

                
c1
{
    // Custom types are user-defined, with user-supplied codecs.
    "custom text"   = |c "cplx(2.94+3i)"|
    "custom binary" = |c 01 f6 28 3c 40 00 00 40 40|
}
                
            

See the Resources page or Github page for more information.

Support Open Source Development!