This document is an informal proposal for the next major revision of the Common Trace Format (CTF) version 2 (hereafter named CTF 2).

This is not a formal reference. Some parts of this document, however, may be formal enough to be elligible for a specification document.

RFC 2119
The key words MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT, MAY, and OPTIONAL in this document, when emphasized, are to be interpreted as described in RFC 2119.

1. Revision history

Significant changes since CTF2-PROP-1.0:

  • Written in Asciidoctor's AsciiDoc syntax.

    The original AsciiDoc project is pretty much deprecated.

    Asciidoctor is also what GitHub, GitLab, and more use to render AsciiDoc documents.

  • Terminology update.

  • Mostly written from the consumer’s perspective, especially the data stream decoding procedure.

  • Metadata types are removed: the document uses JSON directly now.

    This makes the specification easier to read, removing one nonessential abstraction layer.

  • Constant integer JSON object is removed: we can use plain JSON integers.

  • Splitting the specification into three layers is removed: the document specifies the whole format monolithically.

    This makes the specification easier to read.

  • Format simplifications:

  • Optional field class added.

    This is why CTF2-PROP-1.0 introduced the null field class (removed in CTF2-PROP-2.0): in the end, a variant field class with a null field class option and another option really is an optional field class.

  • BLOB field classes added.

    A BLOB field contains a BLOB.

    Having dedicated static- and dynamic-length BLOB field classes instead of using static- and dynamic-length array field classes makes it possible for consumer APIs to provide the values of such fields as byte buffers.

  • An integer field class MAY have a preferred display base, even if it’s insignificant to decode its fields, because this property exists in CTF 1 and makes the transition easier.

  • An event record class MAY have a name and other properties which exist in CTF 1, but which are insignificant to decode its instances, because it makes the transition easier.

  • Extension feature added.

    This is how we plan to add core features and modify existing ones in the future without having to rely on a minor version.

  • Auxiliary stream feature added.

2. CTF publication authority

The DiaMon Workgroup, a Linux Foundation workgroup which creates de facto open standards and tools for tracing, monitoring, and diagnostics, oversees the Common Trace Format design and the publication of official CTF documents.

The DiaMon Workgroup is also responsible for providing a platform for interested parties to comment on CTF-related proposals.

3. CTF 2 document identification

We suggest that all official CTF 2 documents bear a unique document identifier (ID) having the following format:

CTF2-SHORTNAME-MAJOR.MINOR[rREV]
Table 1. Descriptions and roles of CTF 2 document ID parts
Part Description Bump MAY introduce new concepts, procedures, and formats? Bump MAY remove or change existing concepts, procedures, and formats?

SHORTNAME

The capitalized short name of the document, unique amongst all the official CTF 2 documents.

N/A

N/A

MAJOR

The major version number of the document.

Yes

Yes

MINOR

The minor version number of the document.

Yes

No

REV

The revision letter of the document (from A to Z, then ZA to ZZ, and so on).

We’ll use document revisions to add examples, clarify existing concepts, or fix grammar/content mistakes, for example.

No

No

As an example, the short name of this document is PROP, for proposal, and its full document ID is CTF2-PROP-2.0. The next revision would be CTF2-PROP-2.0rA, and the following would be CTF2-PROP-2.0rB.

A CTF 2 document MUST refer to another CTF 2 document using only its ID, for example:

This concept is further explained in CTF2-SOMEID-1.2.

There’s no need to refer to a specific revision: a reference always targets the latest document’s revision.

We suggest the following IDs for the initial documents:

CTF 2 document ID Description

CTF2-DOCID-1.0

CTF 2 document identifier format.

CTF2-FS-2.0

Layout of a CTF 2 trace when stored on a file system.

CTF2-PMETA-2.0

CTF 2 metadata stream packet format.

CTF2-SPEC-2.0

Common Trace Format (CTF), version 2.

We don’t plan to ever bump this document’s minor version as we can use the proposed extension mechanism to add or modify core features.

4. Why CTF 2?

Why do we need a major version bump of the CTF specification?

A major version bump is never an easy choice when it comes to revisiting a software library, a communication protocol, a file format, or anything else that serves as a contract between a producer and a consumer.

When such a decision is taken, it must be justified by solid arguments, since it makes it impossible for old consumers to consume the product of new producers.

In this proposal, for instance, CTF 2 traces are not backward compatible with CTF 1 traces. Although the data stream binary format is compatible, the metadata stream is not: it’s written in a different language (JSON instead of TSDL).

CTF 1 has been used and tested for over ten years now, by different producers and consumers. Over that time, we have noted a few gaps in the trace format, gaps that make it hard to extend CTF 1 as much as we’d like, amongst other things.

This specification proposition designs CTF 2 to overcome those gaps, to the best of our knowledge, and to be flexible enough to gracefully accept future additions while avoiding another major version bump in the upcoming years.

4.1. Design goals

The design goals of CTF 2 are as follows, in order, beginning with the most important:

  1. CTF 2 data streams MUST be backward compatible with CTF 1 data streams.

    Many applications are already written to output valid CTF 1 packets. Modifying the code of those applications to produce different binary packets can be cumbersome, and sometimes impossible if the application passed acceptance tests, for example.

    Making sure that applications producing CTF 1 traces can also produce CTF 2 traces only by changing the metadata stream is therefore a hard requirement.

  2. The CTF 2 data streams MUST be as efficient as possible to produce by a tracer.

    This design goal was also one of the major ones which drove the design of CTF 1.

    In other words, a small embedded system MUST be able to produce CTF 2 data streams natively. Moreover, the tracer MUST be able to copy binary data to the packet buffer of a data stream without altering it.

  3. A CTF 2 metadata stream MUST be extensible by users (including the DiaMon Workgroup) of the specification.

    CTF 1’s TSDL grammar is pretty restrictive when it comes to customizing existing blocks with user-defined attributes.

    Many protocols and declarative languages support custom user data in their payload. For example, HTML5 allows any element to have user attributes with the data- prefix.

    A CTF 2 producer MUST be able to add custom attributes to almost any specified metadata object. This makes it possible for standard consumers to read any CTF 2 trace and ignore unknown user attributes, providing a “bland”, yet complete view of the trace fields, while you can write a special consumer (or extend an existing one) to interpret specific user attributes and use them to render a meaningful visualization.

    This design goal also means that an “old” CTF 2 consumer MUST be able to either:

    • Decode a “new” CTF 2 trace completely.

    • Indicate that it cannot decode a “new” CTF 2 trace completely.

  4. CTF 2’s specification MUST focus on use cases known on its publication date.

    The previous design goal indicates that a CTF 2 metadata stream MUST be extensible by users. Knowing this, the CTF 2 specification itself MUST focus on currently known use cases, leaving anything else to future extensions to be described in other CTF 2 documents.

    As per design goal 1, those known use cases include everything available in CTF 1.

  5. CTF 2’s specification MUST NOT specify how to transport or store a trace.

    In the CTF 2 specification, a CTF 2 trace MUST be defined as a set of streams, without specifying how you transport or store said streams.

    Other official CTF 2 documents published by the DiaMon Workgroup can define standard ways to transport and store CTF 2 streams for targeted use cases. Trace producers and consumers can choose to implement one or more transport/storage strategies following the other documents.

  6. A CTF 2 trace SHOULD be as easy as possible to consume.

    CTF 1 focuses on being easy to produce, which is a good idea since producers are often tracers in this context, and as per design goal 2, a minimal tracer MUST be able to produce a correct CTF trace with minimal code.

    However, because a CTF 1 metadata stream is written in TSDL, a custom, declarative, C-like DSL designed for CTF, writing a minimal consumer of CTF 1 is not an easy task. TSDL is an intricate language, with many special cases and features, many of which are borrowed from the C language, which you can’t ignore when writing a consumer supporting all its features. TSDL was developed to ease the manual (human) production of metadata streams.

    Over time, we realized that, while producing traces is important, consuming them to solve problems with event record analysis is just as important, if not more.

    This is why CTF 2 SHOULD encourage the development of CTF consumers in any programming language by reducing the number of special cases, as well as by using a very simple, yet well-known descriptive language for the metadata stream.

    CTF 1 tries to accomodate other trace formats, which can be converted to CTF without changing the data streams by writing the matching metadata stream. This is also a source of special cases. CTF 2 SHOULD build on binary trace conversion (from another, non-CTF trace format to CTF 2) rather than trying to accomodate other formats.

  7. CTF 2’s model SHOULD be similar to CTF 1’s.

    Some APIs are already written to deal with CTF 1 “objects”, or concepts (event record classes, event records, field classes, and clock classes, for example).

    The model of CTF 2 SHOULD be, as much as possible, compatible with the model of CTF 1, so that those existing APIs can operate on CTF 2 objects too without requiring significant upgrades.

5. Changes since CTF 1

Here is a brief summary of the changes, from CTF 1 to CTF 2, as introduced by this proposal.

  • The terminology of the specification and the binary layouts of the data streams are completely detached from the C language and from the behaviour of any C compiler.

    CTF 2 is a programming language-agnostic trace format.

  • Terminology update:

    CTF 1 term Equivalent CTF 2 term

    Absolute (clock class property)

    Origin is Unix epoch

    Array (field class)

    Static-length array

    Base (integer field class property)

    Preferred display base

    Binary stream

    Data stream

    Clock (when it names a block of metadata which describes actual clocks)

    Clock class

    Content size

    Packet’s content size

    Declaration

    Field class

    Enumeration (field class)

    Fixed-length enumeration

    Event (when it names a block of metadata which describes event records)

    Event record class

    Event (when it names an actual recorded event contained in a packet)

    Event record

    Event context

    Event record specific context

    Event fields

    Event record payload

    Event ID

    Event record class ID

    Events discarded

    Discarded event record counter

    Field (structure field class)

    Structure field member class

    Field (structure field)

    Structure field member

    Floating point (field class)

    Fixed-length floating point number

    Integer (field class)

    Fixed-length integer

    Packet size

    Packet’s total size

    Sequence (field class)

    Dynamic-length array

    Size (integer field class property)

    Length

    Stream (when it names a block of metadata which describes data streams)

    Data stream class

    Stream event context

    Event record common context

    Stream event header

    Event record header

    Stream ID

    Data stream class ID

    Stream instance ID

    Data stream ID

    Stream packet context

    Packet context

    String (field class)

    Null-terminated string

    Tag (variant field class)

    Selector

    Trace (when it names a block of metadata which describes traces)

    Trace class

    Trace packet header

    Packet header

  • The metadata stream is written in JSON, as specified by ECMA-404.

  • The CTF 2 specification doesn’t specify a “packetized” metadata stream format.

    This is a back-end-specific way of wrapping a metadata stream as defined in this document.

    Another document specifies the packetized metadata stream format, which trace storage and transport documents can use if needed.

  • Most objects of a CTF 2 metadata stream MAY contain custom user attributes.

  • Field class aliases are removed.

    This removes indirection, simplifying the format.

  • New field classes:

  • Modified field classes:

  • Relative field locations (variant field class’s tag/selector and dynamic-length field class’s length) are removed.

    In CTF 2, you can specify a location to any valid field with an absolute field location.

    This simplifies the format.

  • “Special” structure field member classes, such as the packet magic number member class, the data stream class ID member class, and the packet’s total size member class, can have any name: a field class MAY have one or more roles instead to indicate the purpose of its instances.

    This means a CTF 2 consumer doesn’t need to rely on reserved names like CTF 1’s magic, stream_id, and packet_size.

    This new approach also makes it possible for a given field class to have multiple roles.

  • The native byte order property is removed from the trace class.

    In CTF 2, each fixed-length bit array field class MUST explicitly indicate the byte order of its instances.

    This removes indirection, simplifying the format.

  • The EMF URI property is removed from the event record class.

    This Eclipse Modeling Framework property is not common enough for the Common Trace Format.

    A producer can still write such an URI as an event record class’s user attribute.

  • The log level property is removed from the event record class.

    Log level integral values are tracer-specific.

    A producer can still write such a log level value as an event record class’s user attribute.

  • A CTF 2 trace MAY have zero or more auxiliary streams.

    An auxiliary stream contains structured information related to a specific trace which doesn’t fit the data stream model, whereas a CTF 2 metadata stream now describes a class of traces (multiple traces can share the same metadata stream).

  • The “call site” block feature is removed.

    We don’t know any producer using this.

6. Common definitions

Common definitions for this specification:

Byte

A group of eight bits operated on as a unit.

Class

A set of values (instances) which share common properties.

For example, a fixed-length unsigned integer field class with an 8-bit length is the set of the all the fixed-length unsigned integer fields from 00000000 to 11111111 (values 0 to 255). .

This document often states that some class describes instances. For example, an event record class describes event records.

Consumer

A software or hardware system which consumes (reads) the streams of a trace.

A trace consumer is often a trace viewer or a trace analyzer.

Namespace

A string of which the purpose is to avoid naming conflicts.

This document doesn’t specify the format of a namespace. It is recommended to use a URI, or at least to include a domain name owned by the organization defining the objects under a namespace.

The std namespace is reserved for the CTF 2 specification.
Producer

A software or hardware system which produces (writes) the streams of a trace.

A trace producer is often a tracer.

Sequence

A set of related items that follow each other in a particular order.

Stream

A sequence of bytes.

7. Trace composition

A trace is:

As a reminder, a stream is defined as a sequence of bytes.

This document doesn’t specify how to transport or store CTF 2 streams. A producer could serialize all streams as a single file on the file system, or it could send the streams over the network using TCP, to name a few examples.

7.1. Metadata stream (overview)

A metadata stream describes trace data streams with JSON objects.

A metadata stream describes things such as:

Multiple traces MAY share the same metadata stream: a given trace MAY contain specific information in its own auxiliary streams.

See Metadata stream for the full metadata stream specification.

7.2. Data stream

A data stream is a sequence of one or more data packets:

ctf trace all

In the metadata stream, a data stream class describes data streams.

A packet MUST contain one or more bytes of data.

Although a packet MAY contain padding at the end itself, from the data stream’s point of view, there’s no padding between packets. In other words, the byte following the last byte of a packet is the first byte of the next packet.

A data stream MAY have, conceptually:

One default, monotonic clock

Described by a clock class in the metadata stream.

Packets and event records MAY contain snapshots, named timestamps, of their data stream’s default clock.

One counter of discarded event records

Indicates the number of event records which the producer needed to discard for different reasons.

For example, a tracer could discard an event record when it doesn’t fit some buffer and there’s no other available buffer.

A packet MAY contain a snapshot of this counter.

See Data stream decoding procedure to learn how to decode a CTF 2 data stream.

7.2.1. Packet

A packet is a part of a data stream.

A packet contains a sequence of data fields or padding (garbage data). In the metadata stream, field classes describe data fields.

A packet P, contained in a data stream S, contains, in this order:

  1. OPTIONAL: A header structure field, described at the trace class level in the metadata stream, which contains:

    1. OPTIONAL: A packet magic number field (0xc1fc1fc1, or 3254525889).

    2. Order not significant:

      • OPTIONAL: A trace class UUID field.

      • OPTIONAL: One or more fields which contain the numeric ID of the class of S.

      • OPTIONAL: One or more fields which contain the numeric ID of S.

  2. OPTIONAL: A context structure field, described at the data stream class level in the metadata stream, which contains (order not significant):

    • OPTIONAL: A field which contains the total size of P, in bits (always a multiple of 8).

    • OPTIONAL: A field which contains the content size of P, in bits.

    • OPTIONAL: A field which contains the beginning timestamp of P.

    • OPTIONAL: A field which contains the end timestamp of P.

    • OPTIONAL: A field which contains a snapshot of the discarded event record counter of S at the end of P.

    • OPTIONAL: A field which contains the sequence number of P within S.

    • OPTIONAL: User fields.

  3. Zero or more event records.

A packet MUST contain one or more bytes of data.

A packet MAY have padding (garbage data) after its last event record. The size of this padding is the difference between its total size and its content size (as found in its context structure field).

Packets are independent of each other: if you remove a packet from a data stream, a consumer can still decode the whole data stream. This is why:

If the packet context fields of a data stream’s packets contain a packet sequence number field, a consumer can recognize missing packets.

See Packet decoding procedure to learn how to decode a CTF 2 packet.

7.2.2. Event record

An event record is the result of a producer writing a record with OPTIONAL user data when an event occurs during its execution.

A packet contains zero or more event records.

An event record class describes the specific parts of event records.

An event record E, contained in a data stream S, contains, in this order:

  1. OPTIONAL: A header structure field, described at the data stream class level in the metadata stream, which contains (order not significant):

    • OPTIONAL: One or more fields which contain the numeric ID of the class of E which has the class of S as its parent.

    • OPTIONAL: One or more fields which contain a timestamp or a partial timestamp.

  2. OPTIONAL: A common context structure field, described at the data stream class level in the metadata stream, which contains user fields.

  3. OPTIONAL: A specific context structure field, described at the event record class level in the metadata stream, which contains user fields.

  4. OPTIONAL: A payload structure field, described at the event record class level in the metadata stream, which contains user fields.

An event record MUST contain one or more bits of data.

The default clock timestamp of an event record, that is, the value of its data stream's default clock after its header field, if any, is encoded/decoded MUST be greater than or equal to the default clock timestamp of the previous event record, if any, within the same data stream.

See Event record decoding procedure to learn how to decode a CTF 2 event record.

7.3. Auxiliary stream

An auxiliary stream is a JSON object, as specified by ECMA-404, which contains extra, structured information about the trace which doesn’t fit the data stream model.

An auxiliary stream has a single property:

Name

Auxiliary stream’s namespace.

Value

A JSON value.

Two auxiliary streams of a given trace MUST NOT have the same namespace.

Example 1. Auxiliary stream with the my.tracer namespace.
{
  "my.tracer": {
    "version": [1, 3, 2],
    "session-name": "hanon"
  }
}
Example 2. Auxiliary stream of which the value is just 42.
{
  "328c7a2d-a959-4f60-bd22-cca74359326f": 42
}

7.3.1. Trace environment

To remain backward compatible with CTF 1, a trace MAY contain an auxiliary stream having the std namespace which contains trace environment variables under the environment property.

The trace environment variables are a single JSON object where each property is:

Name

Trace environment variable name.

Value

Trace environment variable value (any JSON value).

This document doesn’t specify trace environment variable names.

Example 3. std auxiliary stream with trace environment variables.
{
  "std": {
    "environment": {
      "hostname": "hanon",
      "domain": "kernel",
      "sysname": "Linux",
      "kernel_release": "4.12.12-1-ARCH",
      "kernel_version": "#1 SMP PREEMPT Sun Sep 10 09:41:14 CEST 2017",
      "tracer_name": "lttng-modules",
      "tracer_major": 2,
      "tracer_minor": 10,
      "tracer_patchlevel": 0
    }
  }
}

8. Metadata stream

A metadata stream is a JSON array, as specified by ECMA-404, of fragments.

Together, the fragments of a metadata stream contain all the information about the data streams of one or more traces.

A fragment is a JSON object; its allowed properties depend on its type property.

Table 2. Common properties of a fragment F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be one of:

"preamble"

F is a preamble fragment.

"trace-class"

F is a trace class fragment.

"clock-class"

F is a clock class fragment.

"data-stream-class"

F is a data stream class fragment.

"event-record-class"

F is a event record class fragment.

Yes

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

For any fragment except a preamble fragment, any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

The metadata stream is designed as a flat JSON array of fragments instead of a single JSON object containing nested objects to enable real-time, or “live”, tracing: a consumer can always decode event records having known event record classes while a producer can always add new event record classes to a data stream class by appending additional fragments to the metadata stream. Once a producer appends a fragment to a metadata stream, the fragment is considered “frozen”, in that it never needs to change.

A metadata stream:

Example 4. Partial metadata stream.
[
  {
    "type": "preamble",
    "version": 2
  },
  
]

This section doesn’t specify how a metadata stream translates into data stream encoding and decoding rules; it only describes objects and their properties.

See Data stream decoding procedure to learn how to decode a data stream.

8.1. UUID

Both a trace class fragment and a clock class fragment MAY have a UUID property.

Within a metadata stream, a UUID is a JSON string of which the value is the canonical textual representation of the UUID.

Example 5. UUID.
"e53e0ab8-50a1-4f0a-b710-b5f0bba9c4ac"

8.2. Extensions

A producer MAY add extensions to many metadata stream JSON objects.

The purpose of an extension is to add core features to CTF 2 or to modify existing core features, as specified by this document. In other words, an extension MAY alter the format itself. This document doesn’t specify what an extension exactly is.

The metadata stream’s preamble fragment contains extension declarations:

  • Any extension in metadata stream objects MUST be declared, by namespace and name, in the preamble fragment.

    Declaring an extension is said to enable it.

  • If a consumer doesn’t support any declared extension, it MUST NOT consume the trace's data streams.

    The consumer SHOULD report unsupported extensions as an error.

Extensions are a single JSON object, where each property is:

A namespaced extensions object is a JSON object, where each property is:

Name

An extension name

Value

A JSON value

The metadata stream JSON objects which MAY contain extensions as their extensions property are:

Example 6. Three extensions under two namespaces.
{
  "my.tracer": {
    "piano": {
      "keys": 88,
      "temperament": "equal"
    },
    "ramen": 23
  },
  "abc/xyz": {
    "sax": {
      "variant": "alto"
    }
  }
}

8.3. User attributes

A producer MAY add custom user attributes to many metadata stream JSON objects.

This document doesn’t specify what a user attribute exactly is. Unlike extensions, a consumer MUST NOT consider user attributes to decode data streams.

User attributes are a single JSON object, where each property is:

Name

A namespace

Value

A JSON value

The metadata stream JSON objects which MAY contain user attributes as their user-attributes property are:

Example 7. User attributes under two namespaces.
{
  "my.tracer": {
    "max-count": 45,
    "module": "sys"
  },
  "abc/xyz": true
}

8.4. Field classes

A field class describes fields, that is, sequences of bits as found in a data stream.

A field class contains all the properties a consumer needs to decode a given field.

A field is a field class instance.

This document specifies the following types of field classes:

A field class is a JSON object; its properties depend on its type property.

Table 3. Common properties of a field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be one of:

"fixed-length-bit-array"

F is a fixed-length bit array field class.

"fixed-length-boolean"

F is a fixed-length boolean field class.

"fixed-length-unsigned-integer"
"fixed-length-signed-integer"

F is a fixed-length integer field class.

"fixed-length-unsigned-enumeration"
"fixed-length-signed-enumeration"

F is a fixed-length enumeration field class.

"fixed-length-floating-point-number"

F is a fixed-length floating point number field class.

"variable-length-bit-array"

F is a variable-length bit array field class.

"variable-length-unsigned-integer"
"variable-length-signed-integer"

F is a variable-length integer field class.

"variable-length-unsigned-enumeration"
"variable-length-signed-enumeration"

F is a variable-length enumeration field class.

"null-terminated-string"

F is a null-terminated string field class.

"static-length-string"

F is a static-length string field class.

"static-length-blob"

F is a static-length BLOB field class.

"dynamic-length-string"

F is a dynamic-length string field class.

"dynamic-length-blob"

F is a dynamic-length BLOB field class.

"structure"

F is a structure field class.

"static-length-array"

F is a static-length array field class.

"dynamic-length-array"

F is a dynamic-length array field class.

"optional"

F is a optional field class.

"variant"

F is a variant field class.

Yes

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

The following fragment properties require a structure field class as their value:

Trace class fragment

packet-header-field-class

Data stream class fragment
  • packet-context-field-class

  • event-record-header-field-class

  • event-record-common-context-field-class

Event record class fragment
  • specific-context-field-class

  • payload-field-class

8.4.1. Field location

A field location is a means for a consumer to find a field which it needs to decode another field.

A consumer needs to find another field to decode instances of the following classes:

A field location is a JSON array, where, in this order:

  1. The first element is the name (JSON string) of a root field from where to start the lookup, amongst:

    "packet-header"

    Packet header

    "packet-context"

    Packet context

    "event-record-header"

    Event record header

    "event-record-common-context"

    Event record common context

    "event-record-specific-context"

    Event record specific context

    "event-record-payload"

    Event record payload

  2. The following elements are structure field member names (JSON strings) to follow to find the target field.

The length of a field location MUST be greater than or equal to two.

Let T be a field which a consumer needs to decode another field S:

  • If S is in a packet header, then T MUST be in the same packet header.

  • If S is in a packet context, then T MUST be in one of:

    • The packet header the same packet.

    • The same packet context.

  • If S is in an event record header, then T MUST be in one of:

    • The packet header of the same packet.

    • The packet context of the same packet.

    • The same event record header.

  • If S is in an event record common context, then T MUST be in one of:

    • The packet header of the same packet.

    • The packet context of the same packet.

    • The event record header of the same event record.

    • The same event record common context.

  • If S is in an event record specific context, then T MUST be in one of:

    • The packet header of the same packet.

    • The packet context of the same packet.

    • The event record header of the same event record.

    • The event record common context of the same event record.

    • The same event record specific context.

  • If S is in an event record payload, then T MUST be in one of:

    • The packet header of the same packet.

    • The packet context of the same packet.

    • The event record header of the same event record.

    • The event record common context of the same event record.

    • The event record common specific of the same event record.

    • The same event record payload.

  • If S and T are not in the same root field, then T MUST NOT be in any array or optional field.

  • If S and T are in the same root field, then:

    • If S is in an array field, then T must be in the same array field element.

    • If S is in an optional field, then T must be in the same optional field.

If any structure member name N of a field location L names a variant field, then:

If N is not the last element of L

The variant field MUST select a structure field, from which the lookup process can continue, recursively.

If N is the last element of L

The variant field MUST select the target field (fixed-length integer, variable-length integer, or fixed-length boolean), recursively.

In both cases, all the options of the variant field class MUST make it possible for the lookup process to continue.

Example 8. Dynamic-length array field and its length field in the same root field.

The following JSON object is an event record payload structure field class.

{
  "type": "structure",
  "members": [
    {
      "name": "corn", (3)
      "field-class": {
        "type": "fixed-length-unsigned-integer",
        "length": 32,
        "byte-order": "little-endian"
      }
    },
    {
      "name": "inside",
      "field-class": {
        "type": "fixed-length-unsigned-integer",
        "length": 16,
        "byte-order": "little-endian"
      }
    },
    {
      "name": "carbon",
      "field-class": {
        "type": "dynamic-length-array", (1)
        "length-field-location": ["event-record-payload", "corn"], (2)
        "element-field-class": {
          "type": "null-terminated-string"
        }
      }
    }
  ]
}
1 Dynamic-length array field class.
2 Dynamic-length array field class's length field location.
3 Length member class.
Example 9. Dynamic-length array field and its length field in the same root field, within the same array field element.

The following JSON object is an event record payload structure field class.

Both the dynamic-length array field and its length field exist within the same element of the static-length array field named nature.

{
  "type": "structure",
  "members": [
    {
      "name": "norm",
      "field-class": {
        "type": "null-terminated-string"
      }
    },
    {
      "name": "nature",
      "field-class": {
        "type": "static-length-array",
        "length": 43,
        "element-field-class": {
          "type": "structure",
          "members": [
            {
              "name": "laser", (3)
              "field-class": {
                "type": "variable-length-unsigned-integer"
              }
            },
            {
              "name": "joystick",
              "field-class": {
                "type": "dynamic-length-array", (1)
                "length-field-location": [ (2)
                  "event-record-payload",
                  "nature",
                  "laser"
                ],
                "element-field-class": {
                  "type": "null-terminated-string"
                }
              }
            }
          ]
        }
      }
    }
  ]
}
1 Dynamic-length array field class.
2 Dynamic-length array field class's length field location.
3 Length member class.
Example 10. Dynamic-length array and its length field in the same root field, within the same variant field.

The following JSON object is an event record payload structure field class.

Both the dynamic-length array field and its length field exist within the same option of the variant field named clinic.

Moreover, the selector field of the clinic variant field is the lawyer field.

{
  "type": "structure",
  "members": [
    {
      "name": "lawyer", (5)
      "field-class": {
        "type": "fixed-length-signed-integer",
        "length": 16,
        "byte-order": "little-endian"
      }
    },
    {
      "name": "clinic",
      "field-class": {
        "type": "variant",
        "selector-field-location": ["event-record-payload", "lawyer"], (4)
        "options": [
          {
            "selector-field-ranges": [[0, 0]],
            "field-class": {
              "type": "null-terminated-string"
            }
          },
          {
            "selector-field-ranges": [[1, 4]],
            "field-class": {
              "type": "structure",
              "members": [
                {
                  "name": "lemon", (3)
                  "field-class": {
                    "type": "fixed-length-unsigned-integer",
                    "length": 8,
                    "byte-order": "big-endian"
                  }
                },
                {
                  "name": "joystick",
                  "field-class": {
                    "type": "dynamic-length-array", (1)
                    "length-field-location": [ (2)
                      "event-record-payload",
                      "clinic",
                      "lemon"
                    ],
                    "element-field-class": {
                      "type": "null-terminated-string"
                    }
                  }
                }
              ]
            }
          },
          {
            "selector-field-ranges": [[5, 5], [7, 7]],
            "field-class": {
              "type": "fixed-length-boolean",
              "length": 8,
              "byte-order": "little-endian"
            }
          }
        ]
      }
    }
  ]
}
1 Dynamic-length array field class.
2 Dynamic-length array field class's length field location.
3 Length member class.
4 Variant field class’s selector field location.
5 Selector member class.
Example 11. Dynamic-length array and its length field in the same root field; length field is a variant field.

The following JSON object is an event record payload structure field class.

The length field of the dynamic-length array field is a variant field: it can be an 8-bit, a 16-bit, or a 32-bit fixed-length integer field, depending on the variant field’s selection.

Moreover, the selector field of the variant field is located in another root field (event record specific context).

{
  "type": "structure",
  "members": [
    {
      "name": "glass", (3)
      "field-class": {
        "type": "variant",
        "selector-field-location": ["event-record-specific-context", "sel"],
        "options": [
          {
            "selector-field-ranges": [[0, 0]],
            "field-class": {
              "type": "fixed-length-unsigned-integer", (4)
              "length": 8,
              "byte-order": "little-endian"
            }
          },
          {
            "selector-field-ranges": [[1, 1]],
            "field-class": {
              "type": "fixed-length-unsigned-integer", (4)
              "length": 16,
              "byte-order": "little-endian"
            }
          },
          {
            "selector-field-ranges": [[2, 2]],
            "field-class": {
              "type": "fixed-length-unsigned-integer", (4)
              "length": 32,
              "byte-order": "little-endian"
            }
          }
        ]
      }
    },
    {
      "name": "margin",
      "field-class": {
        "type": "dynamic-length-array", (1)
        "length-field-location": ["event-record-payload", "glass"], (2)
        "element-field-class": {
          "type": "null-terminated-string"
        }
      }
    }
  ]
}
1 Dynamic-length array field class.
2 Dynamic-length array field class's length field location.
3 Length member class.
4 Possible length field class.
Example 12. Dynamic-length array and its length field in the same root field; structure field containing length field is a variant field.

The following JSON object is an event record payload structure field class.

The length field of the dynamic-length array field is within a structure field which is a variant field.

Moreover:

  • The selector field of the variant field is located in another root field (event record common context).

  • The field class of the third option of the glass variant field class contains a dynamic-length BLOB field class (lock member); the length field of its instance is the previous member (eagle) within the same structure field.

{
  "type": "structure",
  "members": [
    {
      "name": "glass",
      "field-class": {
        "type": "variant",
        "selector-field-location": ["event-record-common-context", "sel"],
        "options": [
          {
            "selector-field-ranges": [[0, 0]],
            "field-class": {
              "type": "structure",
              "members": [
                {
                  "name": "eagle",
                  "field-class": {
                    "type": "fixed-length-unsigned-integer", (3)
                    "length": 16,
                    "byte-order": "little-endian"
                  }
                },
                {
                  "name": "road",
                  "field-class": {
                    "type": "null-terminated-string"
                  }
                }
              ]
            }
          },
          {
            "selector-field-ranges": [[32, 172]],
            "field-class": {
              "type": "structure",
              "members": [
                {
                  "name": "nuance",
                  "field-class": {
                    "type": "null-terminated-string"
                  }
                },
                {
                  "name": "eagle",
                  "field-class": {
                    "type": "fixed-length-unsigned-integer", (3)
                    "length": 24,
                    "byte-order": "big-endian"
                  }
                }
              ]
            }
          },
          {
            "selector-field-ranges": [[5, 5]],
            "field-class": {
              "type": "structure",
              "members": [
                {
                  "name": "eagle", (5)
                  "field-class": {
                    "type": "variable-length-unsigned-integer" (3)
                  }
                },
                {
                  "name": "lock",
                  "field-class": {
                    "type": "dynamic-length-blob",
                    "length-field-location": [ (4)
                      "event-record-payload",
                      "glass",
                      "eagle"
                    ]
                  }
                }
              ]
            }
          }
        ]
      }
    },
    {
      "name": "margin",
      "field-class": {
        "type": "dynamic-length-array", (1)
        "length-field-location": [ (2)
          "event-record-payload",
          "glass",
          "eagle"
        ],
        "element-field-class": {
          "type": "null-terminated-string"
        }
      }
    }
  ]
}
1 Dynamic-length array field class.
2 Dynamic-length array field class's length field location.
3 Possible length field class.
4 Dynamic-length BLOB field class's length field location.
5 Length field class for dynamic-length BLOB field class.

Note that both the dynamic-length array and dynamic-length BLOB field classes have the same length field location.

Example 13. Dynamic-length array and its length field in another root field.

The following JSON objects are the event record specific context and payload structure field classes of the same event record class.

The length field of the event record payload’s dynamic-length array field is within the event record specific context.

Event record specific context field class.
{
  "type": "structure",
  "members": [
    {
      "name": "cook",
      "field-class": {
        "type": "fixed-length-floating-point-number",
        "length": 64,
        "byte-order": "little-endian"
      }
    },
    {
      "name": "vegetable", (1)
      "field-class": {
        "type": "variable-length-unsigned-integer"
      }
    }
  ]
}
1 Length member class.
Event record payload field class.
{
  "type": "structure",
  "members": [
    {
      "name": "avenue",
      "field-class": {
        "type": "dynamic-length-array", (1)
        "length-field-location": [ (2)
          "event-record-specific-context",
          "vegetable"
        ],
        "element-field-class": {
          "type": "null-terminated-string"
        }
      }
    },
    {
      "name": "railroad",
      "field-class": {
        "type": "null-terminated-string"
      }
    }
  ]
}

8.4.2. Integer range set

An integer range set is a JSON array of integer ranges.

An integer range set MUST contain one or more integer ranges.

An integer range is a JSON array of two elements:

  1. The range’s lower bound (JSON integer, included).

  2. The range’s upper bound (JSON integer, included).

An integer range represents all the integer values from the range’s lower bound to its upper bound.

An integer range’s upper bound MUST be greater than or equal to its lower bound.

If both the lower and upper bounds of an integer range are equal, then the integer range represents a single integer value.

Example 14. Integer ranges.
[3, 67]
[-45, 101]
Single integer value.
[42, 42]
Example 15. Integer range set containing three integer ranges.
[[3, 67], [-45, 1], [42, 42]]

8.4.3. Roles

Some field class instances can have roles.

A role is specific semantics attached to the fields (instances) of a field class. For example, the packet-magic-number role of a fixed-length integer field class indicates that the value of its instances MUST be the packet magic number (0xc1fc1fc1).

Roles are a JSON array of role names (JSON strings).

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

8.4.4. Fixed-length bit array field class

A fixed-length bit array field class describes fixed-length bit array fields.

A fixed-length bit array field is a simple array of contiguous bits, without any attached integer type semantics.

The length, or number of bits, of a fixed-length bit array field is a property (length) of its class.

Table 4. Common properties of a fixed-length bit array field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "fixed-length-bit-array".

Yes

length

JSON integer

Number of bits of an instance of F.

The value of this property MUST be greater than zero.

Yes

byte-order

JSON string

Byte order of an instance of F.

The value of this property MUST be one of:

"big-endian"

Big-endian.

"little-endian"

Little-endian.

Yes

alignment

JSON integer

Alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance.

The value of this property MUST be a positive power of two.

No

1

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 16. Minimal fixed-length bit array field class.
{
  "type": "fixed-length-bit-array",
  "length": 16,
  "byte-order": "little-endian"
}
Example 17. Fixed-length bit array field class with a 32-bit alignment.
{
  "type": "fixed-length-bit-array",
  "length": 48,
  "byte-order": "big-endian",
  "alignment": 32
}
Example 18. Fixed-length bit array field class with user attributes.
{
  "type": "fixed-length-bit-array",
  "length": 16,
  "byte-order": "little-endian",
  "user-attributes": {
    "my.tracer": {
      "is-nice": true
    }
  }
}

8.4.5. Fixed-length boolean field class

A fixed-length boolean field class is a fixed-length bit array field class which describes fixed-length boolean fields.

A fixed-length boolean field is a fixed-length bit array field which has the following semantics:

If all the bit array field’s bits are cleared (zero)

The fixed-length boolean field’s value is false.

Otherwise

The fixed-length boolean field’s value is true.

Table 5. Properties of a fixed-length boolean field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "fixed-length-boolean".

Yes

length

JSON integer

Number of bits of an instance of F.

The value of this property MUST be greater than zero.

Property inherited from the fixed-length bit array field class.

Yes

byte-order

JSON string

Byte order of an instance of F.

The value of this property MUST be one of:

"big-endian"

Big-endian.

"little-endian"

Little-endian.

Property inherited from the fixed-length bit array field class.

Yes

alignment

JSON integer

Alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance.

The value of this property MUST be a positive power of two.

Property inherited from the fixed-length bit array field class.

No

1

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 19. Minimal fixed-length boolean field class.
{
  "type": "fixed-length-boolean",
  "length": 16,
  "byte-order": "little-endian"
}
Example 20. Fixed-length boolean field class with a 32-bit alignment.
{
  "type": "fixed-length-boolean",
  "length": 48,
  "byte-order": "big-endian",
  "alignment": 32
}
Example 21. Fixed-length boolean field class with user attributes.
{
  "type": "fixed-length-boolean",
  "length": 16,
  "byte-order": "little-endian",
  "user-attributes": {
    "my.tracer": {
      "is-nice": true
    }
  }
}

8.4.6. Abstract integer field class

An abstract integer field class is a base of a fixed-length integer field class and a variable-length integer field class.

This field class is abstract in that it only exists to show the relation between different integer field classes in this document: a packet cannot contain an abstract integer field.

Table 6. Common property of an integer field class F.
Name Type Description Required? Default

preferred-display-base

JSON integer

Preferred base to display the value of an instance of F.

The value of this property MUST be one of:

2

Binary base.

8

Octal base.

10

Decimal base.

16

Hexadecimal base.

No

10

8.4.7. Fixed-length integer field class

A fixed-length integer field class is both an abstract integer field class and a fixed-length bit array field class which describes fixed-length integer fields.

A fixed-length integer field is a fixed-length bit array field which has integer semantics.

If the value of a fixed-length integer field class’s type property is "fixed-length-signed-integer", then its instances have the two’s complement format.

A fixed-length integer field class acts as a base of a fixed-length enumeration field class.

Table 7. Common properties of a fixed-length integer field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be one of:

"fixed-length-unsigned-integer"

The instances of F are fixed-length unsigned integer fields.

"fixed-length-signed-integer"

The instances of F are fixed-length signed integer fields.

Yes

length

JSON integer

Number of bits of an instance of F.

The value of this property MUST be greater than zero.

Property inherited from the fixed-length bit array field class.

Yes

byte-order

JSON string

Byte order of an instance of F.

The value of this property MUST be one of:

"big-endian"

Big-endian.

"little-endian"

Little-endian.

Property inherited from the fixed-length bit array field class.

Yes

alignment

JSON integer

Alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance.

The value of this property MUST be a positive power of two.

Property inherited from the fixed-length bit array field class.

No

1

preferred-display-base

JSON integer

Preferred base to display the value of an instance of F.

The value of this property MUST be one of:

2

Binary base.

8

Octal base.

10

Decimal base.

16

Hexadecimal base.

Property inherited from the abstract integer field class.

No

10

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 22. Minimal fixed-length unsigned integer field class.
{
  "type": "fixed-length-unsigned-integer",
  "length": 16,
  "byte-order": "little-endian"
}
Example 23. Fixed-length signed integer field class with a 32-bit alignment.
{
  "type": "fixed-length-signed-integer",
  "length": 48,
  "byte-order": "big-endian",
  "alignment": 32
}
Example 24. Fixed-length unsigned integer field class with a preferred hexadecimal display base.
{
  "type": "fixed-length-unsigned-integer",
  "length": 48,
  "byte-order": "big-endian",
  "preferred-display-base": 16
}
Example 25. Fixed-length signed integer field class with user attributes.
{
  "type": "fixed-length-signed-integer",
  "length": 16,
  "byte-order": "little-endian",
  "user-attributes": {
    "my.tracer": {
      "is-nice": true
    }
  }
}

8.4.8. Abstract enumeration field class

An abstract enumeration field class is a base of a fixed-length enumeration field class and a variable-length enumeration field class.

This field class is abstract in that it only exists to show the relation between different enumeration field classes in this document: a packet cannot contain an abstract enumeration field.

An abstract enumeration field class is an abstract integer field class.

An enumeration field is an integer field which MAY have one or more associated names thanks to its class’s mappings property.

Table 8. Common property of an enumeration field class F.
Name Type Description Required? Default

preferred-display-base

JSON integer

Preferred base to display the value of an instance of F.

The value of this property MUST be one of:

2

Binary base.

8

Octal base.

10

Decimal base.

16

Hexadecimal base.

Property inherited from the abstract integer field class.

No

10

mappings

Enumeration field class mappings

Mappings of F.

The value of this property MUST contain one or more properties.

Yes

8.4.8.1. Enumeration field class mappings

Enumeration field class mappings map names to integer range sets.

Enumeration field class mappings are a JSON object, where each property is:

Name

Mapping name.

Value

Mapped ranges of integers (integer range set).

The integer ranges of two given mappings MAY overlap.

Enumeration field class mappings MUST contain one or more properties.

Example 26. Enumeration field class mappings with three mappings.

In this example, the fortune and building mappings overlap with the values 4 and 5, and the building and journal mappings overlap with the value 80.

{
  "fortune": [[3, 67], [-45, 1], [84, 84]],
  "building": [[4, 5], [75, 82]],
  "journal": [[100, 2305], [80, 80]]
}

8.4.9. Fixed-length enumeration field class

A fixed-length enumeration field class is both an abstract enumeration field class and a fixed-length integer field class which describes fixed-length enumeration fields.

A fixed-length enumeration field is a fixed-length integer field which MAY have one or more associated names thanks to its class’s mappings property.

If the value of a fixed-length enumeration field class’s type property is "fixed-length-signed-enumeration", then its instances have the two’s complement format.

Table 9. Properties of a fixed-length enumeration field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be one of:

"fixed-length-unsigned-enumeration"

The instances of F are fixed-length unsigned enumeration fields.

"fixed-length-signed-enumeration"

The instances of F are fixed-length signed enumeration fields.

Yes

length

JSON integer

Number of bits of an instance of F.

The value of this property MUST be greater than zero.

Property inherited from the fixed-length bit array field class.

Yes

byte-order

JSON string

Byte order of an instance of F.

The value of this property MUST be one of:

"big-endian"

Big-endian.

"little-endian"

Little-endian.

Property inherited from the fixed-length bit array field class.

Yes

alignment

JSON integer

Alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance.

The value of this property MUST be a positive power of two.

Property inherited from the fixed-length bit array field class.

No

1

preferred-display-base

JSON integer

Preferred base to display the value of an instance of F.

The value of this property MUST be one of:

2

Binary base.

8

Octal base.

10

Decimal base.

16

Hexadecimal base.

Property inherited from the abstract integer field class.

No

10

mappings

Enumeration field class mappings

Mappings of F.

The value of this property MUST contain one or more properties.

Property inherited from the abstract enumeration field class.

Yes

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 27. Minimal fixed-length unsigned enumeration field class.
{
  "type": "fixed-length-unsigned-enumeration",
  "length": 16,
  "byte-order": "little-endian",
  "mappings": {
    "apple": [[1, 19]]
  }
}
Example 28. Fixed-length signed enumeration field class with a 32-bit alignment.
{
  "type": "fixed-length-signed-enumeration",
  "length": 48,
  "byte-order": "big-endian",
  "alignment": 32,
  "mappings": {
    "banana": [[-27399, -1882], [8, 199], [101, 101]],
    "orange": [[67, 67], [43, 1534]]
  }
}
Example 29. Fixed-length unsigned enumeration field class with a preferred hexadecimal display base.
{
  "type": "fixed-length-unsigned-enumeration",
  "length": 8,
  "byte-order": "big-endian",
  "preferred-display-base": 16,
  "mappings": {
    "lime": [[3, 3]],
    "kiwi": [[8, 8]],
    "blueberry": [[11, 11]]
  }
}
Example 30. Fixed-length signed enumeration field class with user attributes.
{
  "type": "fixed-length-signed-enumeration",
  "length": 16,
  "byte-order": "little-endian",
  "mappings": {
    "mango": [[23, 42]]
  },
  "user-attributes": {
    "my.tracer": {
      "is-nice": true
    }
  }
}

8.4.10. Fixed-length floating point number field class

A fixed-length floating point number field class is a fixed-length bit array field class which describes fixed-length floating point number fields.

A fixed-length floating point number field is a fixed-length bit array field which has floating point number semantics.

Table 10. Properties of a fixed-length floating point number field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "fixed-length-floating-point-number".

Yes

length

JSON integer

Number of bits of an instance of F.

The value of this property MUST be one of:

16

The instances of F are binary16 floating point numbers, as per IEEE 754-2008's binary interchange format.

32

The instances of F are binary32 floating point numbers.

64

The instances of F are binary64 floating point numbers.

128

The instances of F are binary128 floating point numbers.

K, where K is greater than 128 and a multiple of 32

The instances of F are binaryK floating point numbers.

Property inherited from the fixed-length bit array field class.

Yes

byte-order

JSON string

Byte order of an instance of F.

The value of this property MUST be one of:

"big-endian"

Big-endian.

"little-endian"

Little-endian.

Property inherited from the fixed-length bit array field class.

Yes

alignment

JSON integer

Alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance.

The value of this property MUST be a positive power of two.

Property inherited from the fixed-length bit array field class.

No

1

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 31. Minimal binary32 fixed-length floating point number field class.
{
  "type": "fixed-length-floating-point-number",
  "length": 32,
  "byte-order": "little-endian"
}
Example 32. binary64 fixed-length floating point number field class with a 32-bit alignment.
{
  "type": "fixed-length-floating-point-number",
  "length": 64,
  "byte-order": "big-endian",
  "alignment": 32
}
Example 33. binary192 fixed-length floating point number field class with user attributes.
{
  "type": "fixed-length-floating-point-number",
  "length": 192,
  "byte-order": "little-endian",
  "user-attributes": {
    "my.tracer": {
      "is-nice": true
    }
  }
}

8.4.11. Variable-length bit array field class

A variable-length bit array field class describes variable-length bit array fields.

A variable-length bit array field is a sequence of bytes with a variable length which contains an array of bits of which the length is a multiple of 7. A variable-length bit array field is encoded as per LEB128.

A variable-length bit array field class acts as a base of a variable-length integer field class.

Table 11. Common properties of a variable-length bit array field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "variable-length-bit-array".

Yes

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 34. Minimal variable-length bit array field class.
{
  "type": "variable-length-bit-array"
}
Example 35. Variable-length bit array field class with user attributes.
{
  "type": "variable-length-bit-array",
  "user-attributes": {
    "my.tracer": {
      "is-nice": true
    }
  }
}

8.4.12. Variable-length integer field class

A variable-length integer field class is both an abstract integer field class and a variable-length bit array field class which describes variable-length integer fields.

A variable-length integer field is a variable-length bit array field which has integer semantics.

If the value of a variable-length integer field class’s type property is "variable-length-signed-integer", then its instances have the two’s complement format.

A variable-length integer field class acts as a base of a variable-length enumeration field class.

Table 12. Common properties of a variable-length integer field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be one of:

"variable-length-unsigned-integer"

The instances of F are variable-length unsigned integer fields.

"variable-length-signed-integer"

The instances of F are variable-length signed integer fields.

Yes

preferred-display-base

JSON integer

Preferred base to display the value of an instance of F.

The value of this property MUST be one of:

2

Binary base.

8

Octal base.

10

Decimal base.

16

Hexadecimal base.

Property inherited from the abstract integer field class.

No

10

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 36. Minimal variable-length unsigned integer field class.
{
  "type": "variable-length-unsigned-integer"
}
Example 37. Variable-length signed integer field class with a preferred hexadecimal display base.
{
  "type": "variable-length-signed-integer",
  "preferred-display-base": 16
}
Example 38. Variable-length unsigned integer field class with user attributes.
{
  "type": "variable-length-unsigned-integer",
  "user-attributes": {
    "my.tracer": {
      "is-nice": true
    }
  }
}

8.4.13. Variable-length enumeration field class

A variable-length enumeration field class is both an abstract enumeration field class and a variable-length integer field class which describes variable-length enumeration fields.

A variable-length enumeration field is a variable-length integer field which MAY have one or more associated names thanks to its class’s mappings property.

If the value of a variable-length enumeration field class’s type property is "variable-length-signed-enumeration", then its instances have the two’s complement format.

Table 13. Properties of a variable-length enumeration field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be one of:

"variable-length-unsigned-enumeration"

The instances of F are variable-length unsigned enumeration fields.

"variable-length-signed-enumeration"

The instances of F are variable-length signed enumeration fields.

Yes

preferred-display-base

JSON integer

Preferred base to display the value of an instance of F.

The value of this property MUST be one of:

2

Binary base.

8

Octal base.

10

Decimal base.

16

Hexadecimal base.

Property inherited from the abstract integer field class.

No

10

mappings

Enumeration field class mappings

Mappings of F.

The value of this property MUST contain one or more properties.

Property inherited from the abstract enumeration field class.

Yes

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 39. Minimal variable-length unsigned enumeration field class.
{
  "type": "variable-length-unsigned-enumeration",
  "mappings": {
    "apple": [[1, 19]]
  }
}
Example 40. Variable-length unsigned enumeration field class with a preferred hexadecimal display base.
{
  "type": "variable-length-unsigned-enumeration",
  "preferred-display-base": 16,
  "mappings": {
    "lime": [[3, 3]],
    "kiwi": [[8, 8]],
    "blueberry": [[11, 11]]
  }
}
Example 41. Variable-length signed enumeration field class with user attributes.
{
  "type": "variable-length-signed-enumeration",
  "mappings": {
    "banana": [[-27399, -1882], [8, 199], [101, 101]],
    "orange": [[67, 67], [43, 1534]]
  },
  "user-attributes": {
    "my.tracer": {
      "is-nice": true
    }
  }
}

8.4.14. Null-terminated string field class

A null-terminated string field class describes null-terminated string fields.

A null-terminated string field is, in this order:

  1. Zero or more contiguous non-null (non-zero) bytes which form a UTF-8-encoded string.

  2. One null (zero) byte.

Table 14. Properties of a null-terminated string field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "null-terminated-string".

Yes

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 42. Minimal null-terminated string field class.
{
  "type": "null-terminated-string"
}
Example 43. Null-terminated string field class with user attributes.
{
  "type": "null-terminated-string",
  "user-attributes": {
    "my.tracer": {
      "is-nice": true
    }
  }
}

8.4.15. Static-length string field class

A static-length string field class describes static-length string fields.

A static-length string field is a sequence of zero or more contiguous bytes. All the bytes of a static-length string before the first null (zero) byte, if any, form a UTF-8-encoded string. All the bytes after the first null (zero) byte, if any, are garbage data.

The length, or number of bytes, of a static-length string field is a property (length) of its class.

Table 15. Properties of a static-length string field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "static-length-string".

Yes

length

JSON integer

Number of bytes contained in an instance of F.

The value of this property MUST be greater than or equal to zero.

Yes

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 44. Empty static-length string field class.
{
  "type": "static-length-string",
  "length": 0
}
Example 45. Static-length string field class with instances having 100 bytes.
{
  "type": "static-length-string",
  "length": 100
}
Example 46. Static-length string field class with user attributes.
{
  "type": "static-length-string",
  "length": 13,
  "user-attributes": {
    "my.tracer": null
  }
}

8.4.16. Dynamic-length string field class

A dynamic-length string field class describes dynamic-length string fields.

A dynamic-length string field is a sequence of zero or more contiguous bytes. All the bytes of a dynamic-length string before the first null (zero) byte, if any, form a UTF-8-encoded string. All the bytes after the first null (zero) byte, if any, are garbage data.

The length, or number of bytes, of a dynamic-length string field is the value of another, anterior length field. A consumer can find this length field thanks to the dynamic-length string field class’s length-field-location property.

Table 16. Properties of a dynamic-length string field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "static-length-string".

Yes

length-field-location

Field location

Location of the field of which the value is the number of bytes contained in an instance of F.

The class of the length field MUST be one of:

Yes

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 47. Dynamic-length string field class.
{
  "type": "dynamic-length-string",
  "length-field-location": ["event-record-payload", "length"]
}
Example 48. Dynamic-length string field class with user attributes.
{
  "type": "dynamic-length-string",
  "length-field-location": ["event-record-common-context", "name-length"],
  "user-attributes": {
    "my.tracer": 177
  }
}

8.4.17. Abstract BLOB field class

An abstract BLOB field class is a base of a static-length BLOB field class and a dynamic-length BLOB field class.

This field class is abstract in that it only exists to show the relation between different BLOB field classes in this document: a packet cannot contain an abstract BLOB field.

Table 17. Common properties of a BLOB field class F.
Name Type Description Required? Default

media-type

JSON string

IANA media type of an instance of F.

No

"application/octet-stream"

8.4.18. Static-length BLOB field class

A static-length BLOB field class is an abstract BLOB field class which describes static-length BLOB fields.

A static-length BLOB field is a sequence of zero or more contiguous bytes with an associated IANA media type (given by its class’s media-type property).

The length, or number of bytes, of a static-length BLOB field is a property (length) of its class.

Table 18. Properties of a static-length BLOB field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "static-length-blob".

Yes

length

JSON integer

Number of bytes contained in an instance of F.

The value of this property MUST be greater than or equal to zero.

Yes

media-type

JSON string

IANA media type of an instance of F.

Property inherited from the abstract BLOB field class.

No

"application/octet-stream"

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 49. Empty static-length BLOB field class with a default IANA media type.
{
  "type": "static-length-blob",
  "length": 0
}
Example 50. Static-length TIFF BLOB field class with instances having 511,267 bytes.
{
  "type": "static-length-blob",
  "length": 511267,
  "media-type": "image/tif"
}
Example 51. Static-length CSV BLOB field class with user attributes.
{
  "type": "static-length-blob",
  "length": 2400,
  "media-type": "text/csv",
  "user-attributes": {
    "my.tracer": {
      "csv-cols": 12
    }
  }
}

8.4.19. Dynamic-length BLOB field class

A dynamic-length BLOB field class is an abstract BLOB field class which describes dynamic-length BLOB fields.

A dynamic-length BLOB field is a sequence of zero or more contiguous bytes with an associated IANA media type.

The length, or number of bytes, of a dynamic-length BLOB field is the value of another, anterior length field. A consumer can find this length field thanks to the dynamic-length BLOB field class’s length-field-location property.

Table 19. Properties of a dynamic-length BLOB field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "dynamic-length-blob".

Yes

length-field-location

Field location

Location of the field of which the value is the number of bytes contained in an instance of F.

The class of the length field MUST be one of:

Yes

media-type

JSON string

IANA media type of an instance of F.

Property inherited from the abstract BLOB field class.

No

"application/octet-stream"

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 52. Dynamic-length BLOB field class with a default IANA media type.
{
  "type": "dynamic-length-blob",
  "length-field-location": ["event-record-payload", "length"]
}
Example 53. Dynamic-length JPEG BLOB field class with user attributes.
{
  "type": "dynamic-length-blob",
  "length-field-location": ["event-record-common-context", "length"],
  "media-type": "image/jpeg",
  "user-attributes": {
    "my.tracer": {
      "quality": 85
    }
  }
}

8.4.20. Structure field class

A structure field class describes structure fields.

A structure field is a sequence of zero or more structure field members. A structure field member is a named field.

Table 20. Properties of a structure field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "structure".

Yes

members

JSON array of structure field member classes

Classes of the members of an instance of F.

Each member class’s name property must be unique within this field class’s member class names.

No

[]

minimum-alignment

JSON integer

Minimum alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance.

The value of this property MUST be a positive power of two.

The effective alignment of the first bit of an instance of F MAY be greater than the value of this property.

No

1

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 54. Empty structure field class (no member classes).
{
  "type": "structure"
}
Example 55. Structure field class with three member classes.
{
  "type": "structure",
  "members": [
    {
      "name": "Villeray",
      "field-class": {
        "type": "null-terminated-string"
      }
    },
    {
      "name": "Berri",
      "field-class": {
        "type": "fixed-length-unsigned-integer",
        "length": 32,
        "byte-order": "little-endian",
        "preferred-display-base": 2
      },
      "user-attributes": {
        "my.tracer": {
          "is-mask": true
        }
      }
    },
    {
      "name": "Faillon",
      "field-class": {
        "type": "fixed-length-boolean",
        "length": 8,
        "byte-order": "little-endian"
      }
    }
  ]
}
Example 56. Structure field class with a minimum alignment.
{
  "type": "structure",
  "members": [
    {
      "name": "St-Denis",
      "field-class": {
        "type": "null-terminated-string"
      }
    },
    {
      "name": "Lajeunesse",
      "field-class": {
        "type": "fixed-length-unsigned-integer",
        "length": 32,
        "byte-order": "big-endian",
        "alignment": 32
      }
    }
  ],
  "minimum-alignment": 64
}
Example 57. Structure field class with user attributes.
{
  "type": "structure",
  "members": [
    {
      "name": "Henri-Julien",
      "field-class": {
        "type": "fixed-length-signed-integer",
        "length": 48,
        "byte-order": "little-endian"
      }
    },
    {
      "name": "Casgrain",
      "field-class": {
        "type": "static-length-string",
        "length": 32
      }
    }
  ],
  "user-attributes": {
    "my.tracer": {
      "version": 4
    }
  }
}
8.4.20.1. Structure field member class

A structure field member class describes structure field members.

A structure field member class is a JSON object.

Table 21. Properties of a structure field member class M.
Name Type Description Required? Default

name

JSON string

Name of M.

Yes

field-class

Field class

Field class of M.

Yes

user-attributes

User attributes

User attributes of M.

No

{}

extensions

Extensions

Extensions of M.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 58. Null-terminated string field class member class named cat.
{
  "name": "cat",
  "field-class": {
    "type": "null-terminated-string"
  }
}
Example 59. Variable-length signed integer field class member class named dog with user attributes.
{
  "name": "dog",
  "field-class": {
    "type": "variable-length-signed-integer",
    "preferred-display-base": 8
  },
  "user-attributes": {
    "my.tracer": {
      "uuid": "f36100b8-ec36-4861-8d6b-a9d6ab8973c9",
      "is-pid": true
    }
  }
}

8.4.21. Abstract array field class

An abstract array field class is a base of a static-length array field class and a dynamic-length array field class.

This field class is abstract in that it only exists to show the relation between different array field classes in this document: a packet cannot contain an abstract array field.

Table 22. Common properties of an array field class F.
Name Type Description Required? Default

element-field-class

Field class

Class of the element fields contained in an instance of F.

Yes

8.4.22. Static-length array field class

A static-length array field class is an abstract array field class which describes static-length array fields.

A static-length array field is a sequence of zero or more element fields.

The length, or number of element fields, of a static-length array field is a property (length) of its class.

Table 23. Properties of a static-length array field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "static-length-array".

Yes

element-field-class

Field class

Class of the element fields contained in an instance of F.

Property inherited from the abstract array field class.

Yes

length

JSON integer

Number of element fields contained in an instance of F.

The value of this property MUST be greater than or equal to zero.

Yes

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 60. Empty static-length array field class.
{
  "type": "static-length-array",
  "element-field-class": {
    "type": "fixed-length-signed-integer",
    "length": 16,
    "byte-order": "little-endian",
    "alignment": 16
  },
  "length": 0
}
Example 61. Static-length array field class with instances having 100 null-terminated string fields.
{
  "type": "static-length-array",
  "element-field-class": {
    "type": "null-terminated-string"
  },
  "length": 100
}
Example 62. Static-length array field class with user attributes.
{
  "type": "static-length-array",
  "element-field-class": {
    "type": "variable-length-unsigned-integer"
  },
  "length": 13,
  "user-attributes": {
    "my.tracer": true
  }
}

8.4.23. Dynamic-length array field class

A dynamic-length array field class is an abstract array field class which describes dynamic-length array fields.

A dynamic-length array field is a sequence of zero or more element fields.

The length, or number of element fields, of a dynamic-length array field is the value of another, anterior length field. A consumer can find this length field thanks to the dynamic-length array field class’s length-field-location property.

Table 24. Properties of a dynamic-length array field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "dynamic-length-array".

Yes

element-field-class

Field class

Class of the element fields contained in an instance of F.

Property inherited from the abstract array field class.

Yes

length-field-location

Field location

Location of the field of which the value is the number of element fields contained in an instance of F.

The class of the length field MUST be one of:

Yes

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 63. Dynamic-length array field class.
{
  "type": "dynamic-length-array",
  "element-field-class": {
    "type": "fixed-length-unsigned-integer",
    "length": 32,
    "byte-order": "big-endian",
    "alignment": 16
  },
  "length-field-location": ["event-record-payload", "length"]
}
Example 64. Dynamic-length array field class with user attributes.
{
  "type": "dynamic-length-array",
  "element-field-class": {
    "type": "variable-length-unsigned-integer"
  },
  "length-field-location": ["packet-context", "common-length"],
  "user-attributes": {
    "my.tracer": 177
  }
}

8.4.24. Optional field class

An optional field class describes optional fields.

An optional field is, depending on the value of another, anterior selector field, one of:

  • An instance of a given field class (optional field class’s field-class property).

    In this case, the optional field is said to be enabled.

  • A zero-bit field (no field).

    In this case, the optional field is said to be disabled.

A consumer can find the selector field thanks to the optional field class’s selector-field-location property.

Table 25. Properties of an optional field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "optional".

Yes

field-class

Field class

Class of an instance of F when it’s enabled.

Yes

selector-field-location

Field location

Location of the field of which the value indicates whether or not an instance of F is enabled.

The selector field MUST be an instance of one of:

Fixed-length boolean field class

An instance of F is enabled when the selector field is true.

Fixed-length integer field class
Variable-length integer field class

An instance of F is enabled when the selector field’s value is an element of any of the integer ranges of F's selector-field-ranges property.

Yes

selector-field-ranges

Integer range set

Ranges of integers which the value of a selector field MUST be an element of to enable an instance of F.

Yes, if the selector field is an instance of a fixed-length integer field class or a variable-length integer field class.

None if the selector field is an instance of a fixed-length boolean field class.

roles

Roles

Roles of F.

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 65. Optional static-length array field class with a boolean selector field class.
{
  "type": "optional",
  "selector-field-location": ["event-record-payload", "has-ip"],
  "field-class": {
    "type": "static-length-array",
    "element-field-class": {
      "type": "fixed-length-unsigned-integer",
      "length": 8,
      "byte-order": "little-endian",
      "alignment": 8
    },
    "length": 16
  }
}
Example 66. Optional static-length array field class with a fixed-length signed integer selector field class.
{
  "type": "optional",
  "selector-field-location": ["event-record-payload", "has-ip"],
  "selector-field-ranges": [[-12, -12], [-5, 0], [15, 35]],
  "field-class": {
    "type": "static-length-array",
    "element-field-class": {
      "type": "fixed-length-unsigned-integer",
      "length": 8,
      "byte-order": "little-endian",
      "alignment": 8
    },
    "length": 16
  }
}

8.4.25. Variant field class

A variant field class describes variant fields.

A variant field is, depending on the value of another, anterior selector field, the instance of a specific, effective field class amongst one or more variant field class options.

A consumer can find the selector field thanks to the variant field class’s selector-field-location property.

Table 26. Properties of a variant field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "variant".

Yes

options

JSON array of variant field class options

Options containing the possible effective classes of an instance of F.

This array MUST contain one or more elements.

Each option’s name property, if it’s set, must be unique within this field class’s option names.

The integer ranges (selector-field-ranges property) of two given options MUST NOT intersect.

Yes

selector-field-location

Field location

Location of the field of which the value indicates which option of F contains the effective class of an instance of F.

The selector field MUST be an instance of one of:

Yes

roles

Roles

Roles of F.

See Trace class fragment and Data stream class fragment which indicate accepted roles for their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 67. Variant field class with two options.
{
  "type": "variant",
  "selector-field-location": ["event-record-payload", "sel"],
  "options": [
    {
      "field-class": {
        "type": "null-terminated-string"
      },
      "selector-field-ranges": [[5, 5]]
    },
    {
      "field-class": {
        "type": "fixed-length-signed-integer",
        "length": 16,
        "byte-order": "little-endian",
        "preferred-display-base": 8
      },
      "selector-field-ranges": [[8, 8]]
    }
  ]
}
Example 68. Variant field class within an optional field class which share the same selector field location.

This example shows that an optional field class and a contained variant field class MAY share the same selector field location.

In this example, depending on the selector field’s value:

0

Optional field is not enabled.

1

Optional field is enabled and is a variant field.

Variant field is an instance of a null-terminated string field class (effective class).

2

Optional field is enabled and is a variant field.

Variant field is an instance of a variable-length signed integer field class (effective class).

{
  "type": "optional",
  "selector-field-location": ["event-record-payload", "sel"],
  "selector-field-ranges": [[1, 255]],
  "field-class": {
    "type": "variant",
    "selector-field-location": ["event-record-payload", "sel"],
    "options": [
      {
        "field-class": {
          "type": "null-terminated-string"
        },
        "selector-field-ranges": [[1, 1]]
      },
      {
        "field-class": {
          "type": "variable-length-signed-integer",
          "preferred-display-base": 16
        },
        "selector-field-ranges": [[2, 2]]
      }
    ]
  }
}
Example 69. Variant field class with user attributes.
{
  "type": "variant",
  "selector-field-location": ["event-record-specific-context", "sel"],
  "options": [
    {
      "field-class": {
        "type": "static-length-string",
        "length": 20
      },
      "selector-field-ranges": [[5, 5], [10, 10], [15, 15]]
    },
    {
      "field-class": {
        "type": "fixed-length-floating-point-number",
        "length": 32,
        "byte-order": "big-endian"
      },
      "selector-field-ranges": [[0, 4], [6, 9], [11, 14], [16, 127]]
    }
  ],
  "user-attributes": {
    "my.tracer": {
      "owner": "Jimmy",
      "id": 199990
    }
  }
}
8.4.25.1. Variant field class option

A variant field class option contains a possible effective class of a variant field.

A variant field class option O also contains the ranges of integer values (selector-field-ranges property) of which a selector field’s value MUST be an element of for a variant field’s effective class to be the field class of O.

A variant field class option is a JSON object.

Table 27. Properties of a variant field class option O contained in a variant field class F.
Name Type Description Required? Default

field-class

Field class

Field class of O.

Yes

selector-field-ranges

Integer range set

Ranges of integers which the value of a selector field MUST be an element of for the effective class of an instance of F to be the field class (field-class property) of O.

Yes

name

JSON string

Name of O.

This property exists to remain backward compatible with CTF 1.

No

O is unnamed

user-attributes

User attributes

User attributes of O.

No

{}

extensions

Extensions

Extensions of O.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Example 70. Unnamed null-terminated string field class option.
{
  "field-class": {
    "type": "null-terminated-string"
  },
  "selector-field-ranges": [[3, 9]]
}
Example 71. Variable-length signed integer field class option named juice with user attributes.
{
  "name": "juice",
  "field-class": {
    "type": "variable-length-signed-integer",
    "preferred-display-base": 16
  },
  "selector-field-ranges": [[-4, 4], [9, 9], [100, 200]],
  "user-attributes": {
    "my.tracer": {
      "uuid": "f36100b8-ec36-4861-8d6b-a9d6ab8973c9",
      "is-did": true
    }
  }
}

8.5. Preamble fragment

A preamble fragment indicates:

  • The CTF 2 major version (2).

    CTF 2 doesn’t have a minor version: all new changes are introduced as extensions.

  • Extension declarations.

    An extension declaration is an initial extension of which the purpose is to declare that it’s enabled within the metadata stream.

    Because an extension MAY alter the CTF 2 format itself, and because a preamble fragment is always the first metadata stream fragment, those extension declarations make it possible for a consumer to decline the trace’s data streams gracefully if it doesn’t support any declared extension.

The first fragment of a metadata stream MUST be a preamble fragment.

Table 28. Properties of a preamble fragment F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "preamble".

Yes

version

JSON integer

CTF 2 major version.

The value of this property MUST be 2.

Yes

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extension declarations of F.

The name of each property is a namespace and its value is a namespaced extensions object.

Within a namespaced extensions object, an extension named N is declared when it exists as a property named N, whatever the property’s value.

No

{}

Example 72. Minimal preamble fragment.
{
  "type": "preamble",
  "version": 2
}
Example 73. Preamble fragment with extension declarations.

The following preamble fragment declares the piano and ramen extensions under the my.tracer namespace.

{
  "type": "preamble",
  "version": 2,
  "extensions": {
    "my.tracer": {
      "piano": {
        "keys": 88,
        "temperament": "equal"
      },
      "ramen": null
    }
  }
}

8.6. Trace class fragment

A trace class describes traces.

Within a metadata stream, a trace class fragment MUST occur before any data stream class fragment.

Table 29. Properties of a trace class fragment F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "trace-class".

Yes

uuid

UUID

UUID of F.

No

F has no UUID

packet-header-field-class

Structure field class

Class of all the packet header fields of an instance of F.

Any field class within this property’s value MUST satisfy at least one of:

No

F has no packet header field class

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

8.6.1. Roles

If the packet-header-field-class property of a trace class fragment exists, then its inner field classes MAY have the following roles:

Table 30. Roles of field classes within a packet header field class.
Name Description Field class (F) constraints Other constraints

packet-magic-number

Packet magic number.

The purpose of a packet magic number field is to confirm the beginning of a CTF 2 packet.

Fixed-length unsigned integer field class with the following property value:

length

32

An instance of F MUST be the packet header structure field’s first member.

The value of an instance of F value MUST be 0xc1fc1fc1 (3254525889).

trace-class-uuid

Trace class UUID.

The purpose of a trace class UUID field is to confirm the association between a data stream and a metadata stream.

Static-length BLOB field class with the following property value:

length

16

The uuid property of the trace class must exist.

The value of an instance of F MUST be the binary representation of the uuid property of the trace class.

data-stream-class-id

Data stream class ID.

The purpose of a data stream class ID field is to set the current ID of the class of the packet’s data stream.

data-stream-id

Data stream ID.

The purpose of a data stream ID field is to set the current ID of the packet’s data stream.

Combined with the ID of its class, such a field makes it possible to uniquely identify a data stream within a trace.

Example 74. Trace class fragment.
{
  "type": "trace-class",
  "uuid": "1ec96494-e402-4546-93db-e9222bee6cc7",
  "packet-header-field-class": {
    "type": "structure",
    "members": [
      {
        "name": "the magic!",
        "field-class": {
          "type": "fixed-length-unsigned-integer",
          "length": 32,
          "byte-order": "little-endian",
          "preferred-display-base": 16,
          "roles": ["packet-magic-number"]
        }
      },
      {
        "name": "the UUID",
        "field-class": {
          "type": "static-length-blob",
          "length": 16,
          "roles": ["trace-class-uuid"]
        }
      },
      {
        "name": "my data stream class ID",
        "field-class": {
          "type": "fixed-length-unsigned-integer",
          "length": 8,
          "byte-order": "little-endian",
          "roles": ["data-stream-class-id"]
        }
      },
      {
        "name": "my data stream ID",
        "field-class": {
          "type": "variable-length-unsigned-integer",
          "roles": ["data-stream-id"]
        }
      }
    ]
  }
}

8.7. Clock class fragment

A clock class describes clocks.

A data stream MAY have a default clock.

Within a metadata stream, a clock class fragment MUST occur before any data stream class fragment which refers to it by name with its default-clock-class-name property.

Table 31. Properties of a clock class fragment F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "clock-class".

Yes

frequency

JSON integer

Frequency of an instance of F (Hz).

The value of this property MUST be greater than zero.

Yes

name

JSON string

Name of F.

Yes

description

JSON string

Textual description of F.

This property exists to remain backward compatible with CTF 1.

No

F has no textual description

uuid

UUID

UUID of F.

This property exists to remain backward compatible with CTF 1.

No

F has no UUID

origin-is-unix-epoch

JSON boolean

Whether or not the origin of an instance of F is the Unix epoch.

If the value of this property is false, then the origin of an instance of F is unknown.

No

true

offset

Clock class offset

Offset of an instance of F relative to its origin.

Let:

  • H be the value of F's frequency property.

  • O be the value of this property.

  • S be the value of O's seconds property.

  • C be the value of O's cycles property.

Then the effective offset of an instance of F, in clock cycles, is S × H + C.

No

{"seconds": 0, "cycles": 0}

precision

JSON integer

Precision of an instance of F (clock cycles).

The value of this property MUST be greater than or equal to zero.

Let P be the value of this property and V the value of an instance of F: the range of possible values of the instance is [V − PV + P].

No

0

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Within a metadata stream, two given clock class fragments MUST NOT:

  • Share the same name property value.

  • Share the same uuid property value.

Example 75. Minimal clock class fragment (1 GHz frequency).
{
  "type": "clock-class",
  "name": "my clock class",
  "frequency": 1000000000
}
Example 76. Clock class fragment with a UUID.
{
  "type": "clock-class",
  "name": "my clock class",
  "frequency": 1000000000,
  "uuid": "74d2008c-efff-4d03-8163-e9e2866acf20"
}
Example 77. Clock class fragment with an offset.
{
  "type": "clock-class",
  "name": "my clock class",
  "frequency": 1000000000,
  "offset": {
    "seconds": 1605112699,
    "cycles": 2878388
  }
}
Example 78. Clock class fragment with a precision.
{
  "type": "clock-class",
  "name": "my clock class",
  "frequency": 8000000,
  "precision": 100
}
Example 79. Clock class fragment with an origin which is not the Unix epoch.
{
  "type": "clock-class",
  "name": "my clock class",
  "frequency": 1000000000,
  "origin-is-unix-epoch": false
}
Example 80. Clock class fragment with user attributes.
{
  "type": "clock-class",
  "name": "my clock class",
  "frequency": 16000000,
  "user-attributes": {
    "my.tracer": {
      "sys-name": "SOC23",
      "bus": {
        "name": "LMB5",
        "index": 5
      },
      "propagation-delay-ps": 177
    }
  }
}

8.7.1. Clock class offset

A clock class offset contains the offset of a clock class's instances relative to their origin.

A clock class offset is a JSON object.

Table 32. Properties of a clock class offset contained in a clock class fragment F.
Name Type Description Required? Default

seconds

JSON integer

Offset, in seconds, of an instance of F relative to its origin.

No

0

cycles

JSON integer

Offset, in cycles, of an instance of F relative to its origin.

The value of this property MUST be greater than or equal to zero.

The value of this property MUST be less than the value of F's frequency property.

No

0

Example 81. Minimal clock class offset.
{}
Example 82. Clock class offset with seconds and cycles.
{
  "seconds": 1605112699,
  "cycles": 2878388
}
Example 83. Clock class offset with seconds only.
{
  "seconds": 1605111293
}
Example 84. Negative clock class offset.

This example shows that a clock class offset MAY be negative, that is, before the origin of the clock class instances.

{
  "seconds": -18003,
  "cycles": 11928547
}

8.8. Data stream class fragment

A data stream class describes data streams.

Within a metadata stream, a data stream class fragment F MUST occur before any event record class fragment of which F is the parent.

Table 33. Properties of a data stream class fragment F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "data-stream-class".

Yes

id

JSON integer

Numeric ID of F.

The value of this property MUST be greater than or equal to zero.

No

0

name

JSON string

Name of F.

The purpose of this property, combined with the namespace property, is to uniquely identify a data stream class amongst many producers.

No

F is unnamed

namespace

JSON string

Namespace of F.

The purpose of this property, combined with the name property, is to uniquely identify a data stream class amongst many producers.

No

F has no namespace

default-clock-class-name

JSON string

Name of the class of the default clock of an instance of F.

Within the metadata stream containing F, the clock class fragment which has this property’s value as its name property must occur before F.

No

An instance of F has no default clock

packet-context-field-class

Structure field class

Class of all the packet context fields of an instance of F.

No

F has no packet context field class

event-record-header-field-class

Structure field class

Class of all the event record header fields of an instance of F.

Any field class within this property’s value MUST satisfy at least one of:

No

F has no event record header field class

event-record-common-context-field-class

Structure field class

Class of all the event record common context fields of an instance of F.

No

F has no event record common context field class

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Within a metadata stream, two given data stream class fragments MUST NOT share the same id property value.

8.8.1. Roles

If the packet-context-field-class property of a data stream class fragment exists, then its inner field classes MAY have the following roles:

Table 34. Roles of field classes within a packet context field class.
Name Description Field class (F) constraints Other constraints

packet-total-size

Total size (bits) of the packet.

This size includes any padding after the packet’s content.

The value of an instance of F MUST be greater than or equal to the value of an instance of a field class having the packet-content-size role within the same packet context field.

packet-content-size

Content size (bits) of the packet.

The value of an instance of F MUST be less than or equal to the value of an instance of a field class having the packet-total-size role within the same packet context field.

packet-beginning-default-clock-timestamp

Timestamp of the packet’s data stream's default clock when the packet begins.

The timestamps of all the packet’s event records MUST be greater than or equal to the value of an instance of F.

The value of an instance of F MUST be less than or equal to the value of an instance of a field class having the packet-end-default-clock-timestamp role within the same packet context field.

packet-end-default-clock-timestamp

Timestamp of the packet’s data stream’s default clock when the packet ends.

The timestamps of all the packet’s event records MUST be less than or equal to the value of an instance of F.

The value of an instance of F MUST be greater than or equal to the value of an instance of a field class having the packet-beginning-default-clock-timestamp role within the same packet context field.

discarded-event-record-counter-snapshot

Snapshot of the packet’s data stream’s discarded event record counter when the packet ends.

packet-sequence-number

Sequence number of the packet within its data stream.

If the event-record-header-field-class property of a data stream class fragment exists, then its inner field classes MAY have the following roles:

Table 35. Roles of field classes within an event record header field class.
Name Description Field class (F) constraints

event-record-class-id

Event record class ID.

The purpose of an event record class ID field is to set the current ID of the class of the event record within its parent data stream class.

default-clock-timestamp

Current timestamp of the event record’s data stream's default clock when the event record occurs.

8.9. Event record class fragment

An event record class describes event records.

The data stream class fragment of which the id property’s value matches the value of the data-stream-class-id property of an event record class fragment F is considered the parent of F.

Table 36. Properties of an event record class fragment F having the data stream class P as its parent.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "event-record-class".

Yes

id

JSON integer

Numeric ID of F within P.

The value of this property MUST be greater than or equal to zero.

No

0

data-stream-class-id

JSON integer

Numeric ID of P.

The value of this property MUST be greater than or equal to zero.

Within the metadata stream, P MUST occur before F.

No

0

name

JSON string

Name of F.

The purpose of this property, combined with the namespace property, is to uniquely identify an event record class amongst many producers.

No

F is unnamed

namespace

JSON string

Namespace of F.

The purpose of this property, combined with the name property, is to uniquely identify an event record class amongst many producers.

No

F has no namespace

specific-context-field-class

Structure field class

Class of the event record specific context field of an instance of F.

No

F has no event record specific context field class

payload-field-class

Structure field class

Class of the event record payload field of an instance of F.

No

F has no event record payload field class

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property must also be declared in the metadata stream’s preamble fragment.

No

{}

Within a metadata stream, two given event record class fragments MUST NOT share the same id property value and the same data-stream-class-id property value.

9. Data stream decoding procedure

This section shows how to, procedurally, decode a CTF 2 data stream.

Decoding a data stream is the responsibility of a consumer.

This document doesn’t specify how to encode a data stream, as this procedure implies much more freedom than decoding. You can deduce how to encode a data stream from the decoding procedure.

A consumer needs to keep a data stream decoding state while decoding a data stream. A data stream decoding state comprises the following variable:

Table 37. Variable needed to decode a data stream S.
Name Type Description Initial value

O

Integer

Current decoding offset/position (bits) from the beginning of S.

0

To decode a data stream S:

9.1. Packet decoding procedure

A consumer needs to keep a packet decoding state while decoding a packet. A packet decoding state comprises the following variables:

Table 38. Variables needed to decode a packet P within a data stream S.
Name Type Description Initial value

DEF_CLK_VAL

Integer

Current value (clock cycles) of the default clock of S, if any.

0

DSC_ID

Integer

Current ID of the class of S.

0

DSC

Data stream class

Current class of S.

None

DS_ID

Integer

Current ID of S.

None

PKT_TOTAL_SZ

Integer

Current total size (bits) of P.

PKT_CONTENT_SZ

Integer

Current content size (bits) of P.

LAST_BO

String

Byte order of the last decoded fixed-length bit array field.

None

To decode a packet P within a data stream S:

  1. Let PO be the current value of O.

  2. If the metadata stream’s trace class fragment's packet-header-field-class property exists, then decode the header field of P using this property.

    During the packet header field decoding procedure, after having decoded a field F having the class C with a roles property:

    • If C has the role packet-magic-number, then validate that the integer value of F is 0xc1fc1fc1 (3254525889).

    • If C has the role trace-type-uuid, then validate that the value of F matches the uuid property of the trace class fragment.

    • If C has the role data-stream-class-id, then set DSC_ID to the integer value of F.

    • If C has the role data-stream-id, then set DS_ID to the integer value of F.

    After having decoded the whole packet header field, if DS_ID is set, it contains the ID of S within its class. In other words, two data streams MAY have the same ID if they are instances of different data stream classes.

  3. Set DSC to the data stream class having DSC_ID as the value of its id property.

  4. If the packet-context-field-class property of DSC exists, then decode the context field of P using this property.

    During the packet context field decoding procedure, after having decoded a field F having the class C with a roles property:

    • If C has the role packet-total-size, then set PKT_TOTAL_SZ to the integer value of F.

    • If C has the role packet-content-size, then set PKT_CONTENT_SZ to the integer value of F.

    • If C has the role packet-beginning-default-clock-timestamp, then set DEF_CLK_VAL to the integer value of F.

    • If C has the role packet-end-default-clock-timestamp, then the integer value of F is the value of the default clock of S at the end of P.

    • If C has the role discarded-event-record-counter-snapshot, then the integer value of F is a snapshot of the discarded event record counter of S at the end of P.

    • If C has the role packet-sequence-number, then the integer value of F is the sequence number of P within S.

  5. If PKT_TOTAL_SZ is ∞ and PKT_CONTENT_SZ is not ∞, then set PKT_TOTAL_SZ to PKT_CONTENT_SZ.

  6. If PKT_CONTENT_SZ is ∞ and PKT_TOTAL_SZ is not ∞, then set PKT_CONTENT_SZ to PKT_TOTAL_SZ.

  7. While O < PO + PKT_CONTENT_SZ and there’s remaining data in S:

  8. If PKT_TOTAL_SZ and PKT_CONTENT_SZ both are not ∞, then set O to PO + PKT_TOTAL_SZ, effectively skipping end-of-packet padding.

9.2. Event record decoding procedure

A consumer needs to keep an event record decoding state while decoding an event record. An event record decoding state comprises the following variables:

Table 39. Variables needed to decode an event record E within a data stream S.
Name Type Description Initial value

ERC_ID

Integer

Current ID of the class of E of which the parent is the class of S.

0

ERC

Event record class

Current class of E.

None

To decode an event record E within a data stream S:

  1. If the event-record-header-field-class property of DSC exists, then decode the header field of E using this property.

    During the event record header field decoding procedure, after having decoded a field F having the class C with a roles property:

    • If C has the role event-record-class-id, then set ERC_ID to the integer value of F.

    • If C has the role default-clock-timestamp, then update DEF_CLK_VAL according to Common Trace Format 1.8.3, §8.

      This proposal refers to CTF 1 to remain brief; the CTF 2 specification will show the complete clock value updating procedure.

    After having decoded the whole event record header field, DEF_CLK_VAL is the value of S's default clock when E occurs.

  2. Set ERC to the event record class having:

    • DSC_ID as the value of its data-stream-class-id property.

    • ERC_ID as the value of its id property.

  3. If the event-record-common-context-field-class property of DSC exists, then decode the common context field of E using this property.

  4. If the specific-context-field-class property of ERC exists, then decode the specific context field of E using this property.

  5. If the payload-field-class property of ERC exists, then decode the payload field of E using this property.

9.3. Field decoding procedure

The class of a field contains what’s needed to decode it as a value.

While a field is an actual sequence of bits within a data stream, a value is its conceptual interpretation with attached semantics.

The types of values are:

Value type Possible values

Boolean

True or false.

Integer

Integral quantity.

Real

Continuous quantity.

String

Sequence of Unicode characters.

Array

Sequence of values having the same type.

Structure

Sequence of named values (members) which MAY have different types.

To decode an instance of a field class F, depending on the value of its type property:

Value of F's type property F's decoding procedure

9.3.1. Alignment procedure

The decoding procedure of many fields require O − PO to have a specific alignment.

The alignment requirement of an instance of a field class F is, depending on the value of its type property:

F's type property F's alignment requirement

The value of F's alignment property.

8

The maximum value of:

  • The value of F's minimum-alignment property.

  • The alignment requirements of the instances of the field-class property of each member class of F's members property.

The alignment requirement of an instance of F's element-field-class property

1

To align O − PO to some alignment requirement A (bits):

  • Set O to PO + ((O − PO + A − 1) & −A), where & is the bitwise AND operator.

9.3.2. Fixed-length bit array field decoding procedure

To decode an instance of a fixed-length bit array field class F:

  1. Let:

    • BO be the value of F's byte-order property.

    • L be the value of F's length property.

    • V be an array of booleans of length L.

  2. Align O − PO according to F.

  3. If ((O − PO) mod 8 ≠ 0) and LAST_BO ≠ BO, then fail immediately.

  4. Read L bits of data from S at the offset O following Common Trace Format 1.8.3, §4.1.5, as booleans within V.

    This proposal refers to CTF 1 to remain brief; the CTF 2 specification will show the complete decoding procedures for big-endian and little-endian fixed-length bit array fields.
  5. Set O to O + L.

  6. Set LAST_BO to BO.

V is the decoded value.

9.3.3. Fixed-length boolean field decoding procedure

To decode an instance of a fixed-length boolean field class:

  1. Let VB be a boolean.

  2. Decode the instance as a fixed-length bit array field.

  3. If all the elements of V are false, then set VB to false.

    Else, set VB to true.

VB is the decoded boolean value.

9.3.4. Fixed-length unsigned integer field decoding procedure

To decode an instance of a fixed-length unsigned integer field class:

  1. Let VI be an integer.

  2. Decode the instance as a fixed-length bit array field.

  3. Set VI as the unsigned integer interpretation of V.

VI is the decoded integer value.

9.3.5. Fixed-length signed integer field decoding procedure

To decode an instance of a fixed-length signed integer field class:

  1. Let VI be an integer.

  2. Decode the instance as a fixed-length bit array field.

  3. Set VI as the signed integer interpretation, following the two’s complement format, of V.

VI is the decoded integer value.

9.3.6. Fixed-length floating point number field decoding procedure

  1. Let VR be a real value.

  2. Decode the instance as a fixed-length bit array field.

  3. Set VR to the real number interpretation, following IEEE 754-2008's binary interchange format, of V.

VR is the decoded real value.

9.3.7. Variable-length bit array field decoding procedure

To decode an instance of a variable-length bit array field class F:

  1. Let V be an empty array of booleans.

  2. Align O − PO according to F.

  3. Read N bytes of data from S at the offset O, as many as needed following the unsigned LEB128 format, appending the decoded bits to V as booleans.

    This proposal refers to LEB128 to remain brief; the CTF 2 specification will show the complete decoding procedure.
  4. Set O to O + N × 8.

V is the decoded value.

9.3.8. Variable-length unsigned integer field decoding procedure

To decode an instance of a variable-length unsigned integer field class:

  1. Let VI be an integer.

  2. Decode the instance as a variable-length bit array field.

  3. Set VI as the unsigned integer interpretation of V.

VI is the decoded integer value.

9.3.9. Variable-length signed integer field decoding procedure

To decode an instance of a variable-length signed integer field class:

  1. Let VI be an integer.

  2. Decode the instance as a variable-length bit array field.

  3. Set VI as the signed integer interpretation, following the two’s complement format, of V.

VI is the decoded integer value.

9.3.10. Null-terminated string field decoding procedure

To decode an instance of a null-terminated string field class F:

  1. Let:

    • B be a byte.

    • A be an empty sequence of bytes.

    • V be a string.

  2. Align O − PO according to F.

  3. Read one byte of data from S at the offset O as B.

  4. Set O to O + 8.

  5. While B ≠ 0:

    1. Append B to A.

    2. Read one byte of data from S at the offset O as B.

    3. Set O to O + 8.

  6. Decode A, following UTF-8, as V.

V is the decoded string value.

9.3.11. Static-length string field decoding procedure

To decode an instance of a static-length string field class F:

  1. Let:

    • L be F's length property.

    • I be an unsigned integer initialized to 0.

    • B be a byte.

    • R be a boolean initialized to true.

    • A be an empty sequence of bytes.

    • V be a string.

  2. Align O − PO according to F.

  3. While I < L:

    1. Read one byte of data from S at the offset O as B.

    2. If B = 0, then set R to false.

      Else, if R is true, then append B to A.

    3. Set O to O + 8.

    4. Set I to I + 1.

  4. Decode A, following UTF-8, as V.

V is the decoded string value.

9.3.12. Static-length BLOB field decoding procedure

To decode an instance of a static-length BLOB field class F:

  1. Let:

    • L be F's length property.

    • V be an array of bytes of length L.

  2. Align O − PO according to F.

  3. Read L bytes of data from S at the offset O as V.

  4. Set O to O + L × 8.

V is the decoded BLOB value.

9.3.13. Dynamic-length string field decoding procedure

To decode an instance of a dynamic-length string field class F:

  1. Let:

    • L be the value of the previously decoded unsigned integer field of which F's length-field-location indicates the location.

    • I be an unsigned integer initialized to 0.

    • B be a byte.

    • R be a boolean initialized to true.

    • A be an empty sequence of bytes.

    • V be a string.

  2. Align O − PO according to F.

  3. While I < L:

    1. Read one byte of data from S at the offset O as B.

    2. If B = 0, then set R to false.

      Else, if R is true, then append B to A.

    3. Set O to O + 8.

    4. Set I to I + 1.

  4. Decode A, following UTF-8, as V.

V is the decoded string value.

9.3.14. Dynamic-length BLOB field decoding procedure

To decode an instance of a dynamic-length BLOB field class F:

  1. Let:

    • L be the value of the previously decoded unsigned integer field of which F's length-field-location indicates the location.

    • V be an array of bytes of length L.

  2. Align O − PO according to F.

  3. Read L bytes of data from S at the offset O as V.

  4. Set O to O + L × 8.

V is the decoded BLOB value.

9.3.15. Structure field decoding procedure

To decode an instance of a structure field class F:

  1. Let:

    • M be F's members property.

    • V be an empty structure.

  2. Align O − PO according to F.

  3. For each member class MC of M:

    1. Let:

      • MF be MC's field-class property.

      • MN be MC's name property.

    2. Decode one instance of MF, appending the resulting value as a member of V named MN.

V is the decoded value.

9.3.16. Static-length array field decoding procedure

To decode an instance of a static-length array field class F:

  1. Let:

    • L be F's length property.

    • EF be F's element-field-class property.

    • I be an unsigned integer initialized to 0.

    • V be an array of values of length L.

  2. Align O − PO according to F.

  3. While I < L:

    1. Decode one instance of EF as element I of V.

    2. Set I to I + 1.

V is the decoded value.

9.3.17. Dynamic-length array field decoding procedure

To decode an instance of a dynamic-length array field class F:

  1. Let:

    • L be the value of the previously decoded unsigned integer field of which F's length-field-location indicates the location.

    • EF be F's element-field-class property.

    • I be an unsigned integer initialized to 0.

    • V be an array of values of length L

  2. Align O − PO according to F.

  3. While I < L:

    1. Decode one instance of EF as element I of V.

    2. Set I to I + 1.

V is the decoded value.

9.3.18. Optional field decoding procedure

To decode an instance of an optional field class F:

  1. Let:

    • SEL be the value of the previously decoded boolean or integer field of which F's selector-field-location indicates the location.

    • OF be F's field-class property.

    • V be an optional value, initially not set.

  2. If the class of SEL is a fixed-length boolean field class and SEL is true, then:

    • Decode one instance of OF as V.

    Else, if SEL is an element of any integer range of F's selector-field-ranges property, then:

    • Decode one instance of OF as V.

If V is set, then V is the decoded value. Otherwise, there’s no decoded value.

9.3.19. Variant field decoding procedure

To decode an instance of an variant field class F:

  1. Let:

    • SEL be the value of the previously decoded integer field of which F's selector-field-location indicates the location.

    • OPTS be F's options property.

    • OF be the field-class property of the variant field class option OPT of OPTS of which SEL is an element of any integer range of OPT's selector-field-ranges property.

    • V be a value.

  2. Decode one instance of OF as V.

V is the decoded value.