--- 1/draft-ietf-cbor-7049bis-09.txt 2019-12-18 08:13:22.419954845 -0800 +++ 2/draft-ietf-cbor-7049bis-10.txt 2019-12-18 08:13:22.567958620 -0800 @@ -1,19 +1,19 @@ Network Working Group C. Bormann Internet-Draft Universitaet Bremen TZI Obsoletes: 7049 (if approved) P. Hoffman Intended status: Standards Track ICANN -Expires: May 8, 2020 November 05, 2019 +Expires: June 20, 2020 December 18, 2019 Concise Binary Object Representation (CBOR) - draft-ietf-cbor-7049bis-09 + draft-ietf-cbor-7049bis-10 Abstract The Concise Binary Object Representation (CBOR) is a data format whose design goals include the possibility of extremely small code size, fairly small message size, and extensibility without the need for version negotiation. These design goals make it different from earlier binary serializations such as ASN.1 and MessagePack. This document is a revised edition of RFC 7049, with editorial @@ -42,21 +42,21 @@ Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." - This Internet-Draft will expire on May 8, 2020. + This Internet-Draft will expire on June 20, 2020. Copyright Notice Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents @@ -75,52 +75,51 @@ 2.1. Extended Generic Data Models . . . . . . . . . . . . . . 8 2.2. Specific Data Models . . . . . . . . . . . . . . . . . . 9 3. Specification of the CBOR Encoding . . . . . . . . . . . . . 9 3.1. Major Types . . . . . . . . . . . . . . . . . . . . . . . 11 3.2. Indefinite Lengths for Some Major Types . . . . . . . . . 13 3.2.1. The "break" Stop Code . . . . . . . . . . . . . . . . 13 3.2.2. Indefinite-Length Arrays and Maps . . . . . . . . . . 14 3.2.3. Indefinite-Length Byte Strings and Text Strings . . . 16 3.3. Floating-Point Numbers and Values with No Content . . . . 16 3.4. Tagging of Items . . . . . . . . . . . . . . . . . . . . 18 - 3.4.1. Date and Time . . . . . . . . . . . . . . . . . . . . 21 - 3.4.2. Standard Date/Time String . . . . . . . . . . . . . . 21 - 3.4.3. Epoch-based Date/Time . . . . . . . . . . . . . . . . 21 - 3.4.4. Bignums . . . . . . . . . . . . . . . . . . . . . . . 22 - 3.4.5. Decimal Fractions and Bigfloats . . . . . . . . . . . 22 - 3.4.6. Content Hints . . . . . . . . . . . . . . . . . . . . 24 - 3.4.6.1. Encoded CBOR Data Item . . . . . . . . . . . . . 24 - 3.4.6.2. Expected Later Encoding for CBOR-to-JSON + 3.4.1. Standard Date/Time String . . . . . . . . . . . . . . 21 + 3.4.2. Epoch-based Date/Time . . . . . . . . . . . . . . . . 21 + 3.4.3. Bignums . . . . . . . . . . . . . . . . . . . . . . . 22 + 3.4.4. Decimal Fractions and Bigfloats . . . . . . . . . . . 22 + 3.4.5. Content Hints . . . . . . . . . . . . . . . . . . . . 24 + 3.4.5.1. Encoded CBOR Data Item . . . . . . . . . . . . . 24 + 3.4.5.2. Expected Later Encoding for CBOR-to-JSON Converters . . . . . . . . . . . . . . . . . . . 24 - 3.4.6.3. Encoded Text . . . . . . . . . . . . . . . . . . 25 - 3.4.7. Self-Described CBOR . . . . . . . . . . . . . . . . . 26 + 3.4.5.3. Encoded Text . . . . . . . . . . . . . . . . . . 25 + 3.4.6. Self-Described CBOR . . . . . . . . . . . . . . . . . 26 4. Serialization Considerations . . . . . . . . . . . . . . . . 26 4.1. Preferred Serialization . . . . . . . . . . . . . . . . . 26 4.2. Deterministically Encoded CBOR . . . . . . . . . . . . . 27 4.2.1. Core Deterministic Encoding Requirements . . . . . . 28 4.2.2. Additional Deterministic Encoding Considerations . . 29 4.2.3. Length-first map key ordering . . . . . . . . . . . . 30 5. Creating CBOR-Based Protocols . . . . . . . . . . . . . . . . 31 5.1. CBOR in Streaming Applications . . . . . . . . . . . . . 32 5.2. Generic Encoders and Decoders . . . . . . . . . . . . . . 32 5.3. Validity of Items . . . . . . . . . . . . . . . . . . . . 33 5.3.1. Basic validity . . . . . . . . . . . . . . . . . . . 33 5.3.2. Tag validity . . . . . . . . . . . . . . . . . . . . 34 5.4. Validity and Evolution . . . . . . . . . . . . . . . . . 34 5.5. Numbers . . . . . . . . . . . . . . . . . . . . . . . . . 35 5.6. Specifying Keys for Maps . . . . . . . . . . . . . . . . 36 5.6.1. Equivalence of Keys . . . . . . . . . . . . . . . . . 37 5.7. Undefined Values . . . . . . . . . . . . . . . . . . . . 38 6. Converting Data between CBOR and JSON . . . . . . . . . . . . 38 6.1. Converting from CBOR to JSON . . . . . . . . . . . . . . 38 6.2. Converting from JSON to CBOR . . . . . . . . . . . . . . 39 - 7. Future Evolution of CBOR . . . . . . . . . . . . . . . . . . 40 + 7. Future Evolution of CBOR . . . . . . . . . . . . . . . . . . 41 7.1. Extension Points . . . . . . . . . . . . . . . . . . . . 41 7.2. Curating the Additional Information Space . . . . . . . . 42 8. Diagnostic Notation . . . . . . . . . . . . . . . . . . . . . 42 8.1. Encoding Indicators . . . . . . . . . . . . . . . . . . . 43 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 44 9.1. Simple Values Registry . . . . . . . . . . . . . . . . . 44 9.2. Tags Registry . . . . . . . . . . . . . . . . . . . . . . 44 9.3. Media Type ("MIME Type") . . . . . . . . . . . . . . . . 45 9.4. CoAP Content-Format . . . . . . . . . . . . . . . . . . . 46 9.5. The +cbor Structured Syntax Suffix Registration . . . . . 46 @@ -259,28 +257,28 @@ (that is, most significant byte first, also known as "big-endian"). This specification makes use of the following terminology: Data item: A single piece of CBOR data. The structure of a data item may contain zero, one, or more nested data items. The term is used both for the data item in representation format and for the abstract idea that can be derived from that by a decoder; the former can be addressed specifically by using "encoded data item". - Decoder: A process that decodes a well-formed CBOR data item and - makes it available to an application. Formally speaking, a + Decoder: A process that decodes a well-formed encoded CBOR data item + and makes it available to an application. Formally speaking, a decoder contains a parser to break up the input using the syntax rules of CBOR, as well as a semantic processor to prepare the data in a form suitable to the application. - Encoder: A process that generates the representation format of a - CBOR data item from application information. + Encoder: A process that generates the (well-formed) representation + format of a CBOR data item from application information. Data Stream: A sequence of zero or more data items, not further assembled into a larger containing data item. The independent data items that make up a data stream are sometimes also referred to as "top-level data items". Well-formed: A data item that follows the syntactic structure of CBOR. A well-formed data item uses the initial bytes and the byte strings and/or data items that are implied by their values as defined in CBOR and does not include following extraneous data. @@ -298,27 +296,27 @@ of acceptability. Stream decoder: A process that decodes a data stream and makes each of the data items in the sequence available to an application as they are received. Where bit arithmetic or data types are explained, this document uses the notation familiar from the programming language C, except that "**" denotes exponentiation. Similar to the "0x" notation for hexadecimal numbers, numbers in binary notation are prefixed with - "0b". Underscores can be added to such a number solely for - readability, so 0b00100001 (0x21) might be written 0b001_00001 to - emphasize the desired interpretation of the bits in the byte; in this - case, it is split into three bits and five bits. Encoded CBOR data - items are sometimes given in the "0x" or "0b" notation; these values - are first interpreted as numbers as in C and are then interpreted as - byte strings in network byte order, including any leading zero bytes + "0b". Underscores can be added to a number solely for readability, + so 0b00100001 (0x21) might be written 0b001_00001 to emphasize the + desired interpretation of the bits in the byte; in this case, it is + split into three bits and five bits. Encoded CBOR data items are + sometimes given in the "0x" or "0b" notation; these values are first + interpreted as numbers as in C and are then interpreted as byte + strings in network byte order, including any leading zero bytes expressed in the notation. 2. CBOR Data Models CBOR is explicit about its generic data model, which defines the set of all data items that can be represented in CBOR. Its basic generic data model is extensible by the registration of simple type values and tags. Applications can then subset the resulting extended generic data model to build their specific data models. @@ -349,26 +347,26 @@ o a mapping (mathematical function) from zero or more data items ("keys") each to a data item ("values"), ("map") o a tagged data item ("tag"), comprising a tag number (an integer in the range 0..2**64-1) and a tagged value (a data item) Note that integer and floating-point values are distinct in this model, even if they have the same numeric value. - Also note that serialization variants, such as number of bytes of the - encoded floating value, or the choice of one of the ways in which an - integer, the length of a text or byte string, the number of elements - in an array or pairs in a map, or a tag number, (collectively "the - argument", see Section 3) can be encoded, are not visible at the - generic data model level. + Also note that serialization variants, such as the number of bytes of + the encoded floating value, or the choice of one of the ways in which + an integer, the length of a text or byte string, the number of + elements in an array or pairs in a map, or a tag number, + (collectively "the argument", see Section 3) can be encoded, are not + visible at the generic data model level. 2.1. Extended Generic Data Models This basic generic data model comes pre-extended by the registration of a number of simple values and tag numbers right in this document, such as: o "false", "true", "null", and "undefined" (simple values identified by 20..23) @@ -407,34 +405,36 @@ instead of by referring to aspects of their CBOR representation ("major type 1", "major type 4"). Specific data models can also specify what values (including values of different types) are equivalent for the purposes of map keys and encoder freedom. For example, in the generic data model, a valid map MAY have both "0" and "0.0" as keys, and an encoder MUST NOT encode "0.0" as an integer (major type 0, Section 3.1). However, if a specific data model declares that floating-point and integer representations of integral values are equivalent, using both map - keys "0" and "0.0" in a single map would be considered duplicates and - so invalid, and an encoder could encode integral-valued floats as - integers or vice versa, perhaps to save encoded bytes. + keys "0" and "0.0" in a single map would be considered duplicates, + even while encoded as different major types, and so invalid; and an + encoder could encode integral-valued floats as integers or vice + versa, perhaps to save encoded bytes. 3. Specification of the CBOR Encoding A CBOR data item (Section 2) is encoded to or decoded from a byte string carrying a well-formed encoded data item as described in this - section. The encoding is summarized in Table 6. An encoder MUST - produce only well-formed encoded data items. A decoder MUST NOT - return a decoded data item when it encounters input that is not a - well-formed encoded CBOR data item (this does not detract from the - usefulness of diagnostic and recovery tools that might make available - some information from a damaged encoded CBOR data item). + section. The encoding is summarized in Table 6, indexed by the + initial byte. An encoder MUST produce only well-formed encoded data + items. A decoder MUST NOT return a decoded data item when it + encounters input that is not a well-formed encoded CBOR data item + (this does not detract from the usefulness of diagnostic and recovery + tools that might make available some information from a damaged + encoded CBOR data item). The initial byte of each encoded data item contains both information about the major type (the high-order 3 bits, described in Section 3.1) and additional information (the low-order 5 bits). With a few exceptions, the additional information's value describes how to load an unsigned integer "argument": Less than 24: The argument's value is the value of the additional information. @@ -701,21 +701,22 @@ FF -- "break" 3.2.3. Indefinite-Length Byte Strings and Text Strings Indefinite-length strings are represented by a byte containing the major type and additional information value of 31, followed by a series of zero or more byte or text strings ("chunks") that have definite lengths, followed by the "break" stop code (Section 3.2.1). The data item represented by the indefinite-length string is the concatenation of the chunks (i.e., the empty byte or text string, - respectively, if no chunk is present). + respectively, if no chunk is present). (Note that zero-length + chunks, while not particularly useful, are permitted.) If any item between the indefinite-length string indicator (0b010_11111 or 0b011_11111) and the "break" stop code is not a definite-length string item of the same major type, the string is not well-formed. If any definite-length text string inside an indefinite-length text string is invalid, the indefinite-length text string is invalid. Note that this implies that the bytes of a single UTF-8 character cannot be spread between chunks: a new chunk can only be started at a @@ -812,21 +813,21 @@ In CBOR, a data item can be enclosed by a tag to give it additional semantics while retaining its structure. The tag is major type 6, and represents an unsigned integer as indicated by the tag's argument (Section 3); the (sole) enclosed data item is carried as content data. If a tag requires structured data, this structure is encoded into the nested data item. The definition of a tag number usually restricts what kinds of nested data item or items are valid for tags using this tag number. For example, assume that a byte string of length 12 is marked with a - tag of number 2 to indicate it is a positive bignum (Section 3.4.4). + tag of number 2 to indicate it is a positive bignum (Section 3.4.3). This would be marked as 0b110_00010 (major type 6, additional information 2 for the tag number) followed by 0b010_01100 (major type 2, additional information of 12 for the length) followed by the 12 bytes of the bignum. Decoders do not need to understand tags of every tag number, and tags may be of little value in applications where the implementation creating a particular CBOR data item and the implementation decoding that stream know the semantic meaning of each item in the data flow. Their primary purpose in this specification is to define common data @@ -850,93 +851,93 @@ [RFC7049], with definitions in the rest of this section. Note that many other tag numbers have been defined since the publication of [RFC7049]; see the registry described at Section 9.2 for the complete list. +----------+----------+---------------------------------------------+ | Tag | Data | Semantics | | Number | Item | | +----------+----------+---------------------------------------------+ | 0 | text | Standard date/time string; see | - | | string | Section 3.4.2 | + | | string | Section 3.4.1 | | | | | - | 1 | multiple | Epoch-based date/time; see Section 3.4.3 | + | 1 | multiple | Epoch-based date/time; see Section 3.4.2 | | | | | - | 2 | byte | Positive bignum; see Section 3.4.4 | + | 2 | byte | Positive bignum; see Section 3.4.3 | | | string | | | | | | - | 3 | byte | Negative bignum; see Section 3.4.4 | + | 3 | byte | Negative bignum; see Section 3.4.3 | | | string | | | | | | - | 4 | array | Decimal fraction; see Section 3.4.5 | + | 4 | array | Decimal fraction; see Section 3.4.4 | | | | | - | 5 | array | Bigfloat; see Section 3.4.5 | + | 5 | array | Bigfloat; see Section 3.4.4 | | | | | | 21 | multiple | Expected conversion to base64url encoding; | - | | | see Section 3.4.6.2 | + | | | see Section 3.4.5.2 | | | | | | 22 | multiple | Expected conversion to base64 encoding; see | - | | | Section 3.4.6.2 | + | | | Section 3.4.5.2 | | | | | | 23 | multiple | Expected conversion to base16 encoding; see | - | | | Section 3.4.6.2 | + | | | Section 3.4.5.2 | | | | | - | 24 | byte | Encoded CBOR data item; see Section 3.4.6.1 | + | 24 | byte | Encoded CBOR data item; see Section 3.4.5.1 | | | string | | | | | | - | 32 | text | URI; see Section 3.4.6.3 | + | 32 | text | URI; see Section 3.4.5.3 | | | string | | | | | | - | 33 | text | base64url; see Section 3.4.6.3 | + | 33 | text | base64url; see Section 3.4.5.3 | | | string | | | | | | - | 34 | text | base64; see Section 3.4.6.3 | + | 34 | text | base64; see Section 3.4.5.3 | | | string | | | | | | - | 35 | text | Regular expression; see Section 3.4.6.3 | + | 35 | text | Regular expression; see Section 3.4.5.3 | | | string | | | | | | - | 36 | text | MIME message; see Section 3.4.6.3 | + | 36 | text | MIME message; see Section 3.4.5.3 | | | string | | | | | | - | 55799 | multiple | Self-described CBOR; see Section 3.4.7 | + | 55799 | multiple | Self-described CBOR; see Section 3.4.6 | +----------+----------+---------------------------------------------+ Table 4: Tag numbers defined in RFC 7049 Conceptually, tags are interpreted in the generic data model, not at (de-)serialization time. A small number of tags (specifically, tag number 25 and tag number 29) have been registered with semantics that - do require processing at (de-)serialization time: The decoder needs - to be aware and the encoder needs to be under control of the exact + may require processing at (de-)serialization time: The decoder needs + to be aware and the encoder needs to be in control of the exact sequence in which data items are encoded into the CBOR data stream. This means these tags cannot be implemented on top of every generic CBOR encoder/decoder (which might not reflect the serialization order for entries in a map at the data model level and vice versa); their implementation therefore typically needs to be integrated into the generic encoder/decoder. The definition of new tags with this property is NOT RECOMMENDED. -3.4.1. Date and Time - Protocols using tag numbers 0 and 1 extend the generic data model - (Section 2) with data items representing points in time. + (Section 2) with data items representing points in time; tag numbers + 2 and 3, with arbitrarily sized integers; and tag numbers 4 and 5, + with floating point values of arbitrary size and precision. -3.4.2. Standard Date/Time String +3.4.1. Standard Date/Time String Tag number 0 contains a text string in the standard format described by the "date-time" production in [RFC3339], as refined by Section 3.3 of [RFC4287], representing the point in time described there. A nested item of another type or that doesn't match the [RFC4287] format is invalid. -3.4.3. Epoch-based Date/Time +3.4.2. Epoch-based Date/Time Tag number 1 contains a numerical value counting the number of seconds from 1970-01-01T00:00Z in UTC time to the represented point in civil time. The enclosed item MUST be an unsigned or negative integer (major types 0 and 1), or a floating-point number (major type 7 with additional information 25, 26, or 27). Other contained types are invalid. @@ -957,21 +958,21 @@ non-finite values. To indicate fractional seconds, floating-point values can be used within tag number 1 instead of integer values. Note that this generally requires binary64 support, as binary16 and binary32 provide non-zero fractions of seconds only for a short period of time around early 1970. An application that requires tag number 1 support may restrict the enclosed value to be an integer (or a floating-point value) only. -3.4.4. Bignums +3.4.3. Bignums Protocols using tag numbers 2 and 3 extend the generic data model (Section 2) with "bignums" representing arbitrarily sized integers. In the generic data model, bignum values are not equal to integers from the basic data model, but specific data models can define that equivalence, and preferred encoding never makes use of bignums that also can be expressed as basic integers (see below). Bignums are encoded as a byte string data item, which is interpreted as an unsigned integer n in network byte order. Contained items of @@ -992,21 +993,21 @@ For example, the number 18446744073709551616 (2**64) is represented as 0b110_00010 (major type 6, tag number 2), followed by 0b010_01001 (major type 2, length 9), followed by 0x010000000000000000 (one byte 0x01 and eight bytes 0x00). In hexadecimal: C2 -- Tag 2 49 -- Byte string of length 9 010000000000000000 -- Bytes content -3.4.5. Decimal Fractions and Bigfloats +3.4.4. Decimal Fractions and Bigfloats Protocols using tag number 4 extend the generic data model with data items representing arbitrary-length decimal fractions of the form m*(10**e). Protocols using tag number 5 extend the generic data model with data items representing arbitrary-length binary fractions of the form m*(2**e). As with bignums, values of different types are not equal in the generic data model. Decimal fractions combine an integer mantissa with a base-10 scaling factor. They are most useful if an application needs the exact @@ -1021,21 +1022,21 @@ applications that need some basic binary floating-point capability without the need for supporting IEEE 754. A decimal fraction or a bigfloat is represented as a tagged array that contains exactly two integer numbers: an exponent e and a mantissa m. Decimal fractions (tag number 4) use base-10 exponents; the value of a decimal fraction data item is m*(10**e). Bigfloats (tag number 5) use base-2 exponents; the value of a bigfloat data item is m*(2**e). The exponent e MUST be represented in an integer of major type 0 or 1, while the mantissa also can be a bignum - (Section 3.4.4). Contained items with other structures are invalid. + (Section 3.4.3). Contained items with other structures are invalid. An example of a decimal fraction is that the number 273.15 could be represented as 0b110_00100 (major type of 6 for the tag, additional information of 4 for the number of tag), followed by 0b100_00010 (major type of 4 for the array, additional information of 2 for the length of the array), followed by 0b001_00001 (major type of 1 for the first integer, additional information of 1 for the value of -2), followed by 0b000_11001 (major type of 0 for the second integer, additional information of 25 for a two-byte value), followed by 0b0110101010110011 (27315 in two bytes). In hexadecimal: @@ -1062,38 +1063,38 @@ Decimal fractions and bigfloats provide no representation of Infinity, -Infinity, or NaN; if these are needed in place of a decimal fraction or bigfloat, the IEEE 754 half-precision representations from Section 3.3 can be used. For constrained applications, where there is a choice between representing a specific number as an integer and as a decimal fraction or bigfloat (such as when the exponent is small and non-negative), there is a quality-of- implementation expectation that the integer representation is used directly. -3.4.6. Content Hints +3.4.5. Content Hints The tags in this section are for content hints that might be used by generic CBOR processors. These content hints do not extend the generic data model. -3.4.6.1. Encoded CBOR Data Item +3.4.5.1. Encoded CBOR Data Item Sometimes it is beneficial to carry an embedded CBOR data item that is not meant to be decoded immediately at the time the enclosing data item is being decoded. Tag number 24 (CBOR data item) can be used to tag the embedded byte string as a data item encoded in CBOR format. Contained items that aren't byte strings are invalid. A contained byte string is valid if it encodes a well-formed CBOR item; validity checking of the decoded CBOR item is not required for tag validity (but could be offered by a generic decoder as a special option). -3.4.6.2. Expected Later Encoding for CBOR-to-JSON Converters +3.4.5.2. Expected Later Encoding for CBOR-to-JSON Converters Tags number 21 to 23 indicate that a byte string might require a specific encoding when interoperating with a text-based representation. These tags are useful when an encoder knows that the byte string data it is writing is likely to be later converted to a particular JSON-based usage. That usage specifies that some strings are encoded as base64, base64url, and so on. The encoder uses byte strings instead of doing the encoding itself to reduce the message size, to reduce the code size of the encoder, or both. The encoder does not know whether or not the converter will be generic, and @@ -1109,21 +1110,21 @@ encodings defined in [RFC4648]. For base64url encoding (tag number 21), padding is not used (see Section 3.2 of RFC 4648); that is, all trailing equals signs ("=") are removed from the encoded string. For base64 encoding (tag number 22), padding is used as defined in RFC 4648. For both base64url and base64, padding bits are set to zero (see Section 3.5 of RFC 4648), and encoding is performed without the inclusion of any line breaks, whitespace, or other additional characters. Note that, for all three tag numbers, the encoding of the empty byte string is the empty text string. -3.4.6.3. Encoded Text +3.4.5.3. Encoded Text Some text strings hold data that have formats widely used on the Internet, and sometimes those formats can be validated and presented to the application in appropriate form by the decoder. There are tags for some of these formats. As with tag numbers 21 to 23, if these tags are applied to an item other than a text string, they apply to all text string data items it contains. o Tag number 32 is for URIs, as defined in [RFC3986]. If the text string doesn't match the "URI-reference" production, the string is @@ -1160,21 +1161,21 @@ be offered. Note that many MIME messages are general binary data and can therefore not be represented in a text string; [IANA.cbor-tags] lists a registration for tag number 257 that is similar to tag number 36 but is used with an enclosed byte string.) Note that tag numbers 33 and 34 differ from 21 and 22 in that the data is transported in base-encoded form for the former and in raw byte string form for the latter. -3.4.7. Self-Described CBOR +3.4.6. Self-Described CBOR In many applications, it will be clear from the context that CBOR is being employed for encoding a data item. For instance, a specific protocol might specify the use of CBOR, or a media type is indicated that specifies its use. However, there may be applications where such context information is not available, such as when CBOR data is stored in a file that does not have disambiguating metadata. Here, it may help to have some distinguishing characteristics for the data itself. @@ -1309,21 +1310,22 @@ not distinguish these and might decide to represent all zero values with a positive sign, disallowing negative zero. CBOR tags present additional considerations for deterministic encoding. If a CBOR-based protocol were to provide the same semantics for the presence and absence of a specific tag (e.g., by allowing both tag 1 data items and raw numbers in a date/time position, treating the latter as if they were tagged), the deterministic format would not allow them. In a protocol that requires tags in certain places to obtain specific semantics, the tag - needs to appear in the deterministic format as well. + needs to appear in the deterministic format as well. Deterministic + encoding considerations also apply to the content of tags. Protocols that include floating, big integer, or other complex values need to define extra requirements on their deterministic encodings. For example: o If a protocol includes a field that can express floating-point values (Section 3.3), the protocol's deterministic encoding needs to specify whether the integer 1.0 is encoded as 0x01, 0xf93c00, 0xfa3f800000, or 0xfb3ff0000000000000. Three sensible rules for this are: @@ -1349,26 +1351,26 @@ not possible, specific attention will be needed for NaN handling. Subnormal numbers (nonzero numbers with the lowest possible exponent of a given IEEE 754 number format) may be flushed to zero outputs or be treated as zero inputs in some floating point implementations. A protocol's deterministic encoding may want to exclude them from interchange, interchanging zero instead. o If a protocol includes a field that can express integers with an absolute value of 2^64 or larger using tag numbers 2 or 3 - (Section 3.4.4), the protocol's deterministic encoding needs to + (Section 3.4.3), the protocol's deterministic encoding needs to specify whether small integers are expressed using the tag or major types 0 and 1. o A protocol might give encoders the choice of representing a URL as - either a text string or, using Section 3.4.6.3, tag number 32 + either a text string or, using Section 3.4.5.3, tag number 32 containing a text string. This protocol's deterministic encoding needs to either require that the tag is present or require that it's absent, not allow either one. 4.2.3. Length-first map key ordering The core deterministic encoding requirements sort map keys in a different order from the one suggested by Section 3.9 of [RFC7049] (called "Canonical CBOR" there). Protocols that need to be compatible with [RFC7049]'s order can instead be specified in terms @@ -1472,27 +1474,28 @@ 5.2. Generic Encoders and Decoders A generic CBOR decoder can decode all well-formed CBOR data and present them to an application. See Appendix C. Even though CBOR attempts to minimize these cases, not all well- formed CBOR data is valid: for example, the encoded text string "0x62c0ae" does not contain valid UTF-8 and so is not a valid CBOR item. Also, specific tags may make semantic constraints that may be violated, such as a bignum tag enclosing another tag, or an instance - of tag number 0 containing a byte string or a text string with - contents that do not match [RFC3339]'s "date-time" production. There - is no requirement that generic encoders and decoders make unnatural - choices for their application interface to enable the processing of - invalid data. Generic encoders and decoders are expected to forward - simple values and tags even if their specific codepoints are not - registered at the time the encoder/decoder is written (Section 5.4). + of tag number 0 containing a byte string, or containing a text string + with contents that do not match [RFC3339]'s "date-time" production. + There is no requirement that generic encoders and decoders make + unnatural choices for their application interface to enable the + processing of invalid data. Generic encoders and decoders are + expected to forward simple values and tags even if their specific + codepoints are not registered at the time the encoder/decoder is + written (Section 5.4). Generic decoders provide ways to present well-formed CBOR values, both valid and invalid, to an application. The diagnostic notation (Section 8) may be used to present well-formed CBOR values to humans. Generic encoders provide an application interface that allows the application to specify any well-formed value, including simple values and tags unknown to the encoder. 5.3. Validity of Items @@ -1713,21 +1716,21 @@ (Byte and text) strings are compared byte by byte, arrays element by element, and are equal if they have the same number of bytes/elements and the same values at the same positions. Two maps are equal if they have the same set of pairs regardless of their order; pairs are equal if both the key and value are equal. Tagged values are equal if both the tag number and the enclosed item are equal. (Note that a generic decoder that provides processing for a specific tag may not be able to distinguish some semantically equivalent values, e.g. if leading zeroes occur in the content of tag - 2/3 (Section 3.4.4).) Simple values are equal if they simply have + 2/3 (Section 3.4.3).) Simple values are equal if they simply have the same value. Nothing else is equal in the generic data model, a simple value 2 is not equivalent to an integer 2 and an array is never equivalent to a map. As discussed in Section 2.2, specific data models can make values equivalent for the purpose of comparing map keys that are distinct in the generic data model. Note that this implies that a generic decoder may deliver a decoded map to an application that needs to be checked for duplicate map keys by that application (alternatively, the decoder may provide a programming interface to perform this @@ -1824,27 +1827,28 @@ made with respect to number representation. In a suggested conversion: o JSON numbers without fractional parts (integer numbers) are represented as integers (major types 0 and 1, possibly major type 6 tag number 2 and 3), choosing the shortest form; integers longer than an implementation-defined threshold may instead be represented as floating-point values. The default range that is represented as integer is -2**53+1..2**53-1 (fully exploiting the range for exact integers in the binary64 representation often used - for decoding JSON [RFC7493]), implementations may choose - -2**32..2**32-1 or -2**64..2**64-1 (fully using the integer ranges - available in CBOR with uint32_t or uint64_t, respectively) or even - -2**31..2**31-1 or -2**63..2**63-1 (using popular ranges for two's - complement signed integers). (If the JSON was generated from a - JavaScript implementation, its precision is already limited to 53 - bits maximum.) + for decoding JSON [RFC7493]). A CBOR-based protocol, or a generic + converter implementation, may choose -2**32..2**32-1 or + -2**64..2**64-1 (fully using the integer ranges available in CBOR + with uint32_t or uint64_t, respectively) or even -2**31..2**31-1 + or -2**63..2**63-1 (using popular ranges for two's complement + signed integers). (If the JSON was generated from a JavaScript + implementation, its precision is already limited to 53 bits + maximum.) o Numbers with fractional parts are represented as floating-point values, performing the decimal-to-binary conversion based on the precision provided by IEEE 754 binary64. Then, when encoding in CBOR, the preferred serialization uses the shortest floating-point representation exactly representing this conversion result; for instance, 1.5 is represented in a 16-bit floating-point value (not all implementations will be capable of efficiently finding the minimum form, though). Instead of using the default binary64 precision, there may be an implementation-defined limit to the @@ -2022,20 +2026,23 @@ notated in the form (_ h'0123', h'4567') and (_ "foo", "bar"). 9. IANA Considerations IANA has created two registries for new CBOR values. The registries are separate, that is, not under an umbrella registry, and follow the rules in [RFC8126]. IANA has also assigned a new MIME media type and an associated Constrained Application Protocol (CoAP) Content-Format entry. + [To be removed by RFC editor:] IANA is requested to update these + registries to point to the present document instead of RFC 7049. + 9.1. Simple Values Registry IANA has created the "Concise Binary Object Representation (CBOR) Simple Values" registry at [IANA.cbor-simple-values]. The initial values are shown in Table 3. New entries in the range 0 to 19 are assigned by Standards Action. It is suggested that these Standards Actions allocate values starting with the number 16 in order to reserve the lower numbers for contiguous blocks (if any). @@ -2576,21 +2584,21 @@ Appendix B. Jump Table For brevity, this jump table does not show initial bytes that are reserved for future extension. It also only shows a selection of the initial bytes that can be used for optional features. (All unsigned integers are in network byte order.) +------------+------------------------------------------------------+ | Byte | Structure/Semantics | +------------+------------------------------------------------------+ - | 0x00..0x17 | Integer 0x00..0x17 (0..23) | + | 0x00..0x17 | Unsigned integer 0x00..0x17 (0..23) | | | | | 0x18 | Unsigned integer (one-byte uint8_t follows) | | | | | 0x19 | Unsigned integer (two-byte uint16_t follows) | | | | | 0x1a | Unsigned integer (four-byte uint32_t follows) | | | | | 0x1b | Unsigned integer (eight-byte uint64_t follows) | | | | | 0x20..0x37 | Negative integer -1-0x00..-1-0x17 (-1..-24) | @@ -2668,39 +2676,39 @@ | 0xba | map (four-byte uint32_t for n, and then n pairs of | | | data items follow) | | | | | 0xbb | map (eight-byte uint64_t for n, and then n pairs of | | | data items follow) | | | | | 0xbf | map, pairs of data items follow, terminated by | | | "break" | | | | | 0xc0 | Text-based date/time (data item follows; see | - | | Section 3.4.2) | + | | Section 3.4.1) | | | | | 0xc1 | Epoch-based date/time (data item follows; see | - | | Section 3.4.3) | + | | Section 3.4.2) | | | | | 0xc2 | Positive bignum (data item "byte string" follows) | | | | | 0xc3 | Negative bignum (data item "byte string" follows) | | | | | 0xc4 | Decimal Fraction (data item "array" follows; see | - | | Section 3.4.5) | + | | Section 3.4.4) | | | | | 0xc5 | Bigfloat (data item "array" follows; see | - | | Section 3.4.5) | + | | Section 3.4.4) | | | | | 0xc6..0xd4 | (tag) | | | | | 0xd5..0xd7 | Expected Conversion (data item follows; see | - | | Section 3.4.6.2) | + | | Section 3.4.5.2) | | | | | 0xd8..0xdb | (more tags, 1/2/4/8 bytes and then a data item | | | follow) | | | | | 0xe0..0xf3 | (simple value) | | | | | 0xf4 | False | | | | | 0xf5 | True | | | | @@ -3006,21 +3014,21 @@ o Fixed a bug in the last paragraph of Section 3.6 ("0b000_11101" -> "0b000_11001") Appendix G. Well-formedness errors and examples There are three basic kinds of well-formedness errors that can occur in decoding a CBOR data item: o Too much data: There are input bytes left that were not consumed. This is only an error if the application assumed that the input - bytes would span exexactly one data item. Where the application + bytes would span exactly one data item. Where the application uses the self-delimiting nature of CBOR encoding to permit additional data after the data item, as is for example done in CBOR sequences [I-D.ietf-cbor-sequence], the CBOR decoder can simply indicate what part of the input has not been consumed. o Too little data: The input data available would need additional bytes added at their end for a complete CBOR data item. This may indicate the input is truncated; it is also a common error when trying to decode random data as CBOR. For some applications however, this may not be actually be an error, as the application @@ -3147,21 +3155,21 @@ MessagePack that was developed by Eric Zhang for the binaryjs project. A similar, but different, extension was made by Tim Caswell for his msgpack-js and msgpack-js-browser projects. Many people have contributed to the discussion about extending MessagePack to separate text string representation from byte string representation. The encoding of the additional information in CBOR was inspired by the encoding of length information designed by Klaus Hartke for CoAP. This document also incorporates suggestions made by many people, - notably Dan Frost, James Manger, Jeffrey Yaskin, Joe Hildebrand, + notably Dan Frost, James Manger, Jeffrey Yasskin, Joe Hildebrand, Keith Moore, Laurence Lundblade, Matthew Lepinski, Michael Richardson, Nico Williams, Peter Occil, Phillip Hallam-Baker, Ray Polk, Tim Bray, Tony Finch, Tony Hansen, and Yaron Sheffer. Authors' Addresses Carsten Bormann Universitaet Bremen TZI Postfach 330440 D-28359 Bremen