Status of This Memo

This document is not an Internet Standards Track specification; it is published for informational purposes.¶

This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Not all documents approved by the IESG are candidates for any level of Internet Standard; see Section 2 of RFC 7841.¶

Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at https://www.rfc-editor.org/info/rfc9841.¶

Copyright Notice

Copyright (c) 2025 IETF Trust and the persons identified as the document authors. All rights reserved.¶

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶

Table of Contents

1. Introduction

   +--------+
   |76543210|
   +--------+

   0        1
   +--------+--------+
   |00001000|00000010|
   +--------+--------+
   ^        ^
   |        |
   |        + more significant byte = 2 x 256
   + less significant byte = 8

     byte 2   byte 1   byte 0
   +--------+--------+--------+
   |11100010|11000101|10010110|
   +--------+--------+--------+
    ^            ^ ^   ^   ^
    |            | |   |   |
    |            | |   |   +------ integer value 6
    |            | |   +---------- integer value 2
    |            | +-------------- prefix code 110
    |            +---------------- prefix code 10
    +----------------------------- integer value 3628

2. Shared Brotli Overview

Shared brotli extends brotli [RFC7932] with support for shared dictionaries, a larger LZ77 window, and a framing format.¶

3. Shared Dictionaries

A shared dictionary is a piece of data shared by a compressor and decompressor. The compressor can take advantage of the dictionary context to encode the input in a more compact manner. The compressor and the decompressor must use exactly the same dictionary. A shared dictionary is specially useful to compress short input sequences.¶

A shared brotli dictionary can use two methods of sharing context:¶

LZ77 dictionary:

The encoder and decoder could refer to a given sequence of bytes. Multiple LZ77 dictionaries can be set.¶

Custom static dictionary:

A word list with transforms. The encoder and decoder will replace the static dictionary data with the data in the shared dictionary. The original static dictionary is described in Section 8 in [RFC7932]. The original data from Appendices A and B of [RFC7932] will be replaced. In addition, it is possible to dynamically switch this dictionary based on the data compression context and/or include a reference to the original dictionary in the custom dictionary.¶

If no shared dictionary is set, the decoder behaves the same as in [RFC7932] on a brotli stream.¶

If a shared dictionary is set, then it can set LZ77 dictionaries, override static dictionary words, and/or override transforms.¶

   7-bit SCALAR:
   +--------+
   |0sssssss|
   +--------+

   11-bit SCALAR:
   +--------+--------+
   |110sssss|XXssssss|
   +--------+--------+

   16-bit SCALAR:
   +--------+--------+--------+
   |1110ssss|XXssssss|XXssssss|
   +--------+--------+--------+

   21-bit SCALAR:
   +--------+--------+--------+--------+
   |11110sss|XXssssss|XXssssss|XXssssss|
   +--------+--------+--------+--------+

word_id = distance - (max allowed distance + 1 +
LZ77_DICTIONARY_LENGTH)

4. Varint Encoding

A varint is encoded in base 128 in one or more bytes as follows:¶

   +--------+--------+             +--------+
   |1xxxxxxx|1xxxxxxx| {0-8 times} |0xxxxxxx|
   +--------+--------+             +--------+

where the "x" bits of the first byte are the LSBs of the value and the "x" bits of the last byte are the MSBs of the value. The last byte must have its MSB set to 0, all other bytes to 1 to indicate there is a next byte.¶

The maximum allowed amount of bits to read is 63 bits; if the 9th byte is present and has its MSB set, then the stream must be considered as invalid.¶

5. Shared Dictionary Stream

The shared dictionary stream encodes a custom dictionary for brotli, including custom words and/or custom transformations. A shared dictionary may appear as a standalone or as contents of a resource in a framing format container.¶

A compliant shared brotli dictionary stream must have the following format:¶

2 bytes:

File signature, in hexadecimal the bytes 91, 0.¶

varint:

LZ77_DICTIONARY_LENGTH. The number of bytes for an LZ7711 dictionary or 0 if there is none. The maximum allowed value is the maximum possible sliding window size of brotli or large window brotli.¶

LZ77_DICTIONARY_LENGTH bytes:

Contents of the LZ77 dictionary.¶

1 byte:

NUM_CUSTOM_WORD_LISTS. May have a value of 0 to 64.¶

NUM_CUSTOM_WORD_LISTS times a word list with the following format for each word list:

28 bytes:

SIZE_BITS_BY_LENGTH. An array of 28 unsigned 8-bit integers, indexed by word lengths 4 to 31. The value represents log2(number of words of this length), with the exception of 0 meaning 0 words of this length. The max allowed length value is 15 bits. OFFSETS_BY_LENGTH is computed from this as OFFSETS_BY_LENGTH[i + 1] = OFFSETS_BY_LENGTH[i] + (SIZE_BITS_BY_LENGTH[i] ? (i << SIZE_BITS_BY_LENGTH[i]) : 0).¶

N bytes:

Words dictionary data, where N is OFFSETS_BY_LENGTH[31] + (SIZE_BITS_BY_LENGTH[31] ? (31 << SIZE_BITS_BY_LENGTH[31]) : 0), with all the words of shortest length first, then all words of the next length, and so on, where there are either 0 or a positive power of two number of words for each length.¶

1 byte:

NUM_CUSTOM_TRANSFORM_LISTS. May have a value of 0 to 64.¶

NUM_CUSTOM_TRANSFORM_LISTS times a transform list with the following format for each transform list:

2 bytes:

PREFIX_SUFFIX_LENGTH. The length of prefix/suffix data. Must be at least 1 because the list must always end with a zero-length stringlet even if it is empty.¶

NUM_PREFIX_SUFFIX times:

Prefix/suffix stringlet. NUM_PREFIX_SUFFIX is the number of stringlets parsed and must be in range 1..256.¶

1 byte:

STRING_LENGTH. The length of the entry contents. 0 for the last (terminating) entry of the transform list. For other entries, STRING_LENGTH must be in range 1..255. The 0 entry must be present and must be the last byte of the PREFIX_SUFFIX_LENGTH bytes of prefix/suffix data, else the stream must be rejected as invalid.¶

STRING_LENGTH bytes:

Contents of the prefix/suffix.¶

1 byte:

NTRANSFORMS. Number of transformation triplets.¶

NTRANSFORMS times:

Data for each transform:¶

1 byte:

Index of prefix in prefix/suffix data; must be less than NUM_PREFIX_SUFFIX.¶

1 byte:

Index of suffix in prefix/suffix data; must be less than NUM_PREFIX_SUFFIX.¶

1 byte:

Operation index; must be an index in the table of operations listed in Section 3.1.1.¶

If and only if at least one transform has operation index ShiftFirst or ShiftAll:

NTRANSFORMS times:

2 bytes:

Parameters for the transform. If the transform does not have type ShiftFirst or ShiftAll, the value must be 0. ShiftFirst and ShiftAll interpret these bytes as an unsigned 16-bit integer.¶

If NUM_CUSTOM_WORD_LISTS > 0 or NUM_CUSTOM_TRANSFORM_LISTS > 0 (else implicitly NUM_DICTIONARIES is 1 and points to the brotli built-in and there is no context map):

1 byte:

NUM_DICTIONARIES. May have value 1 to 64. Each dictionary is a combination of a word list and a transform list. Each next dictionary is used when the distance goes beyond the previous. If a CONTEXT_MAP is enabled, then the dictionary matching the context is moved to the front in the order for this context.¶

NUM_DICTIONARIES times:

The DICTIONARY_MAP:¶

1 byte:

Index into a custom word list or value NUM_CUSTOM_WORD_LISTS to indicate using the brotli [RFC7932] built-in default word list.¶

1 byte:

Index into a custom transform list or value NUM_CUSTOM_TRANSFORM_LISTS to indicate using the brotli [RFC7932] built-in default transform list.¶

1 byte:

CONTEXT_ENABLED. If 0, there is no context map. If 1, a context map used to select the dictionary is encoded as below.¶

If CONTEXT_ENABLED is 1, there is a context map for the 64 brotli [RFC7932] literals contexts:

64 bytes:

CONTEXT_MAP. Index into the DICTIONARY_MAP for the first dictionary to use for this context.¶

6. Large Window Brotli Compressed Data Stream

Large window brotli allows a sliding window beyond the 24-bit maximum of regular brotli [RFC7932].¶

The compressed data stream is backwards compatible to brotli [RFC7932] and may optionally have the following differences:¶

Encoding of WBITS in the stream header:

The following new pattern of 14 bits is supported:¶

8 bits:

Value 00010001 to indicate a large window brotli stream.¶

6 bits:

WBITS. Must have value in range 10 to 62.¶

Distance alphabet:

If the stream is a large window brotli stream, the maximum number of extra bits is 62 and the theoretical maximum size of the distance alphabet is (16 + NDIRECT + (124 << NPOSTFIX)). This overrides the value for the distance alphabet size given in Section 3.3 of [RFC7932] and affects the number of bits in the encoding of the Simple Prefix Code for distances as described in Section 3.4 of [RFC7932]. An additional limitation to distances, despite the large allowed alphabet size, is that the alphabet is not allowed to contain a distance symbol able to represent a distance larger than ((1 << 63) - 4) when its extra bits have their maximum value. It depends on NPOSTFIX and NDIRECT when this can occur.¶

A decoder that does not support 64-bit integers may reject a stream if WBITS is higher than 30 or a distance symbol from the distance alphabet is able to encode a distance larger than 2147483644.¶

7. Shared Brotli Compressed Data Stream

The format of a shared brotli compressed data stream without a framing format is backwards compatible with brotli [RFC7932] with the following optional differences:¶

• LZ77 dictionaries as described above are supported.¶
• Custom static dictionaries replacing or extending the static dictionary of brotli [RFC7932] with different words or transforms are supported.¶
• The stream may have the format of regular brotli [RFC7932] or the format of large window brotli as described in Section 6.¶

8. Shared Brotli Framing Format Stream

A compliant shared brotli framing format stream has the format described below.¶

9. Security Considerations

The security considerations for brotli [RFC7932] apply to shared brotli as well.¶

In addition, the same considerations apply to the decoding of new file format streams for shared brotli, including shared dictionaries, the framing format, and the shared brotli format.¶

The dictionary must be treated with the same security precautions as the content because a change to the dictionary can result in a change to the decompressed content.¶

The CRIME attack [CRIME] shows that it's a bad idea to compress data from mixed (e.g., public and private) sources -- the data sources include not only the compressed data but also the dictionaries. For example, if you compress secret cookies using a public-data-only dictionary, you still leak information about the cookies.¶

Not only can the dictionary reveal information about the compressed data, but vice versa; data compressed with the dictionary can reveal the contents of the dictionary when an adversary can control parts of data to compress and see the compressed size. On the other hand, if the adversary can control the dictionary, the adversary can learn information about the compressed data.¶

The most robust defense against CRIME is not to compress private data, e.g., sensitive headers like cookies or any content with personally identifiable information (PII). The challenge has been to identify secrets within a vast amount of data to be compressed. Cloudflare uses a regular expression [CLOUDFLARE]. Another idea is to extend existing web template systems (e.g., Soy [SOY]) to allow developers to mark secrets that must not be compressed.¶

A less robust idea, but easier to implement, is to randomize the compression algorithm, i.e., adding randomly generated padding, varying the compression ratio, etc. The tricky part is to find the right balance between cost and security (i.e., on one hand, we don't want to add too much padding because it adds a cost to data, but on the other hand, we don't want to add too little because the adversary can detect a small amount of padding with traffic analysis).¶

Additionally, another defense is to not use dictionaries for cross- domain requests and to only use shared brotli for the response when the origin is the same as where the content is hosted (using CORS). This prevents an adversary from using a private dictionary with user secrets to compress content hosted on the adversary's origin. It also helps prevent CRIME attacks that try to benefit from a public dictionary by preventing data compression with dictionaries for requests that do not originate from the host itself.¶

The content of the dictionary itself should not be affected by external users; allowing adversaries to control the dictionary allows a form of chosen plaintext attack. Instead, only base the dictionary on content you control or generic large scale content such as a spoken language and update the dictionary with large time intervals (days, not seconds) to prevent fast probing.¶

The use of HighwayHash [HWYHASH] for dictionary identifiers does not guarantee against collisions in an adversarial environment and is intended to be used for identifying the dictionary within a trusted, known set of dictionaries. In an adversarial environment, users of shared brotli should use another mechanism to validate a negotiated dictionary such as a cryptographically proven secure hash.¶

10. IANA Considerations

This document has no IANA actions.¶

Acknowledgments

The authors would like to thank Robert Obryk for suggesting improvements to the format and the text of the specification.¶

Authors' Addresses