James Coglan 29496f6838 Remove ReDoS vulnerability in the Sec-WebSocket-Extensions header parser
There is a regular expression denial of service (ReDoS) vulnerability in
the parser we use to process the `Sec-WebSocket-Extensions` header. It
can be exploited by sending an opening WebSocket handshake to a server
containing a header of the form:

    Sec-WebSocket-Extensions: a;b="\c\c\c\c\c\c\c\c\c\c ...

i.e. a header containing an unclosed string parameter value whose
content is a repeating two-byte sequence of a backslash and some other
character. The parser takes exponential time to reject this header as
invalid, and this can be used to exhaust the server's capacity to
process requests.

This vulnerability has been assigned the identifier CVE-2020-7662 and
was reported by Robert McLaughlin.

We believe this flaw stems from the grammar specified for this header.
[RFC 6455][1] defines the grammar for the header as:

    Sec-WebSocket-Extensions = extension-list

    extension-list    = 1#extension
    extension         = extension-token *( ";" extension-param )
    extension-token   = registered-token
    registered-token  = token
    extension-param   = token [ "=" (token | quoted-string) ]

It refers to [RFC 2616][2] for the definitions of `token` and
`quoted-string`, which are:

    token          = 1*<any CHAR except CTLs or separators>
    separators     = "(" | ")" | "<" | ">" | "@"
                   | "," | ";" | ":" | "\" | <">
                   | "/" | "[" | "]" | "?" | "="
                   | "{" | "}" | SP | HT

    quoted-string  = ( <"> *(qdtext | quoted-pair ) <"> )
    qdtext         = <any TEXT except <">>
    quoted-pair    = "\" CHAR

These rely on the `CHAR`, `CTL` and `TEXT` grammars, which are:

    CHAR           = <any US-ASCII character (octets 0 - 127)>
    CTL            = <any US-ASCII control character (octets 0 - 31) and DEL (127)>
    TEXT           = <any OCTET except CTLs, but including LWS>

Other relevant definitions to support these:

    OCTET          = <any 8-bit sequence of data>
    LWS            = [CRLF] 1*( SP | HT )
    CRLF           = CR LF

    HT             = <US-ASCII HT, horizontal-tab (9)>
    LF             = <US-ASCII LF, linefeed (10)>
    CR             = <US-ASCII CR, carriage return (13)>
    SP             = <US-ASCII SP, space (32)>

To expand some of these terms out and write them as regular expressions:

    OCTET         = [\x00-\xFF]
    CHAR          = [\x00-\x7F]
    TEXT          = [\t \x21-\x7E\x80-\xFF]

The allowable bytes for `token` are [\x00-\x7F], except [\x00-\x1F\x7F]
(leaving [\x20-\x7E]) and `separators`, which leaves the following set
of allowed chars:

    ! # $ % & ' * + - . ^ _ ` | ~ [0-9] [A-Z] [a-z]

`quoted-string` contains a repeated pattern of either `qdtext` or
`quoted-pair`. `qdtext` is any `TEXT` byte except <">, and the <">
character is ASCII 34, or 0x22. The <!> character is 0x21. So `qdtext`
can be written either positively as:

    qdtext        = [\t !\x23-\x7E\x80-\xFF]

or negatively, as:

    qdtext        = [^\x00-\x08\x0A-\x1F\x7F"]

We use the negative definition here. The other alternative in the
`quoted-string` pattern is:

    quoted-pair   = \\[\x00-\x7F]

The problem is that the set of bytes matched by `qdtext` includes <\>,
and intersects with the second element of `quoted-pair`. That means the
sequence \c can be matched as either two `qdtext` bytes, or as a single
`quoted-pair`. When the regex engine fails to find a trailing <"> to
close the string, it back-tracks and tries every alternate parse for the
string, which doubles with each pair of bytes in the input.

To fix the ReDoS flaw we need to rewrite the repeating pattern so that
none of its alternate branches can match the same text. For example, we
could try dividing the set of bytes [\x00-\xFF] into those that must not
follow a <\>, those that may follow a <\>, and those that must be
preceded by <\>, and thereby construct a pattern of the form:

    (A|\?B|\C)*

where A, B and C have no characters in common. In our case the three
branch patterns would be:

    A   =   qdtext - CHAR   =   [\x80-\xFF]
    B   =   qdtext & CHAR   =   [\t !\x23-\x7E]
    C   =   CHAR - qdtext   =   [\x00-\x08\x0A-\x1F\x7F"]

These sets do not intersect, and notice <"> appears in set C so must be
preceded by <\>. But we still have a problem: <\> (0x5C) and all the
alphabetic characters are in set B, so the pattern \?B can match all
these:

    c
    \
    \c

So the sequence \c\c\c... still produces exponential back-tracking. It
also fails to parse input like this correctly:

    Sec-WebSocket-Extensions: a; b="c\", d"

Because the grammar allows a single backslash to appear by itself, this
is arguably a syntax error where the parameter `b` has value `c\` and
then a new extension `d` begins with a <"> appearing where it should
not.

So the core problem is with the grammar itself: `qdtext` matches a
single backslash <\>, and `quoted-pair` matches a pair <\\>. So given a
sequence of backslashes there's no canonical parse and the grammar is
ambiguous.

[RFC 7230][3] remedies this problem and makes the grammar clearer.
First, it defines `token` explicitly rather than implicitly:

    token          = 1*tchar

    tchar          = "!" / "#" / "$" / "%" / "&" / "'" / "*"
                   / "+" / "-" / "." / "^" / "_" / "`" / "|" / "~"
                   / DIGIT / ALPHA

And second, it defines `quoted-string` so that backslashes cannot appear
on their own:

     quoted-string  = DQUOTE *( qdtext / quoted-pair ) DQUOTE
     qdtext         = HTAB / SP /%x21 / %x23-5B / %x5D-7E / obs-text
     obs-text       = %x80-FF
     quoted-pair    = "\" ( HTAB / SP / VCHAR / obs-text )

where VCHAR is any printing ASCII character 0x21-0x7E. Notice `qdtext`
is just our previous definition but with 5C excluded, so it cannot
accept a single backslash.

This commit makes this modification to our matching patterns, and
thereby removes the ReDoS vector. Technically this means it does not
match the grammar of RFC 6455, but we expect this to have little or no
practical impact, especially since the one main protocol extension,
`permessage-deflate` ([RFC 7692][4]), does not have any string-valued
parameters.

[1]: https://tools.ietf.org/html/rfc6455#section-9.1
[2]: https://tools.ietf.org/html/rfc2616#section-2.2
[3]: https://tools.ietf.org/html/rfc7230#section-3.2.6
[4]: https://tools.ietf.org/html/rfc7692
2020-06-02 13:26:04 +01:00
2019-06-10 12:25:53 +01:00
2015-11-08 12:16:15 +00:00
2019-05-24 13:59:25 +01:00
2019-06-10 12:25:53 +01:00

websocket-extensions Build status

A minimal framework that supports the implementation of WebSocket extensions in a way that's decoupled from the main protocol. This library aims to allow a WebSocket extension to be written and used with any protocol library, by defining abstract representations of frames and messages that allow modules to co-operate.

websocket-extensions provides a container for registering extension plugins, and provides all the functions required to negotiate which extensions to use during a session via the Sec-WebSocket-Extensions header. By implementing the APIs defined in this document, an extension may be used by any WebSocket library based on this framework.

Installation

$ npm install websocket-extensions

Usage

There are two main audiences for this library: authors implementing the WebSocket protocol, and authors implementing extensions. End users of a WebSocket library or an extension should be able to use any extension by passing it as an argument to their chosen protocol library, without needing to know how either of them work, or how the websocket-extensions framework operates.

The library is designed with the aim that any protocol implementation and any extension can be used together, so long as they support the same abstract representation of frames and messages.

Data types

The APIs provided by the framework rely on two data types; extensions will expect to be given data and to be able to return data in these formats:

Frame

Frame is a structure representing a single WebSocket frame of any type. Frames are simple objects that must have at least the following properties, which represent the data encoded in the frame:

property description
final true if the FIN bit is set, false otherwise
rsv1 true if the RSV1 bit is set, false otherwise
rsv2 true if the RSV2 bit is set, false otherwise
rsv3 true if the RSV3 bit is set, false otherwise
opcode the numeric opcode (0, 1, 2, 8, 9, or 10) of the frame
masked true if the MASK bit is set, false otherwise
maskingKey a 4-byte Buffer if masked is true, otherwise null
payload a Buffer containing the (unmasked) application data

Message

A Message represents a complete application message, which can be formed from text, binary and continuation frames. It has the following properties:

property description
rsv1 true if the first frame of the message has the RSV1 bit set
rsv2 true if the first frame of the message has the RSV2 bit set
rsv3 true if the first frame of the message has the RSV3 bit set
opcode the numeric opcode (1 or 2) of the first frame of the message
data the concatenation of all the frame payloads in the message

For driver authors

A driver author is someone implementing the WebSocket protocol proper, and who wishes end users to be able to use WebSocket extensions with their library.

At the start of a WebSocket session, on both the client and the server side, they should begin by creating an extension container and adding whichever extensions they want to use.

var Extensions = require('websocket-extensions'),
    deflate    = require('permessage-deflate');

var exts = new Extensions();
exts.add(deflate);

In the following examples, exts refers to this Extensions instance.

Client sessions

Clients will use the methods generateOffer() and activate(header).

As part of the handshake process, the client must send a Sec-WebSocket-Extensions header to advertise that it supports the registered extensions. This header should be generated using:

request.headers['sec-websocket-extensions'] = exts.generateOffer();

This returns a string, for example "permessage-deflate; client_max_window_bits", that represents all the extensions the client is offering to use, and their parameters. This string may contain multiple offers for the same extension.

When the client receives the handshake response from the server, it should pass the incoming Sec-WebSocket-Extensions header in to exts to activate the extensions the server has accepted:

exts.activate(response.headers['sec-websocket-extensions']);

If the server has sent any extension responses that the client does not recognize, or are in conflict with one another for use of RSV bits, or that use invalid parameters for the named extensions, then exts.activate() will throw. In this event, the client driver should fail the connection with closing code 1010.

Server sessions

Servers will use the method generateResponse(header).

A server session needs to generate a Sec-WebSocket-Extensions header to send in its handshake response:

var clientOffer = request.headers['sec-websocket-extensions'],
    extResponse = exts.generateResponse(clientOffer);

response.headers['sec-websocket-extensions'] = extResponse;

Calling exts.generateResponse(header) activates those extensions the client has asked to use, if they are registered, asks each extension for a set of response parameters, and returns a string containing the response parameters for all accepted extensions.

In both directions

Both clients and servers will use the methods validFrameRsv(frame), processIncomingMessage(message) and processOutgoingMessage(message).

The WebSocket protocol requires that frames do not have any of the RSV bits set unless there is an extension in use that allows otherwise. When processing an incoming frame, sessions should pass a Frame object to:

exts.validFrameRsv(frame)

If this method returns false, the session should fail the WebSocket connection with closing code 1002.

To pass incoming messages through the extension stack, a session should construct a Message object according to the above datatype definitions, and call:

exts.processIncomingMessage(message, function(error, msg) {
  // hand the message off to the application
});

If any extensions fail to process the message, then the callback will yield an error and the session should fail the WebSocket connection with closing code 1010. If error is null, then msg should be passed on to the application.

To pass outgoing messages through the extension stack, a session should construct a Message as before, and call:

exts.processOutgoingMessage(message, function(error, msg) {
  // write message to the transport
});

If any extensions fail to process the message, then the callback will yield an error and the session should fail the WebSocket connection with closing code 1010. If error is null, then message should be converted into frames (with the message's rsv1, rsv2, rsv3 and opcode set on the first frame) and written to the transport.

At the end of the WebSocket session (either when the protocol is explicitly ended or the transport connection disconnects), the driver should call:

exts.close(function() {})

The callback is invoked when all extensions have finished processing any messages in the pipeline and it's safe to close the socket.

For extension authors

An extension author is someone implementing an extension that transforms WebSocket messages passing between the client and server. They would like to implement their extension once and have it work with any protocol library.

Extension authors will not install websocket-extensions or call it directly. Instead, they should implement the following API to allow their extension to plug into the websocket-extensions framework.

An Extension is any object that has the following properties:

property description
name a string containing the name of the extension as used in negotiation headers
type a string, must be "permessage"
rsv1 either true if the extension uses the RSV1 bit, false otherwise
rsv2 either true if the extension uses the RSV2 bit, false otherwise
rsv3 either true if the extension uses the RSV3 bit, false otherwise

It must also implement the following methods:

ext.createClientSession()

This returns a ClientSession, whose interface is defined below.

ext.createServerSession(offers)

This takes an array of offer params and returns a ServerSession, whose interface is defined below. For example, if the client handshake contains the offer header:

Sec-WebSocket-Extensions: permessage-deflate; server_no_context_takeover; server_max_window_bits=8, \
                          permessage-deflate; server_max_window_bits=15

then the permessage-deflate extension will receive the call:

ext.createServerSession([
  { server_no_context_takeover: true, server_max_window_bits: 8 },
  { server_max_window_bits: 15 }
]);

The extension must decide which set of parameters it wants to accept, if any, and return a ServerSession if it wants to accept the parameters and null otherwise.

ClientSession

A ClientSession is the type returned by ext.createClientSession(). It must implement the following methods, as well as the Session API listed below.

clientSession.generateOffer()
// e.g.  -> [
//            { server_no_context_takeover: true, server_max_window_bits: 8 },
//            { server_max_window_bits: 15 }
//          ]

This must return a set of parameters to include in the client's Sec-WebSocket-Extensions offer header. If the session wants to offer multiple configurations, it can return an array of sets of parameters as shown above.

clientSession.activate(params) // -> true

This must take a single set of parameters from the server's handshake response and use them to configure the client session. If the client accepts the given parameters, then this method must return true. If it returns any other value, the framework will interpret this as the client rejecting the response, and will throw.

ServerSession

A ServerSession is the type returned by ext.createServerSession(offers). It must implement the following methods, as well as the Session API listed below.

serverSession.generateResponse()
// e.g.  -> { server_max_window_bits: 8 }

This returns the set of parameters the server session wants to send in its Sec-WebSocket-Extensions response header. Only one set of parameters is returned to the client per extension. Server sessions that would confict on their use of RSV bits are not activated.

Session

The Session API must be implemented by both client and server sessions. It contains two methods, processIncomingMessage(message) and processOutgoingMessage(message).

session.processIncomingMessage(message, function(error, msg) { ... })

The session must implement this method to take an incoming Message as defined above, transform it in any way it needs, then return it via the callback. If there is an error processing the message, this method should yield an error as the first argument.

session.processOutgoingMessage(message, function(error, msg) { ... })

The session must implement this method to take an outgoing Message as defined above, transform it in any way it needs, then return it via the callback. If there is an error processing the message, this method should yield an error as the first argument.

Note that both processIncomingMessage() and processOutgoingMessage() can perform their logic asynchronously, are allowed to process multiple messages concurrently, and are not required to complete working on messages in the same order the messages arrive. websocket-extensions will reorder messages as your extension emits them and will make sure every extension is given messages in the order they arrive from the driver. This allows extensions to maintain state that depends on the messages' wire order, for example keeping a DEFLATE compression context between messages.

session.close()

The framework will call this method when the WebSocket session ends, allowing the session to release any resources it's using.

Examples

S
Description
Generic extension management for WebSocket connections
Readme 113 KiB
Languages
JavaScript 100%