RFC 3986
Functions like parse_uri are sufficient for converting URLs but
they require that the entire string is consumed. When URLs appear as
components of a larger grammar, it is desired to use composition
of rules based parsing to process these along with other elements
potentially unrelated to resource locators. To achieve this, the
library provides rules for the top-level BNF productions found
in rfc3986 and a rule for matching percent-encoded strings.
Percent Encoding
The percent-encoding mechanism is used to represent a data octet in a component when the corresponding character is outside the allowed set or is being used as a delimiter of, or within, the component. An encoded octet (also called an escape) is encoded as the percent character ('%') followed by a two-digit hexadecimal number representing the octet’s numeric value:
pct-encoded = "%" HEXDIG HEXDIG
URL components with possible percent-encoded characters are specified in the components BNF using the pct-encoded production, along with the characters which are not considered to be part of the reserved set. For example, this is how path characters are described in 3.3 Path of the RFC:
pchar = unreserved / pct-encoded / sub-delims / ":" / "@"
The library provides the pct_encoded_rule for matching
strings which are percent-encoded. This function is passed the
set of characters that may be used without escapes and returns
a suitable rule. If the input is valid; that is, if there are
no invalid escape sequences, the rule returns a
decode_view. This is a forward range of characters
which performs percent-decoding when iterated. It also supports
equality and comparison to unencoded strings, without allocating
memory. In the example below we parse the string s as a
series of zero or more pchars:
// code_grammar_5_1
These constants are used and provided by the library to specify rules for percent-encoded URL components:
URL Rules
When a URL can appear in the context of a larger grammar, it may be desired to express the enclosing grammar in a single rule that incoporates the URL as an element. To achieve this, the library makes public the rules used to implement high-level parsing of complete strings as URL components, so that these components may be parsed as part of a larger string containing non-URL elements. Here we present a rule suitable for parsing the the HTTP request-line:
// code_grammar_5_2
The library offers these rules to allow custom rule definitions to integrate the various styles of valid URL rules: