URL Encoding Explained: RFC 3986, Percent-Encoding, and Web Security

Why Do URLs Need Encoding in the First Place?

The Uniform Resource Identifier (URI) specification was standardized when the Internet moved across heterogeneous networks using strictly 7-bit US-ASCII channels. Under this design, a URL cannot contain arbitrary binary data, control characters (like line breaks), or non-ASCII glyphs such as Japanese kanji, Arabic script, or accented Latin letters.

More fundamentally, URLs have a strict syntax. Characters like /, ?, #, &, and = serve as architectural delimiters. They tell the browser, proxy, and web server where the protocol ends, where the host begins, where the path segments divide, where query parameters start, and where fragment identifiers anchor.

URL encoding (formally termed percent-encoding) solves this identity problem: it converts any character into a standardized, unambiguous ASCII representation that parsers can transmit without mistaking payload data for structural syntax.

The Anatomy of RFC 3986: Reserved vs. Unreserved Characters

The governing standard for modern URLs is IETF RFC 3986. It partitions all possible characters into two distinct classes:

How Percent-Encoding Works Step-by-Step

The translation rule is straightforward:

1. Take the character and determine its binary byte sequence in UTF-8.
2. Convert each byte into its two-digit hexadecimal representation.
3. Prepend a percent sign (%) to each byte.

For standard 1-byte ASCII characters, this produces a single triplet:

What About Multi-Byte Unicode (Emojis and International Scripts)?

When dealing with characters beyond standard ASCII, modern systems encode the text using UTF-8 first. Since UTF-8 uses between 2 and 4 bytes per character, each byte receives its own percent triplet.

Take the euro symbol (€). In UTF-8, its byte sequence is 0xE2 0x82 0xAC. When percent-encoded, it becomes %E2%82%AC. Similarly, the rocket emoji (🚀), which consists of four bytes (0xF0 0x9F 0x9A 0x80), becomes %F0%9F%9A%80.

The Great Confusion: %20 vs. + (Plus Sign)

One of the most persistent sources of confusion in web engineering is seeing spaces encoded as + in some URLs and %20 in others. They are not interchangeable standards:

JavaScript API Pitfalls: encodeURI vs. encodeURIComponent

JavaScript provides two built-in global functions for encoding. Using the wrong one is one of the most common mistakes in frontend and Node.js development:

Look at what happens when you use encodeURI() on a query parameter value:

// THE WRONG WAY: encodeURI leaves delimiters unencoded
const query = 'shoes & shirts';
const badUrl = 'https://store.com/search?q=' + encodeURI(query);
// Output: https://store.com/search?q=shoes%20&shirts
// Result: Server sees parameter "q" = "shoes ", and a second parameter "shirts" = undefined!

// THE BETTER WAY: encodeURIComponent encodes the ampersand
const goodUrl = 'https://store.com/search?q=' + encodeURIComponent(query);
// Output: https://store.com/search?q=shoes%20%26%20shirts
// Result: Server correctly receives q = "shoes & shirts"

The Modern Best Practice: URL and URLSearchParams

In modern environments (browsers and Node.js 18+), avoid manual string concatenation altogether. Use the WHATWG standard URL and URLSearchParams classes. They handle RFC-compliant percent-encoding automatically without manual escapes:

const url = new URL('https://api.example.com/v1/items');
url.searchParams.set('category', 'Laptops & Hardware');
url.searchParams.set('tag', 'c++');
url.searchParams.set('page', '1');

console.log(url.toString());
// Output: https://api.example.com/v1/items?category=Laptops+%26+Hardware&tag=c%2B%2B&page=1

Web Security Vulnerabilities Linked to URL Encoding

URL encoding is not an encryption mechanism—it provides zero confidentiality. Any observer can reverse percent-encoding instantly. In fact, improper URL decoding routines on servers are among the most frequent culprits behind high-severity web exploits:

The Defensive Developer Checklist: 4 Rules for Safe URL Processing