UTF-8 vs UTF-16: Byte alignments and memory storage properties

The structural difference

UTF-8: variable-width, 1 to 4 bytes per character, with ASCII characters taking exactly 1 byte.

UTF-16: variable-width but coarser-grained — most common characters (the "Basic Multilingual Plane," covering the vast majority of modern-language text) take exactly 2 bytes; characters outside that range (many emoji, some historical/rare scripts) require a 4-byte "surrogate pair."

'A'  → UTF-8: 1 byte (0x41)        UTF-16: 2 bytes (0x0041)
'€'  → UTF-8: 3 bytes              UTF-16: 2 bytes (0x20AC)
'😀' → UTF-8: 4 bytes              UTF-16: 4 bytes (surrogate pair)

Notice € takes 3 bytes in UTF-8 but only 2 in UTF-16 — this is exactly the tradeoff: UTF-8 is more compact for ASCII-heavy text, UTF-16 is more compact for text dominated by non-Latin scripts in the 2-byte range.

Why the web overwhelmingly chose UTF-8

Full ASCII backward-compatibility (UTF-8 encodes ASCII identically to plain ASCII bytes) combined with reasonable compactness for the historically Latin-script-heavy early web made UTF-8 the practical default for HTML, JSON, and most internet protocols — no conversion needed for the enormous existing base of ASCII content, plus full Unicode coverage when needed.

Why UTF-16 persists internally in specific runtimes

This is the detail most people don't expect: JavaScript strings, and several other language/platform string implementations (including Java's and Windows' internal APIs), use UTF-16 as their internal in-memory representation — even though virtually all data exchanged over the web (JSON payloads, HTML) is UTF-8. This isn't a contradiction; internal memory representation and external wire-format encoding are separate decisions, made at different times for different reasons (UTF-16 predates UTF-8's dominance and was already embedded in these systems' string designs before UTF-8 became the web's default).

The practical consequence: JavaScript string length can be surprising

Because JavaScript strings are UTF-16 internally, .length counts UTF-16 code units, not actual Unicode characters — a character requiring a surrogate pair (many emoji) reports as length 2, not 1:

"😀".length // 2, not 1 — it's a surrogate pair in UTF-16

This is a genuinely common source of subtle bugs in string-length validation, substring slicing, and character-counting logic whenever emoji or certain rare characters are involved.

Common mistakes

• Assuming a string's .length in JavaScript equals its visible character count. It counts UTF-16 code units — characters needing a surrogate pair count as 2, not 1.
• Choosing UTF-16 for network transmission or file storage "for consistency" with in-memory representation. The web's external protocols overwhelmingly expect UTF-8 — mixing this up causes real interoperability issues.
• Assuming UTF-8 is always more compact than UTF-16. For content dominated by scripts in UTF-16's 2-byte range (many non-Latin languages), UTF-16 can actually be more compact per character than UTF-8's 3-byte encoding for the same characters.

FAQ

Why does JavaScript's "😀".length return 2 instead of 1?
Because JavaScript strings are UTF-16 internally, and 😀 requires a 4-byte surrogate pair (2 UTF-16 code units) — .length counts code units, not visible characters.

Is UTF-8 always smaller than UTF-16 for the same text?
Not always — it depends on the script; ASCII-heavy text favors UTF-8, while text dominated by scripts in UTF-16's 2-byte range can be more compact in UTF-16.

Why do some programming languages use UTF-16 internally if the web uses UTF-8?
Historical timing — UTF-16 was already embedded in those languages'/platforms' string designs before UTF-8 became the web's dominant external encoding; internal representation and external wire format are separate, independently-made decisions.

Explore character encoding and byte-width behavior with the ASCII Generator — entirely client-side.