Three related but different things
"Character encoding" gets confusing because ASCII, Unicode, and UTF-8 solve different, layered problems:
- ASCII is a fixed 7-bit character set — 128 code points covering English letters, digits, punctuation, and control characters.
- Unicode is a much larger character set — a mapping from characters (including nearly every writing system, symbols, and emoji) to numeric code points, currently over 149,000 assigned.
- UTF-8 is an encoding — a specific way of turning Unicode code points into actual bytes for storage or transmission.
Unicode says "the character 😀 is code point U+1F600"; UTF-8 says "here's how to represent that number as bytes." Confusing the character set with the encoding is the root of most "why doesn't this display right" bugs.
Why UTF-8 won
UTF-8 is a variable-width encoding: 1 byte for ASCII-range characters, up to 4 bytes for less common ones.
'A' → 1 byte: 0x41
'€' → 3 bytes: 0xE2 0x82 0xAC
'😀' → 4 bytes: 0xF0 0x9F 0x98 0x80
The critical design choice: every ASCII byte sequence is also valid UTF-8, byte-for-byte identical. This backward compatibility — plus compactness for English/Latin text and completeness for every other language — is exactly why UTF-8 became the dominant encoding on the web, overtaking older fixed-width and regional encodings almost entirely.
Mojibake: what garbled text actually is
"Mojibake" (文字化け, literally "character transformation") is what happens when bytes encoded in one scheme are decoded using a different one. The bytes themselves never change — only the interpretation does, which is why the result looks like plausible-but-wrong characters rather than random noise:
Correct: café (UTF-8 bytes decoded as UTF-8)
Mojibake: café (same UTF-8 bytes decoded as Latin-1/Windows-1252)
The é character in UTF-8 is two bytes (0xC3 0xA9); a Latin-1 decoder treats those same two bytes as two separate single-byte characters (à and ©), producing the garbled é. This is a decoding mismatch, not data loss — the original bytes are intact, which is why correctly re-decoding with the right encoding fully restores the text.
Where mismatches actually happen
- Missing or wrong
Content-Typecharset header on an HTTP response — the browser guesses, and guesses wrong for non-Latin content often enough to matter. - Saving a file in one editor's default encoding, opening it in another's. Older tools defaulting to a regional encoding (Windows-1252, Shift-JIS) instead of UTF-8 is still a common source of corrupted files, especially across different operating systems' default settings.
- Database columns with the wrong character set/collation, silently truncating or corrupting multi-byte characters on insert.
- Copy-pasting between apps with different default encodings, especially over older clipboard implementations.
Common mistakes
- Assuming "text" is encoding-agnostic. Every string of bytes needs an explicit or assumed encoding to become readable characters — there's no such thing as "just text" without one.
- Truncating a UTF-8 string by byte count instead of code point. Cutting a multi-byte character in half (common when a naive substring operation isn't encoding-aware) produces invalid, undecodable bytes at the cut point.
- Confusing "Unicode" with "UTF-8." Unicode is the character set; UTF-8 (along with UTF-16, UTF-32) is one of several ways to encode it — "convert this to Unicode" is a common but imprecise way of saying "convert this to UTF-8."
- Not declaring encoding explicitly. Relying on a receiving system's default guess, rather than an explicit
charset=utf-8declaration, is exactly how mojibake slips into production.
FAQ
What's the difference between Unicode and UTF-8?
Unicode is the character-to-number mapping (the "what"); UTF-8 is one specific way of turning those numbers into bytes (the "how"). UTF-16 and UTF-32 are other encodings of the same Unicode character set.
Why does text sometimes show up as é or similar garbage?
That's mojibake — the underlying bytes are fine, but they were decoded using the wrong character encoding (commonly Latin-1/Windows-1252 instead of the UTF-8 they were actually encoded in).
Is ASCII a subset of UTF-8?
Yes, exactly — every ASCII byte sequence is byte-for-byte identical to its UTF-8 representation, which is a deliberate design choice for backward compatibility.
Can mojibake corrupt data permanently?
Not if caught early — the original bytes are unchanged, only misinterpreted; re-decoding with the correct encoding restores the text. It becomes permanent only if the corrupted (wrongly re-encoded) version is saved over the original.
Inspect character codes and convert case formats with the ASCII Generator and Text Case Converter — both run entirely in your browser.