Anatomy of a String
The Problem No One Tells You About
You paste a name into a form. It looks fine. The test passes. But two weeks later, support gets a ticket: "My name shows as ??? on the profile page." The input was Ólafur — perfectly valid, visually indistinguishable from ASCII on your screen, but stored as corrupted bytes because somewhere in the pipeline, two systems disagreed about encoding.
This is not a rare edge case. It happens to every system that processes names, addresses, or any user-generated text.
What Is a String, Really?
From a programmer's perspective, a string is a sequence of characters. From a computer's perspective, a string is a sequence of bytes. The encoding is the contract between those two views — it defines which byte sequences represent which characters.
The most common encoding you'll encounter today is UTF-8. It has one elegant property: the 128 ASCII characters (A-Z, a-z, 0-9, punctuation, control characters) are each stored as exactly one byte, identical to their ASCII representation. Every other character — Cyrillic, Arabic, CJK ideographs, emoji, mathematical symbols — takes 2, 3, or 4 bytes.
This means that Hello is 5 bytes in UTF-8. But Héllo is 6 bytes (the é takes 2 bytes). And H😀llo is 8 bytes (the emoji takes 4 bytes).
Why Length Is Deceptive
Here is a concrete example. Consider the JavaScript string "café". How many characters? Four. How many bytes in UTF-8? Five (c=1, a=1, f=1, é=2). How many bytes in Latin-1 (ISO-8859-1)? Also four — but a different sequence. How many bytes in UTF-16? Eight (UTF-16 uses at minimum 2 bytes per character).
Now imagine a backend that validates: if (username.length > 20) return error. What does length mean? In Java, it counts UTF-16 code units. In Python 3, it counts Unicode code points. In C, it counts bytes. An emoji like 🏴 (flag: Scotland, a sequence of 7 code points) might have length 14 in Java, length 7 in Python, and length 28 in C.
The validation that "works" in your test might silently pass or silently block real users depending on which layer does the counting.
The Unicode Standard and Code Points
Unicode is the master list of all characters in every human writing system. Each character has a code point — a number from 0 to 1,114,111 (written as U+0000 to U+10FFFF). The letter A is U+0041. The letter é is U+00E9. The emoji 😀 is U+1F600.
Unicode does not tell you how to store code points as bytes. That is the job of encodings like UTF-8, UTF-16, and UTF-32. This separation is important: a mojibake (garbled text) happens not when a character has no code point, but when you decode bytes with the wrong encoding — you get valid bytes read as the wrong characters.
The Encoding Mismatch Bug
The classic scenario: a form accepts UTF-8 input. The database column is configured with Latin-1 charset (a legacy default in older MySQL databases). The character ñ (U+00F1) is stored as the byte 0xC3 0xB1 in UTF-8. When MySQL reads it back through a Latin-1 lens, it sees two characters: Ã and ±. The user's name is now permanently corrupted.
As a tester, you can catch this:
- Submit a name with non-ASCII characters:
Ólafur,Müller,Søren,Привіт - Navigate away and return to the same profile
- Verify the name is stored and displayed identically
If it comes back changed, you have an encoding bug. The visual test that uses only ASCII names will never find it.
Practical Test Cases for Strings
When testing any text field, you want at least one test from each of these categories:
| Category | Example | What to check |
|---|---|---|
| ASCII only | John Smith |
Baseline — must work |
| Latin extended | Ólafur Sigurðsson |
2-byte UTF-8 chars |
| Cyrillic / Arabic | Привіт / مرحبا |
2-byte UTF-8 chars |
| CJK characters | 李明 |
3-byte UTF-8 chars |
| Emoji | John 😊 |
4-byte UTF-8 chars |
| Mixed script | Café 咖啡 |
Multiple ranges |
| Max length boundary | 50 chars, 50 code points | Where is the limit? |
The goal is not to test "does Unicode work" — it's to find where the system applies length limits, sanitization, or storage in a way that assumes ASCII.
What to Look for in Requirements
When you read a spec that says "username must be 3–20 characters," ask:
- Characters or bytes?
- Characters as Unicode code points or as UTF-16 code units?
- Does the limit apply before or after normalization (e.g., NFD vs NFC form)?
These are not pedantic questions. They determine what test cases you need. A field limited to 20 bytes will silently truncate a 10-character Cyrillic name mid-word. A field limited to 20 UTF-16 code units will accept a 10-emoji username that breaks the database column.
Challenge: Encoding Breaker
The practice challenge for this module asks you to find the edge cases in a profile name field. Some inputs succeed, some trigger errors, and at least one reveals a silent bug — an input that appears to succeed but is actually stored incorrectly.
Use what you know from this lesson to guide your exploration.