🎯 Learning Objectives
By the end of this lesson, you will:
- Explain why release notes are a testable artifact
- Apply the 3-category verification approach to any release notes
- Cross-reference release notes against a build's actual changes
- Identify the most common release notes failures
📋 Release Notes Are a Promise
When a company publishes release notes, it's making a public commitment:
"These are the things we changed. Nothing more, nothing less."
Users, stakeholders, and support teams rely on this document to understand what changed in the build they're receiving. If the release notes are wrong — trust erodes.
Your job in release notes testing: verify the promise was kept.
🗂️ The 3 Categories of Release Notes
Most release notes contain three types of items. Each requires a different verification approach.
Category 1: New Features
The promise: "This feature was added."
How to verify:
- Can you navigate to the feature in the product?
- Does it work as described in the release notes?
- Is it available to the right user roles?
Common failure: Feature is listed but was pulled at the last minute and not removed from the notes. Users try to find it and can't.
Category 2: Bug Fixes
The promise: "This bug was fixed."
How to verify:
- Find the original bug report or ticket
- Reproduce the original bug scenario
- Confirm the fix works
Common failure: The PR was merged to a branch but never made it to the release build. Or the fix addressed a symptom but not the root cause.
💡 Tip: If your team uses JIRA, Linear, or GitHub Issues — check that each listed fix has a corresponding closed ticket. No ticket = no traceability = risky claim.
Category 3: Known Issues
The promise: "These are the bugs we know about but haven't fixed yet."
How to verify:
- Can you reproduce each listed known issue?
- Have any "known issues" from the previous release been fixed? (If yes, they should be in "Bug Fixes", not still listed as known issues.)
- Are there any obvious bugs in the release that are NOT listed as known issues?
Common failure: Known issues list is copied from the previous release without review. Old, already-fixed bugs remain listed.
🔍 Cross-Referencing Against the Build
The most rigorous verification: compare the release notes against the actual changes in the build.
Method 1: Ticket-Based (for teams using Jira/Linear)
- Pull all tickets that were closed/merged in this release
- Compare the list to release notes items
- Flag: tickets closed but not mentioned in release notes (was anything significant missed?)
- Flag: release notes items without a corresponding ticket (what's the source?)
Method 2: Git-Based (for teams with git access)
- Pull the commit log between the previous release tag and the current one
- Read commit messages for changes not covered by release notes
- Look for commits mentioning fixes or features that aren't in the notes
You don't need to be a developer for this. You need to be able to ask: "Is this commit mentioned in the release notes?"
🌍 Real-World Failures
Failure 1 — The Phantom Fix: Release notes: "Fixed: Login fails when email contains uppercase letters." Reality: The fix was committed but a merge conflict reintroduced the bug in the final build. Impact: Users try to log in with uppercase email, still can't. Support calls spike. Team is confused.
Failure 2 — The Missing Feature: Release notes: "New: Dark mode support." Reality: Dark mode is on the roadmap but was descoped from this release. Impact: Users spend hours looking for a setting that doesn't exist.
Failure 3 — The Stale Known Issue: Release notes (v2.5): "Known issue: Search results may not appear immediately on slow connections." The bug was fixed in v2.3. It's still listed in v2.5 known issues because nobody reviewed them. Impact: Users think the bug is still there and submit support tickets for a problem that no longer exists.
💡 Pro Tips
💡 Tip 1: Create a "release notes test checklist" specific to your product. Run through it before every release. It takes 30 minutes and saves far more.
💡 Tip 2: Read release notes from a user perspective, not a developer perspective. Would a non-technical user understand what "Fixed: Race condition in async state reducer" means? Consider the audience.
💡 Tip 3: Check that version numbers are correct and consistent. Nothing erodes trust faster than a release note that says "v2.4" when the build is actually "v2.3.1-hotfix".
📋 Summary
- Release notes are a verifiable contract with users
- 3 categories to test: new features (reachable?), bug fixes (actually fixed?), known issues (still present?)
- Cross-reference against tickets and/or git history
- Common failures: phantom fixes, removed features still listed, stale known issues
Up next: the final lesson — how to give all this feedback professionally, without turning documentation reviews into conflict.