Great work!

XP to next level

BugEater
EN

Testing Specs: Where Are the Holes?

🎯 Learning Objectives

By the end of this lesson, you will:

  • Apply the "negative scenario" technique to any functional spec
  • Identify the most common types of missing scenarios
  • Spot vague "parasite phrases" that fail the testability criterion
  • Ask the right clarifying questions without sounding critical

🕳️ The Hole-Finding Mindset

Every spec describes what should happen when everything goes right.

Your job is to ask: what happens when it goes wrong?

This isn't pessimism. It's professionalism. Real users:

  • Click Cancel on important forms
  • Enter invalid data
  • Close the browser at the worst possible moment
  • Have slow internet
  • Use the app in an unexpected order

If the spec doesn't address these — the developer will guess. And their guess might not match yours.

🔍 Technique 1: Negative Scenario Hunting

For every action described in the spec, ask these questions:

The "What If" Checklist:

Scenario type Question to ask
Cancellation What happens if the user clicks Cancel/Back/X?
Invalid input What happens if the data is wrong, empty, or too long?
Timeout What happens if the operation takes too long?
Duplicate What happens if the user submits the form twice?
No results What does the user see if the search returns nothing?
No permission What happens if the user doesn't have access?
Network failure What happens if the API call fails?

Real Example

Spec says: "Users can reset their password by entering their email address."

Negative scenarios to check for:

  • ❓ Email not registered in the system?
  • ❓ Email field left empty?
  • ❓ User submits the form twice?
  • ❓ The email service is down?
  • ❓ The reset link expires — what happens when clicked?
  • ❓ User resets password but the token is already used?

None of these are in the spec. All of them will be coded by someone. Who decides the behavior? Without a spec, it's the developer's intuition.

🔍 Technique 2: Vague Language Hunt

Some requirements look specific but aren't. The telltale sign: you can't write a pass/fail test for them.

The Classic Parasite Phrases:

Phrase Why it's a problem
"quickly" / "fast" Fast for whom? Under what conditions?
"intuitively" / "user-friendly" Subjective. Unmeasurable.
"appropriately" Appropriate by whose standard?
"easily" Ease is relative.
"correctly" Correct according to what definition?
"as needed" Who decides what's needed?
"etc." What's the rest of the list?
"and so on" Same problem.

How to Find Them:

Read the spec out loud. Every time you say a word and realize you'd need to ask "what does that mean exactly?" — highlight it. That's a vague requirement.

📝 How to Write a Good Review Comment

Bad: "This spec is incomplete and missing negative scenarios."

Good: "I noticed the spec describes the happy path for password reset. Could we add the expected behavior for: (1) unregistered email, (2) double submission, (3) expired reset token? I can draft these if helpful."

The difference:

  • Specific — you name exactly what's missing
  • Collaborative — you offer to help
  • Non-accusatory — you describe an observation, not a verdict

🌍 Real-World Example

Original spec: "The search function shall return results quickly and display them in an intuitive way."

Let's apply both techniques:

Negative scenarios missing:

  • What if there are no results?
  • What if the search term is too long (> 200 chars)?
  • What if search service times out?

Vague language found:

  • "quickly" — needs a number (e.g., "within 1 second")
  • "intuitive way" — needs a visual spec or description of sort order

Review comment:

"Could we clarify what 'quickly' means in measurable terms (e.g., 95th percentile under X ms)? Also, what should the user see when there are no results? I didn't find a design screen or spec for the empty state."

💡 Pro Tips

💡 Tip 1: Build a personal "vague words" list. Add to it whenever you find a new one. Share it with your team at the start of a project.

💡 Tip 2: For every feature, sketch a quick "happy path → sad path" map. If the spec only has the happy path, you know what to ask about.

💡 Tip 3: Negative scenarios aren't just about errors — they're also about boundary conditions. What's the minimum? The maximum? What happens at the exact boundary?

📋 Summary

  • Negative scenario hunting: for every action, ask "what happens when it fails?"
  • Key scenarios to check: cancellation, invalid input, timeout, duplicate, no results, no permission, network failure
  • Vague language hunt: find words that can't be measured or tested — "fast", "intuitive", "appropriate", "correctly"
  • Review comments: be specific, collaborative, and non-accusatory

Up next: User Stories and Acceptance Criteria — the most common format for incomplete requirements.

Quiz

A spec describes a "Send Message" feature but only covers successful message delivery. Which scenario type is most critically missing?

A spec says: "The system will respond appropriately to network errors." What technique would you use to identify the problem?

A requirement states: "The export feature will process data efficiently." Which specific words should you flag as vague language?

During a spec review, you identify that there is no description of what happens when a user tries to register with an email that already exists. What type of missing scenario is this?