gstack Investigate: Root Cause Debugging with AI

The most expensive bugs in software aren't the bugs that crash the application. Those are visible, urgent, and get fixed quickly. The most expensive bugs are the ones that manifest intermittently, resist easy reproduction, and get "fixed" repeatedly without addressing the actual cause.

gstack's Investigate phase is designed for exactly these cases: systematic, AI-driven root cause analysis that doesn't stop at the symptom.

The Root Cause Problem#

Here's the debugging pattern that causes the most damage:

1. User reports a bug
2. Developer reproduces it (or can't reproduce it and closes the ticket)
3. Developer identifies the immediate cause and fixes it
4. PR gets reviewed for the fix, not for whether it addresses the real cause
5. Bug recurs in a different context 3 weeks later, or a related bug emerges
6. Repeat

The mistake at step 3: fixing the symptom without understanding the system. If a race condition caused data corruption, fixing the data corruption without eliminating the race condition means the corruption will happen again under the right circumstances.

Root cause analysis asks "why did this happen?" repeatedly until you reach the actual source. Debugging without root cause analysis is patching, not fixing.

How gstack Investigate Works#

The Investigate phase in DenchClaw's gstack workflow takes a bug report and produces a root cause analysis before any fix is implemented.

Input:

• Bug description (what the user observed)
• Reproduction steps (if known)
• Error logs, stack traces, or screenshots
• Relevant code sections

Process:

1. Characterize the bug: What type of bug is this? (Data corruption, race condition, incorrect logic, external dependency failure, edge case not handled)

2. Trace the execution path: Walk through the code path that leads to the observed behavior. Where does the code diverge from what was expected?

3. Identify the immediate cause: The specific line or block of code that produces the wrong behavior.

4. Ask why: Why did the code do this? Was the logic wrong? Was the input unexpected? Was there a missing guard?

5. Ask why again: Why was the logic wrong? Was the requirement unclear? Was there an assumption baked in that isn't always true?

6. Identify the actual root cause: The underlying reason — be it a design decision, a missing invariant, an unchecked assumption, or a system-level issue.

7. Propose the full fix: Address the root cause, not just the symptom. Often this means fixing the immediate bug AND adding a test that would have caught it, AND updating documentation.

A Concrete Example#

Bug report: "Imported contacts are sometimes missing email addresses."

gstack Investigate process:

Immediate cause: The email field is optional in the import schema, so records without emails are imported as-is. The email appears "missing" because the field is null.

Ask why: Why is the email field optional in the import schema? Review the schema definition — it was made optional because some CSV imports don't include emails.

Ask why again: Why are we importing contacts without emails? Review the code that validates import records. There's no validation that at least one contact method (email or phone) exists.

Root cause: The import validation allows records with no contact methods. These records are technically valid by the schema but practically useless for outreach.

Full fix:

1. Add validation that at least one of email/phone must be present
2. Add a warning in the import UI for records that have neither
3. Add a test for this validation
4. Update the import documentation to specify the requirement
5. Review the existing imported records with no contact methods and flag them for data quality review

The fix that addresses only the immediate cause ("tell users the email field can be empty") doesn't solve the underlying problem. The root cause analysis produces a fix that prevents the class of problem, not just this instance.

Patterns gstack Investigate Looks For#

The Investigate phase looks for common root cause patterns:

Missing invariants: Code assumes a condition is always true that isn't always true. Example: assuming a user always has an email address, but allowing users to be created without one.

Race conditions: Two operations that interact incorrectly when they execute concurrently. Difficult to reproduce because they depend on timing.

Missing error handling: An operation that can fail isn't handled when it does fail. The code assumes success.

Incorrect assumptions about external systems: API returns unexpected data shapes, rate limits aren't handled, timeouts aren't set.

State mutation bugs: A shared piece of state is modified in one place without considering the effects elsewhere.

Off-by-one errors and boundary conditions: The edge of a range isn't handled correctly.

Silent failures: An operation fails but the error is swallowed without logging or user notification.

For each pattern, gstack knows what to look for in the code and what the appropriate fix looks like.

Preventing Recurrence#

Root cause analysis isn't complete without prevention. gstack Investigate always ends with:

Test coverage: What test would have caught this bug before it reached production? Write that test.

Documentation: Was there an assumption that should have been documented but wasn't? Document it.

Monitoring: Should this condition be monitored? Add an alert if the condition occurs again.

Code review: Is there a code review heuristic that would have flagged this pattern? Add it to the engineering review checklist.

The goal: after gstack Investigate, this bug (and the class of bugs it represents) should be impossible to reach production again.

Integration with the Rest of gstack#

Investigate integrates with the gstack workflow:

• Engineering Review runs after Investigate to ensure the proposed fix doesn't introduce new issues
• QA verifies the fix works and tests the specific reproduction steps
• Retro tracks recurring bug patterns across weeks to identify systemic issues

The Retro phase is particularly valuable for recurring bugs: "This is the third time this month we've had a data validation bug in the import flow. Let's talk about what systemic change would prevent this pattern."

Frequently Asked Questions#

When should you run gstack Investigate vs. just fixing a simple bug?#

For bugs with a clear, simple cause (typo, wrong variable name, missing import), a direct fix is appropriate. Run Investigate for: bugs that are hard to reproduce, bugs that have recurred after a previous fix, bugs that indicate a systemic problem, and any bug that caused customer data issues.

How do you handle bugs that can't be reproduced?#

gstack Investigate includes a section on unreproducible bugs: what are the possible causes if the symptom is true but you can't trigger it? Add defensive logging to narrow down the conditions. Add monitoring to detect future occurrences. Sometimes you have to instrument first, wait, and investigate when you have data.

How long does a gstack Investigate session take?#

For a simple bug with clear reproduction: 15-30 minutes. For a complex intermittent bug: 45-90 minutes. The investment is almost always justified by the prevention of recurrence.

Can gstack Investigate be used for performance issues, not just bugs?#

Yes. Performance investigation follows the same pattern: characterize (what specifically is slow, measured how?), trace (what's the code path?), identify the bottleneck (N+1 query, unnecessary computation, blocking operation?), root cause (why was it written this way?), fix and verify.

How do you build a culture of root cause analysis?#

Make it a borrow requirement in code review. "This fixes the symptom — can you add a section on root cause?" After a few sprints, the pattern becomes normal. The Retro phase reinforces this by asking "what are the recurring bug patterns this week?"

Ready to try DenchClaw? Install in one command: npx denchclaw. Full setup guide →