mirror of
https://github.com/zed-industries/zed.git
synced 2026-04-18 07:47:53 +00:00
Add prompts and scripts for automatic crash repro and fix (#49063)
These prompts can be used to automatically diagnose and fix crashes report in Sentry. Usage: 1. Find a crash in Sentry. It will have an ID like ZED-123 2. In an agent, do a prompt like `Follow the instructions in @investigate.md to investigate ZED-123` 3. Once the agent finds a repro, fix it in a new thread by saying `Follow the instructions in @fix.md` Release Notes: - N/A --------- Co-authored-by: Ben Brandt <benjamin.j.brandt@gmail.com>
This commit is contained in:
Eric Holk
GitHub
@@ -0,0 +1,84 @@
|
||||
# Crash Fix
|
||||
|
||||
You are fixing a crash that has been analyzed and has a reproduction test case. Your goal is to implement a minimal, correct fix that resolves the root cause and makes the reproduction test pass.
|
||||
|
||||
## Inputs
|
||||
|
||||
Before starting, you should have:
|
||||
|
||||
1. **ANALYSIS.md** — the crash analysis from the investigation phase. Read it thoroughly.
|
||||
2. **A failing test** — a reproduction test that triggers the crash. Run it first to confirm it fails as expected.
|
||||
|
||||
If either is missing, ask the user to provide them or run the investigation phase first (`/prompt crash/investigate`).
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 1: Confirm the Failing Test
|
||||
|
||||
Run the reproduction test and verify it fails with the expected crash:
|
||||
|
||||
```
|
||||
cargo test -p <crate> <test_name>
|
||||
```
|
||||
|
||||
Read the failure output. Confirm the panic message and stack trace match what ANALYSIS.md describes. If the test doesn't fail, or fails differently than expected, stop and reassess before proceeding.
|
||||
|
||||
### Step 2: Understand the Fix
|
||||
|
||||
Read the "Suggested Fix" section of ANALYSIS.md and the relevant source code. Before writing any code, be clear on:
|
||||
|
||||
1. **What invariant is being violated** — what property of the data does the crashing code assume?
|
||||
2. **Where the invariant breaks** — which function produces the bad state?
|
||||
|
||||
### Step 3: Implement the Fix
|
||||
|
||||
Apply the minimal change needed to resolve the root cause. Guidelines:
|
||||
|
||||
- **Fix the root cause, not the symptom.** Don't just catch the panic with a bounds check if the real problem is an incorrect offset calculation. Fix the calculation.
|
||||
- **Preserve existing behavior** for all non-crashing cases. The fix should only change what happens in the scenario that was previously crashing.
|
||||
- **Don't add unnecessary changes.** No drive-by improvements, keep the diff focused.
|
||||
- **Add a comment only if the fix is non-obvious.** If a reader might wonder "why is this check here?", a brief comment explaining the crash scenario is appropriate.
|
||||
- **Consider long term maintainability** Please make a targeted fix while being sure to consider the long term maintainability and reliability of the codebase
|
||||
|
||||
### Step 4: Verify the Fix
|
||||
|
||||
Run the reproduction test and confirm it passes:
|
||||
|
||||
```
|
||||
cargo test -p <crate> <test_name>
|
||||
```
|
||||
|
||||
Then run the full test suite for the affected crate to check for regressions:
|
||||
|
||||
```
|
||||
cargo test -p <crate>
|
||||
```
|
||||
|
||||
If any tests fail, determine whether the fix introduced a regression. Fix regressions before proceeding.
|
||||
|
||||
### Step 5: Run Clippy
|
||||
|
||||
```
|
||||
./script/clippy
|
||||
```
|
||||
|
||||
Address any new warnings introduced by your change.
|
||||
|
||||
### Step 6: Summarize
|
||||
|
||||
Write a brief summary of the fix for use in a PR description. Include:
|
||||
|
||||
- **What was the bug** — one sentence on the root cause.
|
||||
- **What the fix does** — one sentence on the change.
|
||||
- **How it was verified** — note that the reproduction test now passes.
|
||||
- **Sentry issue link** — if available from ANALYSIS.md.
|
||||
|
||||
We use the following template for pull request descriptions. Please add information to answer the relevant sections, especially for release notes.
|
||||
|
||||
```
|
||||
<Description of change, what the issue was and the fix.>
|
||||
|
||||
Release Notes:
|
||||
|
||||
- N/A *or* Added/Fixed/Improved ...
|
||||
```
|
||||
@@ -0,0 +1,89 @@
|
||||
# Crash Investigation
|
||||
|
||||
You are investigating a crash that was observed in the wild. Your goal is to understand the root cause and produce a minimal reproduction test case that triggers the same crash. This test will be used to verify a fix and prevent regressions.
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 1: Get the Crash Report
|
||||
|
||||
If given a Sentry issue ID (like `ZED-4VS` or a numeric ID), there are several ways to fetch the crash data:
|
||||
|
||||
**Option A: Sentry MCP server (preferred if available)**
|
||||
If the Sentry MCP server is configured as a context server, use its tools directly (e.g., `get_sentry_issue`) to fetch the issue details and stack trace. This is the simplest path — no tokens or scripts needed.
|
||||
|
||||
**Option B: Fetch script**
|
||||
Run the fetch script from the terminal:
|
||||
|
||||
```
|
||||
script/sentry-fetch <issue-id>
|
||||
```
|
||||
|
||||
This reads authentication from `~/.sentryclirc` (set up via `sentry-cli login`) or the `SENTRY_AUTH_TOKEN` environment variable.
|
||||
|
||||
**Option C: Crash report provided directly**
|
||||
If the crash report was provided inline or as a file, read it carefully before proceeding.
|
||||
|
||||
### Step 2: Analyze the Stack Trace
|
||||
|
||||
Read the stack trace bottom-to-top (from crash site upward) and identify:
|
||||
|
||||
1. **The crash site** — the exact function and line where the panic/abort occurs.
|
||||
2. **The immediate cause** — what operation failed (e.g., slice indexing on a non-char-boundary, out-of-bounds access, unwrap on None).
|
||||
3. **The relevant application frames** — filter out crash handler, signal handler, parking_lot, and stdlib frames. Focus on frames marked "(In app)".
|
||||
4. **The data flow** — trace how the invalid data reached the crash site. What computed the bad index, the None value, etc.?
|
||||
|
||||
Find the relevant source files in the repository and read them. Pay close attention to:
|
||||
- The crashing function and its callers
|
||||
- How inputs to the crashing operation are computed
|
||||
- Any assumptions the code makes about its inputs (string encoding, array lengths, option values)
|
||||
|
||||
### Step 3: Identify the Root Cause
|
||||
|
||||
Work backwards from the crash site to determine **what sequence of events or data conditions** produces the invalid state.
|
||||
|
||||
Ask yourself: *What user action or sequence of actions could lead to this state?* The crash came from a real user, so there is some natural usage pattern that triggers it.
|
||||
|
||||
### Step 4: Write a Reproduction Test
|
||||
|
||||
Write a minimal test case that:
|
||||
|
||||
1. **Mimics user actions** rather than constructing corrupt state directly. Work from the top down: what does the user do (open a file, type text, trigger a completion, etc.) that eventually causes the internal state to become invalid?
|
||||
2. **Exercises the same code path** as the crash. The test should fail in the same function with the same kind of error (e.g., same panic message pattern).
|
||||
3. **Is minimal** — include only what's necessary to trigger the crash. Remove anything that isn't load-bearing.
|
||||
4. **Lives in the right place** — add the test to the existing test module of the crate where the bug lives. Follow the existing test patterns in that module.
|
||||
5. **Avoid overly verbose comments** - the test should be self-explanatory and concise. More detailed descriptions of the test can go in ANALYSIS.md (see the next section).
|
||||
|
||||
When the test fails, its stack trace should share the key application frames from the original crash report. The outermost frames (crash handler, signal handling) will differ since we're in a test environment — that's expected.
|
||||
|
||||
If you can't reproduce the exact crash but can demonstrate the same class of bug (e.g., same function panicking with a similar invalid input), that is still valuable. Note the difference in your analysis.
|
||||
|
||||
### Step 5: Write the Analysis
|
||||
|
||||
Create an `ANALYSIS.md` file (in the working directory root, or wherever instructed) with these sections:
|
||||
|
||||
```markdown
|
||||
# Crash Analysis: <short description>
|
||||
|
||||
## Crash Summary
|
||||
- **Sentry Issue:** <ID and link if available>
|
||||
- **Error:** <the panic/error message>
|
||||
- **Crash Site:** <function name and file>
|
||||
|
||||
## Root Cause
|
||||
<Explain what goes wrong and why. Be specific about the data flow.>
|
||||
|
||||
## Reproduction
|
||||
<Describe what the test does and how it triggers the same crash.
|
||||
Include the exact command to run the test, e.g.:
|
||||
`cargo test -p <crate> <test_name>`>
|
||||
|
||||
## Suggested Fix
|
||||
<Describe the fix approach. Be specific: which function, what check to add,
|
||||
what computation to change. If there are multiple options, list them with tradeoffs.>
|
||||
```
|
||||
|
||||
## Guidelines
|
||||
|
||||
- **Don't guess.** If you're unsure about a code path, read the source. Use `grep` to find relevant functions, types, and call sites.
|
||||
- **Check the git history.** If the crash appeared in a specific version, `git log` on the relevant files may reveal a recent change that introduced the bug.
|
||||
- **Look at existing tests.** The crate likely has tests that show how to set up the relevant subsystem. Follow those patterns rather than inventing new test infrastructure.
|
||||
Executable
+357
@@ -0,0 +1,357 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Fetch a crash report from Sentry and output formatted markdown.
|
||||
|
||||
Usage:
|
||||
script/sentry-fetch <issue-short-id-or-numeric-id>
|
||||
script/sentry-fetch ZED-4VS
|
||||
script/sentry-fetch 7243282041
|
||||
|
||||
Authentication (checked in order):
|
||||
1. SENTRY_AUTH_TOKEN environment variable
|
||||
2. Token from ~/.sentryclirc (written by `sentry-cli login`)
|
||||
|
||||
If neither is found, the script will print setup instructions and exit.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import configparser
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
SENTRY_BASE_URL = "https://sentry.io/api/0"
|
||||
DEFAULT_SENTRY_ORG = "zed-dev"
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Fetch a crash report from Sentry and output formatted markdown."
|
||||
)
|
||||
parser.add_argument(
|
||||
"issue",
|
||||
help="Sentry issue short ID (e.g. ZED-4VS) or numeric issue ID",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
token = find_auth_token()
|
||||
if not token:
|
||||
print(
|
||||
"Error: No Sentry auth token found.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
print(
|
||||
"\nSet up authentication using one of these methods:\n"
|
||||
" 1. Run `sentry-cli login` (stores token in ~/.sentryclirc)\n"
|
||||
" 2. Set the SENTRY_AUTH_TOKEN environment variable\n"
|
||||
"\nGet a token at https://sentry.io/settings/auth-tokens/",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
try:
|
||||
issue_id, short_id, issue = resolve_issue(args.issue, token)
|
||||
event = fetch_latest_event(issue_id, token)
|
||||
except FetchError as err:
|
||||
print(f"Error: {err}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
markdown = format_crash_report(issue, event, short_id)
|
||||
print(markdown)
|
||||
|
||||
|
||||
class FetchError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def find_auth_token():
|
||||
"""Find a Sentry auth token from environment or ~/.sentryclirc.
|
||||
|
||||
Checks in order:
|
||||
1. SENTRY_AUTH_TOKEN environment variable
|
||||
2. auth.token in ~/.sentryclirc (INI format, written by `sentry-cli login`)
|
||||
"""
|
||||
token = os.environ.get("SENTRY_AUTH_TOKEN")
|
||||
if token:
|
||||
return token
|
||||
|
||||
sentryclirc_path = os.path.expanduser("~/.sentryclirc")
|
||||
if os.path.isfile(sentryclirc_path):
|
||||
config = configparser.ConfigParser()
|
||||
try:
|
||||
config.read(sentryclirc_path)
|
||||
token = config.get("auth", "token", fallback=None)
|
||||
if token:
|
||||
return token
|
||||
except configparser.Error:
|
||||
pass
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def api_get(path, token):
|
||||
"""Make an authenticated GET request to the Sentry API."""
|
||||
url = f"{SENTRY_BASE_URL}{path}"
|
||||
req = urllib.request.Request(url)
|
||||
req.add_header("Authorization", f"Bearer {token}")
|
||||
req.add_header("Accept", "application/json")
|
||||
try:
|
||||
with urllib.request.urlopen(req) as response:
|
||||
return json.loads(response.read().decode("utf-8"))
|
||||
except urllib.error.HTTPError as err:
|
||||
body = err.read().decode("utf-8", errors="replace")
|
||||
try:
|
||||
detail = json.loads(body).get("detail", body)
|
||||
except (json.JSONDecodeError, AttributeError):
|
||||
detail = body
|
||||
raise FetchError(f"Sentry API returned HTTP {err.code} for {path}: {detail}")
|
||||
except urllib.error.URLError as err:
|
||||
raise FetchError(f"Failed to connect to Sentry API: {err.reason}")
|
||||
|
||||
|
||||
def resolve_issue(identifier, token):
|
||||
"""Resolve a Sentry issue by short ID or numeric ID.
|
||||
|
||||
Returns (issue_id, short_id, issue_data).
|
||||
"""
|
||||
if identifier.isdigit():
|
||||
issue = api_get(f"/issues/{identifier}/", token)
|
||||
return identifier, issue.get("shortId", identifier), issue
|
||||
|
||||
result = api_get(f"/organizations/{DEFAULT_SENTRY_ORG}/shortids/{identifier}/", token)
|
||||
group_id = str(result["groupId"])
|
||||
issue = api_get(f"/issues/{group_id}/", token)
|
||||
return group_id, identifier, issue
|
||||
|
||||
|
||||
def fetch_latest_event(issue_id, token):
|
||||
"""Fetch the latest event for an issue."""
|
||||
return api_get(f"/issues/{issue_id}/events/latest/", token)
|
||||
|
||||
|
||||
def format_crash_report(issue, event, short_id):
|
||||
"""Format a Sentry issue and event as a markdown crash report."""
|
||||
lines = []
|
||||
|
||||
title = issue.get("title", "Unknown Crash")
|
||||
lines.append(f"# {title}")
|
||||
lines.append("")
|
||||
|
||||
issue_id = issue.get("id", "unknown")
|
||||
project = issue.get("project", {})
|
||||
project_slug = (
|
||||
project.get("slug", "unknown") if isinstance(project, dict) else str(project)
|
||||
)
|
||||
first_seen = issue.get("firstSeen", "unknown")
|
||||
last_seen = issue.get("lastSeen", "unknown")
|
||||
count = issue.get("count", "unknown")
|
||||
sentry_url = f"https://sentry.io/organizations/{DEFAULT_SENTRY_ORG}/issues/{issue_id}/"
|
||||
|
||||
lines.append(f"**Short ID:** {short_id}")
|
||||
lines.append(f"**Issue ID:** {issue_id}")
|
||||
lines.append(f"**Project:** {project_slug}")
|
||||
lines.append(f"**Sentry URL:** {sentry_url}")
|
||||
lines.append(f"**First Seen:** {first_seen}")
|
||||
lines.append(f"**Last Seen:** {last_seen}")
|
||||
lines.append(f"**Event Count:** {count}")
|
||||
lines.append("")
|
||||
|
||||
format_tags(lines, event)
|
||||
format_entries(lines, event)
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def format_tags(lines, event):
|
||||
"""Extract and format tags from the event."""
|
||||
tags = event.get("tags", [])
|
||||
if not tags:
|
||||
return
|
||||
|
||||
lines.append("## Tags")
|
||||
lines.append("")
|
||||
for tag in tags:
|
||||
key = tag.get("key", "") if isinstance(tag, dict) else ""
|
||||
value = tag.get("value", "") if isinstance(tag, dict) else ""
|
||||
if key:
|
||||
lines.append(f"- **{key}:** {value}")
|
||||
lines.append("")
|
||||
|
||||
|
||||
def format_entries(lines, event):
|
||||
"""Format exception and thread entries from the event."""
|
||||
entries = event.get("entries", [])
|
||||
|
||||
for entry in entries:
|
||||
entry_type = entry.get("type", "")
|
||||
|
||||
if entry_type == "exception":
|
||||
format_exceptions(lines, entry)
|
||||
elif entry_type == "threads":
|
||||
format_threads(lines, entry)
|
||||
|
||||
|
||||
def format_exceptions(lines, entry):
|
||||
"""Format exception entries."""
|
||||
exceptions = entry.get("data", {}).get("values", [])
|
||||
if not exceptions:
|
||||
return
|
||||
|
||||
lines.append("## Exceptions")
|
||||
lines.append("")
|
||||
|
||||
for i, exc in enumerate(exceptions):
|
||||
exc_type = exc.get("type", "Unknown")
|
||||
exc_value = exc.get("value", "")
|
||||
mechanism = exc.get("mechanism", {})
|
||||
|
||||
lines.append(f"### Exception {i + 1}")
|
||||
lines.append(f"**Type:** {exc_type}")
|
||||
if exc_value:
|
||||
lines.append(f"**Value:** {exc_value}")
|
||||
if mechanism:
|
||||
mech_type = mechanism.get("type", "unknown")
|
||||
handled = mechanism.get("handled")
|
||||
if handled is not None:
|
||||
lines.append(f"**Mechanism:** {mech_type} (handled: {handled})")
|
||||
else:
|
||||
lines.append(f"**Mechanism:** {mech_type}")
|
||||
lines.append("")
|
||||
|
||||
stacktrace = exc.get("stacktrace")
|
||||
if stacktrace:
|
||||
frames = stacktrace.get("frames", [])
|
||||
lines.append("#### Stacktrace")
|
||||
lines.append("")
|
||||
lines.append("```")
|
||||
lines.append(format_frames(frames))
|
||||
lines.append("```")
|
||||
lines.append("")
|
||||
|
||||
|
||||
def format_threads(lines, entry):
|
||||
"""Format thread entries, focusing on crashed and current threads."""
|
||||
threads = entry.get("data", {}).get("values", [])
|
||||
if not threads:
|
||||
return
|
||||
|
||||
crashed_threads = [t for t in threads if t.get("crashed", False)]
|
||||
current_threads = [
|
||||
t for t in threads if t.get("current", False) and not t.get("crashed", False)
|
||||
]
|
||||
other_threads = [
|
||||
t
|
||||
for t in threads
|
||||
if not t.get("crashed", False) and not t.get("current", False)
|
||||
]
|
||||
|
||||
lines.append("## Threads")
|
||||
lines.append("")
|
||||
|
||||
for thread in crashed_threads + current_threads:
|
||||
format_single_thread(lines, thread, show_frames=True)
|
||||
|
||||
if other_threads:
|
||||
lines.append(f"*({len(other_threads)} other threads omitted)*")
|
||||
lines.append("")
|
||||
|
||||
|
||||
def format_single_thread(lines, thread, show_frames=False):
|
||||
"""Format a single thread entry."""
|
||||
thread_id = thread.get("id", "?")
|
||||
thread_name = thread.get("name", "unnamed")
|
||||
crashed = thread.get("crashed", False)
|
||||
current = thread.get("current", False)
|
||||
|
||||
markers = []
|
||||
if crashed:
|
||||
markers.append("CRASHED")
|
||||
if current:
|
||||
markers.append("current")
|
||||
marker_str = f" ({', '.join(markers)})" if markers else ""
|
||||
|
||||
lines.append(f"### Thread {thread_id}: {thread_name}{marker_str}")
|
||||
lines.append("")
|
||||
|
||||
if not show_frames:
|
||||
return
|
||||
|
||||
stacktrace = thread.get("stacktrace")
|
||||
if not stacktrace:
|
||||
return
|
||||
|
||||
frames = stacktrace.get("frames", [])
|
||||
if frames:
|
||||
lines.append("```")
|
||||
lines.append(format_frames(frames))
|
||||
lines.append("```")
|
||||
lines.append("")
|
||||
|
||||
|
||||
def format_frames(frames):
|
||||
"""Format stack trace frames for display.
|
||||
|
||||
Sentry provides frames from outermost caller to innermost callee,
|
||||
so we reverse them to show the most recent (crashing) call first,
|
||||
matching the convention used in most crash report displays.
|
||||
"""
|
||||
output_lines = []
|
||||
|
||||
for frame in reversed(frames):
|
||||
func = frame.get("function") or frame.get("symbol") or "unknown"
|
||||
filename = (
|
||||
frame.get("filename")
|
||||
or frame.get("absPath")
|
||||
or frame.get("abs_path")
|
||||
or "unknown file"
|
||||
)
|
||||
line_no = frame.get("lineNo") or frame.get("lineno")
|
||||
in_app = frame.get("inApp", frame.get("in_app", False))
|
||||
|
||||
app_marker = "(In app)" if in_app else "(Not in app)"
|
||||
line_info = f"Line {line_no}" if line_no else "Line null"
|
||||
|
||||
output_lines.append(f" {func} in {filename} [{line_info}] {app_marker}")
|
||||
|
||||
context_lines = build_context_lines(frame, line_no)
|
||||
output_lines.extend(context_lines)
|
||||
|
||||
return "\n".join(output_lines)
|
||||
|
||||
|
||||
def build_context_lines(frame, suspect_line_no):
|
||||
"""Build context code lines for a single frame.
|
||||
|
||||
Handles both Sentry response formats:
|
||||
- preContext/contextLine/postContext (separate fields)
|
||||
- context as an array of [line_no, code] tuples
|
||||
"""
|
||||
output = []
|
||||
|
||||
pre_context = frame.get("preContext") or frame.get("pre_context") or []
|
||||
context_line = frame.get("contextLine") or frame.get("context_line")
|
||||
post_context = frame.get("postContext") or frame.get("post_context") or []
|
||||
|
||||
if context_line is not None or pre_context or post_context:
|
||||
for code_line in pre_context:
|
||||
output.append(f" {code_line}")
|
||||
if context_line is not None:
|
||||
output.append(f" {context_line} <-- SUSPECT LINE")
|
||||
for code_line in post_context:
|
||||
output.append(f" {code_line}")
|
||||
return output
|
||||
|
||||
context = frame.get("context") or []
|
||||
for ctx_entry in context:
|
||||
if isinstance(ctx_entry, list) and len(ctx_entry) >= 2:
|
||||
ctx_line_no = ctx_entry[0]
|
||||
ctx_code = ctx_entry[1]
|
||||
suspect = " <-- SUSPECT LINE" if ctx_line_no == suspect_line_no else ""
|
||||
output.append(f" {ctx_code}{suspect}")
|
||||
|
||||
return output
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user