How to automate release notes with AI
Writing release notes by hand means someone rereads a week of merged PRs, translates each into user language, and formats the result — reliably the task that slips when a release is late. AI handles the middle of that pipeline well. It does not handle the ends: the quality of what goes in, and the judgment of what goes out.
Here's the pipeline, from cheapest to most automated.
1. Fix the raw material first
Every generator — AI or not — works from PR titles, descriptions, and labels. If your PRs are titled "fix", "wip", and "address review comments", automation will faithfully produce garbage.
Two cheap conventions fix this: PR titles written as user-visible changes where applicable ("Add CSV export to reports" not "refactor exporter"), and a small label set — feature, fix, internal, breaking — applied at review time. Ten seconds per PR, and it's the highest-leverage step in this guide.
2. Start with GitHub's built-in generation
Before adding AI, use GitHub's automatically generated release notes: on any release, it produces the merged-PR list, contributors, and a full-changelog link, with a config file to group PRs by label and exclude noise. No cost, no new tools.
The output is a categorized inventory, not something customers should read. But it's the correct input for the next step, and for internal changelogs it may be all you need — see changelog vs. release notes for where that line sits.
3. Add an LLM summarization pass
The AI step is turning the inventory into prose: batching related PRs into one entry, rewriting titles as user-facing capabilities, and splitting new/improved/fixed. Options ordered by effort:
- Off-the-shelf actions. GitHub's own copilot-release-notes generates notes from the PRs between two refs using Copilot CLI, and the Marketplace has equivalents built on other model APIs that trigger on release creation.
- Agentic workflows. GitHub Agentic Workflows runs an agent on release publish or on a schedule to summarize merged work into a publishable report.
- A prompt in your own CI. Graphite's guide walks through the DIY version: collect merged PRs since the last tag, feed titles and descriptions to a model with your template, commit the draft.
Whichever you pick, pin the output format in the prompt — give it the exact skeleton from our release notes template rather than letting the model invent structure per release.
4. Keep a human on the last read
The draft-review-publish split is where teams get this wrong in both directions: publishing raw AI output (which confidently misstates what a change does, because PR descriptions lie by omission), or reviewing so heavily the automation saves nothing.
The workable contract: a human reads for accuracy and for what to cut, not for wording. Two specific checks — is every breaking change flagged, and is anything listed that didn't actually ship (reverted or feature-flagged off). Five minutes per release.
5. Automate the step after the notes
A published note is still a broadcast. The customers who asked for a feature mostly won't see it — and "tell the requester" is the step no PR-summarizer covers, because the requester's name isn't in the repo. It's in a Slack thread or a support ticket from four months ago.
This is the part we build Modem for, so read this step knowing we sell it. Modem captures requests from Slack, Discord, and support tools with the requester attached, and when the resolving PR merges through the GitHub integration, it matches the merge back to those requests and drafts both the release note entry and the individual follow-ups. Because the drafts pull from a context graph with the original quotes attached, the customer language in each entry is the customer's, not a model's paraphrase of a summary. If nobody identifiable asks for your features — a public devtool with anonymous users, say — steps 1 through 4 are the whole pipeline and Modem adds little.
The smallest version this week
Turn on GitHub's generated release notes and add the four labels from step 1 to your open PRs. Next release, paste the generated inventory plus the template into whatever model you already use, and review the draft for five minutes. That's a working pipeline; the CI wiring and requester matching can come once the habit sticks.
