Write release notes from pull requests without publishing internal noise
Have Beeuni read a connected GitHub repository, check the release range, and draft user-facing notes with references to the changes.
A pull request titled “Refactor export job” tells a reviewer where to look. It tells a customer very little. Release notes need to explain what changed in the product, who it affects, and whether it is available in this release.
Establish what is actually in the release
Give the agent access to the repository through your configured GitHub connection. Ask it to find the previous and current tags, compare their commits, and inspect the associated pull requests. Name the repository and tags yourself when several releases are in progress.
Read the connected repository and compare v1.4.0 with v1.5.0.
Draft release notes for users, grouped into features and fixes.
Inspect the linked PRs and diffs rather than relying on titles alone.
Keep a separate list of internal changes, reversions, and uncertain availability.
Save the draft with PR references. Do not publish a release.A merged PR is not necessarily deployed, and a change merged during the same week may belong to a later release. The tag comparison establishes which code is included. Feature flags and rollout status may still need confirmation from the team.
Ask the agent to create a private working list: source PR, changed behavior, affected users, and whether it appears in the release. Check reverted changes and features that remain disabled. Keep uncertain entries out of the public draft until someone confirms them.
From implementation detail to user-facing change
These illustrative PR notes show a few common editing choices:
| Working note | Release-note draft | Editorial decision |
|---|---|---|
| Persist selected filters in query parameters | Filtered views now keep their filters when you share the link. | Describe the visible behavior |
| Fix duplicate submit on double click | Fixed an issue where clicking Submit twice could create duplicate requests. | State the trigger and effect |
| Reorganize test helpers | Omit from the public notes | Keep maintenance in the internal record |
| Add export behind a disabled feature flag | Hold until the export is available | Confirm availability before announcing |
Group by what readers need to know
Use sections such as new features, improvements, and fixes when the release needs them. Skip empty headings. Combine several PRs that deliver one feature into one entry. Keep distinct fixes separate when they affect different actions.
Do not hide required actions inside a general improvements list. If an API field changes, name the old and new forms and link to migration instructions. If behavior changes only for certain accounts or clients, say who receives it. Have the feature owner confirm those details.
Keep the draft connected to its evidence
- Retain PR or commit references in the review copy, even when they are removed from public notes.
- Check that every announced feature is available in the release being described.
- Remove internal hostnames, credentials, customer details, and unpublished issue content.
- Replace claims such as “faster” with a precise behavior, unless you have a measurement to cite.
Ask for two files: the public draft and the working list showing included, excluded, and unresolved entries. After review, you can ask the agent to update the draft in the repository if the connection permits that write. Publishing the release is a separate instruction. If the notes say a form bug is fixed, the website testing guide describes the reproduction record you can use to check that claim.
More articles
Infrastructure for RSI research: running and evaluating self-improving agents
How Beeuni helps researchers run repeated agent-improvement experiments: edit code and instructions, execute tests, investigate failures, and compare versions.
Use Beeuni to run and evaluate a model fine-tuning experiment
Have an agent prepare training data, run a fine-tuning job on your connected training computer, inspect failures, and compare model outputs.