🚀 New: chi (χ) — an open-source autoresearch harness for fleets of LLM coding agents. Read the announcement.

Treat GitHub Event Streams as a Doorbell, Not a Log

GitHub Event Streams, in preview since November 17, pushes pull request events over SSE with no replay, possible duplicates, no ordering guarantee, and 24-hour subscriptions. A correct consumer treats each event as a prompt to re-read state from the REST API. Deduplicating by api_url, as the docs suggest, can drop a re-run's result.

Contents

Any agent or bot built on GitHub’s new Event Streams preview has to be written as a reconciler. Each event means “something on this pull request changed, go look,” and the client then rebuilds its view from the REST API. A client that treats the stream as an ordered log of what happened will miss events, double-count others, and sooner or later act on a commit that is no longer the head.

That isn’t a criticism of the design. The documentation says all of it plainly. The trouble is in two of its own recommendations, which go wrong if you follow them literally.

What GitHub shipped

Event Streams went into public preview on November 17, behind a wait list, on GitHub.com only. It covers pull requests and nothing else so far. You create a subscription for one PR, open a Server-Sent Events stream on it, and GitHub pushes events while the connection stays up. The client makes an outbound connection, so nothing needs a public URL. That is the selling point for agents running on a laptop or in a CI job.

The API is five routes:

MethodRoutePurpose
POST/event-subscriptionsCreate, with resource_url and event_types
GET/event-subscriptionsList active subscriptions
GET/event-subscriptions/{id}Get one
DELETE/event-subscriptions/{id}Delete one
GET/event-subscriptions/{id}/eventsThe SSE stream

There are five event types: check_suite.status.changed, head.changed, merge_conflict, review.submitted and state.changed, each prefixed com.github.pull_request. and suffixed .v1. Each SSE message has an id, an event naming the type, and data holding a single-line JSON CloudEvent.

The contract is lossy on purpose

These are the delivery properties, all from the docs:

  • There is no replay. You don’t get events from before the stream opens, from “a short window immediately after you create the subscription,” or from any period when you’re disconnected. No resume mechanism is documented, nothing like Last-Event-ID.
  • Events can repeat and arrive out of order: “You may receive the same event more than once, and events for one pull request can arrive out of order.”
  • Payloads are thin: “Events currently tell you that something changed, not what changed.” A state.changed event doesn’t say whether the PR was merged or closed. A review.submitted event doesn’t carry the review state.
  • A subscription expires 24 hours after creation. Each token can hold 100 active ones.
  • A stream opened with an installation token ends at the next access check after the token’s one-hour expiry.

Put together, an event can wake your client up and that’s all it is good for. It can’t serve as a source of truth. Anything you’d want to decide, such as whether CI is green, whether the PR is approved, or whether it merged, has to come from a fresh read of the REST API. Kubernetes controllers work the same way: watch events trigger a reconcile, and the reconcile reads current state rather than replaying deltas.

The docs reach the same conclusion: “Fetch the current state from api_url rather than relying on event order.” The two places where the guidance needs care are both about check suites.

Deduplicating by api_url can drop a re-run

The docs suggest: “For review and check suite events, api_url identifies the review or check suite, so you can use it to deduplicate.” For check suite events, api_url points at the check suite. The same page also says “a re-run produces another [event] when it completes.”

Those two statements conflict once a suite is re-run in place. GitHub’s Rerequest a check suite endpoint takes an existing check_suite_id, and the docs for it say the suite’s “status is reset to queued and the conclusion is cleared.” It’s the same suite with the same ID, so it has the same api_url. If your deduplication is a set of api_url values you have already handled, the event for the first completion (say, failure) gets processed, and the event for the re-run’s completion (say, success) gets thrown away as a duplicate. The agent stays convinced CI is red.

Treating events as wake-ups avoids this, because then deduplication only saves work and never changes an outcome. If you key on anything, key on api_url plus data.conclusion, and still re-read the suite before acting on it.

A conclusion without a commit is not a green build

Check suite events are the one type that carries a result: data.conclusion is always present, with values such as success, failure, cancelled, timed_out, stale or skipped. The docs pitch this as a way to “tell pass from fail without fetching.”

What the documented payload doesn’t promise is which commit the result belongs to. The event fires when a suite “on the pull request’s head commit completes,” but events can arrive out of order. If a developer pushes while CI is running, your client can receive head.changed for the new commit and then, afterwards, a success for a suite that ran on the old one. A client that sets “CI passed” from data.conclusion alone will report the new head as green before any check has run on it.

The check suite object behind api_url has a head_sha. Compare it with the PR’s current head.sha before trusting the conclusion. To decide whether CI is done at all, you need the full list for the head commit (GET /repos/{owner}/{repo}/commits/{ref}/check-suites), because the docs list “no ‘all checks completed’ event” as a known limitation. A PR usually has one suite per app, so one push produces several events.

Gaps you have to close yourself

The subscribe gap is the easiest to forget. Events are dropped for a short window after POST /event-subscriptions, so the docs say to “fetch the pull request once to catch anything you missed.” I would do that after every reconnect too, since a disconnect loses events the same way.

The 24-hour expiry adds another gap. When a subscription expires, the stream ends with a github.stream.end event whose data.outcome is subscription_ended, and you have to create a new subscription, which reopens the post-subscribe window. Renewing early with the same request doesn’t help: if you already have a subscription “with the same pull request and event types, you get that subscription back with 200 OK,” along with its original expires_at. Reading the docs, the only ways to overlap are a different event_types set or accepting the gap and doing the catch-up fetch. Event Streams access is wait-listed, so I haven’t tested which way the server behaves at the boundary.

Merge conflict events depend on someone else’s activity. “GitHub only checks for merge conflicts after someone views or requests the pull request.” The REST reference says the same thing from the other side: a GET on a pull request starts a background job to compute mergeable, which reads null until the job finishes. An agent that only listens may wait a long time for merge_conflict. Its own catch-up fetch is one of the things that triggers the check, so treat mergeable: null as “ask again shortly,” not “no conflict.”

Two cleanup rules are easy to miss. “Deleting a subscription doesn’t close its open streams,” so close the stream before deleting. In GitHub Actions, “a GITHUB_TOKEN can’t be renewed, so delete the subscriptions it created before the job finishes,” or they count against the 100-subscription limit until they expire.

The loop that works

Here is the shape in Python. It’s a sketch built only on documented fields. I haven’t run it against the preview.

def run(pr):
    while True:
        # Returns the existing subscription (200) while it is still active,
        # or a new one (201) after it expires.
        sub = create_subscription(pr, EVENT_TYPES)   # POST /event-subscriptions
        reconcile(pr)                                # close the subscribe or disconnect gap
        outcome = None
        for msg in open_stream(sub["events_url"]):   # SSE; ignore ':' heartbeats
            if msg.event == "github.stream.end":
                outcome = json.loads(msg.data).get("outcome")
                break
            reconcile(pr)                            # any event is a wake-up
        if outcome != "subscription_ended":          # "reconnect", or a drop without stream.end
            backoff_with_jitter()
            refresh_token_if_needed()

def reconcile(pr):
    p = get_pull(pr)                                 # head.sha, state, merged, mergeable
    suites = list_check_suites(p["head"]["sha"])     # only suites on the current head
    reviews = list_reviews(pr)
    decide(p, suites, reviews)                       # idempotent: safe to call repeatedly

reconcile is idempotent and reads everything it needs, so missed events, duplicates and reordering only change how often it runs. They can’t change its result. Coalescing bursts, for example by debouncing a second or two after a push that fires several suite events, saves rate limit and doesn’t affect correctness.

This also shows what Event Streams saves you compared with polling. You still make the same REST calls per change, but you stop making them when nothing has changed. For an agent watching a few dozen PRs that sit idle most of the day, that is most of the polling budget. For a PR that changes constantly, the savings are smaller than the changelog’s “fewer wasted tokens and less rate-limit burn” suggests, because every event still costs a reconcile.

The preview notice says GitHub may make “breaking changes to the API, event types, and limits with short notice.” The .v1 suffix on every event type points the same way. A client built as a reconciler survives that kind of change better, because the only thing it takes from the stream is the fact that something happened.

Sources