How Modem organizes ingested data
Modem’s processing pipeline works on three things:The protocol
Every integration sends the same three things, in this order.1
Create the source container
Send a
source_container.created event the first time you see a channel, ticket, issue, or thread. This is a full snapshot: kind, identity, title, description, and lifecycle state.2
Send messages that reference it
Send a
message.created event for each post or comment. Every message carries a source_container: { kind, external_id } reference pointing at the source container it belongs to.3
Keep the source container up to date
When the source container changes in your tool (a ticket is closed, an issue is reopened, a channel is archived), send a
source_container.updated event with the new snapshot. State changes are evidence Modem uses when it assesses the lifecycle of the topics that source container contributes to.Messages are queued against the source container they reference, and processing starts once that source container exists. It is fine for
a message to arrive before its
source_container.created event; it waits. See Ordering and failure
modes.Authentication
Send your project key in thex-modem-public-key header.
- Open your organization in the Modem dashboard.
- Go to Organization → Projects.
- Open the project you want to ingest into.
- In Public Keys, create a key or copy an existing active key.
Despite the name, treat this key like a server-side bearer credential. Do not ship it in browser code or mobile apps.
Endpoint
Event envelope
Modem assigns
received_at internally when the event is accepted. Do not send it.
created_at is the observation time: the moment your integration looked at the thing it is describing. For a webhook, that is when
the webhook arrived. For a poll, that is when you fetched the record. Never stamp it later than the moment you looked (for example, with
the time you dequeued the event), and never stamp a historical import with “now”. Modem orders source container snapshots by this value,
and a later timestamp on stale data will overwrite fresher data.client
client.name identifies the producer that submitted the event.
- Use a lowercase package-style identifier such as
@your-company/helphub-sync. @modem/*is reserved for Modem-managed integrations.client.nameis descriptive metadata. It is not a trusted provenance boundary.
client.version must be a semantic version, for example 1.0.0.
Event types
All six share the envelope above. Only
body differs.
Choosing a source_name
source_name is the identity of a source inside your organization. It appears on every source container and message event from that source, and Modem hashes it into the deterministic IDs of everything the source produces.
- Use a stable lowercase slug, usually the name of the tool:
helphub,bugtrack,chatterbox. - Use separate names when one upstream app feeds Modem two distinct streams you want to keep apart:
helphub-support,helphub-sales. - Do not prefix custom sources with
custom_. - Built-in integration names (
slack,discord,github,linear,intercom, and so on) are reserved. Sending them with a non-Modemclient.namereturns400. - Do not rename a source later. A new slug is a new source identity, and existing source containers and messages will not carry over.
Responses
Successful requests return200 OK. The event is queued; processing happens asynchronously.
A
200 means the event was accepted, not that it was processed. Schema validation happens synchronously; everything after that is asynchronous.
Billing
Message events and source container events both count toward your ingest usage. Send one source container event per source container (plus one per lifecycle change), not one per message.Source container events are not billed today. This is temporary; they will be billed alongside message events once the current pipeline
rollout completes.
Current limits
- Pull request, transcript, and social feed source containers are produced by Modem’s built-in integrations only. They are not yet available through this API. See Source container kinds.
- This endpoint is not yet part of the OpenAPI spec. The rest of the public API is documented under API Reference.
client.nameprovenance is not enforced by auth yet.
Next steps
Source Containers
Source container kinds, identity, lifecycle state, and the
source_container.* events.Messages
The message payload, the source container reference, threading, and deduplication.
Integrations
Modem’s built-in integrations, which use this same model internally.
Security & Privacy
The security model for data sent to Modem.