Spot Braintrust evaluation regressions in Slack
Turn evaluation results into a quick Slack signal, so your team can notice regressions without adding another dashboard check.
Trigger
Webhook received
Notis starts this workflow when an external tool or custom backend sends an HTTP request.
Action
Add reaction to message
Adds a specified emoji reaction to an existing message in a slack channel, identified by its timestamp; does not remove or retrieve reactions.
Why this helps
Evaluation regressions can slip by when teams have to check Braintrust and Slack separately, especially when no one knows which result needs follow-up.
- Surface a regression for a focused team review.
- Reduce manual context switching between evaluation results and Slack.
- Use a baseline supplied by your team rather than assuming one exists.
Setup
Build it in a few focused steps.
- 1Connect Braintrust and Slack once in the Notis portal.
- 2Create an automation in Automations, New Automation, and name it.
- 3In one instruction, ask Notis to review the evaluation result and add a chosen reaction to the relevant Slack message when it meets your regression criteria; provide the baseline and message reference or channel context.
- 4Choose the notis_webhook trigger and select where run reports should go.
- 5Test with one real evaluation result and confirm the Slack reaction appears on the intended message.
Questions about this workflow
Does Braintrust provide an event trigger here?
No Braintrust trigger event is listed for this workflow, so it starts from a Notis webhook. Send an HTTP request to start it when your process is ready to review an evaluation.
Can Notis decide whether a result is a regression?
It can compare returned results with criteria and a baseline you provide. It should not assume an unavailable historical baseline.
Does the Slack action post a new alert?
No. This action adds a reaction to an existing message. Provide enough message or channel context for Notis to identify the intended message.
When this happens · Trigger
Do this · Action
Supported Triggers and Actions
Notis builds workflows that link Braintrust to Slack. A trigger fires from one place; an action lands in another.
Braintrust triggers
Slack actions
Recurring trigger
Notis starts this workflow on a schedule, such as daily, weekly, or during business hours.
Set snooze duration
Deprecated: turns on do not disturb mode for the current user, or changes its duration. use `set dnd duration` instead.
Webhook trigger
Notis starts this workflow when an external tool or custom backend sends an HTTP request.
Add a custom emoji to a Slack team
Deprecated: adds a custom emoji to a slack workspace given a unique name and an image url. use `add emoji` instead.
Add an emoji alias
Adds an alias for an existing custom emoji in a slack enterprise grid organization.
Add a remote file
Adds a reference to an external file (e.g., google drive, dropbox) to slack for discovery and sharing, requiring a unique `external id` and an `external url` accessible by slack.
Add a star to an item
Stars a channel, file, file comment, or a specific message in slack.
Add call participants
Registers new participants added to a slack call.
Add emoji
Adds a custom emoji to a slack workspace given a unique name and an image url; subject to workspace emoji limits.
Add reaction to message
Adds a specified emoji reaction to an existing message in a slack channel, identified by its timestamp; does not remove or retrieve reactions.
Connect any two apps with Notis in the middle.
Braintrust and Slack, or any other combination from 1,000+ integrations.
When this happens · Trigger
Do this · Action
Save your first hour today.
7-day trial of any paid plan, with 20$ of usage included.
No card. Works with personal or business Slack.