Compare model versions from Slack
React when the review is ready and get a consistent side by side comparison.
Trigger
Message Reaction Added
Triggered when a reaction is added to a message in Slack. Supports optional filtering by channel and emoji name.
Action
Execute Code
Execute python code in a sandbox and return any result, stdout, stderr, and error. use /home/user folder to write/read files. try to not use plt.show() as the code is executed remotely. use files for image/chart output instead.
Why this helps
Model comparisons are easy to postpone when evaluation steps are manual.
- Makes review repeatable
- Highlights metric changes
- Keeps evaluation tied to the decision
Setup
Build it in a few focused steps.
- 1Connect Slackbot and Codeinterpreter once in Notis.
- 2Create an automation for model version comparisons.
- 3Tell Notis to load the named outputs, calculate the requested comparison metrics, and summarize meaningful differences.
- 4Pick Message Reaction Added as the trigger and choose where run reports go.
- 5Test by reacting to one model review message.
Questions about this workflow
What metrics can it compare?
Specify metrics available in the supplied outputs, such as accuracy, latency, or error rates.
What if one version is missing?
The report should identify the missing input and stop the comparison.
When this happens · Trigger
Do this · Action
Supported Triggers and Actions
Notis builds workflows that link Slackbot to Codeinterpreter. A trigger fires from one place; an action lands in another.
Slackbot triggers
Codeinterpreter actions
Recurring trigger
Notis starts this workflow on a schedule, such as daily, weekly, or during business hours.
Create Sandbox
Create a sandbox to execute python code in a jupyter notebook cell. this is useful for agents to communicate, execute code, see output, read files, write files, etc. it's like you own personal computer, but in the cloud. use /home/user folder to write/read files.
Webhook trigger
Notis starts this workflow when an external tool or custom backend sends an HTTP request.
Execute Code
Execute python code in a sandbox and return any result, stdout, stderr, and error. use /home/user folder to write/read files. try to not use plt.show() as the code is executed remotely. use files for image/chart output instead.
New Channel Created Trigger
Triggered when a new channel is created in Slack.
Get File
Get a file from the sandbox and returns the file. the files should be read from /home/user folder.
Channel Message Received
Triggered when a message is posted in a Slack channel (public, private, or multi-party IM). Does NOT match direct messages.
Run Terminal Command
Run a command in the terminal and returns the stdout, stderr, and error code. use /home/user folder to write/read files.
Direct Message Received
Triggered when a new direct message (DM) is sent to a user in Slack. Catches all DMs across all DM channels.
Upload File
Upload a file to the sandbox environment. the files should be uploaded to the /home/user folder.
Message Reaction Added
Triggered when a reaction is added to a message in Slack. Supports optional filtering by channel and emoji name.
Message Reaction Removed
Triggered when a reaction is removed from a message in Slack. Supports optional filtering by channel and emoji name.
Reaction Added Trigger
DEPRECATED: use `SLACK_MESSAGE_REACTION_ADDED` instead. Triggered when a reaction is added to a message in Slack.
Connect any two apps with Notis in the middle.
Slackbot and Codeinterpreter, or any other combination from 1,000+ integrations.
When this happens · Trigger
Do this · Action
Save your first hour today.
7-day trial of any paid plan, with 20$ of usage included.
No card. Works with personal or business Codeinterpreter.