Catch model benchmark regressions before they disrupt your work
Get a focused Jira issue when a model you rely on misses the benchmark threshold you set.
Trigger
Recurring schedule
Notis starts this workflow on a schedule, such as daily, weekly, or during business hours.
Action
Create Issue
Creates a new jira issue (e.g., bug, task, story) in a specified project.
Why this helps
Benchmark changes are easy to miss when you have to check model rankings by hand.
- Review model benchmarks on a predictable schedule.
- Turn only threshold breaches you define into Jira work.
- Keep investigation details in the issue for later follow-up.
Setup
Build it in a few focused steps.
- 1Connect Artificial Analysis and Jira once in the Notis portal.
- 2Create an automation and give it a name.
- 3Tell Notis to review the models and benchmark thresholds you specify, then create a Jira issue for each breach.
- 4Pick a recurring schedule and choose where run reports should go.
- 5Test with one real benchmark example and confirm the issue is useful.
Questions about this workflow
Can Notis decide what counts as a regression?
Provide the benchmark and threshold you want monitored. Notis should not assume a baseline you have not supplied.
How often does it run?
Choose a recurring schedule when you set up the automation.
When this happens · Trigger
Do this · Action
Supported Triggers and Actions
Notis builds workflows that link Artificial Analysis to Jira. A trigger fires from one place; an action lands in another.
Artificial Analysis triggers
Jira actions
Recurring trigger
Notis starts this workflow on a schedule, such as daily, weekly, or during business hours.
Add Attachment
Uploads and attaches a file to a jira issue.
Webhook trigger
Notis starts this workflow when an external tool or custom backend sends an HTTP request.
Add Comment
Adds a comment using atlassian document format (adf) for rich text to an existing jira issue.
Add Watcher to Issue
Adds a user to an issue's watcher list by account id.
Assign Issue
Assigns a jira issue to a user, default assignee, or unassigns; supports email/name lookup.
Bulk Create Issues
Creates multiple jira issues (up to 50 per call) with full feature support including markdown, assignee resolution, and priority handling.
Create Issue
Creates a new jira issue (e.g., bug, task, story) in a specified project.
Link Issues
Links two jira issues using a specified link type with optional comment.
Create Project
Creates a new jira project with required lead, template, and type configuration.
Connect any two apps with Notis in the middle.
Artificial Analysis and Jira, or any other combination from 1,000+ integrations.
When this happens · Trigger
Do this · Action
Save your first hour today.
7-day trial of any paid plan, with 20$ of usage included.
No card. Works with personal or business Jira.