Keep private feedback usable for evaluation
Capture a concrete failure scenario from a private discussion where the team can test it again.
Trigger
Private Team Comment Created
Fires when a new comment is posted on an issue in a private Linear team (polled with the connected user's token).
Action
Add datapoints to dataset
Tool to add datapoints to a dataset. Use when you need to append multiple entries with specified input, ground truth, and history mappings.
Why this helps
Private issue feedback may be hard to revisit, and useful failure details can disappear when a thread is no longer active.
- Preserves useful failure scenarios from private issue discussions.
- Reduces repeated manual transcription of sensitive team context.
- Keeps evaluation material accessible to the authorized connected account.
Setup
Build it in a few focused steps.
- 1Connect Linear and Honeyhive to Notis once in the portal.
- 2Ask Notis to add a concrete private-team AI failure scenario and expected behavior to the matching Honeyhive dataset, using only accessible issue context.
- 3Select the Linear Private Team Comment Created trigger and choose a channel for run reports.
- 4Test with one authorized private comment that contains a safe, representative example.
Questions about this workflow
Will this use comments from teams the account cannot access?
No. The private-team event is polled with the connected user's token, so visibility follows that user's Linear access.
Will every private comment be stored as a datapoint?
Instruct Notis to capture only comments with a concrete, useful evaluation scenario.
When this happens · Trigger
Do this · Action
Supported Triggers and Actions
Notis builds workflows that link Linear to Honeyhive. A trigger fires from one place; an action lands in another.
Linear triggers
Honeyhive actions
Recurring trigger
Notis starts this workflow on a schedule, such as daily, weekly, or during business hours.
Add datapoints to dataset
Tool to add datapoints to a dataset. Use when you need to append multiple entries with specified input, ground truth, and history mappings.
Webhook trigger
Notis starts this workflow when an external tool or custom backend sends an HTTP request.
Create Batch Model Events
Tool to create multiple model events in a single request. Use when you need to log a batch of event interactions to HoneyHive.
Comment Received Trigger
Triggered when a comment is received.
Create Batch Tool Events
Tool to log a batch of external API calls as tool events. Use when you need to record multiple tool events in one request—use after gathering all event data.
Issue Created Trigger
Triggered when a new issue is created.
Create Dataset
Tool to create a dataset. Use when you need to initialize a new dataset within a project.
Issue Updated Trigger
Triggered when an issue is updated. For example labels are changed, issue status is changed, etc.
Create Tool
Tool to create a new tool. Use when you need to register a new function or plugin for invocation.
Private Team Comment Created
Fires when a new comment is posted on an issue in a private Linear team (polled with the connected user's token).
Delete Datapoint
Tool to delete a specific datapoint by its ID. Use when you need to remove a datapoint from HoneyHive after confirming its identifier.
Private Team Issue Created
Fires when a new issue appears in a private Linear team (polled with the connected user's token).
Delete Dataset
Tool to delete a dataset by ID. Use when you need to remove a dataset after confirming its ID.
Private Team Issue Properties Updated
Fires when properties on an issue change in a private Linear team (polled with the connected user's token).
End Evaluation Run
Tool to mark an evaluation run as completed. Use after finishing manual evaluations to update the run status to completed.
Connect any two apps with Notis in the middle.
Linear and Honeyhive, or any other combination from 1,000+ integrations.
When this happens · Trigger
Do this · Action
Save your first hour today.
7-day trial of any paid plan, with 20$ of usage included.
No card. Works with personal or business Honeyhive.