Set up evaluation for private AI work
Private AI initiatives still need an evaluation home. Notis can create a Humanloop project when a relevant issue appears in a private Linear team.
Trigger
Private Team Issue Created
Fires when a new issue appears in a private Linear team (polled with the connected user's token).
Action
Create Project
This tool creates a new project in humanloop. it is an independent action that generates a project by accepting a project's name (required), an optional description, and an optional organization id. upon execution, it returns details of the created project, including the project's id, name, description, created at timestamp, and organization id.
Why this helps
Private initiative setup can be easy to postpone when founders already have a long list of launch details to manage.
- Create a consistent evaluation starting point for private work.
- Reduce setup friction while preserving private-team visibility rules.
- Keep early evaluation planning connected to the initiative issue.
Setup
Build it in a few focused steps.
- 1Connect Linear and Humanloop to Notis once in the portal, ensuring the connected Linear account can see the private team.
- 2Create an automation in the portal, or ask Notis to create one in conversation.
- 3Describe in one instruction which new private-team AI initiative issues should create a Humanloop project.
- 4Select the Private Team Issue Created trigger and a channel for run reports.
- 5Test with one real eligible issue and verify the project was created.
Questions about this workflow
What determines which private issues Notis can see?
The private-team event polls with the connected user's Linear token, so access follows the visibility of that account.
Will this create a project for every private issue?
Explain in the instruction that the workflow applies only to issues representing an AI initiative.
When this happens · Trigger
Do this · Action
Supported Triggers and Actions
Notis builds workflows that link Linear to Humanloop. A trigger fires from one place; an action lands in another.
Linear triggers
Humanloop actions
Recurring trigger
Notis starts this workflow on a schedule, such as daily, weekly, or during business hours.
Create Project
This tool creates a new project in humanloop. it is an independent action that generates a project by accepting a project's name (required), an optional description, and an optional organization id. upon execution, it returns details of the created project, including the project's id, name, description, created at timestamp, and organization id.
Webhook trigger
Notis starts this workflow when an external tool or custom backend sends an HTTP request.
Delete Project
This tool allows you to delete a specific project from your humanloop organization. the deletion is permanent and cannot be undone. all associated data, including sessions, datapoints, and evaluations linked to the project, will be permanently removed.
Comment Received Trigger
Triggered when a comment is received.
List Experiments
This tool retrieves an array of experiments associated with a specific project in humanloop. it requires a project id (starting with 'pr ') and returns details including experiment id, name, description, creation timestamp, status, configuration details, and metrics/results. it is useful for monitoring experiments, analyzing results, tracking model configurations, and comparing experimental setups.
Issue Created Trigger
Triggered when a new issue is created.
List Sessions
This tool retrieves a paginated list of sessions for a specific project in humanloop. it requires a project id (and optionally, page and size for pagination) and returns session details such as id, reference id, project information, datapoints count, first inputs, last output, created at, and updated at. this enables users to monitor and analyze historical project interactions.
Issue Updated Trigger
Triggered when an issue is updated. For example labels are changed, issue status is changed, etc.
Private Team Comment Created
Fires when a new comment is posted on an issue in a private Linear team (polled with the connected user's token).
Private Team Issue Created
Fires when a new issue appears in a private Linear team (polled with the connected user's token).
Private Team Issue Properties Updated
Fires when properties on an issue change in a private Linear team (polled with the connected user's token).
Connect any two apps with Notis in the middle.
Linear and Humanloop, or any other combination from 1,000+ integrations.
When this happens · Trigger
Do this · Action
Save your first hour today.
7-day trial of any paid plan, with 20$ of usage included.
No card. Works with personal or business Humanloop.