Put meaningful feedback schema changes on the radar
When your feedback structure changes in Notion, let Notis start a project to review its impact on evaluation work.
Trigger
Data Source Schema Updated
Triggers when a Notion data source's schema is updated. Fires on column add / remove / rename. Payload includes `data.updated_properties: [{id, name, action}]` so consumers can discriminate the kind of change downstream. Optional `data_source_id` filter scopes to schema changes on a single data source. When omitted, fires for any schema change in the workspace the integration has access to. Note: adding a column also fires `page.properties_updated` once per existing row in the data source. Customers wanting a single structural-change signal should use this trigger.
Action
Create Project
This tool creates a new project in humanloop. it is an independent action that generates a project by accepting a project's name (required), an optional description, and an optional organization id. upon execution, it returns details of the created project, including the project's id, name, description, created at timestamp, and organization id.
Why this helps
Changes to how feedback is collected can go unnoticed until they affect analysis or operations.
- Create a follow-up space for significant schema changes.
- Make structural updates easier to review deliberately.
- Avoid relying on memory to revisit collection changes.
Setup
Build it in a few focused steps.
- 1Connect Notion and Humanloop to Notis once.
- 2Create an automation instructing Notis to create a Humanloop project when a meaningful AI feedback schema change needs evaluation review.
- 3Select Data Source Schema Updated and scope it to the feedback data source if known.
- 4Choose a report channel and test with one actual schema change.
Questions about this workflow
Can this distinguish a rename from a new property?
Notion includes updated property details and action types in the event. Describe which kinds of changes should prompt a project.
Will adding a column also trigger row updates?
Notion may emit page property updates for existing rows as well. This workflow uses the schema change trigger as the structural signal.
When this happens · Trigger
Do this · Action
Supported Triggers and Actions
Notis builds workflows that link Notion to Humanloop. A trigger fires from one place; an action lands in another.
Notion triggers
Humanloop actions
Recurring trigger
Notis starts this workflow on a schedule, such as daily, weekly, or during business hours.
Create Project
This tool creates a new project in humanloop. it is an independent action that generates a project by accepting a project's name (required), an optional description, and an optional organization id. upon execution, it returns details of the created project, including the project's id, name, description, created at timestamp, and organization id.
Webhook trigger
Notis starts this workflow when an external tool or custom backend sends an HTTP request.
Delete Project
This tool allows you to delete a specific project from your humanloop organization. the deletion is permanent and cannot be undone. all associated data, including sessions, datapoints, and evaluations linked to the project, will be permanently removed.
All Page Events
Triggers when any Notion page is created or updated across the workspace.
List Experiments
This tool retrieves an array of experiments associated with a specific project in humanloop. it requires a project id (starting with 'pr ') and returns details including experiment id, name, description, creation timestamp, status, configuration details, and metrics/results. it is useful for monitoring experiments, analyzing results, tracking model configurations, and comparing experimental setups.
Comment Created
Triggers when a new comment is created in Notion. Optional `page_id` filter scopes to comments on a specific page. When omitted, fires for any new comment in the workspace the integration has access to. Requires the 'Read comments' capability on the Notion integration. If a connection was authorized before that capability was enabled, the user must re-authorize the connection for comment events to flow.
List Sessions
This tool retrieves a paginated list of sessions for a specific project in humanloop. it requires a project id (and optionally, page and size for pagination) and returns session details such as id, reference id, project information, datapoints count, first inputs, last output, created at, and updated at. this enables users to monitor and analyze historical project interactions.
New Comment
Triggers when a new comment is added to a specified Notion block or page.
Database Created
Triggers when a new Notion database (the container) is created. A database is the post-2025-09-03 container that holds one or more data sources. This trigger fires for the container's creation event (`database.created`), distinct from `NOTION_DATASOURCE_CREATED` which fires when a new data source is added to an existing database. Most customers calling Notion's `POST /v1/databases` (the legacy API) or creating a database via the Notion UI will see this event. Adding a new data source to an existing database fires `data_source.created` instead — use `NOTION_DATASOURCE_CREATED` for that. Notion's payload puts `entity.type: "block"` (the container is a `child_database` block in the content tree) and `entity.id` is the database id.
Data Source Created
Triggers when a new Notion data source is created. Fires workspace-wide. The payload's `data.parent` carries the data source's tree parent (typically the teamspace) for downstream filtering. A single template-based database creation can fire multiple `data_source.created` events at once — one per data source the template instantiates.
Data Source Schema Updated
Triggers when a Notion data source's schema is updated. Fires on column add / remove / rename. Payload includes `data.updated_properties: [{id, name, action}]` so consumers can discriminate the kind of change downstream. Optional `data_source_id` filter scopes to schema changes on a single data source. When omitted, fires for any schema change in the workspace the integration has access to. Note: adding a column also fires `page.properties_updated` once per existing row in the data source. Customers wanting a single structural-change signal should use this trigger.
Connect any two apps with Notis in the middle.
Notion and Humanloop, or any other combination from 1,000+ integrations.
When this happens · Trigger
Do this · Action
Save your first hour today.
7-day trial of any paid plan, with 20$ of usage included.
No card. Works with personal or business Humanloop.