Track drift in batch predictions fed by browser research
Set up batch monitoring for a defined prediction workflow so changes in performance or data are less likely to go unnoticed.
Trigger
Recurring schedule
Notis starts this workflow on a schedule, such as daily, weekly, or during business hours.
Action
Create Batch Monitoring Job
Create an asynchronous batch monitoring job to track model performance and data drift.
Why this helps
Batch prediction pipelines can keep running while performance changes stay hidden from a busy team.
- Create monitoring for batch prediction performance and data drift.
- Connect the oversight setup to a known Airtop-fed workflow.
- Make the monitoring job creation visible through Notis run reports.
Setup
Build it in a few focused steps.
- 1Connect Airtop and DataRobot once in the Notis portal.
- 2Create an automation in Automations, New Automation, and give it a clear name.
- 3Tell Notis to create a batch monitoring job for the specified prediction workflow and DataRobot deployment.
- 4Pick a recurring schedule, choose where run reports should go, and provide the job context.
- 5Test with a real deployment and batch prediction workflow.
Questions about this workflow
What does the monitoring job track?
The action is described as tracking model performance and data drift on batch predictions at scale.
Does this define a recurring monitoring schedule in DataRobot?
This action creates a batch monitoring job asynchronously. The schedule here controls when Notis asks DataRobot to create that job; it does not establish a recurring DataRobot job definition.
Can this work without a deployment reference?
Provide the applicable deployment and batch prediction context. The action cannot infer missing identifiers.
When this happens · Trigger
Do this · Action
Supported Triggers and Actions
Notis builds workflows that link Airtop to Datarobot. A trigger fires from one place; an action lands in another.
Airtop triggers
Datarobot actions
Recurring trigger
Notis starts this workflow on a schedule, such as daily, weekly, or during business hours.
Add Users to Group
Add one or more existing users to a user group.
Webhook trigger
Notis starts this workflow when an external tool or custom backend sends an HTTP request.
Add User to Organization
Add a user to an existing organization.
Analyze Dataset Definition
Start an analysis job to retrieve a dataset’s metadata, schema and statistics.
Archive Model Package
Archive a model package that is no longer in active use.
Build Java Scoring Code Package
Start an asynchronous build of a Java JAR scoring package for a deployment.
Cancel Notebook Job
Immediately cancel a running or queued notebook job; this cannot be undone.
Cancel Project Job
Cancel a pending project job.
Check Project Status
Check a project’s status and progress.
Connect any two apps with Notis in the middle.
Airtop and Datarobot, or any other combination from 1,000+ integrations.
When this happens · Trigger
Do this · Action
Save your first hour today.
7-day trial of any paid plan, with 20$ of usage included.
No card. Works with personal or business Datarobot.