Set up monitoring for course outcome data
Give recurring outcome monitoring a clear starting point when new graded work arrives, using the deployment details you provide.
Trigger
Canvas Assignment Graded
Polls Canvas for newly graded submissions for a specific assignment.
Action
Create Batch Monitoring Job Definition
Tool to create a Batch Monitoring job definition for tracking deployment performance and data drift. Use when you need to set up scheduled or manual monitoring jobs for a DataRobot deployment.
Why this helps
Monitoring setup can be postponed when course outcome data changes but nobody translates that signal into a DataRobot job definition.
- Prompt a monitoring setup when new grade data arrives
- Keep monitoring configuration tied to a known deployment
- Reduce forgotten setup follow-up
Setup
Build it in a few focused steps.
- 1Connect Canvas and DataRobot once in the Notis portal.
- 2Create an automation in Automations, New Automation, and name it.
- 3In one instruction, ask Notis to create a batch monitoring job definition for the deployment and data source you specify, and report any missing configuration.
- 4Pick Canvas Assignment Graded as the trigger and choose where run reports go.
- 5Test with one real grading event and a valid monitoring configuration.
Questions about this workflow
Does the trigger provide a DataRobot deployment or dataset?
No. Supply the deployment and monitoring configuration in the instruction or another accessible source.
Does this run a monitoring job immediately?
This action creates a job definition. It is distinct from creating a batch monitoring job.
When this happens · Trigger
Do this · Action
Supported Triggers and Actions
Notis builds workflows that link Canvas to Datarobot. A trigger fires from one place; an action lands in another.
Canvas triggers
Datarobot actions
Recurring trigger
Notis starts this workflow on a schedule, such as daily, weekly, or during business hours.
Add Users to Group
Tool to add one or more users to a DataRobot user group by groupId. Use when you have a valid groupId and want to add existing usernames. Limit 100 users per request.
Webhook trigger
Notis starts this workflow when an external tool or custom backend sends an HTTP request.
Add User to Organization
Tool to add a user to an existing organization. Use when you have an organizationId and wish to add or create a user within it.
Canvas Assignment Graded
Polls Canvas for newly graded submissions for a specific assignment.
Analyze Dataset Definition
Tool to analyze a dataset definition by ID. Use when you need to trigger an analysis job that retrieves dataset metadata, schema, and statistics.
New Canvas Assignment Submission
Polls Canvas for new submissions to a specific assignment.
Archive Model Package
Tool to archive a DataRobot model package. Use when you need to archive a model package that is no longer actively used.
New Canvas Discussion Message
Polls Canvas discussion topics for new messages.
Build Java Scoring Code Package
Initiates an asynchronous build of a Java JAR package containing DataRobot Scoring Code for a deployment. The JAR can be executed locally for predictions or used as a library in Java applications. Use after confirming the deployment is active and supports scoring code. Poll the returned statusId or location URL to track build progress.
New Canvas Discussion Topic
Polls a Canvas course for new discussion topics.
Cancel Notebook Job
Tool to cancel a running or pending notebook job execution. Use when you need to stop a notebook job that is currently executing or queued. The cancellation is immediate and cannot be undone.
New Course Created for User
Polls Canvas for newly created courses for a specific user.
Cancel Project Job
Tool to cancel a pending job for a project. Use when you need to stop a queued or running job after confirming its ID.
New Canvas File Upload
Polls a Canvas course for new file uploads.
Check Project Status
Tool to check the status of a DataRobot project. Use after creating or loading a project to monitor its stage and Autopilot completion.
Connect any two apps with Notis in the middle.
Canvas and Datarobot, or any other combination from 1,000+ integrations.
When this happens · Trigger
Do this · Action
Save your first hour today.
7-day trial of any paid plan, with 20$ of usage included.
No card. Works with personal or business Datarobot.