Catch test instability before a campaign launch
Get a recurring view of flaky tests in the project behind your campaign release work, so instability is easier to follow up.
Trigger
Recurring schedule
Notis starts this workflow on a schedule, such as daily, weekly, or during business hours.
Action
Get Flaky Tests
Finds tests that passed and failed in the same commit, across branches.
Why this helps
Intermittent test failures can make launch readiness harder to judge and consume time during a busy campaign window.
- Identify tests that have both passed and failed in the same commit.
- Surface project test instability in a scheduled report.
- Make launch readiness conversations more concrete.
Setup
Build it in a few focused steps.
- 1Connect ActiveTrail and CircleCI once in the Notis portal.
- 2Create an automation in Automations, New Automation, and give it a name.
- 3Tell Notis to retrieve flaky tests for the CircleCI project used by ActiveTrail campaign releases and summarize the results.
- 4Pick a recurring schedule and choose where run reports should go.
- 5Test with one real project and review the flaky-test results.
Questions about this workflow
Does this tell me whether the campaign can launch?
No. It reports flaky tests for a CircleCI project. Your team must decide how those results affect launch readiness.
Can it report on several projects?
The action is described as retrieving flaky tests for a project. Specify the project you want reviewed.
When this happens · Trigger
Do this · Action
Supported Triggers and Actions
Notis builds workflows that link ActiveTrail to Circleci. A trigger fires from one place; an action lands in another.
ActiveTrail triggers
Circleci actions
Recurring trigger
Notis starts this workflow on a schedule, such as daily, weekly, or during business hours.
Get Job Artifacts
Retrieves files produced by a job.
Webhook trigger
Notis starts this workflow when an external tool or custom backend sends an HTTP request.
Create Context
Creates a context for sharing environment variables securely across projects.
Get Job Details
Retrieves status, timing, and executor details for a job.
Create Context (GraphQL)
Creates a context for storing environment variables; returns an error if creation fails.
Get Pipeline Config
Retrieves the source or compiled configuration for a pipeline.
Create Context Restriction
Creates a context restriction to limit access by project, rules, or group.
Get Test Metadata
Retrieves test results, run times, and failure details for a job.
Create Organization Orb Allowlist
Allows URL-based orb references with matching prefixes in an organization’s pipelines.
Connect any two apps with Notis in the middle.
ActiveTrail and Circleci, or any other combination from 1,000+ integrations.
When this happens · Trigger
Do this · Action
Save your first hour today.
7-day trial of any paid plan, with 20$ of usage included.
No card. Works with personal or business Circleci.