Crawl Progress Monitored Automatically Without Polling
Update a crawl monitoring issue in Linear, and automatically check active crawl status and report progress.
Trigger
Issue Updated Trigger
Triggered when an issue is updated. For example labels are changed, issue status is changed, etc.
Action
Check Crawl Status
Tool to check the processing status of a crawl job using snapshot id. call before attempting to download results to ensure data collection is complete.
Why this helps
Manually checking crawl status disrupts focus, and incomplete monitoring leads to missed completion windows and delayed data pipeline handoffs.
- Crawl status checked automatically
- No manual polling required
- Progress updates arrive without interruption
Setup
Build it in a few focused steps.
- 1Connect Linear and Brightdata to Notis.
- 2Create an automation with this prompt: 'When a crawl monitoring issue is updated, check the status of the crawl job and post a status report with progress percentage and estimated time.'
- 3Set the trigger to Issue Updated.
- 4Post status updates as a comment on the issue.
- 5Test by updating a crawl monitoring issue and verify status appears.
Questions about this workflow
How often is status checked?
Status is checked when you update the issue, or you can set up recurring checks (every 30 min, hourly, etc.) if needed.
What status information is included?
Progress percentage, current page count, estimated completion time, and any error messages.
Can you notify when a crawl completes?
Yes, set up a notification so your team is alerted the moment a crawl finishes.
When this happens · Trigger
Do this · Action
Supported Triggers and Actions
Notis builds workflows that link Linear to Brightdata. A trigger fires from one place; an action lands in another.
Linear triggers
Brightdata actions
Recurring trigger
Notis starts this workflow on a schedule, such as daily, weekly, or during business hours.
Trigger Site Crawl
Tool to trigger a site crawl job to extract content across multiple pages or entire domains. use when you need to start a crawl for a given dataset and list of urls.
Webhook trigger
Notis starts this workflow when an external tool or custom backend sends an HTTP request.
Browse Available Scrapers
Tool to list all available pre-made scrapers (datasets) from bright data's marketplace. use when you need to browse available data sources for structured scraping.
Comment Received Trigger
Triggered when a comment is received.
Filter Dataset
Tool to apply custom filter criteria to a marketplace dataset (beta). use after selecting a dataset to generate a filtered snapshot.
Issue Created Trigger
Triggered when a new issue is created.
Get Available Cities
Tool to get available static network cities for a given country. use when you need to configure static proxy endpoints after selecting a country.
Issue Updated Trigger
Triggered when an issue is updated. For example labels are changed, issue status is changed, etc.
Get Available Countries
Tool to list available countries and their iso 3166-1 alpha-2 codes. use when you need to configure zones with valid country codes before provisioning proxies.
Private Team Comment Created
Fires when a new comment is posted on an issue in a private Linear team (polled with the connected user's token).
Download Scraped Data
Tool to retrieve the scraped data from a completed crawl job by snapshot id. use after triggering a crawl or filtering a dataset to download the collected data.
Private Team Issue Created
Fires when a new issue appears in a private Linear team (polled with the connected user's token).
Check Crawl Status
Tool to check the processing status of a crawl job using snapshot id. call before attempting to download results to ensure data collection is complete.
Private Team Issue Properties Updated
Fires when properties on an issue change in a private Linear team (polled with the connected user's token).
List Unlocker Zones
Tool to list your configured web unlocker zones and proxy endpoints. use to view available zones for web scraping and bot protection bypass.
Connect any two apps with Notis in the middle.
Linear and Brightdata, or any other combination from 1,000+ integrations.
When this happens · Trigger
Do this · Action
Save your first hour today.
7-day trial of any paid plan, with 20$ of usage included.
No card. Works with personal or business Brightdata.