Collect structured information across a site on a schedule
Run a recurring site crawl from your chosen seed pages and make broader web research less manual.
Trigger
Recurring schedule
Notis starts this workflow on a schedule, such as daily, weekly, or during business hours.
Action
Start Crawl Job
Tool to spider a site for links and process them with the extract api into a single collection. use when you have seed urls and want to collect structured data across a site. requires a plus plan for crawl api access.
Why this helps
Keeping up with a large site means reopening many pages and piecing together findings by hand.
- Start a crawl from seed URLs on a recurring schedule.
- Process site links through an extraction API into a collection.
- Reduce repetitive collection work across a site.
Setup
Build it in a few focused steps.
- 1Connect Diffbot in the Notis portal and ensure your plan includes crawl API access.
- 2Create a named automation in Automations using New Automation.
- 3Tell Notis to start a crawl from the seed URLs you provide and report the resulting job details.
- 4Choose a recurring schedule and run report destination.
- 5Test with one seed URL and a limited scope.
Questions about this workflow
What plan is needed for crawl access?
Diffbot crawl API access requires a Plus plan.
Where do the seed URLs come from?
Provide the seed URLs in the context available to Notis. The workflow does not assume a separate URL source app.
Will each scheduled run show what changed?
The action starts a crawl and collects structured data. A change report requires prior crawl results or baseline data for comparison.
When this happens · Trigger
Do this · Action
Supported Triggers and Actions
Notis builds workflows that link Agent mail to Diffbot. A trigger fires from one place; an action lands in another.
Agent mail triggers
Diffbot actions
Recurring trigger
Notis starts this workflow on a schedule, such as daily, weekly, or during business hours.
Diffbot Search
Tool to search data extracted by crawl or bulk jobs using dql queries. use after data extraction jobs complete to retrieve search results.
Webhook trigger
Notis starts this workflow when an external tool or custom backend sends an HTTP request.
Get Diffbot Account Details
Tool to retrieve account details, including plan information and usage statistics. use after authenticating to verify subscription and daily quota status.
New Email Received
Trigger for new emails in an AgentMail inbox
Diffbot Analyze
Tool to automatically determine a page's content type and route it to the appropriate extraction api. use when you have only a url and need diffbot to choose the right extractor.
Get Article Data
Tool to extract information from articles, including authors, publication dates, and images. use when you need structured metadata from a web article url.
Get Discussion Thread
Tool to extract threads of content from forums, comment sections, and review pages. use when you need structured discussion data from web pages after identifying the discussion url.
Diffbot Get Event
Tool to extract event details from web pages. use when you need structured event data such as venue, date, and description.
Diffbot Get Image
Tool to extract detailed information about images, including dimensions and recognition data. use after confirming the image url is publicly accessible.
Diffbot Get Product
Tool to extract product information such as specifications, prices, availability, and reviews. use when you need structured product data including specs, pricing, and reviews.
Connect any two apps with Notis in the middle.
Agent mail and Diffbot, or any other combination from 1,000+ integrations.
When this happens · Trigger
Do this · Action
Save your first hour today.
7-day trial of any paid plan, with 20$ of usage included.
No card. Works with personal or business Diffbot.