Find duplicate records in an extraction
Check whether the same extracted item appears more than once before it moves into reporting or review.
Trigger
Webhook received
Notis starts this workflow when an external tool or custom backend sends an HTTP request.
Action
Execute ClickHouse Query
Runs a database query and returns the results.
Why this helps
Duplicates can inflate counts and distract from the real findings in an otherwise useful extraction.
- Identify repeated keys in supplied extraction data.
- Reduce manual duplicate scanning.
- Keep the duplicate rule explicit and reviewable.
Setup
Build it in a few focused steps.
- 1Connect Agenty and ClickHouse once in the Notis portal.
- 2Create an automation and specify the extraction scope, uniqueness key, and duplicate summary you want.
- 3Pick a webhook trigger for the HTTP request that starts the run.
- 4Choose where Notis should send run reports.
- 5Test with a sample extraction that includes a known duplicate.
Questions about this workflow
Will this remove duplicate rows?
No. The query action identifies duplicates and returns results; it does not modify records.
Can it infer the unique key?
State which fields define a duplicate. The correct key depends on your dataset.
When this happens · Trigger
Do this · Action
Supported Triggers and Actions
Notis builds workflows that link Agenty to ClickHouse. A trigger fires from one place; an action lands in another.
Agenty triggers
ClickHouse actions
Recurring trigger
Notis starts this workflow on a schedule, such as daily, weekly, or during business hours.
Execute ClickHouse Query
Runs a database query and returns the results.
Webhook trigger
Notis starts this workflow when an external tool or custom backend sends an HTTP request.
Get Database Schema
Retrieves an overview of a database's schema and tables.
Get Table Schema
Gets the schema and column details for a table.
Get ClickHouse Play Interface
Returns the ClickHouse Play interface for writing queries and viewing results.
List ClickHouse Databases
Lists the databases in an instance.
List ClickHouse Tables
Lists tables in databases with basic details.
Connect any two apps with Notis in the middle.
Agenty and ClickHouse, or any other combination from 1,000+ integrations.
When this happens · Trigger
Do this · Action
Save your first hour today.
7-day trial of any paid plan, with 20$ of usage included.
No card. Works with personal or business ClickHouse.