Google Cloud

Nistor's reference integration. It inventories your Google Cloud data estate and builds lineage from what actually ran, not from what someone documented.

What Nistor collects

• Assets of 25 types: BigQuery datasets and tables (linked datasets and tables written by Pub/Sub subscriptions included), Dataform, Data Fusion instances and pipelines, Cloud Storage, Pub/Sub, Composer (Airflow), Cloud Run services and jobs, Cloud Functions, Cloud Scheduler, Cloud SQL, Compute Engine, Vertex AI, Bigtable instances and tables, Secret Manager and Artifact Registry. Google Sheets come in through the Google Workspace connector.
• Lineage from BigQuery job audit logs, down to the column, plus links inferred between services.
• Runs of Dataform and Data Fusion.
• Cost, with FinOps (Enterprise): bytes billed per job, storage tiers and Cloud Storage usage, in US dollars per asset, team and user.
• Checks, with Data quality (Enterprise): Dataplex data-scan results and Dataform assertion results come in, Nistor's own checks run on BigQuery and Cloud Storage, and failed checks become incidents.

What it can run

Rules can start Dataform runs, Data Fusion pipelines, BigQuery copies and exports to Cloud Storage the moment their upstream data is ready.

Set it up

Audit logs reach Nistor through a Pub/Sub subscription, and the collectors call the Google Cloud APIs with a service account in your project. The deployment guide lists the roles and the Terraform that creates them.