Cotool + Databricks
Connect Databricks to Cotool so agents discover Unity Catalog tables and run read-only SQL over your security lakehouse for triage, hunts, and detections.
The short answer
The Databricks integration lets Cotool agents treat your lakehouse as a security data source: they browse Unity Catalog catalogs, schemas, and tables, inspect columns and sample rows, and run read-only Databricks SQL through a SQL warehouse for investigations, hunts, and code detections.[1]
How it connects: Cotool calls the SQL Statement Execution and Unity Catalog REST APIs as a service principal with OAuth machine-to-machine credentials. A personal access token works as a fallback. Databricks SQL has no read-only session mode, so the service principal's grants keep Cotool read-only.[1][6][7]
What Cotool uses from Databricks
- Unity Catalog metadata
- Agents list catalogs, then schemas, then tables, and describe only the tables they need. The metadata APIs don't wake the SQL warehouse; sampling rows does.[1]
- SQL statements
- Queries run through the Statement Execution API on a SQL warehouse, not an all-purpose cluster. Cotool returns 500 rows by default and at most 10,000, with a 4 MiB result cap and a
truncatedflag when more rows matched.[1][7] - Partition and clustering columns
- For log tables, agents see the event time column and the partition or clustering columns, and are told to filter on them first so scans stay small.[1]
- Legacy Hive metastore tables
- Tables in
hive_metastorealso work. Listing them usesSHOWstatements, which briefly start the warehouse.[1]
Detections, hunts, and triage
Example query
SELECT actor.alternateId AS user,
client.ipAddress AS ip,
COUNT(*) AS failures
FROM security.okta.system_log
WHERE event_date >= current_date() - INTERVAL 1 DAY
AND outcome.result = 'FAILURE'
GROUP BY 1, 2
ORDER BY failures DESC
LIMIT 100The table name is an example. The first filter is on the partition column, which is what Cotool tells agents to do on large log tables. Replace it with your own catalog, schema, and date column.[1]
Setup and permissions
Create a service principal, give it the Databricks SQL access entitlement, and generate an OAuth secret on it with a lifetime of up to 730 days. Databricks shows the secret value once. On Azure Databricks, the secret must come from Databricks, not from an Entra ID app registration.[1][6]
| Permission | Access | Why |
|---|---|---|
| Databricks SQL accessEntitlement that grants access to Databricks SQL features, including SQL warehouses. Without it the principal can't use any warehouse.[1][9] | Read | Entitlement that grants access to Databricks SQL features, including SQL warehouses. Without it the principal can't use any warehouse.[1][9] |
| CAN USEPermission on the SQL warehouse Cotool queries through. Cotool recommends a dedicated serverless warehouse.[1][10] | Read | Permission on the SQL warehouse Cotool queries through. Cotool recommends a dedicated serverless warehouse.[1][10] |
| USE CATALOG, USE SCHEMA, SELECTUnity Catalog grants on the catalog, schemas, or tables that hold your security data.[1][8] | Read | Unity Catalog grants on the catalog, schemas, or tables that hold your security data.[1][8] |
| sql and unity-catalog scopesScopes on the OAuth secret. A scoped secret can't mint tokens beyond its scopes, so Cotool doesn't need all-apis.[1][6] | Read | Scopes on the OAuth secret. A scoped secret can't mint tokens beyond its scopes, so Cotool doesn't need all-apis.[1][6] |
How Cotool works with Databricks
- After you connect, and weekly after that, Cotool maps every catalog, schema, and table the service principal can read, each with a one-sentence summary. Cotool writes the summaries from names, comments, and column names and never reads rows to build the map.[1]
- Each statement has a timeout of 300 seconds by default and at most 30 minutes, and Cotool cancels statements that exceed it so abandoned queries don't keep running.[1]
- Cotool rebuilt the Databricks integration in release 0.73.0 with faster read-only queries, service principal authentication, and step-by-step table discovery.[2]
FAQ
- Can Cotool use an all-purpose Databricks cluster?
- No. Cotool queries through a SQL warehouse. If your HTTP path starts with
sql/protocolv1/o/, it belongs to a cluster; create or pick a SQL warehouse instead.[1] - Why does Cotool recommend a serverless warehouse?
- Serverless warehouses start in seconds, while classic and pro warehouses can take several minutes after auto-stop, and Cotool's first queries after an idle period may time out while they start.[1]
Sources
- Databricks Integration (Cotool docs) · accessed November 16, 2026
- Changelog (Cotool docs) · accessed November 16, 2026
- Detections are code (Cotool Detect) · accessed November 16, 2026
- Hunt threats continuously (Cotool Hunt) · accessed November 16, 2026
- Respond to every signal (Cotool Respond) · accessed November 16, 2026
- Authorize service principal access with OAuth (Databricks docs) · accessed November 16, 2026
- Statement Execution API: Run SQL on warehouses (Databricks docs) · accessed November 16, 2026
- Manage privileges in Unity Catalog (Databricks docs) · accessed November 16, 2026
- Manage entitlements (Databricks docs) · accessed November 16, 2026
- Access control lists (Databricks docs) · accessed November 16, 2026

