Head-to-head · updated 13 September 2026
DataShield vs Databricks: who governs the agent once it leaves the lakehouse?
Databricks is the biggest name in data. It earned that spot. It began as Spark. It became the lakehouse. Now it sells the Data Intelligence Platform. The homepage no longer leads with the lakehouse. It leads with Lakebase, serverless Postgres. The line is "One database for AI, apps and agents." Unity Catalog governs the data. Agent Bricks builds the agents. Genie answers the questions. Over 60% of the Fortune 500 are clients. None of that is in doubt here.
We do not build a lakehouse. We will not pretend to. Instead, we are the proof layer for agents, plus authorization. We run on your own hardware. We hold your own keys. Datasets get tokenized at ingest. Agents query tokenized data over MCP. Detokenization is a privileged step. We audit it. We seal each checked call into a hash chain. You can check it yourself. If your agents only touch Databricks: this page is your last mile. Nothing more. If they reach past it, read on.
The short version
Pick DataShield when
- Your agents reach past Databricks. A Postgres box, an S3 drop, a vendor API, a SaaS record. One policy plane. One log for all of it.
- An examiner asks you to prove a log was not edited. Ours is a SHA-256 chain with signed checkpoints. Break it in a browser. Watch the verifier name the exact failure. Try it.
- You need to pull an agent's rights mid-session. The very next tool call should fail closed. How Auth does it.
- You want the policy engine, the vault, and the audit trail on your own hardware. Databricks runs its control plane in the Databricks account, even when your classic compute runs in yours. See the architecture.
Pick Databricks when
- You have real volume. Spark, Delta, Photon, model training at petabyte scale. We do not play here, and we will not pretend.
- You want the agents built for you. Agent Bricks writes the eval suite, tunes the prompt, and picks the model from a task description. AstraZeneca parsed more than 400,000 clinical trial documents in under an hour with it.
- You want one vendor and one bill. Unity Catalog needs no separate licence. Governance becomes a config change, not a purchase order. Our Unity Catalog page goes deeper.
- You want serverless Postgres beside your lake, with branching you can reset like code. Lakebase is a good product. We have nothing like it.
Bottom line: Databricks is where the data lives and where the agents get built. DataShield is where an agent's rights get checked. It is where the proof gets kept, on hardware you own. In Databricks shops, we expect to run alongside, not instead.
Feature by feature
Competitor cells describe what Databricks' site and docs say. As of the date above. If we've mischaracterised something, email support@myorg.ai. We'll correct it, credited.
| What matters | DataShield | Databricks | Edge |
|---|---|---|---|
| Scale and compute | Ontology is a self-hosted dataset platform. It covers ingest, catalog, analysis, transform. Snapshot, lifecycle, watch sources, workflows too. DuckDB runs over Parquet for analytics. It is not an ELT tool. It is not an orchestrator. It is not a lakehouse. | Spark, Delta Lake, Photon. Model training and serving too. All at a scale we do not try. Over 20,000 clients use it. More than 60% of the Fortune 500 does too. | ◇ |
| Catalog and field classification | Providers register over a live connection. We scan and profile each one in place. No row egress. We cover PostgreSQL today. Not your whole estate. Classification runs 129 field classes. Checksum validators back it. Verdicts stay reproducible. Each one carries a config digest. It covers all 18 HIPAA Safe Harbor identifiers. | One metastore spans each workspace, cloud, and region. It covers tables and files. Models, agents, and MCP servers too. That breadth is real. We found no claim of field-level PII classification. Or PHI. Not like this. Not on the Unity Catalog pages. | ◈ |
| Audit evidence | A SHA-256 hash chain. Ed25519-signed checkpoints, themselves chained. Verification returns a named verdict. It tells tampering apart. From insertion, deletion, and truncation. Ed25519 audit-signing keys stay on your instance. They sit in a key directory you control. Key rotation uses kid keys. An unknown key makes it fail closed. KMS custody is a built seam in Auth. So is HSM custody. Not a shipped integration. Public demo. | Detailed audit events land in system tables. Inference tables capture full model call payloads. Is any of it hash-chained? Signed? Checkable by someone outside the platform? We looked. We found none. | ◆ |
| Agent authorization | Each checked tool call runs a fixed pipeline. Before dispatch. Auth gate first, then consented-tool narrowing. Then token scope ceiling. A declared rights tier comes next, then revocation re-check. Then metering. Delegation uses RFC 8693 token exchange. A scope ceiling stays enforced. | Grants and attribute-based policies sit on catalog objects. Unity AI Gateway adds rules over agents and models. Plus MCP servers. It supports on-behalf-of execution. Evaluation happens when a query or call runs. It does not re-derive as a per-call ceiling. | ◆ |
| Break-glass | Scoped, time-boxed emergency access for agents. It needs admin plus IP allowlist plus step-up. It auto-revokes. Nobody can quietly delete it from the log. | Not described in the docs we read. Ask them. | ◆ |
| GDPR erasure | Crypto-shred of the subject's key material. Plus ISO 27560 consent receipts. Actor identities in the chain carry a commitment. An HMAC one. The proof still checks out. Even after the subject is gone. | Delete the rows. That is the method. The docs don't say. What happens to audit history? Not after that. Neither do they cover inference tables. Neither do they cover data. Data copied out of checked tables. Already copied out. | ◆ |
| Tokenization and data handling | We tokenize datasets at ingest. Agents query tokenized data over MCP. Detokenization is a privileged, audited operation. Tokens are deterministic, join-preserving, vault-reversible. We audit or refuse every lookup. | Masking and redaction apply at query time. By rule. Gateway guardrails cover model traffic too. We found no vault-reversible token architecture. None with a checked detokenization path. | ◆ |
| MCP and agents | More than 200 MCP tools. Across Ontology, Auth, Corpus, and Lighthouse. Auth issues MCP tool tokens with scope ceilings. Per-call metering ties to the agent. The one that made the call. | Managed MCP servers cover Genie and AI Search. Plus Databricks SQL. Unity Catalog functions and a code interpreter too. External MCP servers can register as catalog securables. You can host your own, too. As a Databricks app. | ◈ |
| Building and tuning agents | We govern agents. We do not build them for you. No eval harness, no fine-tuning, no model serving. | Agent Bricks generates eval suites. It auto-tunes prompts and models. Reward models too, from a task description. Mosaic AI sits underneath. This is a real gap on our side. It's covered on our Mosaic AI page. | ◇ |
| Data quality | A 20-section profile per dataset. A weighted trust score too. It covers each entity type, domain, and estate. It recomputes hourly. Change and drift detection come with it. No monitors. No alerts. No freshness targets on warehouse tables. | Lakehouse Monitoring watches tables and model endpoints. With metric tables and dashboards. Different job. Mostly a better fit than ours. For pipeline health. Side by side. | ◇ |
| Reach beyond the platform | Platform-agnostic by construction. Auth issues the token. The checked surfaces confirm it on each call. You need no particular compute vendor. | Federation reaches out to outside sources, read-only. With access controls at the table level. Row and column rules stay a Databricks-object story. Your agents do not stop at that line. | ◆ |
| Deployment and hosting | Self-hosted. Your own cloud, or a data center. Or a dedicated single-tenant server we operate. Docker images cover Auth, Ontology, Corpus, and Lighthouse. Guardian verifies a signed deploy manifest. No vendor control plane sits in the loop. | The docs are clear on this. The control plane "is located in the Databricks account, not your cloud account." Classic compute runs in yours. Serverless compute runs in theirs. Worth knowing which of your workloads is which. | ◆ |
| Maturity and pricing | Live in production. Guardian and Lighthouse have run since April 2026. SOC 2 is not yet certified. We say so. Published pricing, plus an instant quote. No call needed. | A very large company. A deep bench. A long compliance record. Pricing is pay-as-you-go, in DBUs, billed per second. No rate card sits on the pricing page. Cloud compute costs bill separately underneath. You will need a finance person. | ◇ |
◆ DataShield leads◇ Databricks leads◈ comparable
Databricks claims come from two places: databricks.com. And docs.databricks.com. We checked last on 13 September 2026. We link them below. Not a paraphrase from memory.
Three things you get here that you won't get from a data platform
A witness that isn't the defendant
Databricks writes the audit record. It stores that record in its own tables. It holds the keys to the storage too. The telemetry is excellent. But the system under review is also its only witness. No second opinion. That's the catch. In any other part of the business, we'd call that weak. A weak control. Just weak. Our chain sits in a separate plane. It has its own signing keys. Edit a record in your browser. The verifier names the exact failure. Try it, no signup.
Authority that can change mid-flight
An analyst resigns on a Friday afternoon. Their agent is forty minutes into a job. With DataShield, the next checked tool call re-checks rights. Right away. It fails closed. A grant change in a catalog. It doesn't reach work now running. How Auth does it.
No vendor in your control plane
Databricks says it plainly. Its control plane runs in the Databricks account. For most buyers, that is fine. Take a bank with a residency rule. Or a health system. They want the proof plane inside the building. For them, it is the whole conversation. We ship Docker images. We ship a signed deploy manifest. You run all of it. How we deploy.
Where Databricks is genuinely stronger
Nearly everywhere. Anywhere that means moving or modelling data at scale. Spark and Delta are the default answer. For a reason. Photon is fast. The training stack has clients. Clients with real results behind it. Agent Bricks is the part I would not underrate. Describe a task. Get a generated eval suite. Plus a tuned agent. Skip a month of prompt fiddling. AstraZeneca's 400,000 clinical trial documents. In under an hour. A vendor-reported number. But it is the right kind of number. Lakebase is a smart land grab too. Put serverless Postgres next to the lake. With branching. Agents get a place to write, not just read. Unity Catalog comes with the tiers. Ones you now pay for. Control arrives as a checkbox. Against that, a second vendor has to justify itself. In one sentence. That is fair.
Here is our sentence. It is narrow on purpose. Databricks runs the agent. It stores the data. It writes the log. And it keeps the log. Four roles, one party. That is fine until the day it matters. And the day it matters? That's the day someone disputes the record. The rest of the gap follows the same shape. Federation out to other systems is read-only. With table-level controls. A call runs, and it checks agent rights. Nobody re-derives it as a ceiling per call. And no one at Databricks signs anything. We are not asking you to replace any of it. We're asking where the proof lives. That's the one thing we're asking. Does the party holding it have an interest? In the answer.
Questions worth asking both of us
These are the questions we'd want answered. If we were buying. Ask them on each vendor call, including ours.
Can you cryptographically prove an audit log entry wasn't deleted?
DataShield: yes. Each record commits to the one before it. We sign and chain checkpoints. Verification tells deletion from truncation from tampering apart. Try it at /verify. Databricks: audit events are rich and queryable. They land in system tables. Inference tables hold the full payload. Of model calls. We went looking for an integrity claim. We found none. Ask them to show you the proof. Not the query. Our longer argument is in the Unity Catalog write-up.
What happens to a revoked agent mid-session?
DataShield re-checks rights on each checked tool call. A revocation bites on the very next call. Databricks evaluates grants and gateway rules. When a call runs. That covers a lot of ground. We found no description of that. A running session getting downgraded. Not in flight. We also found no rights tiers for agents. Ask how long a hacked agent keeps working. After you pull its access. We wrote up the token side of this for Databricks MCP servers.
How does GDPR erasure interact with the audit trail?
DataShield crypto-shreds the subject's key material. It issues an ISO 27560 consent receipt. Actor identities in the chain carry a commitment. An HMAC one. The chain still checks out. Even after the subject is erased. Deleting rows on Databricks is easy. But the docs skip some cases. What does erasure do to your audit history? Your inference tables? Data copied out already? Ask both of us.
Is DataShield a Databricks alternative?
For the lakehouse job, no. Anyone who tells you otherwise is selling. We have no Spark. No Photon. No model serving. Our in-place catalog scanning covers PostgreSQL today. Not your whole estate. What we do replace is an idea. The idea that the platform's own tables. Proof enough, on their own. We also replace an idea. That control can stop at the platform edge. Most Databricks clients who talk to us. They keep what they have. They just add a layer.
Our compute runs in our own cloud account. Doesn't that make Databricks self-hosted?
Not quite. The docs are honest about it. Classic compute runs in your account. The control plane runs in the Databricks account. Serverless compute runs in a Databricks-managed plane too. So ask which of your workloads are serverless. Ask where Unity Catalog metadata lives, too. Our answer is simpler. You run each part. We hold no keys. Guardian checks the deploy manifest signature before anything starts. Architecture.
Does DataShield have SOC 2?
Not yet. We will not imply otherwise. Databricks will hand you a report. A SOC 2 Type II one. Also ISO certificates, through an account team. That gap is real. Auth is live in production. It ships a public threat model. Anyone can run the verifier. Guardian and Lighthouse have been running since April 2026. Design-partner terms include source escrow. That way, a small vendor stays safe. Never a single point of failure. Details on the security page.
- Homepage hero, verbatim: "One database for AI, apps and agents," with the sub-head "Lakebase is serverless Postgres for applications that scale." Scale claims on the same page: "Over 60% of the Fortune 500 uses Databricks" and "Over 20,000 customers across the globe." - databricks.com, 13 Sep 2026
- Pricing is pay-as-you-go with "no up-front costs. Only pay for the products you use at per second granularity." A DBU is "a normalized unit of processing power on the Databricks Lakehouse Platform." No dollar rate card and no named platform tiers appear on the page. - databricks.com, 13 Sep 2026
- Docs, verbatim: "The control plane includes the backend services that Databricks manages in your Databricks account. The control plane is located in the Databricks account, not your cloud account." Serverless compute also runs inside the Databricks account. - docs.databricks.com, 13 Sep 2026
- Unity Catalog hero, verbatim: "Unified governance for data, apps and AI agents." Licensing, verbatim: "No separate license is required. Unity Catalog is included with Databricks Premium and Enterprise tiers." - databricks.com, 13 Sep 2026
- Agent Bricks launch post, 11 June 2025, reports customer results including AstraZeneca parsing "more than 400,000 clinical trial documents" in "just under 60 minutes," and Flo Health doubling medical accuracy over standard commercial models. - Databricks blog, 11 Jun 2025
- Managed MCP servers give agents "governed access to Genie, AI Search, Databricks SQL, Unity Catalog functions, and the code interpreter," with external MCP servers registered as Unity Catalog securables and governed through Unity Gateway. - docs.databricks.com, 13 Sep 2026
Other head-to-heads
DataShield vs Informatica IDMC
DataShield vs Informatica IDMC: IDMC wins on connectors. And scale. Plus ELT. DataShield adds self-hosting,.
Same marketDataShield vs Snowflake
DataShield vs Snowflake: Snowflake wins scale and Cortex agents. Plus ecosystem. DataShield adds self-hosting,.
Same marketDataShield vs Qlik Talend
DataShield vs Qlik Talend: Qlik Talend Cloud wins here. On data integration. And quality. Plus lineage. DataShield adds.
AllEvery comparison
One honest scorecard per vendor, sources at the bottom.
Keep Databricks. Then see what an independent layer adds. Break a live audit chain. Revoke an agent mid-session. Judge for yourself. Demo Center access is free with a work email.
Get free Demo Center accessYou've seen the proof
Ready for a number? Scope your deployment and we'll price it against your own economics.
Get your quote →