Head-to-head · updated 13 September 2026
DataShield vs Apache Polaris: does an open catalog know which of those tables hold PHI?
Apache Polaris is the open catalog for Iceberg tables. It is good at that job. Spark, Flink, Trino, Doris, StarRocks and Dremio all speak to it. It hands out short-lived cloud keys. An engine can read the files with them. Apache Gravitino sits a layer up. It links many catalogs at once. Plus filesets, Kafka topics, ML models and vector data. Both are free. Both are Apache top-level projects, and run on your own hardware. Want out of catalog lock-in? Pick one of these.
We do a new job. An open catalog notes that a table exists, and who may open it. It does not say which columns hold a patient number. DataShield tags the fields. It tokenizes them. It sets what an agent may do next. It writes proof you can check later. We tokenize data sets at ingest. Agents then query tokenized data over MCP. Turning a token back into real data is a locked step. It's audited, always. Most teams here run both. We think that's the right call.
The short version
Pick DataShield when
- You need to know which columns hold PHI. From a scan, not a sheet. We tag fields against 129 classes. The call repeats each time. See Ontology.
- An agent calls your data, and its rights can shift. We re-check access on each checked tool call. A revoked agent fails on the next one. How Auth does it.
- You have to prove a log wasn't edited. Ours is a hash chain, with signed checkpoints. Run the verifier.
- A GDPR erasure request has to land, and not break the audit trail. Crypto-shred does just that.
Pick Polaris or Gravitino when
- You want one Iceberg catalog every engine can use. That's the whole point of Polaris. And it works.
- Lock-in is the fear. An Apache top-level project, with Snowflake and Dremio both in it, is the safe pick.
- Your estate is wider than tables. Gravitino links Hive, Glue, Kafka topics, filesets, models and vector data, in one place.
- You want free code you can read, fork and run. Both are Apache-2.0. The cost is your own team's time.
Bottom line: Polaris and Gravitino tell your engines where the tables are. Neither claims to know what's in the columns. Neither writes proof a regulator would accept. Run one of them. Put us next to it.
Feature by feature
Competitor cells state what the Polaris and Gravitino docs say. We use the date above. Did we get something wrong? Email support@myorg.ai. We'll fix it and credit you.
| What matters | DataShield | Polaris / Gravitino | Edge |
|---|---|---|---|
| Iceberg REST catalog | Not us. We don't build the Iceberg REST spec. On our side, iceberg_rest is named, but not live yet. Your engines should talk to their catalog, not ours. | Polaris is the model here. It builds the Iceberg REST API. It serves Spark, Flink, Trino, Doris, StarRocks and Dremio. Gravitino ships its own Iceberg REST service, too. | ◇ |
| Field-level classification | 129 field classes. Covers PII, PHI, finance data and secrets. That's all 18 HIPAA Safe Harbor IDs. Regex plus checksum checks, column-name lists and anti-pattern filters. No model runs here, so a call repeats from a config digest. | None in either project. Gravitino has free-form tags you can attach by hand. A tag is a label someone typed, not a scan. | ◆ |
| Access control grain | Column-level. Masking and broad bands are features you switch on, per data set. The masked golden views mask, no matter what the mode says. | Object level, in both. Polaris rights stop at catalog, namespace, table, view and policy. Gravitino stops at the securable object. No column rules, no row filters, no masking in either. | ◆ |
| Policy engine | Cedar decides admin, config, token and vault moves. On the tool path we use a scope ceiling. Plus an access tier, and a mid-session pulled-access check. We don't claim Cedar on each call. | Polaris can hand off to Open Policy Agent. But the docs call it "a preview feature" that "may undergo breaking changes". They tell you to keep rights control native. Gravitino pushes authorization down to Ranger or a JDBC plugin. | ◈ |
| Audit evidence | SHA-256 hash chain, with Ed25519-signed checkpoints. Those checkpoints are chained too. A check names the fault: tampering, insertion, deletion or a cut. Try the verifier. | Real, but not tamper-evident. Polaris 1.7.0 added an OpenTelemetry hook and Kafka events. Gravitino's MCP server writes one JSON line per call to gravitino-mcp-audit.log. Both are files a root user can rewrite. | ◆ |
| Agent authorization | Every checked tool call passes a scope ceiling. Plus an access tier, and a pull check before dispatch. A pulled grant lands mid-session. Delegation is RFC 8693 token exchange, with a fixed scope ceiling. | Gravitino's MCP server takes a fixed token, or forwards the caller's Authorization header. Its own access-control page says Gravitino "won't check the privileges when Gravitino receives the requests". It leans on a plugin instead. | ◆ |
| Break-glass | Scoped, time-boxed access for agents, just for a crisis. It auto-revokes. It can't be quietly wiped from the log. | We found no break-glass idea in either project's docs. In practice the fix is a second admin login. | ◆ |
| GDPR erasure | Crypto-shred of per-person key parts. Plus ISO 27560 consent receipts. The chain still checks out once the person is gone. | No idea of a data subject in either. Erasure is a rewrite of the Iceberg table. The catalog just tracks the new snapshot. | ◆ |
| Tokenization and data handling | Fixed, join-safe, vault-reversible tokens, set at ingest. Plus broad bands for quasi-IDs, as a rule. Dates become year, decade, or age band. ZIPs become 3 or 4 digits. Phones, SSNs and emails get part masked. Each column gets a risk score. Detokenization is a locked, logged step. | Neither one changes the data. Polaris hands out a short-lived cloud key, and the engine reads the raw files. That's by design. It's also the gap. | ◆ |
| Scan and profile a live source | Ontology adds a live PostgreSQL source. It scans it in place, and checks its columns, with no row leaving home. Sample values stay out by default. Storing them in the clear needs a click, and logs an audit row. PostgreSQL today, not your whole estate. | Gravitino links metadata from Hive, Iceberg, Paimon, Hudi, JDBC, ClickHouse, Glue, Kafka and filesets. Far more range than we have. It reads schemas, not column content. | ◇ |
| MCP and agents | More than 200 MCP tools across Ontology, Auth, Corpus and Lighthouse. MCP tool tokens carry scope ceilings. Each call gets logged and tied to the agent. | Both ship one. Gravitino's exposes 30-plus metadata tools. Polaris ships a Python MCP server that wraps its REST APIs. Each hands an agent the map. Neither polices what the agent does with it. | ◈ |
| Deployment | Self-hosted, on your own cloud or in your own room. Or on a single-tenant box we run for you. Docker images cover Auth, Ontology, Corpus and Lighthouse. Guardian checks a signed deploy file first. | Self-hosted by design. Java tools, Helm charts, your own Postgres or MySQL. Gravitino is built for multi-region estates, and says so. | ◈ |
| Maturity signals | Auth, Guardian and Lighthouse are live and in use. Guardian and Lighthouse since April 2026. SOC 2 is not yet in place, and we say so. | Polaris became an Apache top-level project on 19 February 2026. It shipped 1.7.0 on 2 August 2026. Gravitino graduated in June 2025. It runs at Uber and Pinterest, and shipped 1.3.0 on 29 June 2026. Older and busier than us. | ◇ |
| Pricing | Published price, a quick quote, no sales wall. | Free, Apache-2.0. You pay in staff time. Managed Polaris from Snowflake is closed to new sign-ups. Dremio's build is quote-only. | ◇ |
◆ DataShield leads◇ Polaris / Gravitino leads◈ comparable
Polaris and Gravitino claims come from polaris.apache.org and gravitino.apache.org. Also their release notes. Last checked 13 September 2026. We link them below, not just from memory.
Three things you get here that you won't get from an open table catalog
Labels that came from a scan
Gravitino lets you tag a table. Someone still has to know it holds patient data, and type that in. We scan the columns instead. We score them against 129 classes, with checksums and name lists behind the call. No model sits in the path. So the same data gives the same call next quarter. That's the answer when an auditor asks how you knew. See the catalog.
Proof that survives an audit
Polaris can stream audit events to OpenTelemetry. Gravitino writes a JSON line per MCP call. Both are handy. Neither can show that nothing was removed. Ours is a hash chain, with signed checkpoints. The verifier says what broke, not just that it did. EU AI Act Article 12 and HIPAA §164.312(b) checkers look for that trait. Try it in your browser, no signup.
Authority that can change mid-flight
An analyst quits on a Friday. Their agent is 20 minutes into a job, against your lakehouse. A catalog checked the grant when the run began. It has no cause to look again. We check on the next checked tool call, and fail closed. How Auth does it.
Where Polaris and Gravitino are genuinely stronger
Let's be fair. Polaris won its space on merit. It's the Iceberg REST catalog most engines aim for. It hands out scoped cloud keys, so query engines never hold long-lived ones. Its 1.7.0 build in August added Kafka events and an OpenTelemetry audit hook. Snowflake and Dremio both ship it, and both help build it. The Apache board now holds the trademark. Gravitino covers ground we don't. Hive, Glue, ClickHouse, Kafka topics, filesets, ML models and Lance vector data, all linked, across regions. Uber and Pinterest run it. Their reach beats our one live source kind, by years. Neither costs a cent.
Here's the push-back. Both projects use the word governance. Buyers read it as the kind their privacy office means. It's not. Polaris rights stop at the table. Gravitino's own access-control page says it "won't check the privileges when Gravitino receives the requests". It points you at a plugin. Tags get typed by hand. There's no scanner, no masking, no subject, no erasure story, and no chain over the log. None of that is a flaw. None of it is what these projects set out to build. It becomes a problem only when a platform team tells a CISO the catalog has governance covered. Nine months on, an examiner asks which tables held PHI in March. That's the gap we fill. We'd rather you keep Polaris.
Questions worth asking both of us
These are the questions we'd want answered if we were buying. Ask them on every vendor call, including ours.
Can you cryptographically prove an audit log entry wasn't deleted?
DataShield: yes. Each record ties to the one right before it. Checkpoints get signed and chained. A check tells removal apart from a cut or a tamper. Try it at /verify. Polaris sends audit events over OpenTelemetry and Kafka. Gravitino's MCP server writes a JSON line per call. Both are plain records. Whoever holds the file or the topic can edit history. So the proof has to come from somewhere else.
What happens to a revoked agent mid-session?
DataShield re-checks access on each checked tool call. So a pulled grant bites on the next call. Gravitino's MCP server takes a fixed bearer token, or forwards the caller's header. Its docs say Gravitino does not check rights on incoming requests. Polaris checks its own grants per request, at table grain. Neither can cut a running agent short by pulling the token.
How does GDPR erasure interact with the audit trail?
DataShield crypto-shreds per-person key parts and sends an ISO 27560 consent receipt. Actor names in the chain are HMAC-locked, not real names. So the proof still checks out once the person is gone. Neither open catalog has a subject idea. You rewrite the Iceberg table, drop the old snapshots, and hope nobody kept a copy.
Is DataShield an Iceberg REST catalog? Do we drop Polaris?
No, and no. We don't build the Iceberg REST spec. Iceberg is a named but unbuilt source kind for us today. Your engines should keep talking to Polaris. We sit beside it, and answer a new question. Which fields hold risk? What may an agent do with them? Can you prove it later?
Gravitino ships an MCP server. Isn't that agent governance?
It's agent access, which is the easy half. The server exposes 30-plus metadata tools and logs each call. That's genuinely handy. What it does not do: hand out a scoped token per agent. It doesn't narrow that scope at dispatch. It doesn't re-check the grant mid-session, log the call, or seal the record. Read their access-control page next to ours. The split is clear.
Snowflake closed Open Catalog to new sign-ups. Does that hurt Polaris?
Not the project. The docs say new buyers should use Horizon Catalog instead. That reads as Snowflake selling on top of Polaris, not through it. The code is Apache-2.0, and the board holds it. Want managed Polaris today? Look to Dremio. Or run it yourself. Running it yourself is the common path.
Does DataShield have SOC 2?
Not yet, and we won't say otherwise. Auth is live now. It ships with a public threat model and a verifier anyone can run. Guardian and Lighthouse have been live since April 2026. Design-partner terms include source escrow. That way one small vendor isn't a single point of failure. Details on the security page.
- Polaris is "an open-source, fully-featured catalog for Apache Iceberg™. It implements Iceberg's REST API, enabling… multi-engine interoperability" across Doris, Flink, Spark, Dremio, StarRocks and Trino. Its hero line reads: "The open catalog for data and AI assets." - Apache Polaris project site, 13 Sep 2026
- Polaris privileges are granted at catalog, namespace, table, view and policy level. Examples: TABLE_READ_DATA, NAMESPACE_LIST, CATALOG_MANAGE_ACCESS. No column-level or row-level rules appear in the model. No classification or masking, either. - Polaris 1.7.0 access control docs, 13 Sep 2026
- Polaris "policy" means table maintenance. The four predefined system types are system.data-compaction, system.metadata-compaction, system.orphan-file-removal and system.snapshot-expiry. External OPA policy is "currently a preview feature and may undergo breaking changes". - Polaris 1.7.0 policy and external PDP docs, 13 Sep 2026
- "Gravitino only supports authorization for securable objects… Gravitino doesn't support metadata authentication." It means Gravitino "won't check the privileges when Gravitino receives the requests." Built-in authorization is a config flag. The design pushes down to Ranger or JDBC plugins. - Gravitino 1.3.0 access control docs, 13 Sep 2026
- The Gravitino MCP server "provides the ability to manage Gravitino metadata for LLM". It exposes 30-plus tools. It authenticates with a static bearer token, or a forwarded Authorization header. It records "one structured JSON line in gravitino-mcp-audit.log" per tool call. - Gravitino 1.3.0 MCP server docs, 13 Sep 2026
- "Snowflake Open Catalog is a managed service for Apache Polaris™." New customers "can't sign up for their first Open Catalog account". They're pointed at Horizon Catalog instead. Billing on REST API requests "will begin in the first half of 2026". - Snowflake Open Catalog docs, 13 Sep 2026
Other head-to-heads
DataShield vs Amazon DataZone
DataZone catalogs your AWS estate. It grants access via Lake Formation.
Same marketDataShield vs Google Dataplex
Knowledge Catalog wins BigQuery lineage and free context for agents.
Same marketDataShield vs Snowflake Horizon
Horizon governs data inside Snowflake. DataShield adds reversible tokens.
AllEvery comparison
One honest scorecard per vendor, including the rows we lose.
Keep your open catalog. Then see the rest run in your browser. Tag a column. Break a live audit chain. Revoke an agent mid-session. Then decide. Demo Center access is free with a work email.
Get free Demo Center accessYou've seen the proof
Ready for a number? Scope your deployment and we'll price it against your own economics.
Get your quote →