Change the object store table/catalog layout #97
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This change does 2 things: - When a table is created with `catalog=obeject_store`, the catalog file used to be in `defaultPrefix/_catalog/frompg/pg_db.json` after this commit: `defaultPrefix/frompg/catalog/pg_db.json` - When an `catalog=object_store` iceberg table is created, it used to be located in `defaultPrefix/dbname/schema/table_name/oid`. After this commit: `defaultPrefix/frompg/tables/dbname/schema/table_name/oid` The goal is to be able to differentiate the tables written by Postgres, and other systems. The `frompg` allows this. Also, we can control the access of `/catalog` folder such that only priviledged users can access that.
sfc-gh-mslot
approved these changes
Dec 3, 2025
Collaborator
sfc-gh-mslot
left a comment
sfc-gh-mslot
left a comment
There was a problem hiding this comment.
Thanks, this makes sense to me.
sfc-gh-okalaci
deleted the
onder/improve_location_prefixes_another_take
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
And alternative to #96
This change does 2 things:
When a table is created with
catalog=obeject_store, the catalog file used to be indefaultPrefix/_catalog/frompg/pg_db.jsonafter this commit:defaultPrefix/frompg/catalog/pg_db.jsonWhen an
catalog=object_storeiceberg table is created, it used to be located indefaultPrefix/dbname/schema/table_name/oid. After this commit:defaultPrefix/frompg/tables/dbname/schema/table_name/oidThe goal is to be able to differentiate the tables written by Postgres, and other systems. The
frompgallows this. Also, we can control the access of/catalogfolder such that only priviledged users can access that.