Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement strict projection in more transforms #35

Open
rdblue opened this issue Dec 7, 2018 · 1 comment
Open

Implement strict projection in more transforms #35

rdblue opened this issue Dec 7, 2018 · 1 comment

Comments

@rdblue
Copy link
Contributor

@rdblue rdblue commented Dec 7, 2018

Strict projection isn't required and wasn't implemented for several of the partitioning transformations. When strict projection isn't implemented (the projectStrict method returns null) Iceberg will fall back to a safe implementation. For example, residual evaluation will not remove predicates because they cannot be guaranteed to be true, and deletes can't determine that all values in a file match so the file can't be deleted (deletes usually fall back to min/max metrics evaluation).

Implementing strict projection for all transforms where possible will help query efficiency, will make deletes faster, etc.

@waterlx
Copy link
Contributor

@waterlx waterlx commented Jan 17, 2020

I think we could close this issue. Shall we?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.