Good afternoon
When running the dremio-admin clean -o command, it deletes the split orphans. Two questions…What are split orphans and is there a way to query the metadata identifying what split orphans will be deleted?
dremio-admin clean -o
Initial Store Status.
Deleting split orphans…
Completed. Deleted 478 orphans.
Deleting collaboration orphans…
Completed. Deleted 0 orphans.
What are split orphans?
Dremio collects metadata about a dataset from the source on first time query, or if stale, or in the background per metadata policy. This includes metadata about each split of data in a dataset (for example, “row group” is a split of a parquet dataset). If a source supports detecting dataset changes, there may be nothing to collect. “Orphan splits” are the stale metadata about splits that need to be cleaned up internally. This is done in the background during uptime per the metadata policy. During downtime, this cleanup can be more aggressive because there are no queries in progress.
Is there a way to query the metadata identifying what split orphans will be deleted?
No. What is the use case?