How to check if c3 cache is used in query?

Dremio University
1

Hi,

I have enabled cache for Amazon S3 data source and kept CTAS format as ICEBERG.as mentioned in How do we check if cache is used or not - #2 by ben I am not able to see “Operator Metrics” in raw profile nor I am able to see Parquet row group scan
I am getting Operator Metrics in ICEBERG SUB Scan.Is it same?

2

@balaji.ramaswamy Could you please check my question?This is very crucial for us

3

@shraddhagrawal PARQUET_ROW_GROUP_SCAN is only on older versions. Now there are 3 scans

  • ICEBERG_SUB_SCAN is scan of the MANIFEST LIST file
  • TABLE_FUNCTION type=SPLIT_GEN_MANIFEST_SCAN is the MANIFEST scan
  • TABLE_FUNCTION type=DATA_FILE_SCAN is the actual scan of the Parquet files

In all of the above scan you should see if C3 was used. Expand operator metrics and scroll to the right and see NUM_CACHE_HITS and NUM_CACHE_MISSES, like below

image
image3344×1504 280 KB