Dremio cannot pushdown string functions to SQL SERVER

[fbelchior]

Hey guys,

I’ve identified strange behavior on a SQL Server connection. When I use some string function, Dremio doesn’t send that function to the source. Ex.:

This query:
SELECT MAX(COL_TEXT)
FROM TABLE
WHERE DATE = ‘2021-12-23’

Result in the same function in JDBC SQL.

This query:
SELECT MAX(LENGTH(COL_TEXT))
FROM TABLE
WHERE DATE = ‘2021-12-23’

Results in:
SELECT COL_TEXT
FROM TABLE
WHERE DATE = ‘2021-12-23’

Unfortunately my table has over 200 million rows for a date causing slowing results.

[balaji.ramaswamy]

@fbelchior Can you please send us the profile meanwhile we will attempt a repro

[fbelchior]

WITH STRING FUNCTION.zip (13,4,KB)
WITHOUT STRING FUNCTION.zip (12,7,KB)

[fbelchior]

Hi guys! Any news on this issue?

[balaji.ramaswamy]

@fbelchior I was able to reproduce the same behavior, it looks like we are not pushing down the LENGTH into the scan,

Would you be able to send me the explain plan for the same SQL on SQL Server?

[fbelchior]

Hi @balaji.ramaswamy

Unfortunately I don’t have DBA access to SQL Server.

[balaji.ramaswamy]

@fbelchior never mind, this could be something Dremio is currently not pushing down, does the query never complete or you cancelled since it is already missing SLA’s?

[fbelchior]

@balaji.ramaswamy

I canceled the query because it caused problems on my SQL Server

[ljd520cc]

Hi @balaji.ramaswamy ，
I have similar questions about this feature.
I run a SQL ( SELECT c_7 FROM “arp_test”.“my_schema”.aa where c_5=‘6_c_5_data’ ) in Dremio, what I expected is Dremio can push down where clause to JDBC, but the fact is not, see the detail： Jdbc(sql=[SELECT “aa”.“c_5”, “aa”.“c_7” FROM “my_schema”.“aa”]) .

How to config to ask Dremio to push down where clause to JDBC?

[balaji.ramaswamy]

@ljd520cc

This should ideally push down, can we please have the profile?

Thanks
Bali

[ljd520cc]

Hi @balaji.ramaswamy
There is network limitation, I am not able to upload files, please let me know which part you want to check in profile Json, I can paste it here. This is planning part in Json.

“query”: “SELECT c_7 FROM "arp_test"."my_schema".aa where c_5=‘6_c_5_data’\r\n”,
“plan”: “00-00 Screen : rowType = RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData): rowcount = 500.0, cumulative cost = {6050.0 rows, 28150.075 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 297\n00-01 Project(Fragment=[$0], Records=[$1], Path=[$2], Metadata=[$3], Partition=[$4], FileSize=[$5], IcebergMetadata=[$6], fileschema=[$7], PartitionData=[$8]) : rowType = RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData): rowcount = 500.0, cumulative cost = {6000.0 rows, 28100.075 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 296\n00-02 WriterCommitter(final=[/home/infra2/dremio/data/pdfs/results/1b0943bf-994d-9548-1f50-cfd898c98700]) : rowType = RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData): rowcount = 500.0, cumulative cost = {5500.0 rows, 28100.030000000002 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 295\n00-03 Writer : rowType = RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData): rowcount = 500.0, cumulative cost = {5000.0 rows, 27600.030000000002 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 294\n00-04 Project(c_7=[$0]) : rowType = RecordType(VARCHAR(65536) c_7): rowcount = 500.0, cumulative cost = {4500.0 rows, 27100.030000000002 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 293\n00-05 Project(c_7=[$1]) : rowType = RecordType(VARCHAR(65536) c_7): rowcount = 500.0, cumulative cost = {4000.0 rows, 27100.025 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 292\n00-06 SelectionVectorRemover : rowType = RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7): rowcount = 500.0, cumulative cost = {3500.0 rows, 27100.02 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 291\n00-07 Filter(condition=[=($0, ‘6_c_5_data’)]) : rowType = RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7): rowcount = 500.0, cumulative cost = {3000.0 rows, 26600.02 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 290\n00-08 Project(c_5=[$0], c_7=[$1]) : rowType = RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7): rowcount = 1000.0, cumulative cost = {2000.0 rows, 20000.02 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 289\n00-09 Jdbc(sql=[SELECT "aa"."c_5", "aa"."c_7"\nFROM "my_schema"."aa"]) : rowType = RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7): rowcount = 1000.0, cumulative cost = {1000.0 rows, 20000.0 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 189\n”,
“foreman”: {

[balaji.ramaswamy]

@ljd520cc Would need the full contents under final physical transformation, is this query accelerated?

[ljd520cc]

Hi @balaji.ramaswamy ,

{
“phaseName”: “Final Physical Transformation”,
“durationMillis”: 38,
“plan”: “00-00 Screen : rowType = RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData): rowcount = 500.0, cumulative cost = {6050.0 rows, 28150.075 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 297\n00-01 Project(Fragment=[$0], Records=[$1], Path=[$2], Metadata=[$3], Partition=[$4], FileSize=[$5], IcebergMetadata=[$6], fileschema=[$7], PartitionData=[$8]) : rowType = RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData): rowcount = 500.0, cumulative cost = {6000.0 rows, 28100.075 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 296\n00-02 WriterCommitter(final=[/home/infra2/dremio/data/pdfs/results/1b0943bf-994d-9548-1f50-cfd898c98700]) : rowType = RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData): rowcount = 500.0, cumulative cost = {5500.0 rows, 28100.030000000002 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 295\n00-03 Writer : rowType = RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData): rowcount = 500.0, cumulative cost = {5000.0 rows, 27600.030000000002 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 294\n00-04 Project(c_7=[$0]) : rowType = RecordType(VARCHAR(65536) c_7): rowcount = 500.0, cumulative cost = {4500.0 rows, 27100.030000000002 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 293\n00-05 Project(c_7=[$1]) : rowType = RecordType(VARCHAR(65536) c_7): rowcount = 500.0, cumulative cost = {4000.0 rows, 27100.025 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 292\n00-06 SelectionVectorRemover : rowType = RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7): rowcount = 500.0, cumulative cost = {3500.0 rows, 27100.02 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 291\n00-07 Filter(condition=[=($0, ‘6_c_5_data’)]) : rowType = RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7): rowcount = 500.0, cumulative cost = {3000.0 rows, 26600.02 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 290\n00-08 Project(c_5=[$0], c_7=[$1]) : rowType = RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7): rowcount = 1000.0, cumulative cost = {2000.0 rows, 20000.02 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 289\n00-09 Jdbc(sql=[SELECT "aa"."c_5", "aa"."c_7"\nFROM "my_schema"."aa"]) : rowType = RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7): rowcount = 1000.0, cumulative cost = {1000.0 rows, 20000.0 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 189\n”
},

…

"accelerationProfile": {
	"accelerated": false,
	"numSubstitutions": 0,
	"millisTakenGettingMaterializations": 7,
	"millisTakenNormalizing": 0,
	"millisTakenSubstituting": 0,
	"layoutProfiles": [],
	"normalizedQueryPlans": [],
	"accelerationDetails": ""
},

@balaji.ramaswamy, any findings?

[ljd520cc]

@balaji.ramaswamy any findings?

[balaji.ramaswamy]

@ljd520cc Profile would be best to troubleshoot, I tried a local repro and works as expected. Is this the SQL Server source Dremio provides or a custom built one? Are you able to send just the profile.json atleast? What version of Dremio is this?

[ljd520cc]

Hi @balaji.ramaswamy ,
This is a custom JDBC driver. Due to the limitation of network that unable to upload files, I pasted full content of profile include header.json and profile_attempt_0.json.
— header.json ----

{"clusterInfo":{"identity":{"identity":"2d4ee7ea-41a1-4a75-9ec9-0a1d7ce20411","created":1693383564458,"version":{"major":21,"minor":2,"patch":2,"buildNumber":0,"qualifier":"202206231521110575-8e61cb30"}},"version":{"version":"21.2.2-202206231521110575-8e61cb30"},"source":[{"name":"INFORMATION_SCHEMA","type":"INFORMATION_SCHEMA"},{"name":"sys","type":"ESYSFLIGHT"},{"name":"__home","type":"HOME"},{"name":"__accelerator","type":"ACCELERATION"},{"name":"__jobResultsStore","type":"INTERNAL"},{"name":"$scratch","type":"INTERNAL"},{"name":"__datasetDownload","type":"INTERNAL"},{"name":"__metadata","type":"METADATA"},{"name":"__logs","type":"INTERNAL"},{"name":"__support","type":"INTERNAL"},{"name":"test","type":"J_P_A"}],"node":[{"name":"uklvadsb0039.uk.dev.net","role":"executor"},{"name":"uklvadsb0039.uk.dev.net","role":"coordinator"}],"javaVmVersion":"11.0.2+9-LTS","jreVersion":"11","edition":"enterprise"},"submission":{"first":"Jeady","last":"Liang","email":"jeady.liang@sc.com","date":1694050118115,"submissionId":"398adac3-9282-4934-96f2-f457a72e7d72"},"job":{"state":3,"info":{"jobId":{"id":"1b0943bf-994d-9548-1f50-cfd898c98700"},"sql":"SELECT c_7 FROM \"arp_test\".\"my_schema\".aa where c_5='6_c_5_data'\r\n","requestType":7,"user":"1490626","startTime":1693891647787,"finishTime":1693891651363,"datasetPath":["tmp","UNTITLED"],"datasetVersion":"0009912451361506","parents":[{"datasetPath":["arp_test","my_schema","aa"],"type":2}],"queryType":1,"fieldOrigins":[{"name":"c_7","origins":[{"table":["arp_test","my_schema","aa"],"columnName":"c_7","derived":false}]}],"resultMetadata":[{"path":"uklvadsb0039.uk.dev.net@0_0_0.dremarrow1","recordCount":1,"footer":{"field":[{"majorType":{"minorType":24,"mode":0,"precision":65536},"namePart":{"type":0,"name":"c_7"},"child":[{"majorType":{"minorType":20,"mode":1},"namePart":{"type":0,"name":"$bits$"}},{"majorType":{"minorType":24,"mode":1},"namePart":{"type":0,"name":"$values$"},"child":[{"majorType":{"minorType":31,"mode":1},"namePart":{"type":0,"name":"$offsets$"}}]}]}],"batch":[{"recordCount":1,"offset":10}]},"screenNodeEndpoint":{"address":"uklvadsb0039.uk.dev.net","userPort":31010,"fabricPort":45678,"roles":{"sqlQuery":true,"logicalPlan":true,"physicalPlan":true,"javaExecutor":true,"distributedCache":true,"master":true},"startTime":1693891592818,"provisionId":"","maxDirectMemory":8589934592,"availableCores":4,"nodeTag":"","engineId":{"id":""},"subEngineId":{"id":""}},"arrowMetadataVersion":4}],"description":"SELECT c_7 FROM \"arp_test\".\"my_schema\".aa where c_5='6_c_5_data'\r\n","originalCost":7.0877866951005E10,"scanPaths":[{"path":["arp_test","my_schema","aa"]}],"joinAnalysis":{"joinTables":[{"tableId":0,"tableSchemaPath":["arp_test","my_schema","aa"]}]},"context":["@1490626"],"resourceSchedulingInfo":{"queueName":"Low Cost User Queries","queueId":"baa30eec-387e-403c-8fba-69181bf55f41","resourceSchedulingStart":1693891649360,"resourceSchedulingEnd":1693891649524,"queryCost":6000.0,"engineName":""},"outputTable":["__jobResultsStore","1b0943bf-994d-9548-1f50-cfd898c98700"],"batchSchema":"EAAAAAAACgAMAAAACAAEAAoAAAAIAAAACAAAAAAAAAABAAAAGAAAAAAAEgAYABQAEwASAAwAAAAIAAQAEgAAABQAAAAUAAAAGAAAAAAABQEUAAAAAAAAAAAAAAAEAAQABAAAAAMAAABjXzcA","commandPoolWaitMillis":1,"nodeDetails":[{"hostName":"uklvadsb0039.uk.dev.net","hostIp":"uklvadsb0039.uk.dev.net","fabricPort":45678,"maxMemoryUsedKb":10887}],"executionCpuTimeNs":56678055,"setupTimeNs":865812633,"waitTimeNs":7838872,"memoryAllocated":10000000},"stats":{"inputBytes":32044,"outputBytes":19,"inputRecords":1000,"outputRecords":1,"isOutputLimited":false},"details":{"timeSpentInPlanning":1700,"waitInClient":1,"dataVolume":19,"outputRecords":1,"peakMemory":10887296,"topOperations":[{"type":7,"timeConsumed":61.363636},{"type":4,"timeConsumed":32.359306},{"type":5,"timeConsumed":2.4891775},{"type":8,"timeConsumed":1.4069264},{"type":1,"timeConsumed":1.1904762}],"totalMemory":10887296,"cpuUsed":913},"attemptId":"1b0943bf-994d-9548-1f50-cfd898c98700","endpoint":{"address":"uklvadsb0039.uk.dev.net","userPort":31010,"fabricPort":45678,"roles":{"sqlQuery":true,"javaExecutor":true,"master":true},"startTime":1693891592818,"maxDirectMemory":8589934592,"availableCores":4,"nodeTag":"","conduitPort":43231,"dremioVersion":"21.2.2-202206231521110575-8e61cb30"},"accelerationDetails":"","stateList":[{"state":1,"startTime":1693891647785},{"state":2,"startTime":1693891647806},{"state":3,"startTime":1693891648193},{"state":5,"startTime":1693891649476},{"state":4,"startTime":1693891649476},{"state":6,"startTime":1693891649524},{"state":7,"startTime":1693891649730},{"state":8,"startTime":1693891649846},{"state":9,"startTime":1693891651370}]},"dremioVersion":"21.2.2-202206231521110575-8e61cb30"}

[ljd520cc]

@balaji.ramaswamy

— part 1 of profile_attempt_0.json —

{"id":{"part1":1948162804004263240,"part2":2256531942495848192},"start":1693891647787,"end":1693891651363,"query":"SELECT c_7 FROM \"arp_test\".\"my_schema\".aa where c_5='6_c_5_data'\r\n","plan":"00-00    Screen : rowType = RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData): rowcount = 500.0, cumulative cost = {6050.0 rows, 28150.075 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 297\n00-01      Project(Fragment=[$0], Records=[$1], Path=[$2], Metadata=[$3], Partition=[$4], FileSize=[$5], IcebergMetadata=[$6], fileschema=[$7], PartitionData=[$8]) : rowType = RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData): rowcount = 500.0, cumulative cost = {6000.0 rows, 28100.075 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 296\n00-02        WriterCommitter(final=[/home/infra2/dremio/data/pdfs/results/1b0943bf-994d-9548-1f50-cfd898c98700]) : rowType = RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData): rowcount = 500.0, cumulative cost = {5500.0 rows, 28100.030000000002 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 295\n00-03          Writer : rowType = RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData): rowcount = 500.0, cumulative cost = {5000.0 rows, 27600.030000000002 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 294\n00-04            Project(c_7=[$0]) : rowType = RecordType(VARCHAR(65536) c_7): rowcount = 500.0, cumulative cost = {4500.0 rows, 27100.030000000002 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 293\n00-05              Project(c_7=[$1]) : rowType = RecordType(VARCHAR(65536) c_7): rowcount = 500.0, cumulative cost = {4000.0 rows, 27100.025 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 292\n00-06                SelectionVectorRemover : rowType = RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7): rowcount = 500.0, cumulative cost = {3500.0 rows, 27100.02 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 291\n00-07                  Filter(condition=[=($0, '6_c_5_data')]) : rowType = RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7): rowcount = 500.0, cumulative cost = {3000.0 rows, 26600.02 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 290\n00-08                    Project(c_5=[$0], c_7=[$1]) : rowType = RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7): rowcount = 1000.0, cumulative cost = {2000.0 rows, 20000.02 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 289\n00-09                      Jdbc(sql=[SELECT \"aa\".\"c_5\", \"aa\".\"c_7\"\nFROM \"my_schema\".\"aa\"]) : rowType = RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7): rowcount = 1000.0, cumulative cost = {1000.0 rows, 20000.0 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 189\n","foreman":{"address":"uklvadsb0039.uk.dev.net","userPort":31010,"fabricPort":45678,"roles":{"sqlQuery":true,"javaExecutor":true,"master":true},"startTime":1693891592818,"maxDirectMemory":8589934592,"availableCores":4,"nodeTag":"","conduitPort":43231,"dremioVersion":"21.2.2-202206231521110575-8e61cb30"},"state":2,"totalFragments":1,"finishedFragments":1,"fragmentProfile":[{"majorFragmentId":0,"minorFragmentProfile":[{"state":3,"minorFragmentId":0,"operatorProfile":[{"inputProfile":[{"records":1,"batches":1,"size":450}],"operatorId":0,"operatorType":13,"setupNanos":803756,"processNanos":9503114,"peakLocalMemoryAllocated":1000000,"metric":[{"metricId":0,"longValue":450}],"waitNanos":1589142,"operatorSubtype":0},{"inputProfile":[{"records":1,"batches":1,"size":450}],"operatorId":1,"operatorType":10,"setupNanos":1464444,"processNanos":324047,"peakLocalMemoryAllocated":1000000,"metric":[{"metricId":0,"longValue":0},{"metricId":7,"longValue":0},{"metricId":1,"longValue":0},{"metricId":3,"longValue":0},{"metricId":2,"longValue":0},{"metricId":5,"longValue":0},{"metricId":6,"longValue":0},{"metricId":4,"longValue":0}],"waitNanos":0,"details":{"splitInfos":[],"slowIoInfos":[],"slowMetadataIoInfos":[],"runtimefilterDetailsInfos":[],"runtimefilterDetailsInfosInScan":[]},"operatorSubtype":0},{"inputProfile":[{"records":1,"batches":1,"size":450}],"operatorId":2,"operatorType":49,"setupNanos":5245540,"processNanos":368379,"peakLocalMemoryAllocated":1000000,"metric":[{"metricId":0,"longValue":0},{"metricId":1,"longValue":0},{"metricId":3,"longValue":0},{"metricId":4,"longValue":0},{"metricId":7,"longValue":0},{"metricId":2,"longValue":0},{"metricId":6,"longValue":0},{"metricId":5,"longValue":0}],"waitNanos":0,"details":{"splitInfos":[],"slowIoInfos":[],"slowMetadataIoInfos":[],"runtimefilterDetailsInfos":[],"runtimefilterDetailsInfosInScan":[]},"operatorSubtype":0},{"inputProfile":[{"records":1,"batches":1,"size":19}],"operatorId":3,"operatorType":43,"setupNanos":2045818,"processNanos":11139285,"peakLocalMemoryAllocated":1000000,"metric":[{"metricId":0,"longValue":118},{"metricId":1,"longValue":0}],"waitNanos":6249730,"operatorSubtype":0},{"inputProfile":[{"records":1,"batches":1,"size":19}],"operatorId":4,"operatorType":10,"setupNanos":770532,"processNanos":52409,"peakLocalMemoryAllocated":1000000,"metric":[{"metricId":0,"longValue":0},{"metricId":5,"longValue":0},{"metricId":3,"longValue":0},{"metricId":1,"longValue":0},{"metricId":2,"longValue":0},{"metricId":7,"longValue":0},{"metricId":6,"longValue":0},{"metricId":4,"longValue":0}],"waitNanos":0,"details":{"splitInfos":[],"slowIoInfos":[],"slowMetadataIoInfos":[],"runtimefilterDetailsInfos":[],"runtimefilterDetailsInfosInScan":[]},"operatorSubtype":0},{"inputProfile":[{"records":1,"batches":1,"size":38}],"operatorId":5,"operatorType":10,"setupNanos":1436362,"processNanos":108968,"peakLocalMemoryAllocated":1000000,"metric":[{"metricId":0,"longValue":0},{"metricId":5,"longValue":0},{"metricId":4,"longValue":0},{"metricId":6,"longValue":0},{"metricId":1,"longValue":0},{"metricId":2,"longValue":0},{"metricId":3,"longValue":0},{"metricId":7,"longValue":0}],"waitNanos":0,"details":{"splitInfos":[],"slowIoInfos":[],"slowMetadataIoInfos":[],"runtimefilterDetailsInfos":[],"runtimefilterDetailsInfosInScan":[]},"operatorSubtype":0},{"inputProfile":[{"records":1,"batches":1,"size":32044}],"operatorId":6,"operatorType":14,"setupNanos":8906226,"processNanos":1790796,"peakLocalMemoryAllocated":1000000,"metric":[{"metricId":0,"longValue":1}],"waitNanos":0,"operatorSubtype":0},{"inputProfile":[{"records":1000,"batches":1,"size":32044}],"operatorId":7,"operatorType":2,"setupNanos":285973478,"processNanos":3247165,"peakLocalMemoryAllocated":1000000,"metric":[{"metricId":0,"longValue":1},{"metricId":6,"longValue":0},{"metricId":4,"longValue":0},{"metricId":3,"longValue":2},{"metricId":5,"longValue":1},{"metricId":1,"longValue":0},{"metricId":2,"longValue":253}],"waitNanos":0,"details":{"splitInfos":[{"namedExpression":"NamedExpression [expr=FunctionHolderExpression [args=[ValueVectorReadExpression [fieldId=TypedFieldId [fieldIds=[0], remainder=null]], ValueExpression[quoted_string=6_c_5_data]], name=equal, returnType=boolean, isRandom=false], ref=`_filter_`]","inGandiva":true,"outputName":"_filter_","dependsOn":[],"optimize":true}],"slowIoInfos":[],"slowMetadataIoInfos":[],"runtimefilterDetailsInfos":[],"runtimefilterDetailsInfosInScan":[]},"operatorSubtype":0},{"inputProfile":[{"records":1000,"batches":1,"size":32044}],"operatorId":8,"operatorType":10,"setupNanos":21567308,"processNanos":290878,"peakLocalMemoryAllocated":1000000,"metric":[{"metricId":0,"longValue":0},{"metricId":1,"longValue":1},{"metricId":2,"longValue":0},{"metricId":5,"longValue":0},{"metricId":4,"longValue":0},{"metricId":3,"longValue":0},{"metricId":6,"longValue":0},{"metricId":7,"longValue":0}],"waitNanos":0,"details":{"splitInfos":[],"slowIoInfos":[],"slowMetadataIoInfos":[],"runtimefilterDetailsInfos":[],"runtimefilterDetailsInfosInScan":[]},"operatorSubtype":0},{"inputProfile":[{"records":1000,"batches":2,"size":32044}],"operatorId":9,"operatorType":47,"setupNanos":537599169,"processNanos":29853014,"peakLocalMemoryAllocated":1000000,"metric":[{"metricId":27,"longValue":0},{"metricId":25,"longValue":24518297},{"metricId":28,"longValue":0},{"metricId":1,"longValue":1},{"metricId":26,"longValue":501495}],"waitNanos":0,"details":{"splitInfos":[],"slowIoInfos":[],"slowMetadataIoInfos":[],"runtimefilterDetailsInfos":[],"runtimefilterDetailsInfosInScan":[]},"operatorSubtype":0}],"startTime":1693891649792,"endTime":1693891651293,"memoryUsed":0,"maxMemoryUsed":10000000,"endpoint":{"address":"uklvadsb0039.uk.dev.net","fabricPort":45678},"lastUpdate":1693891651307,"lastProgress":1693891651307,"maxIncomingMemoryUsed":0,"sleepingDuration":1,"blockedDuration":40,"firstRun":1693891649841,"runDuration":1439,"numRuns":21,"setupDuration":1335,"finishDuration":43,"blockedOnUpstreamDuration":0,"blockedOnDownstreamDuration":25,"blockedOnSharedResourceDuration":15,"perResourceBlockedDuration":[{"resource":"FRAGMENT_ACTIVATE_SIGNAL","category":2,"duration":15}]}],"nodePhaseProfile":[{"endpoint":{"address":"uklvadsb0039.uk.dev.net","fabricPort":45678},"maxMemoryUsed":10887296}]}],"user":"1490626","planningStart":1693891648030,"planningEnd":1693891649730,"jsonPlan":"{\n  \"\\\"00-00\\\"\": {\n    \"\\\"op\\\"\": \"com.dremio.exec.planner.physical.ScreenPrel\",\n    \"\\\"values\\\"\": {},\n    \"\\\"inputs\\\"\": [\n      \"00-01\"\n    ],\n    \"\\\"rowType\\\"\": \"RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData)\",\n    \"\\\"rowCount\\\"\": 500.0,\n    \"\\\"cumulativeCost\\\"\": \"{6050.0 rows, 28150.075 cpu, 20000.0 io, 20000.0 network, 0.0 memory}\"\n  },\n  \"\\\"00-01\\\"\": {\n    \"\\\"op\\\"\": \"com.dremio.exec.planner.physical.ProjectPrel\",\n    \"\\\"values\\\"\": {\n      \"\\\"fields\\\"\": \"[Fragment, Records, Path, Metadata, Partition, FileSize, IcebergMetadata, fileschema, PartitionData]\",\n      \"\\\"exprs\\\"\": \"[$0, $1, $2, $3, $4, $5, $6, $7, $8]\"\n    },\n    \"\\\"inputs\\\"\": [\n      \"00-02\"\n    ],\n    \"\\\"rowType\\\"\": \"RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData)\",\n    \"\\\"rowCount\\\"\": 500.0,\n    \"\\\"cumulativeCost\\\"\": \"{6000.0 rows, 28100.075 cpu, 20000.0 io, 20000.0 network, 0.0 memory}\"\n  },\n  \"\\\"00-02\\\"\": {\n    \"\\\"op\\\"\": \"com.dremio.exec.planner.physical.WriterCommitterPrel\",\n    \"\\\"values\\\"\": {\n      \"\\\"final\\\"\": \"/home/infra2/dremio/data/pdfs/results/1b0943bf-994d-9548-1f50-cfd898c98700\"\n    },\n    \"\\\"inputs\\\"\": [\n      \"00-03\"\n    ],\n    \"\\\"rowType\\\"\": \"RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData)\",\n    \"\\\"rowCount\\\"\": 500.0,\n    \"\\\"cumulativeCost\\\"\": \"{5500.0 rows, 28100.030000000002 cpu, 20000.0 io, 20000.0 network, 0.0 memory}\"\n  },\n  \"\\\"00-03\\\"\": {\n    \"\\\"op\\\"\": \"com.dremio.exec.planner.physical.WriterPrel\",\n    \"\\\"values\\\"\": {},\n    \"\\\"inputs\\\"\": [\n      \"00-04\"\n    ],\n    \"\\\"rowType\\\"\": \"RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData)\",\n    \"\\\"rowCount\\\"\": 500.0,\n    \"\\\"cumulativeCost\\\"\": \"{5000.0 rows, 27600.030000000002 cpu, 20000.0 io, 20000.0 network, 0.0 memory}\"\n  },\n  \"\\\"00-04\\\"\": {\n    \"\\\"op\\\"\": \"com.dremio.exec.planner.physical.ProjectPrel\",\n    \"\\\"values\\\"\": {\n      \"\\\"fields\\\"\": \"[c_7]\",\n      \"\\\"exprs\\\"\": \"[$0]\"\n    },\n    \"\\\"inputs\\\"\": [\n      \"00-05\"\n    ],\n    \"\\\"rowType\\\"\": \"RecordType(VARCHAR(65536) c_7)\",\n    \"\\\"rowCount\\\"\": 500.0,\n    \"\\\"cumulativeCost\\\"\": \"{4500.0 rows, 27100.030000000002 cpu, 20000.0 io, 20000.0 network, 0.0 memory}\"\n  },\n  \"\\\"00-05\\\"\": {\n    \"\\\"op\\\"\": \"com.dremio.exec.planner.physical.ProjectPrel\",\n    \"\\\"values\\\"\": {\n      \"\\\"fields\\\"\": \"[c_7]\",\n      \"\\\"exprs\\\"\": \"[$1]\"\n    },\n    \"\\\"inputs\\\"\": [\n      \"00-06\"\n    ],\n    \"\\\"rowType\\\"\": \"RecordType(VARCHAR(65536) c_7)\",\n    \"\\\"rowCount\\\"\": 500.0,\n    \"\\\"cumulativeCost\\\"\": \"{4000.0 rows, 27100.025 cpu, 20000.0 io, 20000.0 network, 0.0 memory}\"\n  },\n  \"\\\"00-06\\\"\": {\n    \"\\\"op\\\"\": \"com.dremio.exec.planner.physical.SelectionVectorRemoverPrel\",\n    \"\\\"values\\\"\": {},\n    \"\\\"inputs\\\"\": [\n      \"00-07\"\n    ],\n    \"\\\"rowType\\\"\": \"RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7)\",\n    \"\\\"rowCount\\\"\": 500.0,\n    \"\\\"cumulativeCost\\\"\": \"{3500.0 rows, 27100.02 cpu, 20000.0 io, 20000.0 network, 0.0 memory}\"\n  },\n  \"\\\"00-07\\\"\": {\n    \"\\\"op\\\"\": \"com.dremio.exec.planner.physical.FilterPrel\",\n    \"\\\"values\\\"\": {\n      \"\\\"condition\\\"\": \"=($0, '6_c_5_data')\"\n    },\n    \"\\\"inputs\\\"\": [\n      \"00-08\"\n    ],\n    \"\\\"rowType\\\"\": \"RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7)\",\n    \"\\\"rowCount\\\"\": 500.0,\n    \"\\\"cumulativeCost\\\"\": \"{3000.0 rows, 26600.02 cpu, 20000.0 io, 20000.0 network, 0.0 memory}\"\n  },\n  \"\\\"00-08\\\"\": {\n    \"\\\"op\\\"\": \"com.dremio.exec.planner.physical.ProjectPrel\",\n    \"\\\"values\\\"\": {\n      \"\\\"fields\\\"\": \"[c_5, c_7]\",\n      \"\\\"exprs\\\"\": \"[$0, $1]\"\n    },\n    \"\\\"inputs\\\"\": [\n      \"00-09\"\n    ],\n    \"\\\"rowType\\\"\": \"RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7)\",\n    \"\\\"rowCount\\\"\": 1000.0,\n    \"\\\"cumulativeCost\\\"\": \"{2000.0 rows, 20000.02 cpu, 20000.0 io, 20000.0 network, 0.0 memory}\"\n  },\n  \"\\\"00-09\\\"\": {\n    \"\\\"op\\\"\": \"com.dremio.exec.store.jdbc.rel.JdbcPrel\",\n    \"\\\"values\\\"\": {\n      \"\\\"sql\\\"\": \"SELECT \\\"aa\\\".\\\"c_5\\\", \\\"aa\\\".\\\"c_7\\\"\\nFROM \\\"my_schema\\\".\\\"aa\\\"\"\n    },\n    \"\\\"inputs\\\"\": [],\n    \"\\\"rowType\\\"\": \"RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7)\",\n    \"\\\"rowCount\\\"\": 1000.0,\n    \"\\\"cumulativeCost\\\"\": \"{1000.0 rows, 20000.0 cpu, 20000.0 io, 20000.0 network, 0.0 memory}\"\n  }\n}","clientInfo":{"name":"Dremio Java local client","version":"21.2.2-202206231521110575-8e61cb30","majorVersion":21,"minorVersion":2,"patchVersion":2,"application":"3931056@uklvadsb0039","buildNumber":2,"versionQualifier":"202206231521110575-8e61cb30"},"planPhases":[{"phaseName":"Validation","durationMillis":123,"plan":""},{"phaseName":"Convert To Rel","durationMillis":111,"plan":"LogicalProject(c_7=[$6])\n  LogicalFilter(condition=[=($4, '6_c_5_data')])\n    ScanCrel(table=[arp_test.my_schema.aa], columns=[`c_1`, `c_2`, `c_3`, `c_4`, `c_5`, `c_6`, `c_7`, `c_8`, `c_9`, `c_10`, `c_11`, `c_12`, `c_13`, `c_14`, `c_15`, `c_16`, `c_17`, `c_18`, `c_19`, `c_20`, `c_21`, `c_22`, `c_23`, `c_24`, `c_25`, `c_26`, `c_27`, `c_28`, `c_29`, `c_30`], splits=[1])\n"},{"phaseName":"Reduce Expressions","durationMillis":20,"plan":""},{"phaseName":"Window Function Rewrites","durationMillis":3,"plan":""},{"phaseName":"GroupSet Rewrites","durationMillis":0,"plan":""},{"phaseName":"Project Pushdown","durationMillis":0,"plan":""},{"phaseName":"Pre-Logical Filter Pushdown","durationMillis":0,"plan":""},{"phaseName":"Transitive Predicate Pullup","durationMillis":0,"plan":"LogicalProject(c_7=[$1])\n  LogicalFilter(condition=[=($0, '6_c_5_data')])\n    ScanCrel(table=[arp_test.my_schema.aa], columns=[`c_5`, `c_7`], splits=[1])\n"},{"phaseName":"Pre-Logical Transitive Filter Pushdown","durationMillis":0,"plan":""},{"phaseName":"Find Materializations","durationMillis":7,"plan":""},{"phaseName":"Substitution","durationMillis":0,"plan":""},{"phaseName":"Logical Planning","durationMillis":48,"plan":"ProjectRel(c_7=[$1])\n  FilterRel(condition=[=($0, '6_c_5_data')])\n    JdbcCrel\n      JdbcTableScan(table=[arp_test.my_schema.aa], columns=[`c_5`, `c_7`], splits=[1])\n"},{"phaseName":"Relational Planning","durationMillis":17,"plan":""},{"phaseName":"Multi-Join analysis","durationMillis":2,"plan":""},{"phaseName":"LOPT Join Planning","durationMillis":0,"plan":""},{"phaseName":"Post Join Optimization","durationMillis":1,"plan":""},{"phaseName":"Physical Planning","durationMillis":44,"plan":""},{"phaseName":"Physical Heuristic Planning","durationMillis":0,"plan":""},{"phaseName":"Final Physical Transformation","durationMillis":38,"plan":"00-00    Screen : rowType = RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData): rowcount = 500.0, cumulative cost = {6050.0 rows, 28150.075 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 297\n00-01      Project(Fragment=[$0], Records=[$1], Path=[$2], Metadata=[$3], Partition=[$4], FileSize=[$5], IcebergMetadata=[$6], fileschema=[$7], PartitionData=[$8]) : rowType = RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData): rowcount = 500.0, cumulative cost = {6000.0 rows, 28100.075 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 296\n00-02        WriterCommitter(final=[/home/infra2/dremio/data/pdfs/results/1b0943bf-994d-9548-1f50-cfd898c98700]) : rowType = RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData): rowcount = 500.0, cumulative cost = {5500.0 rows, 28100.030000000002 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 295\n00-03          Writer : rowType = RecordType(VARCHAR(65536) Fragment, BIGINT Records, VARCHAR(65536) Path, VARBINARY(65536) Metadata, INTEGER Partition, BIGINT FileSize, VARBINARY(65536) IcebergMetadata, VARBINARY(65536) fileschema, VARBINARY(65536) ARRAY PartitionData): rowcount = 500.0, cumulative cost = {5000.0 rows, 27600.030000000002 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 294\n00-04            Project(c_7=[$0]) : rowType = RecordType(VARCHAR(65536) c_7): rowcount = 500.0, cumulative cost = {4500.0 rows, 27100.030000000002 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 293\n00-05              Project(c_7=[$1]) : rowType = RecordType(VARCHAR(65536) c_7): rowcount = 500.0, cumulative cost = {4000.0 rows, 27100.025 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 292\n00-06                SelectionVectorRemover : rowType = RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7): rowcount = 500.0, cumulative cost = {3500.0 rows, 27100.02 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 291\n00-07                  Filter(condition=[=($0, '6_c_5_data')]) : rowType = RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7): rowcount = 500.0, cumulative cost = {3000.0 rows, 26600.02 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 290\n00-08                    Project(c_5=[$0], c_7=[$1]) : rowType = RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7): rowcount = 1000.0, cumulative cost = {2000.0 rows, 20000.02 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 289\n00-09                      Jdbc(sql=[SELECT \"aa\".\"c_5\", \"aa\".\"c_7\"\nFROM \"my_schema\".\"aa\"]) : rowType = RecordType(VARCHAR(65536) c_5, VARCHAR(65536) c_7): rowcount = 1000.0, cumulative cost = {1000.0 rows, 20000.0 cpu, 20000.0 io, 20000.0 network, 0.0 memory}, id = 189\n"},{"phaseName":"Execution Plan: Executor Selection","durationMillis":1,"plan":"idealNumFragments: 1\nidealNumNodes    : 1\nnumExecutors     : 1\ndetails          : defaultEngine:1,blacklist(0): selectedEndpoints: uklvadsb0039.uk.dev.net:45678 hardAffinity: false"},{"phaseName":"Execution Plan: Fragment Assignment","durationMillis":21,"plan":""},{"phaseName":"Execution Plan: Plan Generation","durationMillis":170,"plan":""},{"phaseName":"Fragment Start RPCs","durationMillis":87,"sizeStats":{"sizePerNode":3572,"fragments":[{"majorId":0,"majorPortionSize":2855,"minorPortionSize":45}],"minorSpecificAttrs":[],"sharedAttrs":[]}},{"phaseName":"Fragment Activate RPCs","durationMillis":2},{"phaseName":"arp_test.my_schema.aa: PERMISSION_CACHE_MISS","durationMillis":3},{"phaseName":"arp_test.my_schema.aa: CACHED_METADATA","durationMillis":13}],"accelerationProfile":{"accelerated":false,"numSubstitutions":0,"millisTakenGettingMaterializations":7,"millisTakenNormalizing":0,"millisTakenSubstituting":0,"layoutProfiles":[],"normalizedQueryPlans":[],"accelerationDetails":""},"nonDefaultOptionsJSON":"[ {\n  \"kind\" : \"STRING\",\n  \"type\" : \"QUERY\",\n  \"name\" : \"planner.query_results_store_path\",\n  \"string_val\" : \"__jobResultsStore\"\n}, {\n  \"kind\" : \"BOOLEAN\",\n  \"type\" : \"QUERY\",\n  \"name\" : \"planner.output_limit_enable\",\n  \"bool_val\" : true\n}, {\n  \"kind\" : \"STRING\",\n  \"type\" : \"QUERY\",\n  \"name\" : \"planner.store_query_results\",\n  \"string_val\" : \"PATH_AND_ATTEMPT_ID\"\n}, {\n  \"kind\" : \"BOOLEAN\",\n  \"type\" : \"SYSTEM\",\n  \"name\" : \"support.admins.email\",\n  \"bool_val\" : false\n}, {\n  \"kind\" : \"BOOLEAN\",\n  \"type\" : \"SYSTEM\",\n  \"name\" : \"client.tools.powerbi\",\n  \"bool_val\" : false\n}, {\n  \"kind\" : \"BOOLEAN\",\n  \"type\" : \"SYSTEM\",\n  \"name\" : \"support.users.email\",\n  \"bool_val\" : false\n}, {\n  \"kind\" : \"BOOLEAN\",\n  \"type\" : \"SYSTEM\",\n  \"name\" : \"client.tools.tableau\",\n  \"bool_val\" : false\n} ]","datasetProfile":[{"datasetPath":"arp_test.my_schema.aa","type":1,"batchSchema":"EAAAAAAACgAMAAAACAAEAAoAAAAIAAAACAAAAAAAAAAeAAAA3AUAAJgFAABoBQAAOAUAAAgFAADYBAAAqAQAAHgEAABIBAAAFAQAAOADAACsAwAAeAMAAEQDAAAQAwAA3AIAAKgCAAB0AgAAQAIAAAwCAADYAQAApAEAAHABAAA8AQAACAEAANQAAACgAAAAbAAAADgAAAAEAAAArvr//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAACc+v//BAAAAGNfMzAAAAAA3vr//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAADM+v//BAAAAGNfMjkAAAAADvv//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAD8+v//BAAAAGNfMjgAAAAAPvv//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAAs+///BAAAAGNfMjcAAAAAbvv//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAABc+///BAAAAGNfMjYAAAAAnvv//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAACM+///BAAAAGNfMjUAAAAAzvv//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAC8+///BAAAAGNfMjQAAAAA/vv//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAADs+///BAAAAGNfMjMAAAAALvz//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAAc/P//BAAAAGNfMjIAAAAAXvz//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAABM/P//BAAAAGNfMjEAAAAAjvz//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAB8/P//BAAAAGNfMjAAAAAAvvz//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAACs/P//BAAAAGNfMTkAAAAA7vz//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAADc/P//BAAAAGNfMTgAAAAAHv3//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAAM/f//BAAAAGNfMTcAAAAATv3//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAA8/f//BAAAAGNfMTYAAAAAfv3//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAABs/f//BAAAAGNfMTUAAAAArv3//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAACc/f//BAAAAGNfMTQAAAAA3v3//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAADM/f//BAAAAGNfMTMAAAAADv7//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAD8/f//BAAAAGNfMTIAAAAAPv7//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAAs/v//BAAAAGNfMTEAAAAAbv7//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAABc/v//BAAAAGNfMTAAAAAAnv7//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAACM/v//AwAAAGNfOQDK/v//FAAAABQAAAAUAAAAAAAFARAAAAAAAAAAAAAAALj+//8DAAAAY184APb+//8UAAAAFAAAABQAAAAAAAUBEAAAAAAAAAAAAAAA5P7//wMAAABjXzcAIv///xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAAQ////AwAAAGNfNgBO////FAAAABQAAAAUAAAAAAAFARAAAAAAAAAAAAAAADz///8DAAAAY181AHr///8UAAAAFAAAABQAAAAAAAUBEAAAAAAAAAAAAAAAaP///wMAAABjXzQApv///xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAACU////AwAAAGNfMwDS////FAAAABQAAAAUAAAAAAAFARAAAAAAAAAAAAAAAMD///8DAAAAY18yAAAAEgAYABQAEwASAAwAAAAIAAQAEgAAABQAAAAUAAAAGAAAAAAABQEUAAAAAAAAAAAAAAAEAAQABAAAAAMAAABjXzEA","allowApproxStats":false},{"datasetPath":"arp_test.my_schema.aa","type":1,"batchSchema":"EAAAAAAACgAMAAAACAAEAAoAAAAIAAAACAAAAAAAAAAeAAAA3AUAAJgFAABoBQAAOAUAAAgFAADYBAAAqAQAAHgEAABIBAAAFAQAAOADAACsAwAAeAMAAEQDAAAQAwAA3AIAAKgCAAB0AgAAQAIAAAwCAADYAQAApAEAAHABAAA8AQAACAEAANQAAACgAAAAbAAAADgAAAAEAAAArvr//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAACc+v//BAAAAGNfMzAAAAAA3vr//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAADM+v//BAAAAGNfMjkAAAAADvv//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAD8+v//BAAAAGNfMjgAAAAAPvv//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAAs+///BAAAAGNfMjcAAAAAbvv//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAABc+///BAAAAGNfMjYAAAAAnvv//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAACM+///BAAAAGNfMjUAAAAAzvv//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAC8+///BAAAAGNfMjQAAAAA/vv//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAADs+///BAAAAGNfMjMAAAAALvz//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAAc/P//BAAAAGNfMjIAAAAAXvz//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAABM/P//BAAAAGNfMjEAAAAAjvz//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAB8/P//BAAAAGNfMjAAAAAAvvz//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAACs/P//BAAAAGNfMTkAAAAA7vz//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAADc/P//BAAAAGNfMTgAAAAAHv3//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAAM/f//BAAAAGNfMTcAAAAATv3//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAA8/f//BAAAAGNfMTYAAAAAfv3//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAABs/f//BAAAAGNfMTUAAAAArv3//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAACc/f//BAAAAGNfMTQAAAAA3v3//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAADM/f//BAAAAGNfMTMAAAAADv7//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAD8/f//BAAAAGNfMTIAAAAAPv7//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAAs/v//BAAAAGNfMTEAAAAAbv7//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAABc/v//BAAAAGNfMTAAAAAAnv7//xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAACM/v//AwAAAGNfOQDK/v//FAAAABQAAAAUAAAAAAAFARAAAAAAAAAAAAAAALj+//8DAAAAY184APb+//8UAAAAFAAAABQAAAAAAAUBEAAAAAAAAAAAAAAA5P7//wMAAABjXzcAIv///xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAAAQ////AwAAAGNfNgBO////FAAAABQAAAAUAAAAAAAFARAAAAAAAAAAAAAAADz///8DAAAAY181AHr///8UAAAAFAAAABQAAAAAAAUBEAAAAAAAAAAAAAAAaP///wMAAABjXzQApv///xQAAAAUAAAAFAAAAAAABQEQAAAAAAAAAAAAAACU////AwAAAGNfMwDS////FAAAABQAAAAUAAAAAAAFARAAAAAAAAAAAAAAAMD///8DAAAAY18yAAAAEgAYABQAEwASAAwAAAAIAAQAEgAAABQAAAAUAAAAGAAAAAAABQEUAAAAAAAAAAAAAAAEAAQABAAAAAMAAABjXzEA","allowApproxStats":false}],"nodeProfile":[{"endpoint":{"address":"uklvadsb0039.uk.dev.net","fabricPort":45678},"maxMemoryUsed":10887296,"timeEnqueuedBeforeSubmitMs":0}],"dremioVersion":"21.2.2-202206231521110575-8e61cb30","operatorTypeMetricsMap":{"metricsDef":[{"metricDef":[{"id":0,"name":"BYTES_SENT"}]},{"metricDef":[{"id":0,"name":"N_RECEIVERS"},{"id":1,"name":"BYTES_SENT"}]},{"metricDef":[{"id":0,"name":"JAVA_BUILD_TIME"},{"id":1,"name":"JAVA_EXECUTE_TIME"},{"id":2,"name":"GANDIVA_BUILD_TIME"},{"id":3,"name":"GANDIVA_EXECUTE_TIME"},{"id":4,"name":"JAVA_EXPRESSIONS"},{"id":5,"name":"GANDIVA_EXPRESSIONS"},{"id":6,"name":"MIXED_SPLITS"}]},{"metricDef":[{"id":0,"name":"NUM_BUCKETS"},{"id":1,"name":"NUM_ENTRIES"},{"id":2,"name":"NUM_RESIZING"},{"id":3,"name":"RESIZING_TIME"},{"id":4,"name":"PIVOT_TIME"},{"id":5,"name":"INSERT_TIME"},{"id":6,"name":"ACCUMULATE_TIME"},{"id":7,"name":"REVERSE_TIME"},{"id":8,"name":"UNPIVOT_TIME"},{"id":9,"name":"VECTORIZED"},{"id":10,"name":"NUM_HASH_PARTITIONS"},{"id":11,"name":"MAX_HASHTABLE_BATCH_SIZE"},{"id":12,"name":"MIN_HASHTABLE_ENTRIES"},{"id":13,"name":"MAX_HASHTABLE_ENTRIES"},{"id":14,"name":"MIN_REHASH_COUNT"},{"id":15,"name":"MAX_REHASH_COUNT"},{"id":16,"name":"PREALLOCATED_MEMORY"},{"id":17,"name":"SPILL_COUNT"},{"id":18,"name":"PARTITIONS_SPILLED"},{"id":19,"name":"ITERATIONS"},{"id":20,"name":"RAN_OUT_OF_MEMORY"},{"id":21,"name":"SPILL_TIME"},{"id":22,"name":"READ_SPILLED_BATCH_TIME"},{"id":23,"name":"TOTAL_BATCHES_SPILLED"},{"id":24,"name":"MAX_BATCHES_SPILLED"},{"id":25,"name":"TOTAL_RECORDS_SPILLED"},{"id":26,"name":"MAX_RECORDS_SPILLED"},{"id":27,"name":"RECURSION_DEPTH"},{"id":28,"name":"TOTAL_SPILLED_DATA_SIZE"},{"id":29,"name":"MAX_SPILLED_DATA_SIZE"},{"id":30,"name":"MAX_TOTAL_NUM_BUCKETS"},{"id":31,"name":"MAX_TOTAL_NUM_ENTRIES"},{"id":32,"name":"ALLOCATED_FOR_FIXED_KEYS"},{"id":33,"name":"UNUSED_FOR_FIXED_KEYS"},{"id":34,"name":"ALLOCATED_FOR_VARIABLE_KEYS"},{"id":35,"name":"UNUSED_FOR_VARIABLE_KEYS"},{"id":36,"name":"MAX_VARIABLE_BLOCK_LENGTH"},{"id":37,"name":"NUM_SPLICE"},{"id":38,"name":"SPLICE_TIME_NS"},{"id":39,"name":"NUM_FORCE_ACCUM"},{"id":40,"name":"FORCE_ACCUM_TIME_NS"},{"id":41,"name":"NUM_ACCUM_COMPACTS"},{"id":42,"name":"ACCUM_COMPACTS_TIME_NS"},{"id":43,"name":"MAX_VARLEN_KEY_SIZE"},{"id":44,"name":"OOB_SENDS"},{"id":45,"name":"OOB_RECEIVES"},{"id":46,"name":"OOB_DROP_LOCAL"},{"id":47,"name":"OOB_DROP_WRONG_STATE"},{"id":48,"name":"OOB_DROP_UNDER_THRESHOLD"},{"id":49,"name":"OOB_DROP_NO_VICTIM"},{"id":50,"name":"OOB_SPILL"},{"id":51,"name":"OOB_DROP_ALREADY_SPILLING"}]},{"metricDef":[{"id":0,"name":"NUM_BUCKETS"},{"id":1,"name":"NUM_ENTRIES"},{"id":2,"name":"NUM_RESIZING"},{"id":3,"name":"RESIZING_TIME_NANOS"},{"id":4,"name":"PIVOT_TIME_NANOS"},{"id":5,"name":"INSERT_TIME_NANOS"},{"id":6,"name":"ACCUMULATE_TIME_NANOS"},{"id":7,"name":"REVERSE_TIME_NANOS"},{"id":8,"name":"UNPIVOT_TIME_NANOS"},{"id":9,"name":"VECTORIZED"},{"id":10,"name":"PROBE_PIVOT_NANOS"},{"id":11,"name":"PROBE_MATCH_NANOS"},{"id":12,"name":"PROBE_LIST_NANOS"},{"id":13,"name":"PROBE_FIND_NANOS"},{"id":14,"name":"PROBE_COPY_NANOS"},{"id":15,"name":"BUILD_COPY_NANOS"},{"id":16,"name":"BUILD_COPY_NOMATCH_NANOS"},{"id":17,"name":"LINK_TIME_NANOS"},{"id":18,"name":"UNMATCHED_BUILD_KEY_COUNT"},{"id":19,"name":"UNMATCHED_PROBE_COUNT"},{"id":20,"name":"OUTPUT_RECORDS"},{"id":21,"name":"HASHCOMPUTATION_TIME_NANOS"},{"id":22,"name":"PROBE_HASHCOMPUTATION_TIME_NANOS"},{"id":23,"name":"RUNTIME_FILTER_DROP_COUNT"},{"id":24,"name":"RUNTIME_COL_FILTER_DROP_COUNT"},{"id":25,"name":"DUPLICATE_BUILD_RECORD_COUNT"},{"id":26,"name":"EXTRA_CONDITION_EVALUATION_COUNT"},{"id":27,"name":"EXTRA_CONDITION_EVALUATION_MATCHED"},{"id":28,"name":"EXTRA_CONDITION_SETUP_NANOS"},{"id":29,"name":"BUILD_CARRYOVER_COPY_NANOS"}]},{"metricDef":[]},{"metricDef":[{"id":0,"name":"BATCHES_SENT"},{"id":1,"name":"RECORDS_SENT"},{"id":2,"name":"MIN_RECORDS"},{"id":3,"name":"MAX_RECORDS"},{"id":4,"name":"N_RECEIVERS"},{"id":5,"name":"BYTES_SENT"},{"id":6,"name":"SENDING_THREADS_COUNT"},{"id":7,"name":"COST"},{"id":8,"name":"NUM_COPIES"},{"id":9,"name":"FIXED_COPY_NS"},{"id":10,"name":"VARIABLE_COPY_NS"},{"id":11,"name":"BINARY_COPY_NS"},{"id":12,"name":"GENERIC_COPY_NS"},{"id":13,"name":"PRECOPY_NS"},{"id":14,"name":"FLUSH_NS"},{"id":15,"name":"NUM_FLUSHES"},{"id":16,"name":"BUCKET_SIZE"}]},{"metricDef":[]},{"metricDef":[{"id":0,"name":"BYTES_RECEIVED"},{"id":1,"name":"NUM_SENDERS"},{"id":2,"name":"NEXT_WAIT_NANOS"}]},{"metricDef":[]},{"metricDef":[{"id":0,"name":"JAVA_EXPRESSIONS"},{"id":1,"name":"JAVA_BUILD_TIME"},{"id":2,"name":"JAVA_EVALUATE_TIME"},{"id":3,"name":"GANDIVA_EXPRESSIONS"},{"id":4,"name":"GANDIVA_BUILD_TIME"},{"id":5,"name":"GANDIVA_EVALUATE_TIME"},{"id":6,"name":"MIXED_EXPRESSIONS"},{"id":7,"name":"MIXED_SPLITS"}]},

[ljd520cc]

@balaji.ramaswamy
----- part 2 of profile_attempt_0.json —

{"metricDef":[{"id":0,"name":"BYTES_RECEIVED"},{"id":1,"name":"NUM_SENDERS"}]},{"metricDef":[]},{"metricDef":[{"id":0,"name":"BYTES_SENT"}]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[{"id":0,"name":"SPILL_COUNT"},{"id":1,"name":"MERGE_COUNT"},{"id":2,"name":"PEAK_BATCHES_IN_MEMORY"},{"id":3,"name":"MAX_BATCH_SIZE"},{"id":4,"name":"AVG_BATCH_SIZE"},{"id":5,"name":"SPILL_TIME_NANOS"},{"id":6,"name":"MERGE_TIME_NANOS"},{"id":7,"name":"TOTAL_SPILLED_DATA_SIZE"},{"id":8,"name":"BATCHES_SPILLED"},{"id":9,"name":"OOB_SENDS"},{"id":10,"name":"OOB_RECEIVES"},{"id":11,"name":"OOB_DROP_LOCAL"},{"id":12,"name":"OOB_DROP_WRONG_STATE"},{"id":13,"name":"OOB_DROP_UNDER_THRESHOLD"},{"id":14,"name":"OOB_SPILL"},{"id":15,"name":"UNCOMPRESSED_BYTES_WRITTEN"},{"id":16,"name":"IO_BYTES_WRITTEN"},{"id":17,"name":"UNCOMPRESSED_BYTES_READ"},{"id":18,"name":"IO_BYTES_READ"},{"id":19,"name":"COMPRESSION_NANOS"},{"id":20,"name":"DECOMPRESSION_NANOS"},{"id":21,"name":"IO_WRITE_WAIT_NANOS"},{"id":22,"name":"IO_READ_WAIT_NANOS"},{"id":23,"name":"SPILL_COPY_NANOS"},{"id":24,"name":"OOM_ALLOCATE_COUNT"},{"id":25,"name":"OOM_COPY_COUNT"}]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[{"id":0,"name":"SETUP_NS"},{"id":1,"name":"NUM_READERS"},{"id":2,"name":"NUM_REMOTE_READERS"},{"id":3,"name":"NUM_ROW_GROUPS"},{"id":4,"name":"NUM_VECTORIZED_COLUMNS"},{"id":5,"name":"NUM_NON_VECTORIZED_COLUMNS"},{"id":6,"name":"COPY_NS"},{"id":7,"name":"FILTER_NS"},{"id":8,"name":"PARQUET_EXEC_PATH"},{"id":9,"name":"FILTER_EXISTS"},{"id":10,"name":"PARQUET_BYTES_READ"},{"id":11,"name":"NUM_ASYNC_STREAMS"},{"id":12,"name":"NUM_ASYNC_READS"},{"id":13,"name":"NUM_ASYNC_BYTES_READ"},{"id":14,"name":"NUM_EXTENDING_READS"},{"id":15,"name":"EXTENDING_READ_BYTES"},{"id":16,"name":"TOTAL_BYTES_READ"},{"id":17,"name":"LOCAL_BYTES_READ"},{"id":18,"name":"SHORT_CIRCUIT_BYTES_READ"},{"id":19,"name":"PRELOADED_BYTES"},{"id":20,"name":"NUM_CACHE_HITS"},{"id":21,"name":"NUM_CACHE_MISSES"},{"id":22,"name":"BLOCK_AFFINITY_CACHE_HITS"},{"id":23,"name":"BLOCK_AFFINITY_CACHE_MISSES"},{"id":24,"name":"AVG_PROCESSING_TIME_NS"},{"id":25,"name":"JAVA_BUILD_TIME_NS"},{"id":26,"name":"JAVA_EXECUTE_TIME_NS"},{"id":27,"name":"GANDIVA_BUILD_TIME_NS"},{"id":28,"name":"GANDIVA_EXECUTE_TIME_NS"},{"id":29,"name":"NUM_FILTERS_MODIFIED"},{"id":30,"name":"NUM_HIVE_PARQUET_TRUNCATE_VARCHAR"},{"id":31,"name":"TOTAL_HIVE_PARQUET_TRUNCATE_VARCHAR"},{"id":32,"name":"TOTAL_HIVE_PARQUET_TRANSFER_VARCHAR"},{"id":33,"name":"HIVE_PARQUET_CHECK_VARCHAR_CAST_TIME_NS"},{"id":34,"name":"NUM_ROW_GROUPS_PRUNED"},{"id":35,"name":"MAX_ROW_GROUPS_IN_HIVE_FILE_SPLITS"},{"id":36,"name":"NUM_HIVE_FILE_SPLITS_WITH_NO_ROWGROUPS"},{"id":37,"name":"MIN_IO_READ_TIME_NS"},{"id":38,"name":"MAX_IO_READ_TIME_NS"},{"id":39,"name":"AVG_IO_READ_TIME_NS"},{"id":40,"name":"NUM_IO_READ"},{"id":41,"name":"NUM_HIVE_PARQUET_DECIMAL_COERCIONS"},{"id":42,"name":"NUM_ROW_GROUPS_TRIMMED"},{"id":43,"name":"NUM_COLUMNS_TRIMMED"},{"id":44,"name":"NUM_PARTITIONS_PRUNED"},{"id":45,"name":"NUM_BOOSTED_FILE_READS"},{"id":46,"name":"MAX_BOOSTED_FILE_READ_TIME_NS"},{"id":47,"name":"AVG_BOOSTED_FILE_READ_TIME_NS"},{"id":48,"name":"TOTAL_BOOSTED_BYTES_READ"},{"id":49,"name":"NUM_COLUMNS_BOOSTED"},{"id":50,"name":"OFFSET_INDEX_READ"},{"id":51,"name":"COLUMN_INDEX_READ"},{"id":52,"name":"NUM_BOOSTED_RECORD_BATCHES_PRUNED"},{"id":53,"name":"NUM_PAGES_PRUNED"},{"id":54,"name":"NUM_PAGES_READ"},{"id":55,"name":"PAGE_DECOMPRESSION_TIME_NS"},{"id":56,"name":"NUM_RUNTIME_FILTERS"},{"id":57,"name":"RUNTIME_COL_FILTER_DROP_COUNT"},{"id":58,"name":"ROW_GROUPS_SCANNED_WITH_RUNTIME_FILTER"},{"id":59,"name":"RLE_PAGE_DECODING_READ_TIME_NS"},{"id":60,"name":"RLE_PAGE_DECODING_WRITE_TIME_NS"},{"id":61,"name":"PACKED_PAGE_DECODING_READ_TIME_NS"},{"id":62,"name":"PACKED_PAGE_DECODING_WRITE_TIME_NS"},{"id":63,"name":"MIN_METADATA_IO_READ_TIME_NS"},{"id":64,"name":"MAX_METADATA_IO_READ_TIME_NS"},{"id":65,"name":"AVG_METADATA_IO_READ_TIME_NS"},{"id":66,"name":"NUM_METADATA_IO_READ"},{"id":67,"name":"HIVE_FILE_FORMATS"},{"id":68,"name":"NUM_ZERO_SIZED_COLUMN"},{"id":69,"name":"NUM_EXTRA_FOOTER_READS"}]},{"metricDef":[{"id":0,"name":"SETUP_NS"},{"id":1,"name":"NUM_READERS"},{"id":2,"name":"NUM_REMOTE_READERS"},{"id":3,"name":"NUM_ROW_GROUPS"},{"id":4,"name":"NUM_VECTORIZED_COLUMNS"},{"id":5,"name":"NUM_NON_VECTORIZED_COLUMNS"},{"id":6,"name":"COPY_NS"},{"id":7,"name":"FILTER_NS"},{"id":8,"name":"PARQUET_EXEC_PATH"},{"id":9,"name":"FILTER_EXISTS"},{"id":10,"name":"PARQUET_BYTES_READ"},{"id":11,"name":"NUM_ASYNC_STREAMS"},{"id":12,"name":"NUM_ASYNC_READS"},{"id":13,"name":"NUM_ASYNC_BYTES_READ"},{"id":14,"name":"NUM_EXTENDING_READS"},{"id":15,"name":"EXTENDING_READ_BYTES"},{"id":16,"name":"TOTAL_BYTES_READ"},{"id":17,"name":"LOCAL_BYTES_READ"},{"id":18,"name":"SHORT_CIRCUIT_BYTES_READ"},{"id":19,"name":"PRELOADED_BYTES"},{"id":20,"name":"NUM_CACHE_HITS"},{"id":21,"name":"NUM_CACHE_MISSES"},{"id":22,"name":"BLOCK_AFFINITY_CACHE_HITS"},{"id":23,"name":"BLOCK_AFFINITY_CACHE_MISSES"},{"id":24,"name":"AVG_PROCESSING_TIME_NS"},{"id":25,"name":"JAVA_BUILD_TIME_NS"},{"id":26,"name":"JAVA_EXECUTE_TIME_NS"},{"id":27,"name":"GANDIVA_BUILD_TIME_NS"},{"id":28,"name":"GANDIVA_EXECUTE_TIME_NS"},{"id":29,"name":"NUM_FILTERS_MODIFIED"},{"id":30,"name":"NUM_HIVE_PARQUET_TRUNCATE_VARCHAR"},{"id":31,"name":"TOTAL_HIVE_PARQUET_TRUNCATE_VARCHAR"},{"id":32,"name":"TOTAL_HIVE_PARQUET_TRANSFER_VARCHAR"},{"id":33,"name":"HIVE_PARQUET_CHECK_VARCHAR_CAST_TIME_NS"},{"id":34,"name":"NUM_ROW_GROUPS_PRUNED"},{"id":35,"name":"MAX_ROW_GROUPS_IN_HIVE_FILE_SPLITS"},{"id":36,"name":"NUM_HIVE_FILE_SPLITS_WITH_NO_ROWGROUPS"},{"id":37,"name":"MIN_IO_READ_TIME_NS"},{"id":38,"name":"MAX_IO_READ_TIME_NS"},{"id":39,"name":"AVG_IO_READ_TIME_NS"},{"id":40,"name":"NUM_IO_READ"},{"id":41,"name":"NUM_HIVE_PARQUET_DECIMAL_COERCIONS"},{"id":42,"name":"NUM_ROW_GROUPS_TRIMMED"},{"id":43,"name":"NUM_COLUMNS_TRIMMED"},{"id":44,"name":"NUM_PARTITIONS_PRUNED"},{"id":45,"name":"NUM_BOOSTED_FILE_READS"},{"id":46,"name":"MAX_BOOSTED_FILE_READ_TIME_NS"},{"id":47,"name":"AVG_BOOSTED_FILE_READ_TIME_NS"},{"id":48,"name":"TOTAL_BOOSTED_BYTES_READ"},{"id":49,"name":"NUM_COLUMNS_BOOSTED"},{"id":50,"name":"OFFSET_INDEX_READ"},{"id":51,"name":"COLUMN_INDEX_READ"},{"id":52,"name":"NUM_BOOSTED_RECORD_BATCHES_PRUNED"},{"id":53,"name":"NUM_PAGES_PRUNED"},{"id":54,"name":"NUM_PAGES_READ"},{"id":55,"name":"PAGE_DECOMPRESSION_TIME_NS"},{"id":56,"name":"NUM_RUNTIME_FILTERS"},{"id":57,"name":"RUNTIME_COL_FILTER_DROP_COUNT"},{"id":58,"name":"ROW_GROUPS_SCANNED_WITH_RUNTIME_FILTER"},{"id":59,"name":"RLE_PAGE_DECODING_READ_TIME_NS"},{"id":60,"name":"RLE_PAGE_DECODING_WRITE_TIME_NS"},{"id":61,"name":"PACKED_PAGE_DECODING_READ_TIME_NS"},{"id":62,"name":"PACKED_PAGE_DECODING_WRITE_TIME_NS"},{"id":63,"name":"MIN_METADATA_IO_READ_TIME_NS"},{"id":64,"name":"MAX_METADATA_IO_READ_TIME_NS"},{"id":65,"name":"AVG_METADATA_IO_READ_TIME_NS"},{"id":66,"name":"NUM_METADATA_IO_READ"},{"id":67,"name":"HIVE_FILE_FORMATS"},{"id":68,"name":"NUM_ZERO_SIZED_COLUMN"},{"id":69,"name":"NUM_EXTRA_FOOTER_READS"}]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[{"id":0,"name":"NUM_FILES_WRITTEN"},{"id":1,"name":"MIN_FILE_SIZE"},{"id":2,"name":"MAX_FILE_SIZE"},{"id":3,"name":"AVG_FILE_SIZE"},{"id":4,"name":"MIN_RECORD_COUNT_IN_FILE"},{"id":5,"name":"MAX_RECORD_COUNT_IN_FILE"},{"id":6,"name":"MIN_IO_WRITE_TIME"},{"id":7,"name":"MAX_IO_WRITE_TIME"},{"id":8,"name":"AVG_IO_WRITE_TIME"},{"id":9,"name":"NUM_IO_WRITE"}]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[{"id":0,"name":"SETUP_NS"},{"id":1,"name":"NUM_READERS"},{"id":2,"name":"NUM_REMOTE_READERS"},{"id":3,"name":"NUM_ROW_GROUPS"},{"id":4,"name":"NUM_VECTORIZED_COLUMNS"},{"id":5,"name":"NUM_NON_VECTORIZED_COLUMNS"},{"id":6,"name":"COPY_NS"},{"id":7,"name":"FILTER_NS"},{"id":8,"name":"PARQUET_EXEC_PATH"},{"id":9,"name":"FILTER_EXISTS"},{"id":10,"name":"PARQUET_BYTES_READ"},{"id":11,"name":"NUM_ASYNC_STREAMS"},{"id":12,"name":"NUM_ASYNC_READS"},{"id":13,"name":"NUM_ASYNC_BYTES_READ"},{"id":14,"name":"NUM_EXTENDING_READS"},{"id":15,"name":"EXTENDING_READ_BYTES"},{"id":16,"name":"TOTAL_BYTES_READ"},{"id":17,"name":"LOCAL_BYTES_READ"},{"id":18,"name":"SHORT_CIRCUIT_BYTES_READ"},{"id":19,"name":"PRELOADED_BYTES"},{"id":20,"name":"NUM_CACHE_HITS"},{"id":21,"name":"NUM_CACHE_MISSES"},{"id":22,"name":"BLOCK_AFFINITY_CACHE_HITS"},{"id":23,"name":"BLOCK_AFFINITY_CACHE_MISSES"},{"id":24,"name":"AVG_PROCESSING_TIME_NS"},{"id":25,"name":"JAVA_BUILD_TIME_NS"},{"id":26,"name":"JAVA_EXECUTE_TIME_NS"},{"id":27,"name":"GANDIVA_BUILD_TIME_NS"},{"id":28,"name":"GANDIVA_EXECUTE_TIME_NS"},{"id":29,"name":"NUM_FILTERS_MODIFIED"},{"id":30,"name":"NUM_HIVE_PARQUET_TRUNCATE_VARCHAR"},{"id":31,"name":"TOTAL_HIVE_PARQUET_TRUNCATE_VARCHAR"},{"id":32,"name":"TOTAL_HIVE_PARQUET_TRANSFER_VARCHAR"},{"id":33,"name":"HIVE_PARQUET_CHECK_VARCHAR_CAST_TIME_NS"},{"id":34,"name":"NUM_ROW_GROUPS_PRUNED"},{"id":35,"name":"MAX_ROW_GROUPS_IN_HIVE_FILE_SPLITS"},{"id":36,"name":"NUM_HIVE_FILE_SPLITS_WITH_NO_ROWGROUPS"},{"id":37,"name":"MIN_IO_READ_TIME_NS"},{"id":38,"name":"MAX_IO_READ_TIME_NS"},{"id":39,"name":"AVG_IO_READ_TIME_NS"},{"id":40,"name":"NUM_IO_READ"},{"id":41,"name":"NUM_HIVE_PARQUET_DECIMAL_COERCIONS"},{"id":42,"name":"NUM_ROW_GROUPS_TRIMMED"},{"id":43,"name":"NUM_COLUMNS_TRIMMED"},{"id":44,"name":"NUM_PARTITIONS_PRUNED"},{"id":45,"name":"NUM_BOOSTED_FILE_READS"},{"id":46,"name":"MAX_BOOSTED_FILE_READ_TIME_NS"},{"id":47,"name":"AVG_BOOSTED_FILE_READ_TIME_NS"},{"id":48,"name":"TOTAL_BOOSTED_BYTES_READ"},{"id":49,"name":"NUM_COLUMNS_BOOSTED"},{"id":50,"name":"OFFSET_INDEX_READ"},{"id":51,"name":"COLUMN_INDEX_READ"},{"id":52,"name":"NUM_BOOSTED_RECORD_BATCHES_PRUNED"},{"id":53,"name":"NUM_PAGES_PRUNED"},{"id":54,"name":"NUM_PAGES_READ"},{"id":55,"name":"PAGE_DECOMPRESSION_TIME_NS"},{"id":56,"name":"NUM_RUNTIME_FILTERS"},{"id":57,"name":"RUNTIME_COL_FILTER_DROP_COUNT"},{"id":58,"name":"ROW_GROUPS_SCANNED_WITH_RUNTIME_FILTER"},{"id":59,"name":"RLE_PAGE_DECODING_READ_TIME_NS"},{"id":60,"name":"RLE_PAGE_DECODING_WRITE_TIME_NS"},{"id":61,"name":"PACKED_PAGE_DECODING_READ_TIME_NS"},{"id":62,"name":"PACKED_PAGE_DECODING_WRITE_TIME_NS"},{"id":63,"name":"MIN_METADATA_IO_READ_TIME_NS"},{"id":64,"name":"MAX_METADATA_IO_READ_TIME_NS"},{"id":65,"name":"AVG_METADATA_IO_READ_TIME_NS"},{"id":66,"name":"NUM_METADATA_IO_READ"},{"id":67,"name":"HIVE_FILE_FORMATS"},{"id":68,"name":"NUM_ZERO_SIZED_COLUMN"},{"id":69,"name":"NUM_EXTRA_FOOTER_READS"}]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[{"id":0,"name":"MATCH_NANOS"},{"id":1,"name":"COPY_NANOS"},{"id":2,"name":"COMPILE_NANOS"},{"id":3,"name":"PROBE_COUNT"}]},{"metricDef":[]},{"metricDef":[{"id":0,"name":"TOTAL_RECORDS_READ"},{"id":1,"name":"NUM_LOCAL_RECORDS_READ"},{"id":2,"name":"NUM_REMOTE_RECORDS_READ"},{"id":3,"name":"TOTAL_BYTES_READ"}]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[{"id":0,"name":"BYTES_WRITTEN"},{"id":1,"name":"OUTPUT_LIMITED"}]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[{"id":0,"name":"ICEBERG_METADATA_WRITE_TIME"},{"id":1,"name":"ICEBERG_CATALOG_UPDATE_TIME"},{"id":2,"name":"ICEBERG_COMMIT_TIME"}]},{"metricDef":[{"id":0,"name":"N_RECEIVERS"},{"id":1,"name":"BYTES_SENT"}]},{"metricDef":[]},{"metricDef":[{"id":0,"name":"SETUP_NS"},{"id":1,"name":"NUM_READERS"},{"id":2,"name":"NUM_REMOTE_READERS"},{"id":3,"name":"NUM_ROW_GROUPS"},{"id":4,"name":"NUM_VECTORIZED_COLUMNS"},{"id":5,"name":"NUM_NON_VECTORIZED_COLUMNS"},{"id":6,"name":"COPY_NS"},{"id":7,"name":"FILTER_NS"},{"id":8,"name":"PARQUET_EXEC_PATH"},{"id":9,"name":"FILTER_EXISTS"},{"id":10,"name":"PARQUET_BYTES_READ"},{"id":11,"name":"NUM_ASYNC_STREAMS"},{"id":12,"name":"NUM_ASYNC_READS"},{"id":13,"name":"NUM_ASYNC_BYTES_READ"},{"id":14,"name":"NUM_EXTENDING_READS"},{"id":15,"name":"EXTENDING_READ_BYTES"},{"id":16,"name":"TOTAL_BYTES_READ"},{"id":17,"name":"LOCAL_BYTES_READ"},{"id":18,"name":"SHORT_CIRCUIT_BYTES_READ"},{"id":19,"name":"PRELOADED_BYTES"},{"id":20,"name":"NUM_CACHE_HITS"},{"id":21,"name":"NUM_CACHE_MISSES"},{"id":22,"name":"BLOCK_AFFINITY_CACHE_HITS"},{"id":23,"name":"BLOCK_AFFINITY_CACHE_MISSES"},{"id":24,"name":"AVG_PROCESSING_TIME_NS"},{"id":25,"name":"JAVA_BUILD_TIME_NS"},{"id":26,"name":"JAVA_EXECUTE_TIME_NS"},{"id":27,"name":"GANDIVA_BUILD_TIME_NS"},{"id":28,"name":"GANDIVA_EXECUTE_TIME_NS"},{"id":29,"name":"NUM_FILTERS_MODIFIED"},{"id":30,"name":"NUM_HIVE_PARQUET_TRUNCATE_VARCHAR"},{"id":31,"name":"TOTAL_HIVE_PARQUET_TRUNCATE_VARCHAR"},{"id":32,"name":"TOTAL_HIVE_PARQUET_TRANSFER_VARCHAR"},{"id":33,"name":"HIVE_PARQUET_CHECK_VARCHAR_CAST_TIME_NS"},{"id":34,"name":"NUM_ROW_GROUPS_PRUNED"},{"id":35,"name":"MAX_ROW_GROUPS_IN_HIVE_FILE_SPLITS"},{"id":36,"name":"NUM_HIVE_FILE_SPLITS_WITH_NO_ROWGROUPS"},{"id":37,"name":"MIN_IO_READ_TIME_NS"},{"id":38,"name":"MAX_IO_READ_TIME_NS"},{"id":39,"name":"AVG_IO_READ_TIME_NS"},{"id":40,"name":"NUM_IO_READ"},{"id":41,"name":"NUM_HIVE_PARQUET_DECIMAL_COERCIONS"},{"id":42,"name":"NUM_ROW_GROUPS_TRIMMED"},{"id":43,"name":"NUM_COLUMNS_TRIMMED"},{"id":44,"name":"NUM_PARTITIONS_PRUNED"},{"id":45,"name":"NUM_BOOSTED_FILE_READS"},{"id":46,"name":"MAX_BOOSTED_FILE_READ_TIME_NS"},{"id":47,"name":"AVG_BOOSTED_FILE_READ_TIME_NS"},{"id":48,"name":"TOTAL_BOOSTED_BYTES_READ"},{"id":49,"name":"NUM_COLUMNS_BOOSTED"},{"id":50,"name":"OFFSET_INDEX_READ"},{"id":51,"name":"COLUMN_INDEX_READ"},{"id":52,"name":"NUM_BOOSTED_RECORD_BATCHES_PRUNED"},{"id":53,"name":"NUM_PAGES_PRUNED"},{"id":54,"name":"NUM_PAGES_READ"},{"id":55,"name":"PAGE_DECOMPRESSION_TIME_NS"},{"id":56,"name":"NUM_RUNTIME_FILTERS"},{"id":57,"name":"RUNTIME_COL_FILTER_DROP_COUNT"},{"id":58,"name":"ROW_GROUPS_SCANNED_WITH_RUNTIME_FILTER"},{"id":59,"name":"RLE_PAGE_DECODING_READ_TIME_NS"},{"id":60,"name":"RLE_PAGE_DECODING_WRITE_TIME_NS"},{"id":61,"name":"PACKED_PAGE_DECODING_READ_TIME_NS"},{"id":62,"name":"PACKED_PAGE_DECODING_WRITE_TIME_NS"},{"id":63,"name":"MIN_METADATA_IO_READ_TIME_NS"},{"id":64,"name":"MAX_METADATA_IO_READ_TIME_NS"},{"id":65,"name":"AVG_METADATA_IO_READ_TIME_NS"},{"id":66,"name":"NUM_METADATA_IO_READ"},{"id":67,"name":"HIVE_FILE_FORMATS"},{"id":68,"name":"NUM_ZERO_SIZED_COLUMN"},{"id":69,"name":"NUM_EXTRA_FOOTER_READS"}]},{"metricDef":[{"id":0,"name":"SETUP_NS"},{"id":1,"name":"NUM_READERS"},{"id":2,"name":"NUM_REMOTE_READERS"},{"id":3,"name":"NUM_ROW_GROUPS"},{"id":4,"name":"NUM_VECTORIZED_COLUMNS"},{"id":5,"name":"NUM_NON_VECTORIZED_COLUMNS"},{"id":6,"name":"COPY_NS"},{"id":7,"name":"FILTER_NS"},{"id":8,"name":"PARQUET_EXEC_PATH"},{"id":9,"name":"FILTER_EXISTS"},{"id":10,"name":"PARQUET_BYTES_READ"},{"id":11,"name":"NUM_ASYNC_STREAMS"},{"id":12,"name":"NUM_ASYNC_READS"},{"id":13,"name":"NUM_ASYNC_BYTES_READ"},{"id":14,"name":"NUM_EXTENDING_READS"},{"id":15,"name":"EXTENDING_READ_BYTES"},{"id":16,"name":"TOTAL_BYTES_READ"},{"id":17,"name":"LOCAL_BYTES_READ"},{"id":18,"name":"SHORT_CIRCUIT_BYTES_READ"},{"id":19,"name":"PRELOADED_BYTES"},{"id":20,"name":"NUM_CACHE_HITS"},{"id":21,"name":"NUM_CACHE_MISSES"},{"id":22,"name":"BLOCK_AFFINITY_CACHE_HITS"},{"id":23,"name":"BLOCK_AFFINITY_CACHE_MISSES"},{"id":24,"name":"AVG_PROCESSING_TIME_NS"},{"id":25,"name":"JAVA_BUILD_TIME_NS"},{"id":26,"name":"JAVA_EXECUTE_TIME_NS"},{"id":27,"name":"GANDIVA_BUILD_TIME_NS"},{"id":28,"name":"GANDIVA_EXECUTE_TIME_NS"},{"id":29,"name":"NUM_FILTERS_MODIFIED"},{"id":30,"name":"NUM_HIVE_PARQUET_TRUNCATE_VARCHAR"},{"id":31,"name":"TOTAL_HIVE_PARQUET_TRUNCATE_VARCHAR"},{"id":32,"name":"TOTAL_HIVE_PARQUET_TRANSFER_VARCHAR"},{"id":33,"name":"HIVE_PARQUET_CHECK_VARCHAR_CAST_TIME_NS"},{"id":34,"name":"NUM_ROW_GROUPS_PRUNED"},{"id":35,"name":"MAX_ROW_GROUPS_IN_HIVE_FILE_SPLITS"},{"id":36,"name":"NUM_HIVE_FILE_SPLITS_WITH_NO_ROWGROUPS"},{"id":37,"name":"MIN_IO_READ_TIME_NS"},{"id":38,"name":"MAX_IO_READ_TIME_NS"},{"id":39,"name":"AVG_IO_READ_TIME_NS"},{"id":40,"name":"NUM_IO_READ"},{"id":41,"name":"NUM_HIVE_PARQUET_DECIMAL_COERCIONS"},{"id":42,"name":"NUM_ROW_GROUPS_TRIMMED"},{"id":43,"name":"NUM_COLUMNS_TRIMMED"},{"id":44,"name":"NUM_PARTITIONS_PRUNED"},{"id":45,"name":"NUM_BOOSTED_FILE_READS"},{"id":46,"name":"MAX_BOOSTED_FILE_READ_TIME_NS"},{"id":47,"name":"AVG_BOOSTED_FILE_READ_TIME_NS"},{"id":48,"name":"TOTAL_BOOSTED_BYTES_READ"},{"id":49,"name":"NUM_COLUMNS_BOOSTED"},{"id":50,"name":"OFFSET_INDEX_READ"},{"id":51,"name":"COLUMN_INDEX_READ"},{"id":52,"name":"NUM_BOOSTED_RECORD_BATCHES_PRUNED"},{"id":53,"name":"NUM_PAGES_PRUNED"},{"id":54,"name":"NUM_PAGES_READ"},{"id":55,"name":"PAGE_DECOMPRESSION_TIME_NS"},{"id":56,"name":"NUM_RUNTIME_FILTERS"},{"id":57,"name":"RUNTIME_COL_FILTER_DROP_COUNT"},{"id":58,"name":"ROW_GROUPS_SCANNED_WITH_RUNTIME_FILTER"},{"id":59,"name":"RLE_PAGE_DECODING_READ_TIME_NS"},{"id":60,"name":"RLE_PAGE_DECODING_WRITE_TIME_NS"},{"id":61,"name":"PACKED_PAGE_DECODING_READ_TIME_NS"},{"id":62,"name":"PACKED_PAGE_DECODING_WRITE_TIME_NS"},{"id":63,"name":"MIN_METADATA_IO_READ_TIME_NS"},{"id":64,"name":"MAX_METADATA_IO_READ_TIME_NS"},{"id":65,"name":"AVG_METADATA_IO_READ_TIME_NS"},{"id":66,"name":"NUM_METADATA_IO_READ"},{"id":67,"name":"HIVE_FILE_FORMATS"},{"id":68,"name":"NUM_ZERO_SIZED_COLUMN"},{"id":69,"name":"NUM_EXTRA_FOOTER_READS"},{"id":70,"name":"NUM_DATA_FILE"},{"id":71,"name":"NUM_MANIFEST_FILE"},{"id":72,"name":"MAX_SCHEMA_WIDTH"}]},{"metricDef":[{"id":0,"name":"SETUP_NS"},{"id":1,"name":"NUM_READERS"},{"id":2,"name":"NUM_REMOTE_READERS"},{"id":3,"name":"NUM_ROW_GROUPS"},{"id":4,"name":"NUM_VECTORIZED_COLUMNS"},{"id":5,"name":"NUM_NON_VECTORIZED_COLUMNS"},{"id":6,"name":"COPY_NS"},{"id":7,"name":"FILTER_NS"},{"id":8,"name":"PARQUET_EXEC_PATH"},{"id":9,"name":"FILTER_EXISTS"},{"id":10,"name":"PARQUET_BYTES_READ"},{"id":11,"name":"NUM_ASYNC_STREAMS"},{"id":12,"name":"NUM_ASYNC_READS"},{"id":13,"name":"NUM_ASYNC_BYTES_READ"},{"id":14,"name":"NUM_EXTENDING_READS"},{"id":15,"name":"EXTENDING_READ_BYTES"},{"id":16,"name":"TOTAL_BYTES_READ"},{"id":17,"name":"LOCAL_BYTES_READ"},{"id":18,"name":"SHORT_CIRCUIT_BYTES_READ"},{"id":19,"name":"PRELOADED_BYTES"},{"id":20,"name":"NUM_CACHE_HITS"},{"id":21,"name":"NUM_CACHE_MISSES"},{"id":22,"name":"BLOCK_AFFINITY_CACHE_HITS"},{"id":23,"name":"BLOCK_AFFINITY_CACHE_MISSES"},{"id":24,"name":"AVG_PROCESSING_TIME_NS"},{"id":25,"name":"JAVA_BUILD_TIME_NS"},{"id":26,"name":"JAVA_EXECUTE_TIME_NS"},{"id":27,"name":"GANDIVA_BUILD_TIME_NS"},{"id":28,"name":"GANDIVA_EXECUTE_TIME_NS"},{"id":29,"name":"NUM_FILTERS_MODIFIED"},{"id":30,"name":"NUM_HIVE_PARQUET_TRUNCATE_VARCHAR"},{"id":31,"name":"TOTAL_HIVE_PARQUET_TRUNCATE_VARCHAR"},{"id":32,"name":"TOTAL_HIVE_PARQUET_TRANSFER_VARCHAR"},{"id":33,"name":"HIVE_PARQUET_CHECK_VARCHAR_CAST_TIME_NS"},{"id":34,"name":"NUM_ROW_GROUPS_PRUNED"},{"id":35,"name":"MAX_ROW_GROUPS_IN_HIVE_FILE_SPLITS"},{"id":36,"name":"NUM_HIVE_FILE_SPLITS_WITH_NO_ROWGROUPS"},{"id":37,"name":"MIN_IO_READ_TIME_NS"},{"id":38,"name":"MAX_IO_READ_TIME_NS"},{"id":39,"name":"AVG_IO_READ_TIME_NS"},{"id":40,"name":"NUM_IO_READ"},{"id":41,"name":"NUM_HIVE_PARQUET_DECIMAL_COERCIONS"},{"id":42,"name":"NUM_ROW_GROUPS_TRIMMED"},{"id":43,"name":"NUM_COLUMNS_TRIMMED"},{"id":44,"name":"NUM_PARTITIONS_PRUNED"},{"id":45,"name":"NUM_BOOSTED_FILE_READS"},{"id":46,"name":"MAX_BOOSTED_FILE_READ_TIME_NS"},{"id":47,"name":"AVG_BOOSTED_FILE_READ_TIME_NS"},{"id":48,"name":"TOTAL_BOOSTED_BYTES_READ"},{"id":49,"name":"NUM_COLUMNS_BOOSTED"},{"id":50,"name":"OFFSET_INDEX_READ"},{"id":51,"name":"COLUMN_INDEX_READ"},{"id":52,"name":"NUM_BOOSTED_RECORD_BATCHES_PRUNED"},{"id":53,"name":"NUM_PAGES_PRUNED"},{"id":54,"name":"NUM_PAGES_READ"},{"id":55,"name":"PAGE_DECOMPRESSION_TIME_NS"},{"id":56,"name":"NUM_RUNTIME_FILTERS"},{"id":57,"name":"RUNTIME_COL_FILTER_DROP_COUNT"},{"id":58,"name":"ROW_GROUPS_SCANNED_WITH_RUNTIME_FILTER"},{"id":59,"name":"RLE_PAGE_DECODING_READ_TIME_NS"},{"id":60,"name":"RLE_PAGE_DECODING_WRITE_TIME_NS"},{"id":61,"name":"PACKED_PAGE_DECODING_READ_TIME_NS"},{"id":62,"name":"PACKED_PAGE_DECODING_WRITE_TIME_NS"},{"id":63,"name":"MIN_METADATA_IO_READ_TIME_NS"},{"id":64,"name":"MAX_METADATA_IO_READ_TIME_NS"},{"id":65,"name":"AVG_METADATA_IO_READ_TIME_NS"},{"id":66,"name":"NUM_METADATA_IO_READ"},{"id":67,"name":"HIVE_FILE_FORMATS"},{"id":68,"name":"NUM_ZERO_SIZED_COLUMN"},{"id":69,"name":"NUM_EXTRA_FOOTER_READS"}]},{"metricDef":[{"id":0,"name":"SETUP_NS"},{"id":1,"name":"NUM_READERS"},{"id":2,"name":"NUM_REMOTE_READERS"},{"id":3,"name":"NUM_ROW_GROUPS"},{"id":4,"name":"NUM_VECTORIZED_COLUMNS"},{"id":5,"name":"NUM_NON_VECTORIZED_COLUMNS"},{"id":6,"name":"COPY_NS"},{"id":7,"name":"FILTER_NS"},{"id":8,"name":"PARQUET_EXEC_PATH"},{"id":9,"name":"FILTER_EXISTS"},{"id":10,"name":"PARQUET_BYTES_READ"},{"id":11,"name":"NUM_ASYNC_STREAMS"},{"id":12,"name":"NUM_ASYNC_READS"},{"id":13,"name":"NUM_ASYNC_BYTES_READ"},{"id":14,"name":"NUM_EXTENDING_READS"},{"id":15,"name":"EXTENDING_READ_BYTES"},{"id":16,"name":"TOTAL_BYTES_READ"},{"id":17,"name":"LOCAL_BYTES_READ"},{"id":18,"name":"SHORT_CIRCUIT_BYTES_READ"},{"id":19,"name":"PRELOADED_BYTES"},{"id":20,"name":"NUM_CACHE_HITS"},{"id":21,"name":"NUM_CACHE_MISSES"},{"id":22,"name":"BLOCK_AFFINITY_CACHE_HITS"},{"id":23,"name":"BLOCK_AFFINITY_CACHE_MISSES"},{"id":24,"name":"AVG_PROCESSING_TIME_NS"},{"id":25,"name":"JAVA_BUILD_TIME_NS"},{"id":26,"name":"JAVA_EXECUTE_TIME_NS"},{"id":27,"name":"GANDIVA_BUILD_TIME_NS"},{"id":28,"name":"GANDIVA_EXECUTE_TIME_NS"},{"id":29,"name":"NUM_FILTERS_MODIFIED"},{"id":30,"name":"NUM_HIVE_PARQUET_TRUNCATE_VARCHAR"},{"id":31,"name":"TOTAL_HIVE_PARQUET_TRUNCATE_VARCHAR"},{"id":32,"name":"TOTAL_HIVE_PARQUET_TRANSFER_VARCHAR"},{"id":33,"name":"HIVE_PARQUET_CHECK_VARCHAR_CAST_TIME_NS"},{"id":34,"name":"NUM_ROW_GROUPS_PRUNED"},{"id":35,"name":"MAX_ROW_GROUPS_IN_HIVE_FILE_SPLITS"},{"id":36,"name":"NUM_HIVE_FILE_SPLITS_WITH_NO_ROWGROUPS"},{"id":37,"name":"MIN_IO_READ_TIME_NS"},{"id":38,"name":"MAX_IO_READ_TIME_NS"},{"id":39,"name":"AVG_IO_READ_TIME_NS"},{"id":40,"name":"NUM_IO_READ"},{"id":41,"name":"NUM_HIVE_PARQUET_DECIMAL_COERCIONS"},{"id":42,"name":"NUM_ROW_GROUPS_TRIMMED"},{"id":43,"name":"NUM_COLUMNS_TRIMMED"},{"id":44,"name":"NUM_PARTITIONS_PRUNED"},{"id":45,"name":"NUM_BOOSTED_FILE_READS"},{"id":46,"name":"MAX_BOOSTED_FILE_READ_TIME_NS"},{"id":47,"name":"AVG_BOOSTED_FILE_READ_TIME_NS"},{"id":48,"name":"TOTAL_BOOSTED_BYTES_READ"},{"id":49,"name":"NUM_COLUMNS_BOOSTED"},{"id":50,"name":"OFFSET_INDEX_READ"},{"id":51,"name":"COLUMN_INDEX_READ"},{"id":52,"name":"NUM_BOOSTED_RECORD_BATCHES_PRUNED"},{"id":53,"name":"NUM_PAGES_PRUNED"},{"id":54,"name":"NUM_PAGES_READ"},{"id":55,"name":"PAGE_DECOMPRESSION_TIME_NS"},{"id":56,"name":"NUM_RUNTIME_FILTERS"},{"id":57,"name":"RUNTIME_COL_FILTER_DROP_COUNT"},{"id":58,"name":"ROW_GROUPS_SCANNED_WITH_RUNTIME_FILTER"},{"id":59,"name":"RLE_PAGE_DECODING_READ_TIME_NS"},{"id":60,"name":"RLE_PAGE_DECODING_WRITE_TIME_NS"},{"id":61,"name":"PACKED_PAGE_DECODING_READ_TIME_NS"},{"id":62,"name":"PACKED_PAGE_DECODING_WRITE_TIME_NS"},{"id":63,"name":"MIN_METADATA_IO_READ_TIME_NS"},{"id":64,"name":"MAX_METADATA_IO_READ_TIME_NS"},{"id":65,"name":"AVG_METADATA_IO_READ_TIME_NS"},{"id":66,"name":"NUM_METADATA_IO_READ"},{"id":67,"name":"HIVE_FILE_FORMATS"},{"id":68,"name":"NUM_ZERO_SIZED_COLUMN"},{"id":69,"name":"NUM_EXTRA_FOOTER_READS"}]},{"metricDef":[]},{"metricDef":[]},{"metricDef":[{"id":0,"name":"NUM_FILES_WRITTEN"},{"id":1,"name":"MIN_FILE_SIZE"},{"id":2,"name":"MAX_FILE_SIZE"},{"id":3,"name":"AVG_FILE_SIZE"},{"id":4,"name":"MIN_RECORD_COUNT_IN_FILE"},{"id":5,"name":"MAX_RECORD_COUNT_IN_FILE"},{"id":6,"name":"MIN_IO_WRITE_TIME"},{"id":7,"name":"MAX_IO_WRITE_TIME"},{"id":8,"name":"AVG_IO_WRITE_TIME"},{"id":9,"name":"NUM_IO_WRITE"}]},{"metricDef":[]},{"metricDef":[{"id":0,"name":"BYTES_SENT"}]},{"metricDef":[{"id":0,"name":"BYTES_RECEIVED"}]},{"metricDef":[]},{"metricDef":[]}]},"resourceSchedulingProfile":{"queueName":"Low Cost User Queries","queueId":"baa30eec-387e-403c-8fba-69181bf55f41","ruleContent":"query_cost() < 30000000","ruleName":"Low Cost User Queries","ruleAction":"PLACE","schedulingProperties":{"queryCost":6000.0,"queryType":"UI Run"},"resourceSchedulingStart":1693891649360,"resourceSchedulingEnd":1693891649524},"commandPoolWaitMillis":1,"serializedPlan":"CkcKKnR5cGUuZ29vZ2xlYXBpcy5jb20vcGxhbi5QTG9naWNhbFRhYmxlU2NhbhIZCghhcnBfdGVzdAoJbXlfc2NoZW1hCgJhYQrAAQondHlwZS5nb29nbGVhcGlzLmNvbS9wbGFuLlBMb2dpY2FsRmlsdGVyEpQBGpEBKo4BCgQIARABEggKBkVRVUFMUxo4EjYKMggBEB0ggIAEKhwKGElTTy04ODU5LTEkZW5fVVMkcHJpbWFyeSgCMgpJU08tODg1OS0xEAQaQiJACjAQHSCAgAQqHAoYSVNPLTg4NTktMSRlbl9VUyRwcmltYXJ5KAIyCklTTy04ODU5LTEaCjZfY181X2RhdGFQHAptCih0eXBlLmdvb2dsZWFwaXMuY29tL3BsYW4uUExvZ2ljYWxQcm9qZWN0EkEIARIDY183GjgSNgoyCAEQHSCAgAQqHAoYSVNPLTg4NTktMSRlbl9VUyRwcmltYXJ5KAIyCklTTy04ODU5LTEQBg==","stateList":[{"state":1,"startTime":1693891647785},{"state":2,"startTime":1693891647806},{"state":3,"startTime":1693891648193},{"state":5,"startTime":1693891649476},{"state":4,"startTime":1693891649476},{"state":6,"startTime":1693891649524},{"state":7,"startTime":1693891649730},{"state":8,"startTime":1693891649846},{"state":9,"startTime":1693891651370}],"numPlanCacheUsed":0}

[balaji.ramaswamy]

@ljd520cc When you say customer JDBC driver, this source is not created via the regular “New Microsoft SQL Server Source” ? Like the screenshot below?

[ljd520cc]

Hi @balaji.ramaswamy ,
No, it’s not a MS SQL source or any built-in data source in Dremio. It’s a customized data source built via ARP connector connect to a customized JDBC Driver.