Hi, getting the following error with a new setup when I try to run a query
Setup
AKS, 3 nodes, Azure Storage Gen 2
Cloud Cache enabled
all running under dremio user via ps -ef | grep dremo
Storage Container
ERROR
SYSTEM ERROR: NativeIOException: Operation not permitted
SqlOperatorImpl ARROW_WRITER
Location 0:0:3
SqlOperatorImpl ARROW_WRITER
Location 0:0:3
Fragment 0:0
[Error Id: a2b66576-8fca-407f-9df4-89aebbd816b5 on dremio-executor-0.dremio-cluster-pod.bd-dremio.svc.cluster.local:0]
(org.apache.hadoop.io.nativeio.NativeIOException) Operation not permitted
org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl():-2
org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod():234
org.apache.hadoop.fs.RawLocalFileSystem.setPermission():861
org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode():547
org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission():587
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs():559
org.apache.hadoop.fs.RawLocalFileSystem.create():316
org.apache.hadoop.fs.RawLocalFileSystem.create():351
com.dremio.exec.store.dfs.PseudoDistributedFileSystem.create():369
org.apache.hadoop.fs.FileSystem.create():1118
org.apache.hadoop.fs.FileSystem.create():1098
org.apache.hadoop.fs.FileSystem.create():987
org.apache.hadoop.fs.FileSystem.create():975
com.dremio.exec.hadoop.HadoopFileSystem.create():212
com.dremio.exec.store.easy.arrow.ArrowRecordWriter.setup():98
com.dremio.sabot.op.writer.WriterOperator.setup():107
com.dremio.sabot.driver.SmartOp$SmartSingleInput.setup():255
com.dremio.sabot.driver.Pipe$SetupVisitor.visitSingleInput():73
com.dremio.sabot.driver.Pipe$SetupVisitor.visitSingleInput():63
com.dremio.sabot.driver.SmartOp$SmartSingleInput.accept():200
com.dremio.sabot.driver.StraightPipe.setup():103
com.dremio.sabot.driver.StraightPipe.setup():102
com.dremio.sabot.driver.StraightPipe.setup():102
com.dremio.sabot.driver.StraightPipe.setup():102
com.dremio.sabot.driver.Pipeline.setup():68
com.dremio.sabot.exec.fragment.FragmentExecutor.setupExecution():391
com.dremio.sabot.exec.fragment.FragmentExecutor.run():273
com.dremio.sabot.exec.fragment.FragmentExecutor.access$1400():94
com.dremio.sabot.exec.fragment.FragmentExecutor$AsyncTaskImpl.run():711
com.dremio.sabot.task.AsyncTaskWrapper.run():112
com.dremio.sabot.task.slicing.SlicingThread.mainExecutionLoop():228
com.dremio.sabot.task.slicing.SlicingThread.run():159