I have one Query regrading dremio with apache Arrow ,as of now my understanding is apache arrow used for in-memory buffer for Dremio.
so,my question is what happend if apache arrow in-memory full
for Example: let’s say i have 10K Records in memory from diffrent diffrent datasource and i have added some more records in that case in-memory full
so, how apache arrow handle that situation i need guidline on above query
Is there a particular query you are referring to? If memory is indeed full, Dremio will either spill to disk (more around this area in coming months) or query would fail with OOM