I have layers of VDS’s set up for a specific usecase
The Preparation VDS name is Table1
the underlying SQL is:
Select column1,column2,and Column3
from
ClouderaHive.Table_name
Column1 in this table has always exceeded the 32000KB Limit so we truncate the underlying PDS and the VDS above works fine when we run it on its own.
The problem occurs when I join this VDS to others
select a.column1, b.field1, b.field2
from Table1 a
inner join
Table2 b
on a.date = b.date and a.id = b.id
When we Run the above it fails with the 32000KB error
How can the underlying VDS run fine alone but fail when joined? Is dremio holding onto some old metadata when the original VDS was built?