How do I use Dremio to query and return results for large unstructured text files like logs, etc. I have tried to load the data as Iceberg, Delta, Parquet, and JSON and I keep hitting the 32k query limit. I was under the impression that Dremio can query unstructured data but I can’t seem to figure out the best way to do it. If anyone can help, it would be much appreciated.
Thanks
Uddie
@kapadiup There is currently a product limit of field width max of 32K, so if the field cannot be separated based on a delimiter then that limit is probably going to be an issue
We’ve PDF data in our object storage (S3 and Azure Blob Storage), we’d like to know if there’s any way within dremio to be able to query the contents through a view. We understand that we need to convert the PDFs into an organized data structure like JSON, Parquet etc. but our data isn’t organized like that. It can be brochures, whitepapers etc. So, what’s the best way to go about this?
@balaji.ramaswamy could you help us with this?
@balaji.ramaswamy , any expedited help on this would be much appreciated
@rohith.krishnakumar Currently Dremio does not support reading PDFs