Hi, I am trying to explore if Dremio is suitable for my usecase. I have a scenario where we write data into PostgreSQL and we would like to use an Analytics tool like Superset to create dashboards. Is Dremio a good usecase to just query a single datasource?
Also the data in Postgres has JSONB columns. I understand that Dremio does not recognise JSON datatype and it has to be explicitly converted on the fly during query time? This seems to take a lot of time for execution.
Is there a workaround - will reflection enhance the query performance?
@pruk I would be most performant if the data is on Iceberg or Parquet. Postgres is fine but the query will be pushdown to Postgres. Also Dremio should be able to read the JSONB column but if you need to extract elements then you would beed to flatten/extract if it has lists or structs inside. Creating reflections at the VDS level with the joins/agss in place would be much more performant as the query will read from Iceberg/Parquet
@balaji.ramaswamy Thank you for your response. Since Dremio does not recognise Postgres JSONB column. When I query with CONVERT_JSON and use reflection , it throws an error. Any advice on why the query works without refelction and when the reflection is turened on , it throws this?
@pruk What happens if you do in 2 steps
- Create a VDS with the convert from
- Create the reflection on the vds