I’m utilizing Dremio Flight Endpoint in Python to execute TPCH SQL queries and create a view in Dremio. However, each attempt results in the following error message:
python
from dremio.arguments.parse import get_config
from dremio.flight.endpoint import DremioFlightEndpoint
import os
if __name__ == "__main__":
# Parse the config file.
args = get_config()
sql_query = read_sql_from_file(sql_file)
# Set the query in args
args['query'] = sql_query
# print(args)
# Instantiate DremioFlightEndpoint object with updated query
dremio_flight_endpoint = DremioFlightEndpoint(args)
flight_client = dremio_flight_endpoint.connect()
# print(dremio_flight_endpoint)
# Get reader
reader = dremio_flight_endpoint.get_reader(flight_client)
# Print out the data as a dataframe
df = reader.read_pandas()
print(f"Data from {sql_file}:")
print(df)
sql
create view "poc".tpch.revenue0 as
select
l_suppkey as supplier_no ,
sum(l_extendedprice * (1 - l_discount)) as total_revenue
from
lineitem
where
l_shipdate >= date '1995-02-01'
and l_shipdate < date '1995-02-01' + interval '3' month
group by
l_suppkey;
pyarrow.lib.ArrowInvalid: Flight returned invalid argument error, with message: Validation of view sql failed. null
I’ve ensured that the submitted SQL query statements comply with Dremio’s syntax requirements, yet the view creation still fails. I’ve also inspected communication with the Dremio server, but haven’t identified any apparent issues.
UI surface :
It’s worth noting that the same SQL queries executed successfully in the Dremio UI interface, and the view was created without any problems.
I’m reaching out here to see if anyone has encountered a similar issue or if anyone can provide further guidance to resolve this problem. Are there any additional details or solutions regarding this error? Thank you!