When we create virtual tables using the REST API, it can be quite slow. Sometimes the API will take up to 30-45s (or longer) before it returns. We have about ~125 virtual queries (and growing) and its really slow. Is Dremio running the query as part of the API call? Is there a way to speed this up?
There are no jobs (that I can find) when we create them via REST API.
I think I found by changing in the job UI to “internal” and they now show up.
Here’s one that is ~11s. It’s sometimes slow, sometimes faster.
e8708773-fe2b-4a8b-9159-4e4a9c0f69af.zip (79.5 KB)
I can see from this screen that it is actually running these queries to create a prepared statement. I wish there was a way that we could do these async since we have currently 128 virtual tables and we are likely to create another 50-75 in the next few weeks. Its making it really super slow to start up a container to run our software.
Hi @jhaynie
We should not be doing prepare statements for REST API calls. Did you mean ODBC client. If you this is really REST calls what is the client you are using?
Thanks
@balaji.ramaswamy
We do some work when generating a VDS (like figuring out the schema) and your query is quite complex. You can run multiple API calls in parallel to speed things up for now to help speed things up.
we are just hitting the API from Go. Nothing special.