Best practice to perform queries via the REST API

[jorge]

Is there a way for my app to be notified when the query job is completed as opposed to calling the GET /job endpoint? Dremio’s browser interface to preview and run queries, available on http://localhost:9047, seems to be notified when executing queries because it is not polling the “job” endpoint. My app, on the other hand, calls the “job” endpoint lots of times, potentially loading the server and slowing down other processes.

This is how my app is executing Dremio queries:

• This call sends the query to execute
  POST http://localhost:9047/api/v3/sql

• This call checks for the status of the job until its state is “COMPLETED”
  GET http://localhost:9047/api/v3/job/24e8a8e9-6e53-4a81-33af-6886aea003

• This call retrieves the results
  GET http://localhost:9047/api/v3/job/24e8a8e9-6e53-4a81-33af-6886aea00300/results?offset=0&limit=499

Thanks!

[doron]

Hi,

The flow you describe is currently the only supported way to monitor a job. The UI uses websockets to monitor progress and we currently do not have plans to expose it as a stable API.

We are always interested in hearing about how Dremio is being utilized - is there any specific reasons why the current flow doesn’t work for you?

thanks,
Doron

[jorge]

Doron, thanks very much for your quick reply. Dremio looks great. I’d love to utilize it in a way that users can visualize and interact with data directly from web browsers and mobile devices.

The current REST API flow to perform queries is doing the job. I’ll implement a backoff strategy to avoid calling the job endpoint too many times while waiting for results. However, it would be great to also have a query endpoint that returns results if the query completes within a specified timeout.

As a side note, another feature that I’d love to see is being able to authenticate data app users to Dremio using OAuth implicit workflow.

Thanks!