Failure during setup error when from REST Application

1

Hello, we are getting Failure during setup when running query through REST Application. INVALID_DATASET_METADATA ERROR in profile.
Running same query friom UI works fine and afterwards REST Application runnin same query is also green in logs.
Changing WHERE clause(id in (different value)) in query makes REST app fail again.
Running REFRESH METADATA over whole table helps to resolve the issue for any WHERE clause, but than error returns after a while.

We are querying Aws Glue table directly(simple SELECT)

select * from "Aws glue".xxxx w
here id in  (50372)

Log attached 63121f32-633a-4ffe-972a-96f4c7d1176e.zip (19,0 )

Build
25.0.7-202407181421320869-2632b04f
Edition
AWS Edition (activated)

2

@vladislav-stolyarov This is the actual error

Caused By (java.io.FileNotFoundException) Version of file changed data/media_segments/data.gz.parquet

UI query may work as it might be truncating the query. Can you please send us the UI profile too?

Any chance you are overwriting this file?

3

Yes you are right - we overwrite this file occasionally.
Here how it looks like. So UI run was able to resolve metadata change and agter that API calls succeed.

image
image1450×201 52 KB

This is ui call profile
d122ba14-9c6c-4ab0-92b8-fb2370558d03.zip (33,9 КБ)

Than my question is why UI call was able to reset error and start with new metadata, but api call not? is it possible to work this around in Rest api call?

Are there any other workarounds?

We have single file glue table without partitions, so can not do partition swicth of whatever, seems on prev versions of dremio this issue was not so critical.
Should we call refresh metadata right after we changed file?

4

@vladislav-stolyarov This is expected behavior, if you see, the jobs has 2 attempts

attempt 1 - File not found
attempt 2 - successful query

But if you also select job type internal, you will see another “REFRESH DATASET” job that was triggered between the 2 attempts, that is why the second attempt started 20 seconds later. I do not see any issues with this. The other option is what you are saying

  • Delete file
  • Add file
  • Run “ALTER PDS REFRESH METADATA”
  • Run UI or REST query
5

Yep but why this type of job is not triggered when query failed multiply times from REST? Only after UI query.

6

@vladislav-stolyarov Understood, it looks like the retry did not work via REST. Can you try via JDBC and see if the retry works? Just want to make sure it is indeed REST

7

For ODBC Client it does 2 attempts with metadata refresh behind the scenes.

82025f33-19d8-4531-b859-dc2d3842ba7b.zip (30,8 КБ)

8

Thanks for testing this, looks like REST is not automatically firing, we would need to prioritize and repro this internally, meanwhile, can u do below?

  • File removed
  • run ALTER PDS <PDS_NAME> REFRESH METADATA
  • run REST query