Hello ,
I have enabled raw reflection on PhysicalDataset with some of the Field i have stored Distributed filesystem to Azure ADL Gen2 where Result,Acceleation stored some times when i have restart dremio-master its not able to find job_result scanner in that case i have deleted all folder mounted on Azure ADL and restart dremio cluster.after run Query from dremio gui in i am gettimg below error
com.microsoft.azure.storage.v10.adlsg2.models.DataLakeStorageErrorException: Status code 404, "{“error”:{“code”:"PathNotFound
I am attaching job Profile for this
to Resolve this i have delete old reflection on Physical DataSet and also disable enabled on VirtualDataset .please guide me how to solve this type of issuesb9933a49-fb69-4a6c-9638-289c57b1f2d2.zip (14.0 KB)
@Vikash_Singh
Can you search for this “f92059e9-904c-4b4d-9f18-3943911e0977” in your jobs page and see if there is a DELETE JOB?
@Vikash_Singh
Do you actually find these files on Gen2 or are they gone?. For example in gen2 you would be storing reflections under a folder. Do you find a folder called “2b4aea35-dca4-46bf-ac5a-e722bdc03b13” and under that “f92059e9-904c-4b4d-9f18-3943911e0977”?
No i could not find file in adl gen2 as i have deleted those because dremio master is not starting of some job search issue after that i have faced above issue
@Vikash_Singh
What where the order of events? Did you first delete files from Gen2?
order of event
- started dremio cluster ,dremion master failed as unable to find job search result
- stopped cluster ,deleted all folder from adl gen2
- restarted dremio cluster
- after that error Occured
note: i have enabled Reflection of Orignal Data Source PDS
@Vikash_Singh
Dremio should not fail if there are missing reflection files or job results. Do you still have the original log file when Dremio did not start first time? I am thinking Dremio failed to start due to a different reason