Reflection/acceleration errors

1

I’m having lots of trouble trying to add reflections to our datasets.

  1. In one case, when I click ‘Reflections’ on a virtual data set, I immediately get an error “Something went wrong” with more detail “acceleration at [/unable to find acceleration at (UUID here)] not found”.

55%20AM
Screen Shot 2018-02-23 at 10.53.55 AM.png2400×372 28.9 KB

I tried to run ALTER TABLE <TABLE_PATH> DROP ACCELERATION, it says “Table is not currently accelerated.”

  1. In another case, I set up a Raw Reflection on a primary data set. The job to create the acceleration reports successful completion after a few minutes. When I click ‘Show Reflection’, the ‘ready’ fire symbol is greyed out and there is a warning symbol with the message ‘Expired’. A ‘calculate refresh’ job reports success after less than a second, resulting in the same state–Expired and reflection not ready.

For the one acceleration I can get to work, the create job displays output like:
Input
Input Bytes:
Input Records:
Output
Output Bytes: 2.49 MB
Output Records: 5,082

The create acceleration job for the problematic dataset does not have this output.

We are using Dremio 1.4.4. Thanks for any assistance you can provide!

2

Hi @jkissinger,

For #1, can you try to clear all reflections and retry?
For #2 Can you please check the server.log for errors or upload the server.log?

Thanks,
@balaji.ramaswamy

3

We cleared all reflections, data sources, etc and started over trying to create one acceleration. The server.log 's don’t seem to display any errors, but I’ve attached them just in case.

It seems to be an issue of size. One of the datasets that I’m trying to accelerate is 56 GB. The acceleration job ends after a couple minutes with a success checkmark, but there’s no “Output” on the job. From my data source, I can see that the acceleration query continues running. Eventually we get to the point of “No executors currently available” with the executors crashed. How should we create a reflection on large datasets?

We are running two executors and one coordinator–is there a recommendation of how many nodes should run?

dremio_logs.zip (60.0 KB)

4

@balaji.ramaswamy Any ideas? We tried again on a 11GB dataset and see the same failure scenario.

5

HI @jkissinger

Sorry about the delay. What happens when you try to create reflections on smaller tables or a say a reflection a subset of columns?

Can you also please send me a copy of your dremio-env and dremio.conf from your coordinator and executor nodes?

Also can you please send me the DESCRIBE of the table that is 11 GB?

I am seeing a few exceptions in the server.log that you uploaded and will get back to you on that

Thanks,
@balaji.ramaswamy