A data sources table is set to refresh every 2 hours and always shows expired in the reflection window, even though the refreshes are successful.
Help me understand please.
At the MySQL data source level
Then at the data source table level, if you hover over the orange triangle with the error, it states “Expired”
Then we click on jobs to review when the last time the table was queried for incremental updates. This shows that Accelerators are being created every 2 hours, as planned.
We click on the settings of the data sources table and then click on refresh policy. We then see that the refresh policy is set at 2 hours and to expire after 4 hours. We are looking at the error of “Expired” within the hour of when the Accelerator was created.
What am I missing?
I jumped into the server.log, then I find a “Unable to cache materialzation” error. However, when I search for the job, it shows completed.
Here is the server log
2018-03-15 16:30:16,860 [pool-11-thread-1] WARN c.d.s.a.CachedMaterializationProvider - Unable to cache materialization a4fd6347-f865-47c1-ac37-a909b06632b8
java.lang.AssertionError: null
at org.apache.calcite.util.Pair.zip(Pair.java:202) ~[calcite-core-1.12.0-201802162217200854-87dbbf1.jar:1.12.0-201802162217200854-87dbbf1]
at org.apache.calcite.rex.RexUtil.generateCastExpressions(RexUtil.java:177) ~[calcite-core-1.12.0-201802162217200854-87dbbf1.jar:1.12.0-201802162217200854-87dbbf1]
at org.apache.calcite.rex.RexUtil.generateCastExpressions(RexUtil.java:159) ~[calcite-core-1.12.0-201802162217200854-87dbbf1.jar:1.12.0-201802162217200854-87dbbf1]
at com.dremio.exec.planner.common.MoreRelOptUtil.createCastRel(MoreRelOptUtil.java:239) ~[dremio-sabot-kernel-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at com.dremio.exec.planner.common.MoreRelOptUtil.createCastRel(MoreRelOptUtil.java:214) ~[dremio-sabot-kernel-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at com.dremio.exec.planner.sql.MaterializationExpander.expand(MaterializationExpander.java:90) ~[dremio-sabot-kernel-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at com.dremio.exec.planner.sql.MaterializationDescriptor.getMaterializationFor(MaterializationDescriptor.java:148) ~[dremio-sabot-kernel-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at com.dremio.service.accelerator.CachedMaterializationProvider.expand(CachedMaterializationProvider.java:288) [dremio-services-accelerator-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at com.dremio.service.accelerator.CachedMaterializationProvider.updateCache(CachedMaterializationProvider.java:239) [dremio-services-accelerator-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at com.dremio.service.accelerator.CachedMaterializationProvider.compareAndSetCache(CachedMaterializationProvider.java:182) [dremio-services-accelerator-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at com.dremio.service.accelerator.CachedMaterializationProvider.refresh(CachedMaterializationProvider.java:164) [dremio-services-accelerator-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at com.dremio.service.accelerator.CachedMaterializationProvider.access$000(CachedMaterializationProvider.java:68) [dremio-services-accelerator-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at com.dremio.service.accelerator.CachedMaterializationProvider$1.run(CachedMaterializationProvider.java:155) [dremio-services-accelerator-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_151]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_151]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_151]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_151]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151]
2018-03-15 16:30:16,875 [pool-11-thread-1] WARN c.d.s.a.CachedMaterializationProvider - Unable to cache materialization 6732821f-973b-47fe-8887-785e1a149469
java.lang.AssertionError: null
at org.apache.calcite.util.Pair.zip(Pair.java:202) ~[calcite-core-1.12.0-201802162217200854-87dbbf1.jar:1.12.0-201802162217200854-87dbbf1]
at org.apache.calcite.rex.RexUtil.generateCastExpressions(RexUtil.java:177) ~[calcite-core-1.12.0-201802162217200854-87dbbf1.jar:1.12.0-201802162217200854-87dbbf1]
at org.apache.calcite.rex.RexUtil.generateCastExpressions(RexUtil.java:159) ~[calcite-core-1.12.0-201802162217200854-87dbbf1.jar:1.12.0-201802162217200854-87dbbf1]
at com.dremio.exec.planner.common.MoreRelOptUtil.createCastRel(MoreRelOptUtil.java:239) ~[dremio-sabot-kernel-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at com.dremio.exec.planner.common.MoreRelOptUtil.createCastRel(MoreRelOptUtil.java:214) ~[dremio-sabot-kernel-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at com.dremio.exec.planner.sql.MaterializationExpander.expand(MaterializationExpander.java:90) ~[dremio-sabot-kernel-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at com.dremio.exec.planner.sql.MaterializationDescriptor.getMaterializationFor(MaterializationDescriptor.java:148) ~[dremio-sabot-kernel-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at com.dremio.service.accelerator.CachedMaterializationProvider.expand(CachedMaterializationProvider.java:288) [dremio-services-accelerator-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at com.dremio.service.accelerator.CachedMaterializationProvider.updateCache(CachedMaterializationProvider.java:239) [dremio-services-accelerator-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at com.dremio.service.accelerator.CachedMaterializationProvider.compareAndSetCache(CachedMaterializationProvider.java:182) [dremio-services-accelerator-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at com.dremio.service.accelerator.CachedMaterializationProvider.refresh(CachedMaterializationProvider.java:164) [dremio-services-accelerator-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at com.dremio.service.accelerator.CachedMaterializationProvider.access$000(CachedMaterializationProvider.java:68) [dremio-services-accelerator-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at com.dremio.service.accelerator.CachedMaterializationProvider$1.run(CachedMaterializationProvider.java:155) [dremio-services-accelerator-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_151]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_151]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_151]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_151]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151]
2018-03-15 16:30:16,893 [pool-11-thread-1] WARN c.d.s.a.CachedMaterializationProvider - Unable to cache materialization 9504017d-533b-4d7a-86ab-0ddeb3d98903
java.lang.AssertionError: null
at org.apache.calcite.util.Pair.zip(Pair.java:202) ~[calcite-core-1.12.0-201802162217200854-87dbbf1.jar:1.12.0-201802162217200854-87dbbf1]
at org.apache.calcite.rex.RexUtil.generateCastExpressions(RexUtil.java:177) ~[calcite-core-1.12.0-201802162217200854-87dbbf1.jar:1.12.0-201802162217200854-87dbbf1]
at org.apache.calcite.rex.RexUtil.generateCastExpressions(RexUtil.java:159) ~[calcite-core-1.12.0-201802162217200854-87dbbf1.jar:1.12.0-201802162217200854-87dbbf1]
at com.dremio.exec.planner.common.MoreRelOptUtil.createCastRel(MoreRelOptUtil.java:239) ~[dremio-sabot-kernel-1.4.9-201802191836310213-7195059.jar:1.4.9-201802191836310213-7195059]
at com.dremio.exec.planner.common.MoreRelOptUtil.c