Hello, I got an error says: Cannot add expression of different type to set: set type is RecordType .... expression type is RecordType.... with two long record type definitions. I did a diff on it, the only difference is the second record type has a NOT NULL on one of the field definition. I added the same data source with a different name and it returns the table fine.
Attached please find the log. Thank you so much.
log.zip (3.5 KB)
Can you talk more about what you are trying to do when this occurs? Do you have a query profile to share?
Hello,
Attached please find the query profile.We were just trying to preview a virtual datasets with a physical MS SQL server behind.
a1ec7758-9516-4a94-b4d2-2771556ba6e9.zip (9.3 KB)
I have the same error with another dataset. I think the problem is that the equivalenceClass always add a “NOT NULL” qualifier to one of the field in the query expression thus throw the AssertionException.
Refercece here:
I found the reason that cause the problem. We have a dataset with a hardcoded value for a column, eg: SELECT 'a' AS 'col'. When we query it, the plan expression add NOT NULL to this field and cause the assertion exception to be thrown as shown in the attached. We removed the hardcoded column and the problem is gone. Thank you.
I have the same issue occuring sing the migration to 2.1.
I think that this happens in certain cases when querying a reflection that is referenced by another VDS, which has itself reflections.
I needed to deactivate the depending reflections, and reactivate them, from the parent to the children.
The error log mentions columns that have nothing to do with the query executed.
Original: RecordType(BIGINT distri_id, VARCHAR(65536) swntid, VARCHAR(65536) rfr, BIGINT nbrefret, DOUBLE prc, BIGINT is_preview, BIGINT auto, BIGINT is_bot_traffic, BIGINT v, VARCHAR(65536) pid, VARCHAR(65536) module, VARCHAR(65536) rfr2, VARCHAR(65536) iid, BIGINT is_tablet, VARCHAR(65536) c_cfduid, VARCHAR(65536) utm_source, VARBINARY(65536) opscode, DOUBLE avgdist, DOUBLE avgpc, ANY lang, ANY area, BIGINT nbres, VARCHAR(65536) pageid, VARBINARY(65536) currency, VARBINARY(65536) geoip_city, VARCHAR(65536) os, VARBINARY(65536) geoip_country, VARCHAR(65536) sids, BIGINT is_bot, TIMESTAMP(3) ts, VARCHAR(65536) wtbid, VARCHAR(65536) reqpid, BIGINT is_mobile, VARCHAR(65536) sid, VARCHAR(65536) action, VARCHAR(65536) type, VARCHAR(65536) fmt, VARCHAR(65536) browser, VARCHAR(65536) dir0, VARCHAR(65536) dir1, VARCHAR(65536) dir2, VARCHAR(65536) dir3, VARCHAR(65536) geoip.country, BIGINT is_pc, VARCHAR(65536) ct, VARCHAR(65536) utm_campaign, VARCHAR(65536) os_version, VARCHAR(65536) utm_content, ANY avpid, VARCHAR(65536) browser_version, VARCHAR(65536) rfr2_domain, VARCHAR(65536) c_swntid, VARCHAR(65536) utm_term, VARCHAR(65536) utm_medium, ANY dlv, VARCHAR(65536) rfr_domain, BIGINT $dremio$update$)
Replacement: RecordType(BIGINT distri_id, VARCHAR(65536) swntid, VARCHAR(65536) rfr, BIGINT nbrefret, DOUBLE prc, BIGINT is_preview, BIGINT auto, BIGINT is_bot_traffic, BIGINT v, VARCHAR(65536) pid, VARCHAR(65536) module, VARCHAR(65536) rfr2, VARCHAR(65536) iid, BIGINT is_tablet, VARCHAR(65536) c_cfduid, VARCHAR(65536) utm_source, ANY opscode, DOUBLE avgdist, DOUBLE avgpc, ANY lang, ANY area, BIGINT nbres, VARCHAR(65536) pageid, VARBINARY(65536) currency, ANY geoip_city, VARCHAR(65536) os, ANY geoip_country, VARCHAR(65536) sids, BIGINT is_bot, TIMESTAMP(3) ts, VARCHAR(65536) wtbid, VARCHAR(65536) reqpid, BIGINT is_mobile, VARCHAR(65536) sid, VARCHAR(65536) action, VARCHAR(65536) type, VARCHAR(65536) fmt, VARCHAR(65536) browser, VARCHAR(65536) dir0, VARCHAR(65536) dir1, VARCHAR(65536) dir2, VARCHAR(65536) dir3, VARCHAR(65536) geoip.country, BIGINT is_pc, VARCHAR(65536) ct, VARCHAR(65536) utm_campaign, VARCHAR(65536) os_version, VARCHAR(65536) utm_content, ANY avpid, VARCHAR(65536) browser_version, VARCHAR(65536) rfr2_domain, VARCHAR(65536) utm_term, VARCHAR(65536) utm_medium, ANY dlv, VARCHAR(65536) rfr_domain, BIGINT $dremio$update$).
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:383) ~[guava-20.0.jar:na]
at com.dremio.reflection.bup.ReplacementShuttle.replace(ReplacementShuttle.java:37) ~[dremio-extra-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.reflection.bup.ReplacementShuttle.replace(ReplacementShuttle.java:33) ~[dremio-extra-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.reflection.bup.SubIterator.replicateWithReplace(SubIterator.java:100) ~[dremio-extra-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.reflection.bup.MatBlock.generateAlternativeBlocks(MatBlock.java:544) [dremio-extra-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.reflection.bup.MatBlock.access$100(MatBlock.java:47) [dremio-extra-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.reflection.bup.MatBlock$2.apply(MatBlock.java:199) ~[dremio-extra-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.reflection.bup.MatBlock$2.apply(MatBlock.java:195) ~[dremio-extra-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.google.common.collect.Iterators$7.transform(Iterators.java:750) ~[guava-20.0.jar:na]
at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:47) ~[guava-20.0.jar:na]
at com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:252) ~[guava-20.0.jar:na]
at com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:209) ~[guava-20.0.jar:na]
at com.google.common.collect.FluentIterable.toList(FluentIterable.java:614) ~[guava-20.0.jar:na]
at com.dremio.reflection.bup.MatBlock.matchLeafNode(MatBlock.java:200) [dremio-extra-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.reflection.bup.MatBlock.determineLeafIterators(MatBlock.java:183) [dremio-extra-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.reflection.bup.MatBlock.generateAlternativeBlocks(MatBlock.java:490) [dremio-extra-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.reflection.bup.MatBlock.generateCandidates(MatBlock.java:441) [dremio-extra-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
… 16 common frames omitted
2018-09-07 07:47:30,824 [246dd2ec-9fa0-0f0c-a8a3-2ac959ad8f00:foreman] INFO com.dremio.reflection.bup.MatBlock - User Error Occurred [ErrorId: 6b3067a4-5a27-4d57-ab7d-3cc85a151001]
com.dremio.common.exceptions.UserException: Failure while attempting candidate reflection substitution for 8ca4112a-16bd-4734-9c25-851b36bcfc76
at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:746) ~[dremio-common-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.reflection.bup.MatBlock.generateCandidates(MatBlock.java:474) [dremio-extra-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.reflection.bup.BupFinder.generateAlternatives(BupFinder.java:125) [dremio-extra-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.exec.planner.acceleration.substitution.DremioSubstitutionProvider.generateNewAlternatives(DremioSubstitutionProvider.java:179) [dremio-extra-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.exec.planner.acceleration.substitution.DremioSubstitutionProvider.findSubstitutions(DremioSubstitutionProvider.java:141) [dremio-extra-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.exec.planner.acceleration.substitution.AccelerationAwareSubstitutionProvider.findSubstitutions(AccelerationAwareSubstitutionProvider.java:67) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.exec.planner.DremioVolcanoPlanner.registerMaterializations(DremioVolcanoPlanner.java:70) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:605) [calcite-core-1.16.0-201807312206090258-e1b02e7.jar:1.16.0-201807312206090258-e1b02e7]
at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:368) [calcite-core-1.16.0-201807312206090258-e1b02e7.jar:1.16.0-201807312206090258-e1b02e7]
at com.dremio.exec.planner.sql.handlers.PrelTransformer.transform(PrelTransformer.java:444) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.exec.planner.sql.handlers.PrelTransformer.convertToDrel(PrelTransformer.java:222) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.exec.planner.sql.handlers.PrelTransformer.convertToDrel(PrelTransformer.java:280) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.exec.planner.sql.handlers.query.NormalHandler.getPlan(NormalHandler.java:47) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.exec.planner.sql.handlers.commands.HandlerToExec.plan(HandlerToExec.java:69) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at com.dremio.exec.work.foreman.AttemptManager.run(AttemptManager.java:291) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]
Caused by: java.lang.IllegalArgumentException: Row types didn’t match.
Bump! Same issue here.
I also am getting this issue, no reflections, am not able to remove the column with the static value.