# Dremio 23 not match any reflections with MINIOS3

**URL:** <https://community.dremio.com/t/dremio-23-not-match-any-reflections-with-minios3/9982>\
**Category:** Uncategorized\
**Created:** [November 30, 2022, 1:57pm UTC](https://community.dremio.com/t/dremio-23-not-match-any-reflections-with-minios3/9982 "2022-11-30T13:57:19Z")\
**Posts on this page:** 12\
**Page:** 2

<div class="post-metadata">

**Author:** ![jeffreyslort](https://avatars.discourse-cdn.com/v4/letter/j/58f4c7/32.png) [@jeffreyslort](https://community.dremio.com/u/jeffreyslort)\
**Post date:** [January 2, 2023, 7:39am UTC](https://community.dremio.com/t/dremio-23-not-match-any-reflections-with-minios3/9982/21 "2023-01-02T07:39:15Z")

</div>

I did not test the reflections between upgrades. So as far as I know the reflections worked in V20 and did not work on V23.

---

<div class="post-metadata">

**Author:** ![Benny\_Chow](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/benny_chow/32/5527_2.png) [@Benny\_Chow](https://community.dremio.com/u/Benny_Chow)\
**Post date:** [January 3, 2023, 4:58pm UTC](https://community.dremio.com/t/dremio-23-not-match-any-reflections-with-minios3/9982/22 "2023-01-03T16:58:53Z")

</div>

@jeffreyslort In the v23 profile you sent, you ran:

SELECT \* FROM Raw.TOS.“che\_master”.“che\_master\_current”

And the reflection substitution failed with the error:

Caught exception during default reflection substitution for Raw.TOS.che\_master.che\_master\_current

Is it possible for you to run the same query with the same reflection on a prior working version and give us the profile?

Thanks

---

<div class="post-metadata">

**Author:** ![jeffreyslort](https://avatars.discourse-cdn.com/v4/letter/j/58f4c7/32.png) [@jeffreyslort](https://community.dremio.com/u/jeffreyslort)\
**Post date:** [January 4, 2023, 7:28am UTC](https://community.dremio.com/t/dremio-23-not-match-any-reflections-with-minios3/9982/23 "2023-01-04T07:28:30Z")

</div>

@Benny_Chow I could not downgrade the dremio instance with this problem so I set up a new environment where I connected to the same azure storage account installed V22 setup the datasets and reflections. Run a “SELECT \* FROM” reflection is used. Then I upgraded, ran the dremio-admin upgrade and restarted. Ran the same “SELECT \* FROM” reflection is not used.

[v23-job-profile.zip](https://community.dremio.com/uploads/short-url/cBvO3ZB1mlHIGqnphgLXkCnqtWs.zip) (19,0 KB)  
[v22-job-profile.zip](https://community.dremio.com/uploads/short-url/cv7TE4h9AIqFwMPBYePGiXbwEcx.zip) (16,4 KB)

---

<div class="post-metadata">

**Author:** ![Benny\_Chow](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/benny_chow/32/5527_2.png) [@Benny\_Chow](https://community.dremio.com/u/Benny_Chow)\
**Post date:** [January 4, 2023, 5:25pm UTC](https://community.dremio.com/t/dremio-23-not-match-any-reflections-with-minios3/9982/24 "2023-01-04T17:25:14Z")

</div>

@jeffreyslort Thank you for the profiles. I logged an internal ticket and will try to reproduce.

---

<div class="post-metadata">

**Author:** ![Benny\_Chow](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/benny_chow/32/5527_2.png) [@Benny\_Chow](https://community.dremio.com/u/Benny_Chow)\
**Post date:** [January 4, 2023, 8:25pm UTC](https://community.dremio.com/t/dremio-23-not-match-any-reflections-with-minios3/9982/25 "2023-01-04T20:25:34Z")

</div>

@jeffreyslort I took your v22 profile, extracted the PDS and VDS definitions, re-created them on a 22 server as Iceberg tables in S3. Then, I created a raw reflection on the VDS (also stored as Iceberg in S3) and verified that the “select \* on VDS” was accelerated. Then, I shutdown the server, upgraded to 23.1.0 and re-ran the query on the 23.1.0 server. For me, the “select \* on VDS” continues to get accelerated with the 22 reflection.

So, right now, I still can’t reproduce the substitution error. 🙃

Could you add the following to your v23 logback.xml, re-run the “select \*” and then send me reflection.log, acceleration.log, server.log and dremio.conf? Thanks

```auto
<appender name="reflection" class="ch.qos.logback.core.rolling.RollingFileAppender">
  <file>${dremio.log.path}/reflection.log</file>
  <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
    <fileNamePattern>${dremio.log.path}/archive/reflection.%d{yyyy-MM-dd}.log.gz</fileNamePattern>
    <maxHistory>30</maxHistory>
  </rollingPolicy>
  <encoder>
    <pattern>%date{ISO8601} [%thread] %-5level %logger{36} - %msg%n</pattern>
  </encoder>
</appender>
  
<logger name="com.dremio.service.reflection" additivity="false">
  <level value="DEBUG"/>
    <appender-ref ref="reflection"/>
</logger>

<appender name="acceleration" class="ch.qos.logback.core.rolling.RollingFileAppender">
  <file>${dremio.log.path}/acceleration.log</file>
  <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
    <fileNamePattern>${dremio.log.path}/archive/acceleration.%d{yyyy-MM-dd}.log.gz</fileNamePattern>
    <maxHistory>30</maxHistory>
  </rollingPolicy>
  <encoder>
    <pattern>%date{ISO8601} [%thread] %-5level %logger{36} - %msg%n</pattern>
  </encoder>
</appender>

<logger name="com.dremio.exec.planner.acceleration" additivity="false">
    <level value="debug" />
    <appender-ref ref="acceleration" />
</logger>
<logger name="com.dremio.reflection.bup" additivity="false">
  <level value="debug" />
  <appender-ref ref="acceleration" />
</logger>
<logger name="com.dremio.reflection.ReflectionTracer" additivity="false">
  <level value="debug" />
  <appender-ref ref="acceleration" />
</logger>

```

---

<div class="post-metadata">

**Author:** ![jeffreyslort](https://avatars.discourse-cdn.com/v4/letter/j/58f4c7/32.png) [@jeffreyslort](https://community.dremio.com/u/jeffreyslort)\
**Post date:** [January 5, 2023, 2:34pm UTC](https://community.dremio.com/t/dremio-23-not-match-any-reflections-with-minios3/9982/26 "2023-01-05T14:34:59Z")

</div>

@Benny_Chow, So for me the files are stored on a Azure storage account and the files are just Parquet files, not Iceberg.

See the zip for an extract of my log files 🙂

[logs.zip](https://community.dremio.com/uploads/short-url/2XrSSYiFdznodNAIoidR2rfTh8U.zip) (6,6 KB)

---

<div class="post-metadata">

**Author:** ![Benny\_Chow](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/benny_chow/32/5527_2.png) [@Benny\_Chow](https://community.dremio.com/u/Benny_Chow)\
**Post date:** [January 12, 2023, 10:07pm UTC](https://community.dremio.com/t/dremio-23-not-match-any-reflections-with-minios3/9982/27 "2023-01-12T22:07:44Z")

</div>

Hi @jeffreyslort , just want to give you an update. We were able to reproduce and fix the problem with reflections not matching starting on V23. I would recommend waiting until V24 comes out and upgrade to that.

---

<div class="post-metadata">

**Author:** ![jeffreyslort](https://avatars.discourse-cdn.com/v4/letter/j/58f4c7/32.png) [@jeffreyslort](https://community.dremio.com/u/jeffreyslort)\
**Post date:** [January 13, 2023, 10:20am UTC](https://community.dremio.com/t/dremio-23-not-match-any-reflections-with-minios3/9982/28 "2023-01-13T10:20:59Z")

</div>

@Benny_Chow alright that sounds promising. Any idea when v24 will be released ?

---

<div class="post-metadata">

**Author:** ![Benny\_Chow](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/benny_chow/32/5527_2.png) [@Benny\_Chow](https://community.dremio.com/u/Benny_Chow)\
**Post date:** [January 13, 2023, 6:51pm UTC](https://community.dremio.com/t/dremio-23-not-match-any-reflections-with-minios3/9982/29 "2023-01-13T18:51:02Z")

</div>

Very soon! Also, I want to reiterate the appreciation for you and others in this group for sharing the profiles and/or logs so that we could finally get to the bottom of this and fix this issue. This problem was also related to substitution errors, upgrade and matching issues in V23.

---

<div class="post-metadata">

**Author:** ![Benny\_Chow](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/benny_chow/32/5527_2.png) [@Benny\_Chow](https://community.dremio.com/u/Benny_Chow)\
**Post date:** [February 15, 2023, 8:18pm UTC](https://community.dremio.com/t/dremio-23-not-match-any-reflections-with-minios3/9982/30 "2023-02-15T20:18:47Z")

</div>

@jeffreyslort v24 is released. The issue with reflections containing VARCHAR columns not matching has been fixed. [Index of community-server/24.0.0-202302100528110223-3a169b7c/](https://download.dremio.com/community-server/24.0.0-202302100528110223-3a169b7c/)

---

<div class="post-metadata">

**Author:** ![jeffreyslort](https://avatars.discourse-cdn.com/v4/letter/j/58f4c7/32.png) [@jeffreyslort](https://community.dremio.com/u/jeffreyslort)\
**Post date:** [February 16, 2023, 7:20am UTC](https://community.dremio.com/t/dremio-23-not-match-any-reflections-with-minios3/9982/31 "2023-02-16T07:20:27Z")

</div>

@Benny_Chow Thanks for the headsup will test it right away 🙂

---

<div class="post-metadata">

**Author:** ![jeffreyslort](https://avatars.discourse-cdn.com/v4/letter/j/58f4c7/32.png) [@jeffreyslort](https://community.dremio.com/u/jeffreyslort)\
**Post date:** [February 16, 2023, 8:44am UTC](https://community.dremio.com/t/dremio-23-not-match-any-reflections-with-minios3/9982/32 "2023-02-16T08:44:58Z")

</div>

Alright I can confirm that it works with dremio v24

[Previous page](https://community.dremio.com/t/dremio-23-not-match-any-reflections-with-minios3/9982.md?page=1)