# “Enable partition column inference” option doesn’t work

**URL:** <https://community.dremio.com/t/enable-partition-column-inference-option-doesn-t-work/10262>\
**Category:** Dremio University\
**Created:** [February 2, 2023, 2:58pm UTC](https://community.dremio.com/t/enable-partition-column-inference-option-doesn-t-work/10262 "2023-02-02T14:58:26Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pawel](https://avatars.discourse-cdn.com/v4/letter/p/ecae2f/32.png) [@Pawel](https://community.dremio.com/u/Pawel)\
**Post date:** [February 2, 2023, 2:58pm UTC](https://community.dremio.com/t/enable-partition-column-inference-option-doesn-t-work/10262/1 "2023-02-02T14:58:26Z")

</div>

We would like to start using option **“Enable partition column inference”** to have well formatted columns for partitioned parquet files but looks like this option doesn’t work like is described on [documentation paged](https://docs.dremio.com/software/data-sources/entity-promotion/#partitioned-data)

No new well formatted columns visible only standard old dir0

 ![image](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/2X/9/98b7b49a0e477e595fc51ab6b953ab138c0645b2.png)

Tested on latest community edition 23.1.0 with simple dummy partitioned parquet file.  
I’ve created new Object Storages with option selected (tested on S3 and NAS object storage)

Could you check and advice?

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [February 13, 2023, 4:23am UTC](https://community.dremio.com/t/enable-partition-column-inference-option-doesn-t-work/10262/2 "2023-02-13T04:23:11Z")

</div>

@Pawel It requires you to forget metadata on the PDS, so you can either remove format/add format back or do the below

```auto
alter pds <pds-name> forget metadata;
alter pds <pds-name> refresh metadata;

```

Thanks  
Bali

---

<div class="post-metadata">

**Author:** ![Pawel](https://avatars.discourse-cdn.com/v4/letter/p/ecae2f/32.png) [@Pawel](https://community.dremio.com/u/Pawel)\
**Post date:** [February 14, 2023, 2:55pm UTC](https://community.dremio.com/t/enable-partition-column-inference-option-doesn-t-work/10262/3 "2023-02-14T14:55:55Z")

</div>

@balaji.ramaswamy Yes from that reason I tested on new Object Storage “local” with option selected from creation time.  
Additionally I also tested sequences:

_select \* from “local”.parquet\_test_  
_alter table “local”.parquet\_test forget metadata_  
_alter table “local”.parquet\_test refresh metadata_

But this also dosn’t help.

Could you check if this is working on any dummy parquet file - I’ve attached my test one here  
[parquet\_test.zip](https://community.dremio.com/uploads/short-url/kopKGJjqfHrVePEVK8tDmVu4rqa.zip) (3.3 KB)

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [February 15, 2023, 5:19am UTC](https://community.dremio.com/t/enable-partition-column-inference-option-doesn-t-work/10262/4 "2023-02-15T05:19:40Z")

</div>

@Pawel Works as expected on your dataset, let me try on community edition as I tried on enterprise

 ![parquet-test](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/2X/e/e57e734174cc5a564557627d07685bef85112c16.png)

---

<div class="post-metadata">

**Author:** ![Pawel](https://avatars.discourse-cdn.com/v4/letter/p/ecae2f/32.png) [@Pawel](https://community.dremio.com/u/Pawel)\
**Post date:** [February 22, 2023, 7:08am UTC](https://community.dremio.com/t/enable-partition-column-inference-option-doesn-t-work/10262/5 "2023-02-22T07:08:34Z")

</div>

@balaji.ramaswamy - Could you check on community edition and let us know - we will know that we don’t do anything incorrectly - thanks

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [February 24, 2023, 3:50am UTC](https://community.dremio.com/t/enable-partition-column-inference-option-doesn-t-work/10262/6 "2023-02-24T03:50:43Z")

</div>

@Pawel I do not see the feature working in Community edition. Let me find out if it is a bug or EE only feature

---

<div class="post-metadata">

**Author:** ![fetanchaud](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/fetanchaud/32/4192_2.png) [@fetanchaud](https://community.dremio.com/u/fetanchaud)\
**Post date:** [February 24, 2023, 12:33pm UTC](https://community.dremio.com/t/enable-partition-column-inference-option-doesn-t-work/10262/7 "2023-02-24T12:33:22Z")

</div>

Hi, all,  
by the way, I tried on CE v24, with no luck.  
Could you please tell us if this also works with the TABLE() construct ?

Thanks !

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [February 24, 2023, 10:29pm UTC](https://community.dremio.com/t/enable-partition-column-inference-option-doesn-t-work/10262/8 "2023-02-24T22:29:25Z")

</div>

@fetanchaud This is only for file system sources like HDFS/S3/AZ/GCS

---

<div class="post-metadata">

**Author:** ![fetanchaud](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/fetanchaud/32/4192_2.png) [@fetanchaud](https://community.dremio.com/u/fetanchaud)\
**Post date:** [February 25, 2023, 1:55pm UTC](https://community.dremio.com/t/enable-partition-column-inference-option-doesn-t-work/10262/9 "2023-02-25T13:55:12Z")

</div>

Sorry @balaji.ramaswamy , let me explain, I meant this kind of construct :

```auto

select * from table( "cbp"."chevoperiod"."rtjepman"."rtemanf" ( type=>'text', fieldDelimiter=>';', extractHeader=>True, trimHeader=>True ) )

```

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [February 27, 2023, 3:59am UTC](https://community.dremio.com/t/enable-partition-column-inference-option-doesn-t-work/10262/10 "2023-02-27T03:59:17Z")

</div>

@fetanchaud

Promotion has to be either done with UI or via REST [Dremio](https://docs.dremio.com/software/rest-api/catalog/post-catalog-id/) or enable auto promotion, are you trying to promote via this command

Also this feature for partition inference is only for PARQUET, is your file format PARQUET?

---

<div class="post-metadata">

**Author:** ![Pawel](https://avatars.discourse-cdn.com/v4/letter/p/ecae2f/32.png) [@Pawel](https://community.dremio.com/u/Pawel)\
**Post date:** [March 31, 2023, 7:22am UTC](https://community.dremio.com/t/enable-partition-column-inference-option-doesn-t-work/10262/11 "2023-03-31T07:22:37Z")

</div>

@balaji.ramaswamy Could you let us know if this is bug or EE only feature

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [April 9, 2023, 12:08am UTC](https://community.dremio.com/t/enable-partition-column-inference-option-doesn-t-work/10262/12 "2023-04-09T00:08:09Z")

</div>

@Pawel

Although initially it did not work, I then had to enable unlimited splits in order for it to show up on CE. In 22.x, unlimited splits is on by default but the dist:/// on dremio.conf needs to be configured to write to something like S3/HDFS/AZ/GCS etc

 ![Screen Shot 2023-04-08 at 5.06.36 PM](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/2X/e/e80faba7f06d23273fc770e6006663ffe81a80dd.png)

---

<div class="post-metadata">

**Author:** ![Pawel](https://avatars.discourse-cdn.com/v4/letter/p/ecae2f/32.png) [@Pawel](https://community.dremio.com/u/Pawel)\
**Post date:** [April 11, 2023, 6:24am UTC](https://community.dremio.com/t/enable-partition-column-inference-option-doesn-t-work/10262/13 "2023-04-11T06:24:09Z")

</div>

@balaji.ramaswamy Many thanks  
It also works with NAS storage for example I could use local path /opt/test:  
dist: “file:///opt/test”

 ![image](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/2X/9/900a79cf821b0a18f53bf72a9aa48d1d1df22a54.png)

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [April 11, 2023, 3:58pm UTC](https://community.dremio.com/t/enable-partition-column-inference-option-doesn-t-work/10262/14 "2023-04-11T15:58:02Z")

</div>

@Pawel So works with NAS and not via S3?
