Helm install error on EKS

1

Hi,

When i try to install dremio as mentioned in tutorials in EKS I am facing error at helm install . --wait.

I have not made any changes to the config files (as it is from git). The following is the error:
(Copied first few lines and last few lines)

Error: template: dremio/templates/dremio-configmap.yaml:6:5: executing “dremio/templates/dremio-configmap.yaml” at <tpl (.Files.Glob “config/*”).AsConfig .>: error calling tpl: error during tpl function execution for "core-site.xml: "<?xml version=\\\"1.0\\\"?>\r\n\r\n \r\n{{- if and .Values.distStorage.type (ne .Values.distStorage.type \“local\”)\n }}\r\n \r\n {{- if eq .Values.distStorage.type\n \“aws\” }}\r\n \r\n fs.dremioS3.impl\r\n The\n FileSystem implementation. Must be set to com.dremio.plugins.s3.store.S3FileSystem\r\n\n \

…
<rollingPolicy class=\“ch.qos.logback.core.rolling.TimeBasedRollingPolicy\”>\r\n\n \ {dremio.log.path}/archive/hive.deprecated.function.warning.%d{yyyy-MM-dd}.%i.log.gz</fileNamePattern>\\r\\n\n \\ <maxHistory>30</maxHistory>\\r\\n <timeBasedFileNamingAndTriggeringPolicy\n class=\\\"ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP\\\">\\r\\n <maxFileSize>100MB</maxFileSize>\\r\\n\n \\ </timeBasedFileNamingAndTriggeringPolicy>\\r\\n </rollingPolicy>\\r\\n\\r\\n\n \\ <encoder>\\r\\n <pattern>%date{ISO8601} [%thread] %-5level %logger{36}\n - %msg%n</pattern>\\r\\n </encoder>\\r\\n </appender>\\r\\n </then>\\r\\n\n \\ </if>\\r\\n </logger>\\r\\n\\r\\n <root>\\r\\n <level value=\\\"{dremio.log.root.level:-error}\"/>\r\n\n \ \r\n \r\n <appender-ref\n ref=\“text\”/>\r\n <appender-ref ref=\“json\”/>\r\n \r\n \r\n\n \ <appender-ref ref=\“console\”/>\r\n \r\n \r\n \r\n\r\n\r\n"": parse error at (dremio/templates/dremio-configmap.yaml:3): unexpected “\” in operand

I am stuck, couldn’t move forward, please help.

Govi

2

@govi-drem

It looks like the filesystem schema on core-site.xml is having an issue. Are you storing reflections on S3? See Documentation below

https://docs.dremio.com/deployment/dist-store-config.html#configuring-dremio-for-amazon-s3

3

Dear @balaji.ramaswamy

Thanks for your response. No I am not setting it to s3 but it is the default one (local). In fact, haven’t made any changes to cloud tools folder from git when i try to do helm install. However i did try setting up with S3 today and still see the same error.

Govi

4

@govi-drem

Cn you please send me your dremio.comf?

5

Yes. First, I used the original one that came from GIT (without changing the values anywhere) with the default setting to local. Then per your suggestion, I modified it to use s3. And attached the file here.

dremio.zip (1.4 KB)

6

@govi-drem

In order to use S3 as a distributed store you would need to follow below steps (have you copied core-site.xml?)

Configuring S3 as a distributed storage

7

Dear @balaji.ramaswamy,

Yes, of course, i have followed the link and configured (updated core-site.xml with accessid and accesskey. Herewith attached core-site.xml file as well for your reference (obscuring my ID and key). I also tried to remove false cases (azure parts) just to see if that is causing issues, but still getting the same error.
I have also updated the values.xml with S3.

distStorage:
type: “aws”
aws: # S3
bucketName: “dremio-lake”
path: “/”
accessKey: “[My access key]”
secret: “[my access secret]”

The S3 bucket also has necessary permission as mentioned in the links.

But, the point is it was throwing the same error even when I leave it to local (default configs). So not sure if S3 config has anything to do with this issue.

core-site.zip (591 Bytes)

8

Hi, probably this is not a final solution, but it allowed me to continue working.
With the current code in dremio-configmap.yaml, it does not matter what you put in config folder, it will fail.
I modified the last line of dremio-configmap.yaml (in templates folder):
{{ (tpl (.Files.Glob “configs/*”).AsConfig . ) | nindent 2 }}

9

@govi-drem,

Have you tried downloading the latest charts and trying this again? Another user reported something similar around the time of your original post. Or do you still see this error?

Error: template: dremio/templates/dremio-configmap.yaml:6:5: executing “dremio/templates/dremio-configmap.yaml” at <tpl (.Files.Glob “config/*”).AsConfig .>: error calling tpl: error during tpl function execution for "core-site.xml: "<?xml version=\\\"1.0\\\"?>\r\n\r\n \r\n{{- if and .Values.distStorage.type (ne .Values.distStorage.type \“local\”)\n }}\r\n \r\n {{- if eq .Values.distStorage.type\n \“aws\” }}\r\n \r\n fs.dremioS3.impl\r\n The\n FileSystem implementation. Must be set to com.dremio.plugins.s3.store.S3FileSystem\r\n\n \

Also, which version of Helm are you using?

10

Hi,

I discovered that the problem are the windows style carriage return.
I deleted all the /r in the config files and the original code works fine for me.

11

Hi,
Could you please elaborate?

12

@vipul5111

Seems like for @rafaleon15 there were some special characters like carriage return that crept into the file. If you vi the helm chart, does it look ok?

13

@balaji.ramaswamy

How do I remove the carriage return from all the files?

This is the dremio-configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
name: dremio-config
data:
{{- tpl ($.Files.Glob “config/*”).AsConfig . | nindent 2 }}

apiVersion: v1
kind: ConfigMap
metadata:
name: dremio-hive2-config
data:
{{- tpl ($.Files.Glob “config/hive2/*”).AsConfig . | nindent 2 }}

apiVersion: v1
kind: ConfigMap
metadata:
name: dremio-hive3-config
data:
{{- tpl ($.Files.Glob “config/hive3/*”).AsConfig . | nindent 2 }}

{{- originalValues := mustDeepCopy .Values -}}
{{- range engine := keys (default (dict) .Values.executor.engineOverride) -}}
{{- if has engine .Values.executor.engines -}}
{{- _ := mustMergeOverwrite .Values $originalValues -}}
{{- engineAwareConfig := mustDeepCopy .Values -}}
{{- $_ := set $engineAwareConfig “currentEngine” engine }} {{- _ := mustMergeOverwrite engineAwareConfig.executor (get .Values.executor.engineOverride engine) -}} {{- _ := mustMergeOverwrite $.Values $engineAwareConfig -}}
apiVersion: v1
kind: ConfigMap
metadata:
name: dremio-config-{{ engine }} data: {{- tpl (.Files.Glob “config/*”).AsConfig $ | nindent 2 }}

{{- end -}}
{{- end -}}
~
~
~
~
~

This is the helm chart

apiVersion: “v1”
name: “dremio”
version: “2.0.0”
keywords:

14

I did it in a Windows text editor (Notepad++). Replace /r for nothing.