Automate user and source creation on Dremio docker

1

I am using the official docker dremio image Docker Hub

I need to create a user and datalake (HDFS,S3) when container start,avoiding ui creation. Is it possible via API?

I have try to create the user and lakes, then export data (“data” folder) and mount the data on another instance(new container) but the ui still require first user creation.

I need the user and sources for some automation tests where I can’t create the resources trhough the ui.

2

@stefano.castoldi

No API but you can add this to dremio.conf on the coordinator and it will create a user “dremio” password “dremio123”

debug: {
addDefaultUser: true
}

Currently available user API’s are below

http://docs.dremio.com/rest-api/user/

3

Thanks! This solve the problem. I have update the thread. Is it possible create the sources in the same way (on dremio conf) without using the API?

4

@stefano.castoldi Not for source unfortunately

5

Hi,
I am trying out Dremio docker for local development.

By inspecting what is happening in Chrome when I perform actions via the GUI - I discovered can create the admin user thus:

Create admin user on dremio

curl ‘http://localhost:9047/apiv2/bootstrap/firstuser’
-X ‘PUT’
-H ‘Content-Type: application/json’
-H ‘Authorization: null’
–data-raw ‘{“userName”:“Luke”,“firstName”:“Luke”,“lastName”:“Skywalker”,“email”:“LS@rebelalliance.universe”,“createdAt”:1694089769453,“password”:“Skywalker01”,“extra”:null}’
–compressed

I can also create a Space (Tatooine) like this:

curl ‘http://localhost:9047/api/v3/catalog’
-H ‘Accept: /’
-H ‘Authorization: _dremiolnps9vrhgho9sdo1mvam25hvrl’
-H ‘Connection: keep-alive’
-H ‘Content-Type: application/json’
–data-raw ‘{“name”:“Tatooine”,“entityType”:“space”}’
–compressed ;

However, I’m stuck at trying to get a hold of the correct token for the authorisation (so the bit after “_dremio”) for my automation script.

After a few goes - the tokens seem to change, but I have seen them repeat and also the value seems to be hiding in plain sight in a log file in the container.

Can anyone help with this, please?

Thanks

6

Answering my own question here, but this seems to work (based on another post I saw getting the token using Python):

# Get token for admin user to use with api
output=$(curl -X POST 'http://localhost:9047/apiv2/login' \
  -H 'Accept: */*' \
  -H 'Connection: keep-alive' \
  -H 'Content-Type: application/json' \
  --data-raw '{"userName":"Luke","password":"Skywalker01"}' \
  --compressed ;
)
dremio_token=$(echo "$output" | python -c "import sys, json; print(json.load(sys.stdin)['token'])")
echo "$dremio_token"

# Create Dremio Tatooine Space and folders
curl 'http://localhost:9047/api/v3/catalog' \
  -H 'Accept: */*' \
  -H "Authorization: _dremio${dremio_token}" \
  -H 'Connection: keep-alive' \
  -H 'Content-Type: application/json' \
  --data-raw '{"name":"Tatooine","entityType":"space"}' \
  --compressed ;

curl 'http://localhost:9047/apiv2/space/Tatooine/folder/?nocache=1694170905671' \
  -H 'Accept: */*' \
  -H "Authorization: _dremio${dremio_token}" \
  -H 'Connection: keep-alive' \
  -H 'Content-Type: application/json' \
  --data-raw '{"name":"Mos Eisley"}' \
  --compressed ;
7

@Jamie Thanks a lot for posting your learnings, super useful when someone hits the same issue