Cannot create a new project since there is a project already running on this instance

1

Hello,

I have an AWS instance with a currently opened project.

I want to be able to clone the instance+project to another test environment.

I have tried to follow the instructions in Dremio

which asks me to create a custom project on a new dremio AWS instance and then try to run the restore.

because we use encrypted s3 for the volumes, I have to try to create a custom project as per the instructions at Dremio

I run the following curl stmt on the aws dremio host:

curl -v -X POST -H “Content-Type:application/json” --data @a.json http://localhost:9047/aws/gateway/customProject

content of a.json is below:

{
“name”: “CA1”,
“id”: “facb3c66-xxxxxxx”,
“engineSize”: “SMALL”,
“customNodeCount”: 0,
“enableAutoBackups”: false,
“engineNodeType”: “STANDARD”,
“iamInstanceProfile”: “”,
“instanceId”: “i-05e1cf6be63149ex1”,
“disablePublicIp”: true,
“ebsVolumeId”: “vol-908e4178325e0e24b”,
“efsVolumeName”: “fs-9503897a252be16ce”,
“s3BucketName”: “dremio-me-xxxxx…”
}

The project name, ebs and efs Volume (some literals changed) and s3 bucket all have been created with the required tags. However, I keep getting this error, although this instance does not have an active project created. The existing active project is running on another Dremio instance.

2023-03-06 14:46:25,822 [aws-project-action-pool1] INFO com.dremio.dac.server.AwsUtil - Executing command, bash -c sudo chown -R 2000:2000 /etc/dremio/dremio.conf
2023-03-06 14:46:25,846 [aws-project-action-pool1] INFO com.dremio.dac.server.AwsUtil - Executing command, bash -c sudo cp /etc/dremio/dremio.conf /etc/dremio/dremio.conf.back
2023-03-06 14:46:26,003 [aws-project-action-pool1] INFO c.dremio.dac.server.AwsConfigurator - createCustomProject attachedVolumes size 1
2023-03-06 14:46:26,003 [aws-project-action-pool1] WARN c.dremio.dac.server.AwsConfigurator - Cannot create a new project since there is a project already running on this instance.
2023-03-06 14:46:26,004 [aws-project-action-pool1] INFO com.dremio.dac.server.AwsUtil - Executing command, bash -c sudo cp /etc/dremio/dremio.conf.back /etc/dremio/dremio.conf
2023-03-06 14:46:26,020 [aws-project-action-pool1] INFO c.d.dac.resource.AwsSetupResource - Failed to create acustom project
java.lang.UnsupportedOperationException: Cannot create a new project since there is a project already running on this instance.
** at com.dremio.dac.server.AwsConfigurator.createCustomProject(AwsConfigurator.java:555)**
at com.dremio.dac.resource.AwsSetupResource.lambda$createCustomProject$3(AwsSetupResource.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)

Any ideas why the second custom project on a separate instance does not get allowed to be created?

2

@Muneer Have asked internally for someone to respond

3

@Muneer The error message “createCustomProject attachedVolumes size 1” shows that there is already an EBS mount to the instance on which you are trying to launch a custom project.

Can you walk me through the steps you followed on that instance on which you are trying to create a custom project? I want to know why and how there is a volume attached to it.

Alternatively, you can try to spin up a new instance and launch the custom project on it.

4

Thanks @Rafay . What had happened was I had created a template from the existing AWS EC2 instance that had my current project and using it to launch a new EC2 instance.

this was causing an EBS volume to get created and get attached to my new EC2 instance and that obviously explains the error message that there was already an EBS mount. (I couldn’t remember from my first custom project creation that I had created a separate EBS volume and assumed it was a necessary part of the install and didn’t question it).

Went back to creating a fresh EC2 instance from the AMI and that then let the process complete successfully and i could create the project and spin up the new instance.

Thanks to you and @balaji.ramaswamy for your help in resolving this.