Fabric Layer Provides no Security (Disclosure)

NOTICE: I have discussed this security concern with Dremio (the company), they do not consider it a security vulnerability, and instead a limitation of the CE version. I have also received appropriate permission to discuss this issue with the online community.

The Fabric Layer
Dremio uses a protocol called the Fabric Layer to communicate between nodes in a Dremio cluster. Whether you are running a cluster or only a single node Dremio runs the Fabric Layer service, and opens a port for communication (45678 by default). In Dremio CE, Fabric Layer has NO security. Anyone who knows which libraries to import can write a simple piece of code that connects to your nodes exposed Fabric Layer and makes remote command calls.

Security Implications
One of the services exposed by the Fabric Layer is external access to Dremio’s Key Value store (database). With this access you can read/write any key/value in Dremio’s backend. For example, you could create access tokens as any user, modify stores, submit jobs, read passwords for data stores, or change anything without the normal logic checks put in place by the front end. And all of this with full anonymity and no security checks.

The following sample code demonstrates creating a remote connection to the Fabric Layer of a Dremio coordinator node and creating an API access token that never expires. It then finds a job and deletes it: https://gist.github.com/patricker/4bf3d668b301e0872a7ff994c5c82683#file-securityrunner-java

Resolution
Here are two suggestions for ways you can help to resolve this issue; I say “help”, because neither of them is perfect.

  • Firewall rules: Depending on your environment you can setup firewall rules to prevent access to the port either from all hosts, or only allow access from known cluster nodes.
  • Fabric Layer Wire Encryption: A while ago I released a free, open source, version of Wire Encrypytion for Dremio. Ref: Open Source Wire Encryption (With Code!).

Wire Encryption, which was the suggestion provided by Dremio (the company) before I mentioned I was discussing the CE version, is not a perfect solution for this problem either, as security is only enforced at the CA level, and there is no way to authenticate/deny specific client certificates short of certificate revocation. In most environments this means that anyone who can get a client certificate issued from the same CA that you used to setup wire encryption can also connect to your Fabric Layer. So if you use an enterprise CA to issue certificates to both servers and users it’s likely a malicious user could still use this access path. For this reason Dremio (the company) suggested using an isolated CA created for the purpose.

4 Likes