I can see that the query had been executed successfully according to the corresponding job status which is COMPLETED as shown in one of the attachment. However, the result can not be rendered out properly in Data Viewer
I have went through both server.log and server.out and didn’t find any error in the log entries. I am thinking maybe this is an issue UI/Dashboard related?
Many thanks if someone could help me on this.
Best regards,
Sean
Hi @xiao-cui,
Hi Ben, I have one coordinator and one executor, didn’t adopt a load balancer, is that a problem?
No, the lack of a load balancer is not a problem, but this issue looks superficially like an issue another user was facing where the load balancer involved.
How is your Dremio cluster deployed, for example, are you useing AWS EC2 instances, GCE, on premise hosts?
Does this happen with all queries or just queries against one particular dataset?
Please share your {{dremio-env}} and {{dremio.conf}} if possible.
Thanks for your help @ben, here is my dremio.conf:
coordinator:
paths: {
# the local path for dremio to store data.
local: "/var/lib/dremio"
# the distributed path Dremio data including job results, downloads, uploads, etc
dist: "pdfs://"${paths.local}"/pdfs"
}
services: {
coordinator.enabled: true,
coordinator.master.enabled: true,
executor.enabled: false
}
zookeeper: "ip-172-31-12-111.ap-southeast-2.compute.internal:2181/dremio"
executor:
paths: {
# the local path for dremio to store data.
local: "/var/lib/dremio"
# the distributed path Dremio data including job results, downloads, uploads, etc
dist: "pdfs://"${paths.local}"/pdfs"
}
services: {
coordinator.enabled: false,
coordinator.master.enabled: false,
executor.enabled: true,
}
zookeeper: "ip-172-31-12-111.ap-southeast-2.compute.internal:2181/dremio"
and this is my dremio-env (mostly default value is used):
DREMIO_LOG_DIR=/var/log/dremio
DREMIO_PID_DIR=/var/run/dremio
i deployed dremio cluster on 2 ec2 instance, one for cooridnator and one for executor, and as shown below, is my security group:
Through the log on server.log and server.out, I can see the job has been finished successfully without any error, but the front-end returned webscoket connection error in the console log, and no data are presented as shown in the previous attachment (loading annimation keep spinning)
From the screenshot, it looks like the coordinator and executor are in the same availability zone, is that correct?
Also, does this happen with every query or just a select few? Have you tried refreshing the browser when this occurs?
Please server.log and server.out if you can.
Yes, this happens for all queries, (failure due to websocket connection error), after a preview of the sample dataset, I can not loaded (which is the execution of a Select * statement)
Here is the server.log and server.out:
server.log on executor
2019-02-18 03:38:30,080 [FABRIC-rpc-event-queue] INFO c.d.sabot.exec.FragmentExecutors - Received remote fragment start instruction for 2395d749-e8be-4835-0038-4a19ec9c6400:1:0
2019-02-18 03:38:30,081 [FABRIC-rpc-event-queue] INFO c.d.sabot.exec.FragmentExecutors - Received remote fragment start instruction for 2395d749-e8be-4835-0038-4a19ec9c6400:0:0
2019-02-18 03:38:30,082 [FABRIC-rpc-event-queue] INFO c.d.sabot.exec.FragmentExecutors - Received remote fragment start instruction for 2395d749-e8be-4835-0038-4a19ec9c6400:2:0
2019-02-18 03:38:30,086 [FABRIC-rpc-event-queue] INFO c.d.sabot.exec.FragmentExecutors - Received remote fragment start instruction for 2395d749-e8be-4835-0038-4a19ec9c6400:4:0
2019-02-18 03:38:30,087 [FABRIC-rpc-event-queue] INFO c.d.sabot.exec.FragmentExecutors - Received remote fragment start instruction for 2395d749-e8be-4835-0038-4a19ec9c6400:3:0
2019-02-18 03:38:30,110 [e0 - 2395d749-e8be-4835-0038-4a19ec9c6400:frag:3:0] INFO c.d.s.e.f.FragmentStatusReporter - 2395d749-e8be-4835-0038-4a19ec9c6400:3:0: State to report: FINISHED
2019-02-18 03:38:30,130 [e1 - 2395d749-e8be-4835-0038-4a19ec9c6400:frag:4:0] INFO c.d.s.e.f.FragmentStatusReporter - 2395d749-e8be-4835-0038-4a19ec9c6400:4:0: State to report: FINISHED
2019-02-18 03:38:30,134 [e1 - 2395d749-e8be-4835-0038-4a19ec9c6400:frag:2:0] INFO c.d.s.e.f.FragmentStatusReporter - 2395d749-e8be-4835-0038-4a19ec9c6400:2:0: State to report: FINISHED
2019-02-18 03:38:30,140 [e0 - 2395d749-e8be-4835-0038-4a19ec9c6400:frag:1:0] INFO c.d.s.e.f.FragmentStatusReporter - 2395d749-e8be-4835-0038-4a19ec9c6400:1:0: State to report: FINISHED
2019-02-18 03:38:30,149 [e0 - 2395d749-e8be-4835-0038-4a19ec9c6400:frag:0:0] INFO c.d.s.e.f.FragmentStatusReporter - 2395d749-e8be-4835-0038-4a19ec9c6400:0:0: State to report: FINISHED
server.log on coordinator:
2019-02-18 02:27:21,656 [out-of-band-observer] INFO query.logger - {"queryId":"2395e7f6-7dcb-bfc7-ed0a-b4faad7ba400","schema":"[sys]","queryText":"select\n 'green' as status,\n nodes.hostname name,\n nodes.ip_address ip,\n nodes.fabric_port port,\n cpu cpu,\n memory memory \nfrom\n sys.nodes,\n (select\n hostname,\n fabric_port,\n sum(cast(cpuTime as float) / cores) cpu \n from\n sys.threads \n group by\n hostname,\n fabric_port) cpu,\n (select\n hostname,\n fabric_port,\n direct_current * 100.0 / direct_max as memory \n from\n sys.memory) memory \nwhere\n nodes.hostname = cpu.hostname \n and nodes.fabric_port = cpu.fabric_port \n and nodes.hostname = memory.hostname \n and nodes.fabric_port = memory.fabric_port \norder by\n name,\n port","start":1550456841451,"finish":1550456841652,"outcome":"COMPLETED","username":"admin"}
2019-02-18 03:38:03,033 [qtp1357067993-94] INFO c.dremio.dac.resource.SystemResource - address: ip-172-31-12-111.ap-southeast-2.compute.internal fabric port: 45678
2019-02-18 03:38:03,154 [2395d764-3644-20b6-69f9-10d1f8906200:foreman] WARN c.d.exec.planner.logical.RexToExpr - Converting exact decimal into approximate decimal. Should be fixed once decimal is implemented.
2019-02-18 03:38:03,232 [out-of-band-observer] INFO query.logger - {"queryId":"2395d764-3644-20b6-69f9-10d1f8906200","schema":"[sys]","queryText":"select\n 'green' as status,\n nodes.hostname name,\n nodes.ip_address ip,\n nodes.fabric_port port,\n cpu cpu,\n memory memory \nfrom\n sys.nodes,\n (select\n hostname,\n fabric_port,\n sum(cast(cpuTime as float) / cores) cpu \n from\n sys.threads \n group by\n hostname,\n fabric_port) cpu,\n (select\n hostname,\n fabric_port,\n direct_current * 100.0 / direct_max as memory \n from\n sys.memory) memory \nwhere\n nodes.hostname = cpu.hostname \n and nodes.fabric_port = cpu.fabric_port \n and nodes.hostname = memory.hostname \n and nodes.fabric_port = memory.fabric_port \norder by\n name,\n port","start":1550461083037,"finish":1550461083228,"outcome":"COMPLETED","username":"admin"}
2019-02-18 03:38:29,934 [qtp1357067993-95] INFO c.dremio.dac.resource.SystemResource - address: ip-172-31-12-111.ap-southeast-2.compute.internal fabric port: 45678
2019-02-18 03:38:30,063 [2395d749-e8be-4835-0038-4a19ec9c6400:foreman] WARN c.d.exec.planner.logical.RexToExpr - Converting exact decimal into approximate decimal. Should be fixed once decimal is implemented.
2019-02-18 03:38:30,155 [out-of-band-observer] INFO query.logger - {"queryId":"2395d749-e8be-4835-0038-4a19ec9c6400","schema":"[sys]","queryText":"select\n 'green' as status,\n nodes.hostname name,\n nodes.ip_address ip,\n nodes.fabric_port port,\n cpu cpu,\n memory memory \nfrom\n sys.nodes,\n (select\n hostname,\n fabric_port,\n sum(cast(cpuTime as float) / cores) cpu \n from\n sys.threads \n group by\n hostname,\n fabric_port) cpu,\n (select\n hostname,\n fabric_port,\n direct_current * 100.0 / direct_max as memory \n from\n sys.memory) memory \nwhere\n nodes.hostname = cpu.hostname \n and nodes.fabric_port = cpu.fabric_port \n and nodes.hostname = memory.hostname \n and nodes.fabric_port = memory.fabric_port \norder by\n name,\n port","start":1550461109938,"finish":1550461110151,"outcome":"COMPLETED","username":"admin"}
we are doing some trail for company use, but due to this deployment issue, our data scientist couldn’t try it on cloud properly. would be appreciate if you could help.
@xiao-cui, do you have a Firewall or reverse proxy servers between you and Dremio?
@ben thanks, it is a firewall issue, I use a load balancer to forward the request from port 80 to 9047 then it works.
Do you know how can I configure the tableau so as to overwrite the default connection configuration generated by the dashboard when I click export to tableau? for example, my domain for the dashboard is abc.com, but i have another domain for db connection which is 123.com, the tableau connection should use 123.com as the hostname rather than abc.com.
@xiao-cui, are you creating a “live” connection to Dremio in Tableau, using these steps:
https://docs.dremio.com/client-applications/tableau.html#creating-a-live-connection-to-a-dataset-from-dremio
Or are you setting up a Dremio datasource in Tableau following these:
https://docs.dremio.com/client-applications/tableau.html#creating-a-dremio-source-in-tableau
Are you trying to connect to 2 separate Dremio instances from the same Tableau?
Hi @ben, yes, I am creating a live connection using the steps as shown here https://docs.dremio.com/client-applications/tableau.html#creating-a-live-connection-to-a-dataset-from-dremio
But the problem I have is due to that my db server is under a different subdomain (eg. odbc.abc.com) compared with my dashboard (eg. client.abc.com), the configuration created by the system is not correct as shown in below:
To set up a Dremio datasource in Tableau following the steps https://docs.dremio.com/client-applications/tableau.html#creating-a-dremio-source-in-tableau, it works perfectly as long as I give the host and port number I am using (because I am using a load balancer to redirect the requests)
Hi @xiao-cui, if by “db server” you mean the Dremio coordinator host, that can be configured in the system DSN when you install the Dremio ODBC connector:
https://docs.dremio.com/drivers/dremio-connector.html
Are you using Tableau Desktop or Tableau Server?
Hello,
I’m facing the same problem using Docker Image (One container).
I can see that the job was executed correctly, and no error message on my logs (docker logs). But the table is not rendered.
When I check the network traffic on my navigator the following errors are displayed:
SOCKET CONNECTION ERROR type error /apiv2/socket event phase2 failed
DEVTOOLS FAILED TO LOAD SOURCEMAP: COULD NOT LOAD CONTENT HTTP ERROR STATUS CODE 404
establishing a tunnel via proxy server failed
My docker container is on a server, the ports are published and the access is opened (Telnet is working).
Please, any help would be much appreciated.
