Error uploading file

1

Hello,

We got the error below when uploading files to Dremio. Can you help to identify the problem? Thank you so much.

2018-08-03 15:20:01,954 [249b8d5d-cbf0-c360-5933-71d6d7f04000:foreman] ERROR c.d.e.p.f.HardAffinityFragmentParallelizer - Number of mandatory endpoints (2) that require an assignment is more than the allowed fragment max width (1).
2018-08-03 15:20:01,954 [249b8d5d-cbf0-c360-5933-71d6d7f04000:foreman] INFO  query.logger - {"queryId":"249b8d5d-cbf0-c360-5933-71d6d7f04000","schema":"","queryText":"select * from table(\"__home\".dremio_data_dist.pdfs.uploads.\"_staging.e18f00f7c8f4\".\"@admin\".\"test_csv-054a7338-78cc-4593-a2b4-8de390bd7a13\" (type => 'text', fieldDelimiter => ',', comment => '#', \"escape\" => '\"', quote => '\"', lineDelimiter => '\r\n', extractHeader => false, skipFirstLine => false, autoGenerateColumnNames => true, trimHeader => true)) limit 500","start":1533309601830,"finish":1533309601954,"outcome":"FAILED","username":"admin","commandDescription":"<unknown>"}
2018-08-03 15:20:01,954 [249b8d5d-cbf0-c360-5933-71d6d7f04000:foreman] INFO  query.logger - {"queryId":"249b8d5d-cbf0-c360-5933-71d6d7f04000","schema":"","queryText":"select * from table(\"__home\".dremio_data_dist.pdfs.uploads.\"_staging.e18f00f7c8f4\".\"@admin\".\"test_csv-054a7338-78cc-4593-a2b4-8de390bd7a13\" (type => 'text', fieldDelimiter => ',', comment => '#', \"escape\" => '\"', quote => '\"', lineDelimiter => '\r\n', extractHeader => false, skipFirstLine => false, autoGenerateColumnNames => true, trimHeader => true)) limit 500","start":1533309601830,"finish":1533309601954,"outcome":"FAILED","username":"admin","commandDescription":"<unknown>"}
2018-08-03 15:20:01,954 [249b8d5d-cbf0-c360-5933-71d6d7f04000:foreman] ERROR c.d.exec.work.foreman.AttemptManager - PhysicalOperatorSetupException: Can not parallelize fragment.
com.dremio.common.exceptions.UserException: PhysicalOperatorSetupException: Can not parallelize fragment.
	at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:746) ~[dremio-common-2.0.5-201806021755080191-767cfb5.jar:2.0.5-201806021755080191-767cfb5]
	at com.dremio.exec.work.foreman.AttemptManager$AttemptResult.close(AttemptManager.java:563) [dremio-sabot-kernel-2.0.5-201806021755080191-767cfb5.jar:2.0.5-201806021755080191-767cfb5]
	at com.dremio.exec.work.foreman.AttemptManager.moveToState(AttemptManager.java:618) [dremio-sabot-kernel-2.0.5-201806021755080191-767cfb5.jar:2.0.5-201806021755080191-767cfb5]
	at com.dremio.exec.work.foreman.AttemptManager.run(AttemptManager.java:341) [dremio-sabot-kernel-2.0.5-201806021755080191-767cfb5.jar:2.0.5-201806021755080191-767cfb5]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_171]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_171]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_171]
Caused by: com.dremio.exec.work.foreman.ForemanException: Unexpected exception during fragment initialization: Can not parallelize fragment.
	at com.dremio.exec.work.foreman.AttemptManager.run(AttemptManager.java:342) [dremio-sabot-kernel-2.0.5-201806021755080191-767cfb5.jar:2.0.5-201806021755080191-767cfb5]
	... 3 common frames omitted
Caused by: com.dremio.exec.physical.PhysicalOperatorSetupException: Can not parallelize fragment.
	at com.dremio.exec.planner.fragment.HardAffinityFragmentParallelizer.checkOrThrow(HardAffinityFragmentParallelizer.java:146) ~[dremio-sabot-kernel-2.0.5-201806021755080191-767cfb5.jar:2.0.5-201806021755080191-767cfb5]
	at com.dremio.exec.planner.fragment.HardAffinityFragmentParallelizer.parallelizeFragment(HardAffinityFragmentParallelizer.java:78) ~[dremio-sabot-kernel-2.0.5-201806021755080191-767cfb5.jar:2.0.5-201806021755080191-767cfb5]
	at com.dremio.exec.planner.fragment.SimpleParallelizer.parallelizeFragment(SimpleParallelizer.java:289) ~[dremio-sabot-kernel-2.0.5-201806021755080191-767cfb5.jar:2.0.5-201806021755080191-767cfb5]
	at com.dremio.exec.planner.fragment.SimpleParallelizer.getFragmentsHelper(SimpleParallelizer.java:203) ~[dremio-sabot-kernel-2.0.5-201806021755080191-767cfb5.jar:2.0.5-201806021755080191-767cfb5]
	at com.dremio.exec.planner.fragment.SimpleParallelizer.getFragments(SimpleParallelizer.java:174) ~[dremio-sabot-kernel-2.0.5-201806021755080191-767cfb5.jar:2.0.5-201806021755080191-767cfb5]
	at com.dremio.exec.planner.sql.handlers.commands.ExecutionPlanCreator.getExecutionPlan(ExecutionPlanCreator.java:85) ~[dremio-sabot-kernel-2.0.5-201806021755080191-767cfb5.jar:2.0.5-201806021755080191-767cfb5]
	at com.dremio.exec.planner.sql.handlers.commands.HandlerToExec.plan(HandlerToExec.java:68) ~[dremio-sabot-kernel-2.0.5-201806021755080191-767cfb5.jar:2.0.5-201806021755080191-767cfb5]
	at com.dremio.exec.work.foreman.AttemptManager.run(AttemptManager.java:293) [dremio-sabot-kernel-2.0.5-201806021755080191-767cfb5.jar:2.0.5-201806021755080191-767cfb5]
	... 3 common frames omitted
2

Could you take a look/share profile of this job (upload)?: How to share profile
The only caveat you may not see this job in list of jobs unless you add to job types selection “Internal”
34%20AM

Also to may be overcome an issue in the interim try to add “NAS” Source so you can get file you try to upload from local file system directly

55%20AM
Screen Shot 2018-08-03 at 11.40.55 AM.png929×971 66.8 KB

3

Cool, I got it working now! Thank you.

4

I also have this problem on our Kubernetes installation.

Profile is attached - uploads fail but NAS works as expected.

Any ideas?

8502804d-4f4d-4763-a2ad-874566376516.zip (6.4 KB)

5

@mg66

This looks like a known issue, does the NAS workaround work?

Thanks
@balaji.ramaswamy

6

Yes, the workaround works.

Any ideas on a resolution?