Dremio java.lang.OutOfMemoryError on build

Hello,

I am trying to build Dremio with mvn clean install -DskipTests but I am receiving a java.lang.OutOfMemoryError: Java heap space error. Follow the stack:

The system is out of resources.
Consult the following stack trace for details.
java.lang.OutOfMemoryError: Java heap space
	at java.util.LinkedHashMap$LinkedValues.iterator(LinkedHashMap.java:607)
	at com.sun.tools.javac.code.SymbolMetadata.getAttributesForCompletion(SymbolMetadata.java:203)
	at com.sun.tools.javac.code.SymbolMetadata.setDeclarationAttributesWithCompletion(SymbolMetadata.java:190)
	at com.sun.tools.javac.code.Symbol.setDeclarationAttributesWithCompletion(Symbol.java:227)
	at com.sun.tools.javac.comp.MemberEnter.actualEnterAnnotations(MemberEnter.java:964)
	at com.sun.tools.javac.comp.MemberEnter.access$600(MemberEnter.java:76)
	at com.sun.tools.javac.comp.MemberEnter$5.run(MemberEnter.java:888)
	at com.sun.tools.javac.comp.Annotate.flush(Annotate.java:155)
	at com.sun.tools.javac.comp.Annotate.enterDone(Annotate.java:141)
	at com.sun.tools.javac.comp.Enter.complete(Enter.java:524)
	at com.sun.tools.javac.comp.Enter.main(Enter.java:483)
	at com.sun.tools.javac.main.JavaCompiler.enterTrees(JavaCompiler.java:994)
	at com.sun.tools.javac.main.JavaCompiler.enterTreesIfNeeded(JavaCompiler.java:977)
	at com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing(JavacProcessingEnvironment.java:1254)
	at com.sun.tools.javac.main.JavaCompiler.processAnnotations(JavaCompiler.java:1182)
	at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:868)
	at com.sun.tools.javac.main.Main.compile(Main.java:535)
	at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:141)
	at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:150)
	at org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess(JavaxToolsCompiler.java:125)
	at org.codehaus.plexus.compiler.javac.JavacCompiler.performCompile(JavacCompiler.java:169)
	at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:823)
	at org.apache.maven.plugin.compiler.CompilerMojo.execute(CompilerMojo.java:129)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:956)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:192)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
	at java.lang.reflect.Method.invoke(Method.java:508)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] An unknown compilation problem occurred
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Dremio - Parent 4.2.1-202004111451200819-0c3ecaea:
[INFO] 
[INFO] Tools - Build tools ................................ SUCCESS [  1.541 s]
[INFO] Dremio - Parent .................................... SUCCESS [  2.956 s]
[INFO] Tools .............................................. SUCCESS [  0.504 s]
[INFO] Tools - Protostuff Compiler Plugin ................. SUCCESS [  3.531 s]
[INFO] Protocol ........................................... SUCCESS [ 23.127 s]
[INFO] Tools - Freemarker Codegen Tooling ................. SUCCESS [  1.552 s]
[INFO] Common ............................................. SUCCESS [  9.765 s]
[INFO] Source Connector SPI ............................... SUCCESS [  2.067 s]
[INFO] Services ........................................... SUCCESS [  0.228 s]
[INFO] Services - Coordinator ............................. SUCCESS [  1.239 s]
[INFO] Services - Telemetry - Utils ....................... SUCCESS [  0.727 s]
[INFO] Services - Telemetry - API ......................... SUCCESS [  0.957 s]
[INFO] Services - Base RPC ................................ SUCCESS [  1.701 s]
[INFO] Sabot .............................................. SUCCESS [  0.291 s]
[INFO] Sabot - Vector Tools ............................... SUCCESS [  2.221 s]
[INFO] Client - Parent .................................... SUCCESS [  0.276 s]
[INFO] Client - Base ...................................... SUCCESS [  1.046 s]
[INFO] Services - Options ................................. SUCCESS [  0.810 s]
[INFO] Services - Executor Service ........................ SUCCESS [  0.911 s]
[INFO] Services - gRPC .................................... SUCCESS [  0.517 s]
[INFO] Services - Maestro Service ......................... SUCCESS [  0.396 s]
[INFO] Services - Maestro common .......................... SUCCESS [  0.992 s]
[INFO] Services - Maestro Client .......................... SUCCESS [  0.268 s]
[INFO] Services - Telemetry - Impl ........................ SUCCESS [  0.798 s]
[INFO] Contrib - Parent ................................... SUCCESS [  0.166 s]
[INFO] Contrib - Slim Shaded Dependency JAR ............... SUCCESS [  0.701 s]
[INFO] Services - Fabric RPC .............................. SUCCESS [  0.792 s]
[INFO] Services - Datastore ............................... SUCCESS [ 13.285 s]
[INFO] Services - Task Scheduler .......................... SUCCESS [  1.108 s]
[INFO] Services - Users ................................... SUCCESS [  0.668 s]
[INFO] Services - Namespace ............................... SUCCESS [  3.353 s]
[INFO] Sabot - Logical Plan ............................... SUCCESS [  5.402 s]
[INFO] Services - Spill management ........................ SUCCESS [  1.257 s]
[INFO] Services - Resource Scheduler ...................... SUCCESS [  0.762 s]
[INFO] Services - Command Pool ............................ SUCCESS [  0.400 s]
[INFO] Analyst Center ..................................... SUCCESS [  0.092 s]
[INFO] Analyst Center - common ............................ SUCCESS [  0.511 s]
[INFO] Provisioning API - Parent .......................... SUCCESS [  0.114 s]
[INFO] Provisioning API - common .......................... SUCCESS [  2.439 s]
[INFO] Services - Executor Selector ....................... SUCCESS [  0.604 s]
[INFO] dremio-services-jobtelemetry ....................... SUCCESS [  0.161 s]
[INFO] Services - JobTelemetry - Common ................... SUCCESS [  0.827 s]
[INFO] Services - Job Telemetry - Server .................. SUCCESS [  1.072 s]
[INFO] Services - Job Telemetry Client .................... SUCCESS [  0.179 s]
[INFO] Sabot - Kernel ..................................... FAILURE [02:10 min]
[INFO] Client - JDBC Driver ............................... SKIPPED
[INFO] Provisioning API - Yarn Provisioning ............... SKIPPED
[INFO] Provisioning API - Shaded Twill JAR ................ SKIPPED
[INFO] Provisioning API - Yarn/Twill integration .......... SKIPPED
[INFO] Contrib - Shaded Hive 2 JAR ........................ SKIPPED
[INFO] Contrib - Shaded Hive 3 JAR ........................ SKIPPED
[INFO] Services - Jobs .................................... SKIPPED
[INFO] Plugins - Parent ................................... SKIPPED
[INFO] Plugins - Pseudo-Distributed FileSystem Plugin ..... SKIPPED
[INFO] Services - Accelerator ............................. SKIPPED
[INFO] Services - Configuration ........................... SKIPPED
[INFO] Plugins - NAS Plugin ............................... SKIPPED
[INFO] Analyst Center - backend ........................... SKIPPED
[INFO] Analyst Center - UI ................................ SKIPPED
[INFO] Plugins - Elasticsearch Plugin ..................... SKIPPED
[INFO] Plugins - HDFS Plugin .............................. SKIPPED
[INFO] Plugins - Mongo Plugin ............................. SKIPPED
[INFO] Storage Plugins - Hive Plugin Proxy ................ SKIPPED
[INFO] Plugins - Hive Plugin .............................. SKIPPED
[INFO] Plugins - Hive Functions ........................... SKIPPED
[INFO] Plugins - JDBC Plugin .............................. SKIPPED
[INFO] Tools - Attach Tool ................................ SKIPPED
[INFO] Plugins - S3 Plugin ................................ SKIPPED
[INFO] Plugins - Azure Storage Plugin ..................... SKIPPED
[INFO] Plugins - Azure Data Lake Store Plugin ............. SKIPPED
[INFO] Analyst Center - daemon ............................ SKIPPED
[INFO] Distribution - Parent .............................. SKIPPED
[INFO] Distribution - Scripts and Resources ............... SKIPPED
[INFO] Distribution - All-in-one JDBC JAR ................. SKIPPED
[INFO] Plugins - Hive 2.x Parent .......................... SKIPPED
[INFO] Storage Plugins - Hive 2.x Plugin Launcher ......... SKIPPED
[INFO] Plugins - Hive 3.x Parent .......................... SKIPPED
[INFO] Storage Plugins - Hive 3.x Plugin Launcher ......... SKIPPED
[INFO] Distribution - Server Packaging .................... SKIPPED
[INFO] Storage Plugins - Hive 2.x Plugin .................. SKIPPED
[INFO] Storage Plugins - Hive 3.x Plugin .................. SKIPPED
[INFO] Tools - Mongo Test Runner .......................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  03:49 min
[INFO] Finished at: 2020-04-13T12:29:49Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project dremio-sabot-kernel: Compilation failure
[ERROR] An unknown compilation problem occurred
[ERROR] 
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :dremio-sabot-kernel

Thanks

@eduardoslopes,

have you tried to increase the heap?

https://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError

Thanks

@balaji.ramaswamy,

I had tried increase Java heap space but I was doing it the wrong way.

Thank you. That worked.