Upgrade Dremio 11 Community Edtion to Dremio 11 AWS Edition

3

Removed arrow flight jar and occured other error:

2020-12-12 13:34:27,909 [main] DEBUG c.d.common.scanner.ClassPathScanner - - collected resource’s classpath root URL jar:file:/opt/dremio/jars/dremio-services-jobs-11.0.0-202011171636110752-16ab953d.jar!/.
2020-12-12 13:34:27,909 [main] DEBUG c.d.common.scanner.ClassPathScanner - - collected resource’s classpath root URL jar:file:/opt/dremio/jars/dremio-services-datastore-11.0.0-202011171636110752-16ab953d.jar!/.
2020-12-12 13:34:27,909 [main] DEBUG c.d.common.scanner.ClassPathScanner - - collected resource’s classpath root URL jar:file:/opt/dremio/jars/dremio-services-tokens-11.0.0-202011171636110752-16ab953d.jar!/.
2020-12-12 13:34:27,909 [main] DEBUG c.d.common.scanner.ClassPathScanner - - collected resource’s classpath root URL jar:file:/opt/dremio/jars/dremio-sabot-kernel-11.0.0-202011171636110752-16ab953d.jar!/.
2020-12-12 13:34:27,909 [main] DEBUG c.d.common.scanner.ClassPathScanner - - collected resource’s classpath root URL jar:file:/opt/dremio/jars/dremio-sabot-logical-11.0.0-202011171636110752-16ab953d.jar!/.
2020-12-12 13:34:27,909 [main] DEBUG c.d.common.scanner.ClassPathScanner - - collected resource’s classpath root URL jar:file:/opt/dremio/jars/dremio-services-accelerator-11.0.0-202011171636110752-16ab953d.jar!/.
2020-12-12 13:34:27,910 [main] DEBUG c.d.common.scanner.ClassPathScanner - - collected resource’s classpath root URL jar:file:/opt/dremio/jars/dremio-services-jobtelemetry-server-11.0.0-202011171636110752-16ab953d.jar!/.
2020-12-12 13:34:28,459 [main] INFO c.d.common.scanner.ClassPathScanner - Scanning packages [com.dremio.activation, com.dremio.service.namespace, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.server.options, com.dremio.dac.resource, com.dremio.dac.resource.AwsShutdownProjectResource, com.dremio.service.roles, com.dremio.aws.provision, com.dremio.dac, com.dremio.dac.support.SupportService, com.dremio.dac.support.BasicSupportService, com.dremio.activation, com.dremio.service.usergroup, com.dremio.plugins.azure, com.dremio.service.cachemanager, com.dremio.dac, com.dremio.dac.support.SupportService, com.dremio.dac.cmd, com.dremio.dac.cmd.upgrade, com.dremio.exec.store.jdbc, com.dremio.plugins.awsglue.store, com.dremio.plugins.mongo, com.dremio.exec.store.jdbc, com.dremio.dac, com.dremio.sabot.rpc.user, com.dremio.joust.geo, com.dremio.exec.store.jdbc, com.dremio.extra.exec.store.dfs, com.dremio.exec.planner.acceleration.substitution, com.dremio.sabot.task.slicing.SlicingTaskPool, com.dremio.exec.store.dfs, com.dremio.exec.planner.serializer, com.dremio.extras.exec.store.hive, com.dremio.extras.exec.store.hive, com.dremio.service.reflection, com.dremio.service.execselector, com.dremio.extusr, com.dremio.rulesservice, com.dremio.rulesservice.resource, com.dremio.service.coordinator, com.dremio.resource.wlm.scheduler, com.dremio.resource.wlm.requester, com.dremio.resource.wlm.scheduler.common.WLMConstants, com.dremio.resource.wlm.queuemanager, com.dremio.resource.wlm.apiresource, com.dremio.exec.expr.fn.impl.wlm, com.dremio.service.jobs, com.dremio.exec.store.dfs, com.dremio.exec.store.hive, com.dremio.exec.store.hive.exec, com.dremio.datastore, com.dremio.exec.store.hive, com.dremio.service.execselector.ExecutorSelectionService, com.dremio.exec.store.hive, com.dremio.service.namespace, com.dremio.plugins.mongo, com.dremio.options, com.dremio.exec.store.dfs, com.dremio.exec.store, com.dremio.provision, com.dremio.telemetry.api, com.dremio.plugins.s3.store, com.dremio.service.tokens, com.dremio.exec.ExecConstants, com.dremio.exec.catalog, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.sabot.op.aggregate.vectorized, com.dremio.sabot.rpc.user, com.dremio.service.users, com.dremio.services.configuration,com.dremio.services.configuration.ConfigurationStore, com.dremio.service.accelerator, com.dremio.service.reflection, com.dremio.service.voting, com.dremio.service.jobtelemetry.server.store, com.dremio.resource,com.dremio.resource.basic, com.dremio.telemetry.utils, com.dremio.provision.yarn.service] in locations [jar:file:/opt/dremio/jars/dremio-ee-hive-plugin-common-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ee-services-activation-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ee-services-namespace-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-aws-dac-backend-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-aws-dac-daemon-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ee-services-roles-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-aws-provision-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-aws-services-license-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ee-services-usergroups-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-azure-storage-plugin-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ce-services-cachemanager-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ce-hive2-plugin-launcher-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-client-base-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ce-hive3-plugin-launcher-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-dac-daemon-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ce-jdbc-plugin-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ee-awsglue-plugin-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ce-mongo-plugin-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ee-dac-tools-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ce-sabot-joust-java-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ee-jdbc-plugin-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ce-sabot-kernel-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ce-sabot-serializer-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ee-hive3-plugin-launcher-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ee-hive2-plugin-launcher-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ee-services-accelerator-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ee-services-execselector-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ee-services-external-users-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ee-services-rulesservice-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-services-coordinator-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ee-services-wlm-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-ee-services-wlmfunctions-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-hdfs-plugin-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-hive-plugin-common-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-hive2-plugin-launcher-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-services-execselector-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-hive3-plugin-launcher-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-services-namespace-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-mongo-plugin-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-services-options-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-nas-plugin-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-pdfs-plugin-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-provision-common-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-services-telemetry-api-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-s3-plugin-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-services-users-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-services-configuration-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-services-resourcescheduler-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-services-telemetry-utils-11.0.0-202011171636110752-16ab953d.jar!/, jar:file:/opt/dremio/jars/dremio-yarn-11.0.0-202011171636110752-16ab953d.jar!/] took 547ms
2020-12-12 13:34:28,550 [main] INFO c.d.datastore.LocalKVStoreProvider - Starting LocalKVStoreProvider
2020-12-12 13:34:28,643 [main] DEBUG c.dremio.datastore.ByteStoreManager - WAL settings: size: ‘0 MB’, TTL: ‘300’ seconds
2020-12-12 13:34:30,511 [main] DEBUG c.d.datastore.utility.StoreLoader - Loaded the following StoreCreationFunctions: [class com.dremio.service.reflection.voting.store.ReflectionVotesStore$VotesStoreCreator, class com.dremio.activation.ActivationService$ActivationStoreCreator, class com.dremio.service.reflection.store.ReflectionSettingsStore$StoreCreator, class com.dremio.service.reflection.store.RefreshRequestsStore$StoreCreator, class com.dremio.service.roles.LocalRolesService$RolesStoreBuilder, class com.dremio.service.reflection.store.MaterializationStore$RefreshStoreCreator, class com.dremio.service.namespace.NamespaceServiceImpl$NamespaceStoreCreator, class com.dremio.exec.server.options.SystemOptionManager$OptionStoreCreator, class com.dremio.exec.catalog.CatalogSourceDataCreator, class com.dremio.service.reflection.store.MaterializationStore$MaterializationStoreCreator, class com.dremio.service.reflection.store.DependenciesStore$StoreCreator, class com.dremio.service.reflection.store.ExternalReflectionStore$StoreCreator, class com.dremio.service.reflection.voting.store.ReflectionVotesStore$DatasetStoreCreator, class com.dremio.services.configuration.ConfigurationStore$ConfigurationStoreCreator, class com.dremio.service.reflection.store.ReflectionGoalsStore$StoreCreator, class com.dremio.dac.service.collaboration.CollaborationWikiStore$CollaborationWikiStoreStoreCreator, class com.dremio.resource.wlm.queuemanager.QueueConfigurationServiceImpl$QueueConfigContainerCreator, class com.dremio.service.usergroup.SimpleAccessTokenService$AccessTokenStoreBuilder, class com.dremio.service.execselector.NodeBlacklistStore$StoreCreator, class com.dremio.service.jobs.LocalJobsService$JobsStoreCreator, class com.dremio.provision.service.ProvisioningServiceImpl$ProvisioningStoreCreator, class com.dremio.dac.service.collaboration.CollaborationTagStore$CollaborationTagsStoreCreator, class com.dremio.dac.support.UpgradeStore$UpgradeTaskStoreCreator, class com.dremio.rulesservice.RulesManagerServiceImpl$RulesConfigurationStoreCreator, class com.dremio.service.namespace.NamespaceServiceImpl$PartitionChunkCreator, class com.dremio.exec.server.options.SystemOptionManager$LegacyProtoOptionStoreCreator, class com.dremio.dac.support.BasicSupportService$OldSupportStoreCreator, class com.dremio.service.users.SimpleUserService$UserGroupStoreBuilder, class com.dremio.exec.server.options.SystemOptionManager$LegacyJacksonOptionStoreCreator, class com.dremio.service.jobtelemetry.server.store.LocalProfileStore$KVProfileStoreCreator, class com.dremio.dac.service.datasets.DatasetVersionMutator$VersionStoreCreator, class com.dremio.resource.wlm.queuemanager.QueueConfigurationServiceImpl$QueueConfigurationStoreCreator, class com.dremio.service.execselector.NodeCollectionStore$StoreCreator, class com.dremio.service.reflection.store.ReflectionEntriesStore$StoreCreator, class com.dremio.service.tokens.TokenStoreCreator, class com.dremio.service.namespace.NamespaceServiceImpl$MultiSplitStoreCreator].
2020-12-12 13:34:30,531 [main] INFO c.d.d.a.LegacyKVStoreProviderAdapter - Starting LegacyKVStoreProviderAdapter.
2020-12-12 13:34:30,531 [main] INFO c.d.datastore.LocalKVStoreProvider - LocalKVStoreProvider is up

2020-12-12 13:34:30,671 [main] INFO c.d.datastore.LocalKVStoreProvider - Stopping LocalKVStoreProvider
2020-12-12 13:34:30,788 [main] INFO c.d.datastore.LocalKVStoreProvider - Stopped LocalKVStoreProvider
2020-12-12 13:34:31,133 [main] INFO com.dremio.dac.daemon.DACDaemon - Dremio daemon write path: /var/lib/dremio
2020-12-12 13:34:31,134 [main] INFO com.dremio.dac.daemon.DACDaemon - This node is the master node, ip-###-###-###-###.ec2.internal. This node acts as a coordinator.
2020-12-12 13:34:31,303 [main] WARN com.dremio.telemetry.api.Telemetry - Failure reading telemetry configuration. Leaving telemetry as is.
java.lang.IllegalArgumentException: resource dremio-telemetry.yaml not found.
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:217)
at com.google.common.io.Resources.getResource(Resources.java:195)
at com.dremio.telemetry.api.Telemetry$1.get(Telemetry.java:90)
at com.dremio.telemetry.api.Telemetry$1.get(Telemetry.java:80)
at com.dremio.telemetry.api.config.AutoRefreshConfigurator.refreshOnce(AutoRefreshConfigurator.java:74)
at com.dremio.telemetry.api.config.AutoRefreshConfigurator.(AutoRefreshConfigurator.java:54)
at com.dremio.telemetry.api.config.AutoRefreshConfigurator.(AutoRefreshConfigurator.java:44)
at com.dremio.telemetry.api.Telemetry.startTelemetry(Telemetry.java:105)
at com.dremio.exec.server.BootStrapContext.(BootStrapContext.java:62)
at com.dremio.dac.daemon.DACDaemonModule.bootstrap(DACDaemonModule.java:247)
at com.dremio.dac.daemon.DACDaemon.(DACDaemon.java:184)
at com.dremio.dac.daemon.DACDaemon.newDremioDaemon(DACDaemon.java:312)
at com.dremio.dac.daemon.DACDaemon.newDremioDaemon(DACDaemon.java:320)
at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:145)

2020-12-12 13:34:31,607 [main] DEBUG c.d.c.l.data.LogicalOperatorBase - Found 16 logical operator classes: [class com.dremio.common.logical.data.GroupingAggregate, class com.dremio.common.logical.data.Flatten, class com.dremio.common.logical.data.Writer, class com.dremio.common.logical.data.Limit, class com.dremio.common.logical.data.Store, class com.dremio.common.logical.data.Join, class com.dremio.common.logical.data.Window, class com.dremio.common.logical.data.Union, class com.dremio.common.logical.data.Order, class com.dremio.common.logical.data.Filter, class com.dremio.common.logical.data.Transform, class com.dremio.common.logical.data.Sequence, class com.dremio.common.logical.data.Scan, class com.dremio.common.logical.data.RunningAggregate, class com.dremio.common.logical.data.Project, class com.dremio.common.logical.data.Values].
2020-12-12 13:34:31,619 [main] DEBUG c.d.c.l.StoragePluginConfigBase - Found 2 logical operator classes: [class com.dremio.plugins.mongo.MongoStoragePluginConfig, class com.dremio.exec.store.dfs.FileSystemConfig].
2020-12-12 13:34:31,623 [main] DEBUG c.d.c.logical.FormatPluginConfigBase - Found 7 format plugin configuration classes:
com.dremio.exec.store.dfs.NamedFormatPluginConfig
com.dremio.exec.store.easy.text.TextFormatPlugin$TextFormatConfig
com.dremio.exec.store.easy.arrow.ArrowFormatPluginConfig
com.dremio.exec.store.parquet.ParquetFormatConfig
com.dremio.exec.store.easy.json.JSONFormatPlugin$JSONFormatConfig
com.dremio.exec.store.easy.excel.ExcelFormatPluginConfig
com.dremio.exec.store.iceberg.IcebergFormatConfig

2020-12-12 13:34:33,455 [main] INFO c.d.d.d.EnterpriseDACDaemonModule - Internal user/group service is configured.
2020-12-12 13:34:34,100 [main] INFO c.d.s.commandpool.CommandPoolFactory - Starting bound command pool of size 7
2020-12-12 13:34:34,490 [main] ERROR ROOT - Dremio is exiting. Failure while starting services.
java.lang.NoClassDefFoundError: com/dremio/service/flight/DremioFlightService
at com.dremio.dac.daemon.DACDaemonModule.build(DACDaemonModule.java:1057)
at com.dremio.dac.daemon.EnterpriseDACDaemonModule.build(EnterpriseDACDaemonModule.java:201)
at com.dremio.dac.daemon.AwsDACDaemonModule.build(AwsDACDaemonModule.java:75)
at com.dremio.dac.daemon.DACDaemon.(DACDaemon.java:185)
at com.dremio.dac.daemon.DACDaemon.newDremioDaemon(DACDaemon.java:312)
at com.dremio.dac.daemon.DACDaemon.newDremioDaemon(DACDaemon.java:320)
at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:145)
Caused by: java.lang.ClassNotFoundException: com.dremio.service.flight.DremioFlightService
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
… 7 common frames omitted