When I clone dremio repository and simply try to build it:
mvn install
it fails. I run the tests on a Mac OX machine. Is it supposed to work?
Module
[INFO] Services - Namespace … FAILURE [ 0.986 s]
java -version
openjdk version “1.8.0_222”
mvn -version
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Separately tried to install “Services - Namespace”, seemed like it failed because of the tools package. Switched to the tools package and installed it, after that “services - namespace” went throgh. Then “sabot-kernel” failed with:
[ERROR] Errors:
[ERROR] TestNewDateFunctions.testLocalTimestamp:184 » After matching 0 records, did n…
Test set: com.dremio.exec.store.iceberg.TestIcebergSchemaEvolution
Tests run: 10, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 68.147 s <<< FAILURE! - in com.dremio.exec.store.iceberg.TestIcebergSchemaEvolution
testColumnRenamePrimitive(com.dremio.exec.store.iceberg.TestIcebergSchemaEvolution) Time elapsed: 3.965 s <<< ERROR!
java.lang.Exception:.
After matching 0 records, did not find expected record in result set:
col1 : true, col2 : 1, col3 : 1, col4 : 1.0, col5 : 1.0, col6 : 1.000, col7 : 2019-12-25T08:00:00.000, col8 : 1970-01-01T20:00:00.000, col9 : 2019-12-25T20:00:00.000, col10 : abc,.
So, I wonder, what am I doing wrong.
This build itself seem to be unstable.
Kirill
so, I read documentation saying that Windows and Max OS is not supported and tried to build the software and run tests on a linux box:
/opt/jdk1.8.0_251/bin/java -version
java version “1.8.0_251”
Java™ SE Runtime Environment (build 1.8.0_251-b08)
Java HotSpot™ 64-Bit Server VM (build 25.251-b08, mixed mode)
mvn -version
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /Users/kpetrov/opt/apache-maven-3.6.3
Java version: 1.8.0_251, vendor: Oracle Corporation,
Default locale: en_US, platform encoding: UTF-8
OS name: “linux”, version: “4.1.12-124.20.7.el6uek.x86_64”, arch: “amd64”, family: “unix”
cleaned the mvn repository etc
Nevertheless, unit tests kept failing in sabot.
[INFO] Sabot - Kernel … FAILURE [14:29 min]
So, I still have two questions:
- why do unit tests fail? isn’t it a release?
- I noticed in the documentation it says that windows/mac osx are not supported for installation. Is it the case just for installation, right? Development/building can still be done safely in Mac OS?
thank you,
Kirill
You need to run the mvn command from the Dremio root and build in proper order, sounds like you are building modules adhoc?
You can develop and run in MacOS just fine, we just don’t provide windows/mac installers anymore.
You can also build without tests (https://github.com/dremio/dremio-oss#c-build-the-code) to speed up the build.
I did not do any “out of order” building/testing of the project. Did “mvn install”, if the project failed in a particular module, I tried to run “test” in that particular module.
I understand that I can build without tests but this is not the point, if I am to do the development, I want to know that I start from a stable state. On a mac machine, the tests consistently fail in the sabot kernel module.
[INFO] Sabot - Kernel … FAILURE [14:29 min]
If Macs are not used by the core developers of dremio, could it be the case that the build is indeed broken for Mac OS. would it be better to do the development on a linux machine then?
thank you,
Kirill
It looks like the OSS code is indeed failing on a test, we will look at it. For now you can disable that test.
looks like at least 4-5 of them are unstable, those that fail consistently (both under Mac OS and RHEL) are:
sabot/kernel/src/test/java/com/dremio/exec/fn/impl/TestNewDateFunctions.java
sabot/kernel/src/test/java/com/dremio/exec/store/iceberg/TestIcebergSchemaEvolution.java
provision/yarn/yarntwill/src/test/java/com/dremio/provision/yarn/TestYarnWatchdog.java