Dremio 1.4 Released!

1

Today we shipped Dremio 1.4!

This is a major release that includes many new features - see the highlights and more details below. Our products team will host a live discussion of these new features - you can register here.

  • Enhanced vectorization performance. Dremio’s in-memory query execution engine is based on columnar data structures and formats provided by Apache Arrow. We improved the Java implementation of Arrow, and incorporated these enhancements into this release. You can read more about the changes here and the performance benefits here.
  • Azure Data Lake Store Support. Azure Data Lake Store can now be used as a source and distributed storage option in Dremio.
  • Enhanced query push downs for Elasticsearch. Dremio can now push down the NOT LIKE operator into Elasticsearch. This is also supported when Boolean expressions are part of a larger complex expression. In addition, parts of a predicate conjunction that can be pushed down will be pushed down, with the remaining components handled in the Dremio execution engine.
  • Improved auto failover resiliency. Dremio now handles detecting master failures and electing a new master node more robustly.
  • Better handling of non UTF-8 characters. Dremio now supports ISUTF8() and CONVERT_FROM() functions for dealing with non UTF-8 string values. Users can programmatically validate and replace/omit invalid characters as needed.
  • Enhanced sampling for MongoDB. Dremio now uses the $sample stage for MongoDB versions 3.2 and later when previewing data from a collection.
  • Learn more in the release notes here.

You can download here.

Be sure to follow the upgrade instructions for your deployment method.

If you have questions, you know where to ask. :slight_smile:

2 Likes
5

I have had a big issue when running the RPM migration.

When executing upgrade, i have the following message :

Exception in thread “main” java.lang.RuntimeException: Failure reading configuration file. The following properties were invalid:
services.coordinator.master.enabled

although I had changed my dremio.conf file to follow instructions for 1.4 new configuration file.
My dremio.conf was exactly like in documentation link https://docs.dremio.com/deployment/dremio-config.html:

paths: {

the local path for dremio to store data.

local: “/var/lib/dremio”

the distributed path Dremio data including job results, downloads, uploads, etc

#dist: “pdfs://”${paths.local}“/pdfs”
}
services: {
coordinator.enabled: true,
coordinator.master.enabled: true,
executor.enabled: true
}

6

hi @dfleckinger,

can you confirm dremio version by running rpm -qi dremio-community and also by returning the output of the following command: ls /opt/dremio/jars/dremio-common-*.jar?

The configuration change mentioned in the doc applies to 1.4.x releases and later.

7

Careful review of the ES pushdown the relevant documents, very good, very helpful to us.

1 Like
8

Thanks for the answer. It indeed appears that sudo yum install did run partially, updating the dremio.conf but not everything.
I did run again the sudo yum install after downloading again the 1.4.4 version, and that time Dremio could start successfully.

1 Like
9

Glad it worked that time!

Enjoy the new version.