Some quick user feedback about dremio-oss

1

Hi,

Dremio is a very interesting project.
I gave it a quick try using a pre-installed docker and version 1.1.0-201708121825170680-436784e,
and tried it against a local dockerized Hive install.
I wanted to give you some quick feedback to help you improve the project:

Compatibility with Protobuf 2.6+ ?

I tried to compile the source code but it failed with ClassNotFoundErrors for classes from the com.google.protobuf package. I do have protobuf installed as protoc --version returns libprotoc 2.6.1.
I was wondering if it is because Dremio requires Protobuf 2.5 and not 2.6?
In that case it would be good to precise it in the readme.

Missing UDFs:

  • It would be nice to have a regexp_extract UDF similar to Hive’s.

Error messages for syntax error are not clear at all:

(line, column) isn’t enough information, especially since lines are not numbered in the editor.
For example SELECT T.time FROM table T gives this message Failure parsing the query
instead of something explaining that time is a reserved keyword.

Coloration

Since simple and double quotes don’t have the same meaning, they should not be colored with the same color.

Example:

SELECT 
  T."day", 
  T.col 
FROM table T 
WHERE "day" = '2017-01-01'

It would be better to color the "day" in green like T.col instead of coloring it in blue like strings.
I looked into it a little, and it does not seem to be that easy to do, as the CodeMirror sql mode
seems to make no difference between strings escaped with simple or double quotes.

no NULL ???

SELECT NULL FROM table gives me the error message Illegal use of "NULL"

‘Run’ shortkey

It would be nice to have something like a ‘CTRL + ENTER’ shortkey to execute or preview the query.

(Hive) No partition pruning for Preview

I tried doing a SELECT * FROM table WHERE partition = ... ,
and the ‘Preview’ told me the results were empty, while the ‘Run’ button gave me results.
It would be nice if it could actually be capable of fetching a sample from the right partitions,
when a filter on partitions is applied.

no graph feature (yet)?

I am trying Dremio version 1.1.0-201708121825170680-436784e against Hive, but I don’t see the 'Graph’
button that is mentionned in the documentation. Maybe it is only available in the premium edition?
Or maybe my docker installation has some issues…

And that’s all. I hope this feedback will be useful to you.
It is a really interesting project, keep up the good work guys!

2

Thanks for this, lots of useful feedback. I’ll respond to a few things here, and others might have additional feedback/questions.

The graph feature is part of Enterprise Edition. Let us know if that’s something you’d like to evaluate. More about the differences here: https://www.dremio.com/enterprise-edition/

As for regexp_extract, there are several functions documented here: http://docs.dremio.com/sql-reference/sql-functions/string.html

In addition, you might look at extract_pattern, eg:

extract_pattern(full_address, ‘\w+’, 0, ‘INDEX_BACKWARDS’)

There’s also regex_split, eg:

regexp_split(full_address, ‘\Q \E’, ‘FIRST’, -1)

Run shortkey has been on my wish list too. :slight_smile: Hoping to get that in soon.

3

To add to @kelly’s reply:

  • protobuf 2.5 is a hard prerequisite: it’s actually mentionned in the README.md prerequisites section
  • for error messages, we are currently tracking those messages and will come up with better ones in the next releases
  • typeless NULL is not supported. You can use CAST operator however: SELECT CAST(NULL AS ...) FROM table
4

Hello, Could you pls tell me what is the meaning of this syntax ?
regexp_split(col1, ‘\Q,\E’, ‘ALL’, 10)

Thanks