Dremio Datasets metadata

Hi everyone,
I’m quite new to dremio, so far I’ve installed it successfully on single node mode on Ubuntu 16.04. I would like to know is there any way by which I can find out all datasets loaded by users, all the columns within each dataset?
Its similar to Oracle/MS SQL Server etc RDBMSs that have system tables which store information about all the tables/columns etc in a table.

Rational is that users want to have some sort of catalogue describing all datasets, columns within each dataset. Right now we’re still evaluating Dremio, there are initial requests, I tried DBeaver, which users find too technical for them.

Please advise if anyone of you know.

Thank you.

Dremio does support INFORMATION_SCHEMA schema which allows you to query catalog (CATALOGS), schemas (SCHEMATA), tables (TABLES) and columns (COLUMNS).

You can get the list of columns with the following query for example:

SELECT * FROM INFORMATION_SCHEMA.COLUMNS

If you use a JDBC or an ODBC client, you might want consider using the native metadata functions which are slightly faster.

1 Like

Awesome, thank you very much Laurent for precise reply. Do you know whether similar information is there on Dremio official site etc, I didn’t manage to find it in documentation.

Documentation seems to be lacking on that topic. I’m opening an internal issue to improve it.

1 Like

thank you Laurent :slight_smile:

Hi @Laurent, Is there any to query reflections. I would like to be able to quickly know which tables have reflections, and which columns are included in reflections (both raw and aggregation reflections)

Thanks

SELECT * FROM sys.accelerations (to be renamed to sys.reflections in future versions)

@dfleckinger This query should give you the information you are looking for.

Yes thanks. I saw that sys.layouts and sys.accelerations are providing the informations i was expected.
Thanks

Hi Laurent, Can I understand, Once I update the table/view structure, such as new view created, new column adding, or column data type change, the metadata table INFORMATION_SCHEMA.COLUMNS will get the timely update ASAP? For example, how much time I can see the updated metadata after my structure change of my view? thanks!

@dolphinlei

Have answered your new post on the same question, lets chat there