Hi Laurent - I am running a Dremio sql query on an ElasticSearch index which has my application log details (pls see sample below). Basically i am extracting required details for my daily metrics from the logs. I can rename the daily index files (ES-196"."logstash-order-2018.03.04-log) which is created on a daily basis to a common name like ‘test-alias’ as described in one of your forum.
My requirement is to shcedule this query at 1:00 AM early morning. Are you saying Dremio supports Crontab ? If so how to configure it. I understand you cannot comment on the specific external tools, but how to integrate external scheduler tools to Dremio. Is there any documenation available ?
Any help on this is highly appreciated.
SELECT operation, serviceName, COUNT() AS Count_Star
FROM (
SELECT extract_pattern(log_message, '(?<=Label=)(.)(?=,LastValue)’, 0, ‘INDEX’) AS operation,extract_pattern(log_message, ‘\d+’, 0, ‘INDEX’) AS lastvalue, extract_pattern(log_message, ‘(?<=Last_Access=)(.)(?=))’, 0, ‘INDEX’) AS lastaccess, serviceName, cspUserId, serviceVersion
FROM “ES-196”.“logstash-order-2018.03.04-log”.syslog AS syslog
WHERE regexp_like(log_message, '.?\QLabel=\E.*?’) ) nested_0
GROUP BY operation , serviceName