Backup Restore Util -field "SQL" too long

1

When calling the dremio-admin restore command on an existing backup during the restore process, an error occurs:

java.lang.IllegalArgumentException: Document contains at least one immense term in field=“SQL” (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: ‘[info about term]’, original message: bytes can be at most 32766 in length; got 58929

As far as I understand, this is due to a request in backup that is too long. At the same time, at first glance, recovery is proceeding normally - all existing requests are found, including the one on which the error occurred.

A search through the error text showed that something like this occurs in Lucene (Solr). My guess is that when I do restore, the utility interacts with Lucene, apparently trying to index the SQL field, but gets an error (because the token is out of range).

I have two questions:

  1. When restoring, the utility returns an error, but the data is restored (it looks like this). Does this mean that I can ignore these errors, or did something go wrong?
  2. Is there a way to work around this error? In relation to such an error, ElasticSearch has options with adding the ignore_aboved parameter, changing the field type, etc. What to do in the framework of the dremo is not clear to me.
2

@Arol

#1 If your Dremio is working as expected, then should be fine
#2 Looks like you rightly said, this is coming from elasticsearch


Will see, if there is anything we can do from the Dremio side

Thanks
Bali

3

The question of whether everything is normal, and worries me. Visually, everything is fine, a long request is restored (according to monkey testing) and works. No matter how it caused problems later.
I don’t know why the query text is saved in elastic - for system needs, for advanced text search, or the like. We can ignore this error in the log, I see how to change it (to a less terrible Warning) in the sources from the github, the question is whether it will be correct.
Thx