How to load oversized data using the restful API

1

Using the restful API /apiv2/job/ to query data, when the data is too long, it does not return all the data, is there any setting to solve this problem?

My data has a field that stores WKT data, and the data is very large, so I first made the following settings:

alter system set limits.single_field_size_bytes = 104857600; // 100M

Then using the interface to load the data normally, I found that the overly long data was not loaded completely and it provided a link to tell me how to load all the data for that row for that field, but that is not what I want.

I know that JDBC can query all the data, but I would like to ask for the restful API, is there a setting to change this behavior?

Thanks!

image
image664×548 26.3 KB

2

@bigfacewo When you say data is too long, do you mean

  • Single field width is > 104857600

(OR)

  • Total number of rows is too many
3

Single field width too long.

I checked the code and it seems that only a single field string returned to the front end is allowed with a maximum length of 100. I couldn’t find a place to change this setting

4

@bigfacewo Did you already not increase?

alter system set limits.single_field_size_bytes = 104857600;
5

I used this setting, but the problem is data truncation in the REST API.

The relevant code should be in the serialize method of the JobDataFragmentSerializer class, and the maximum value of ‘MAX_CELL_SIZE_KEY’ seems to be only 100