Fetching data from a couple different Flight/FlightSQL endpoints at the same time

1

Hi,

I would like to test one scenario where
I am getting data from various different endpoints using Flight SQL.

Here is example how to get data using FlightSql

FlightInfo flightInfo; // Use a FlightSqlClient method to get a FlightInfo

// 1. Fetch each partition sequentially (though this can be done in parallel)
for (FlightEndpoint endpoint : flightInfo.getEndpoints()) {

  // 2. Get a stream of results as Arrow vectors
  try (FlightStream stream = flightSqlClient.getStream(endpoint.getTicket())) {

    // 3. Iterate through the stream until the end
    while (stream.next()) {

      // 4. Get a chunk of results (VectorSchemaRoot) and print it to the console
      VectorSchemaRoot vectorSchemaRoot = stream.getRoot();
      System.out.println(vectorSchemaRoot.contentToTSVString());
    }
  }
}

and I am talking specifically about this line:

// 1. Fetch each partition sequentially (though this can be done in parallel)
for (FlightEndpoint endpoint : flightInfo.getEndpoints()) {

I always get the same single endpoint and I would like to employ several partitions instead.
Could you please guide me what I should do in regards to cluster configuration and data source
to achieve getting many endpoints from flight info.

Thank you

2

The ability of have Dremio return multiple endpoints using Flight is not yet supported.