ELT Process Minio to Nessie

1

Hi there I am trying to ingest data from Minio Json files to Nessie.

I managed to load my data from Minio and transform it to a “@dip”.“User_Followers_new_profiles” VIEW but I am struggling to merge this data into Nessie.

My queries look like:

CREATE BRANCH IF NOT EXISTS elt IN Silver;
USE BRANCH elt IN Silver;
DROP TABLE IF EXISTS Silver.socials.new_followers_profiles AT BRANCH elt;
CREATE TABLE Silver.socials.new_followers_profiles AS SELECT * FROM "@dip"."User_Followers_new_profiles";

MERGE INTO Silver.socials.profiles AT BRANCH elt AS target
    USING Silver.socials.new_followers_profiles AT BRANCH elt AS source
    ON target."@id" = source."""@id""" AND target.owner_id = source.owner_id AND target.execution_id = source.execution_id
    WHEN MATCHED THEN
        UPDATE SET
            "@type" = source."""@type""",
            "as:name" = source."""as:name""",
            "as:url" = source."""as:url""",
            owner_id = source.owner_id,
            execution_id = source.execution_id
    WHEN NOT MATCHED THEN
        INSERT ("@type", "@id",  "as:name", "as:url", inserted_at, owner_id, execution_id)
        VALUES (source."""@type""", source."""@id""", source."""as:name""", source."""as:url""", source.inserted_at, source.owner_id, source.execution_id);

The last query fails returning following error: IllegalArgumentException: DML commit does not specify starting snapshot id