Anyone has source code of some dremio sql functions (for eample hash, hash64)?

quangbilly79 · August 14, 2025, 7:07am

Does anyone have the source code of the below SQL function hash/hash64

Need to make sure there is no Uniqueness/Collision after hashing value.

Try to find in the dremio-oss GitHub project, but no luck so far

dacopan · August 14, 2025, 2:19pm

you can find code in Hash64Functions dremio-oss/sabot/kernel/src/main/java/com/dremio/exec/expr/fn/impl/Hash64Functions.java at master · dremio/dremio-oss · GitHub

that use Murmur algorithms

quangbilly79 · August 15, 2025, 1:20am

There is this part

@Override
public void eval() {
  out.isSet = 1;
  if (in.isSet == 0) {
    out.value = 0;
  } else {
    out.value = com.dremio.common.expression.fn.impl.HashHelper.hash64(in.value, 0);
  }
}

Which call

com.dremio.common.expression.fn.impl.HashHelper.hash64

Which I think is the main logic of the function. But I can’t find it anywhere in the code?
There is no main logic in the

Nvm I found it

github.com/dremio/dremio-oss

sabot/logical/src/main/java/com/dremio/common/expression/fn/impl/HashHelper.java

master

/*
 * Copyright (C) 2017-2019 Dremio Corporation
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package com.dremio.common.expression.fn.impl;

import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import org.apache.arrow.memory.ArrowBuf;

This file has been truncated. show original

It uses the MurmurHash3, which logic stores in this file

github.com/dremio/dremio-oss

sabot/logical/src/main/java/com/dremio/common/expression/fn/impl/MurmurHash3.java

master

/*
 * Copyright (C) 2017-2019 Dremio Corporation
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package com.dremio.common.expression.fn.impl;

import io.netty.util.internal.PlatformDependent;
import org.apache.arrow.memory.ArrowBuf;

This file has been truncated. show original

I think hash64 is safe for collision at < 1 billion items

Topic		Replies	Views
Dremio SQL, need to gen UNIQUE hash value from different values	0	32	July 29, 2025
Built-in SHA256 hashing function in Dremio? Dremio University	2	2459	August 19, 2021
How to generate sequences in Dremio Dremio Cloud	1	508	January 15, 2024
Hash exceeded the memory limits	3	1790	August 11, 2021
Bit String Function in Dremio	0	1078	November 27, 2018

Anyone has source code of some dremio sql functions (for eample hash, hash64)?

Related topics