# Kafka connector

**URL:** <https://community.dremio.com/t/kafka-connector/6689>\
**Category:** Uncategorized\
**Created:** [December 4, 2020, 7:52am UTC](https://community.dremio.com/t/kafka-connector/6689 "2020-12-04T07:52:45Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![jacek](https://avatars.discourse-cdn.com/v4/letter/j/e19adc/32.png) [@jacek](https://community.dremio.com/u/jacek)\
**Post date:** [December 4, 2020, 7:52am UTC](https://community.dremio.com/t/kafka-connector/6689/1 "2020-12-04T07:52:45Z")

</div>

Related to 2/3 years old questions:

> [@KTable connector?](https://community.dremio.com/t/ktable-connector/1124):
>
> Hello, is it planned to use Kafka KTable as source for dremio? It could be great for building an IOT plaform with Kafka and Dremio where Kafka is used as a streaming datastore and Dremio as a query engine and data governance tool. No more need of hadoop then.

> [@Is dremio suitable for big streaming data?](https://community.dremio.com/t/is-dremio-suitable-for-big-streaming-data/311):
>
> Hi all first of all congratulations for a really great piece of infrastructure. The premises are great. I was wondering if Dremio is meant/suitable for big STREAMING data, for example 1TB a day of streaming logs. would one (realistically) be able to operate and query the data as its streaming? so: are there streaming connectors? would Dremio understand when data is appended to a file on HDFS? how about if the data is e.g. in Elasticsearch and grows continuosly and switches index name every…

You mentioned there that Kafka is on your roadmap, but it is still not documented as a supported data source.  
Any news here? Target release date?  
Regards

Jacek

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [December 4, 2020, 8:04am UTC](https://community.dremio.com/t/kafka-connector/6689/2 "2020-12-04T08:04:43Z")

</div>

@jacek

For building a Kafka connector, please take a look at the below link

> **[Dremio Hub](https://www.dremio.com/hub/)**
>
> Dremio Hub provides community-supported connectors.

Do you have any questions on reading logs, elastic search?

Thanks  
Bali

---

<div class="post-metadata">

**Author:** ![jacek](https://avatars.discourse-cdn.com/v4/letter/j/e19adc/32.png) [@jacek](https://community.dremio.com/u/jacek)\
**Post date:** [December 4, 2020, 8:08am UTC](https://community.dremio.com/t/kafka-connector/6689/3 "2020-12-04T08:08:07Z")

</div>

Well, what happened that it is no longer on your roadmap?  
Why it stopped to be your priority? Any technical reasons?

Writing connector could be quite a complex task, could not be?  
I mean reading from Kafka topic could be easy, but to add full support for reflections could be quite complex, right?

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [December 4, 2020, 5:25pm UTC](https://community.dremio.com/t/kafka-connector/6689/4 "2020-12-04T17:25:27Z")

</div>

@jacek

Currently we do not have plans for an inbuilt Kafka connector

Thanks

---

<div class="post-metadata">

**Author:** ![jacek](https://avatars.discourse-cdn.com/v4/letter/j/e19adc/32.png) [@jacek](https://community.dremio.com/u/jacek)\
**Post date:** [December 4, 2020, 5:41pm UTC](https://community.dremio.com/t/kafka-connector/6689/5 "2020-12-04T17:41:58Z")

</div>

I analyzed Vertica connector and now I see that it is not too complex.  
Moreover I start considering an option to utilize ksqldb (JDBC driver available).  
Is this out of your radar as well?

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [December 5, 2020, 7:11am UTC](https://community.dremio.com/t/kafka-connector/6689/6 "2020-12-05T07:11:13Z")

</div>

@jacek

Currently it is not in the radar

Thanks  
Bali
