Dataverse for Apache Kafka®
What is Dataverse for Apache Kafka®?
Dataverse for Apache Kafka® is an open-source distributed event streaming platform for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications, deployable in the cloud of your choice, which can bring unlimited scalability and high-availability to your environment and other applications.
Why Apache Kafka?
CORE CAPABILITIES
HIGH THROUGHPUT
Deliver messages at network limited throughput using a cluster of machines with latencies as low as 2ms.
SCALABLE
Scale production clusters up to a thousand brokers, trillions of messages per day, petabytes of data, hundreds of thousands of partitions. Elastically expand and contract storage and processing.
PERMANENT STORAGE
Store streams of data safely in a distributed, durable, fault-tolerant cluster.
HIGH AVAILABILITY
Stretch clusters efficiently over availability zones or connect separate clusters across geographic regions.
ECOSYSTEM
BUILT-IN STREAM PROCESSING
Process streams of events with joins, aggregations, filters, transformations, and more, using event-time and exactly-once processing.
CONNECT TO ALMOST ANYTHING
Kafka’s out-of-the-box Connect interface integrates with hundreds of event sources and event sinks including Postgres, JMS, Elasticsearch, AWS S3, and more.
CLIENT LIBRARIES
Read, write, and process streams of events in a vast array of programming languages.
LARGE ECOSYSTEM OPEN SOURCE TOOLS
Large ecosystem of open source tools: Leverage a vast array of community-driven tooling.
TRUST & EASE OF USE
MISSION CRITICAL
Support mission-critical use cases with guaranteed ordering, zero message loss, and efficient exactly-once processing.
TRUSTED BY THOUSANDS OF ORGS
Thousands of organizations use Kafka, from internet giants to car manufacturers to stock exchanges. More than 5 million unique lifetime downloads.
VAST USER COMMUNITY
Kafka is one of the five most active projects of the Apache Software Foundation, with hundreds of meetups around the world.
RICH ONLINE RESOURCES
Rich documentation, online training, guided tutorials, videos, sample projects, Stack Overflow, etc.
Integrates with other Dataverse building blocks
Apache Kafka is highly compatible with other Dataverse blocks.
