Apache Kafka Windows Service Wrapper

We’ve started exploring different tools at work for dealing with increasing scale; focusing on using existing technology and enterprise integration patterns to build a solid foundation.

One we’ve been looking into lately is Apache Kafka; a high throughput distributed messaging system that inspired AWS Kinesis. Some of it’s benefits include:

  • Cluster management within Apache Zookeeper
  • Partitioning of messages across groups
  • Replication and fault tolerance
  • Strong ordering guarantees within a topic and partition
  • Append only commit log semantics
  • High throughput

One thing that was missing in the core project was a way to easily set up an instance on Windows for development.

To help with this we’ve written a Windows Service wrapper using TopShelf that upon starting:

  1. Downloads Kafka
  2. Adjusts the settings for a Windows environment
  3. Runs Zookeeper in a separate process
  4. Runs Kafka in a separate process

TopShelf then makes it easy to install and manage the service from the command line.

Check it out here: https://github.com/lukemerrett/Kafka-Windows-Service and feel free to contribute!