Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kafka 0.11 on debian:stretch #5

Merged
merged 31 commits into from
Jul 26, 2017
Merged

Kafka 0.11 on debian:stretch #5

merged 31 commits into from
Jul 26, 2017

Conversation

solsson
Copy link
Owner

@solsson solsson commented Jul 20, 2017

Replaces #3 and #4.

At Yolean we recently evaluated Alpine (3.6) across our services. The small base image size and impressively fast apk saves time in development, but we had to conclude we get more glitches in production (e.g. kubernetes DNS name resolution, default shell not forwarding signals) and a bit more maintenance of our dockerfiles (e.g. 3rd party build scripts not tested with Alpine).

Before I continue with the tuning in Yolean/kubernetes-kafka#30 I wanted to minimize the gambling. To do that I've decided to look at https://github.com/confluentinc/cp-docker-images as a reference setup. We still want to maintain our own Kafka image though, compatible with Docker Hub's automated build feature.

@solsson
Copy link
Owner Author

solsson commented Jul 20, 2017

Kafka docs are quite light on JVM requirements. Based on "use the latest released version of JDK 1.8" and confluentinc/cp-docker-images#21 I conclude they assume vendor is Oracle. I don't know anything about Zulu but let's do like Confluent (99f4d45).

Still deviating from Confluent's image builds in many aspects. For example they are:

  • Not using the official Kafka distribution.
  • Tested only with --net=host (judging by the docs) which we don't want in Kubernetes.
  • Encouraging config by env, which is docker-ish but we've had to conclude in for example Multizone rack awareness Yolean/kubernetes-kafka#41 that we need .properties files modified by init containers.

@solsson
Copy link
Owner Author

solsson commented Jul 20, 2017

Ouch... 99f4d45 added almost 200MB to image size. 231MB instead of 37MB for the installation layer.

This was referenced Jul 20, 2017
@solsson
Copy link
Owner Author

solsson commented Jul 23, 2017

RC build: solsson/kafka:0.11.0.0@sha256:be80af0f63b6c980d557193fec7927c3903d07ef3ab01ead6a33cff35b0440ce

TODO document

  • Upgrade from previous image versions
  • log4j.properties
  • Pointer towards how to set up clusters

solsson added a commit to Yolean/kubernetes-kafka that referenced this pull request Jul 23, 2017
@solsson
Copy link
Owner Author

solsson commented Jul 23, 2017

It would save some pull time at minor kafka upgrades if Zulu and Kafka were installed in separate layers. However due to the lack of build cache with Docker Hub automated build, the way to do this would be to maintain a Zulu image. For now let's stick with the single build strategy.

@solsson
Copy link
Owner Author

solsson commented Jul 23, 2017

@solsson
Copy link
Owner Author

solsson commented Jul 26, 2017

Merging this without a decision on #10. Let's keep Kafka as vanilla as possible for now (though with 2MB site-docs removed). At /opt/kafka that is.

@solsson solsson merged commit fdae47c into master Jul 26, 2017
@solsson
Copy link
Owner Author

solsson commented Jul 26, 2017

Built from master: solsson/kafka@sha256:75ecbf6c7fb9f814a0ab0e44c5d48d3cb4dd688a244a377f6986eefcb1498614 and solsson/kafka:0.11.0.0@sha256:e09d493ae027bbe45d82dc5b9ceaae40adad23efb2936116b233b7dce48136ba.

@solsson
Copy link
Owner Author

solsson commented Jul 26, 2017

The docs TODOs above were fixed in master, 4a7b8da.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant