Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

came across error from kafka console #2

Open
xiaods opened this issue Nov 6, 2017 · 7 comments
Open

came across error from kafka console #2

xiaods opened this issue Nov 6, 2017 · 7 comments

Comments

@xiaods
Copy link

xiaods commented Nov 6, 2017

i follow your use case to testing the kafka on openshift,
when i create topic on this testing kafka, it report error:

bash-4.2$ bin/kafka-console-producer.sh --broker-list apache-kafka:9092 --topic test
1
[2017-11-06 17:01:13,401] WARN Error while fetching metadata with correlation id 0 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:13,499] WARN Error while fetching metadata with correlation id 1 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:13,607] WARN Error while fetching metadata with correlation id 2 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:13,712] WARN Error while fetching metadata with correlation id 3 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:13,817] WARN Error while fetching metadata with correlation id 4 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:13,923] WARN Error while fetching metadata with correlation id 5 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:14,031] WARN Error while fetching metadata with correlation id 6 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:14,136] WARN Error while fetching metadata with correlation id 7 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:14,242] WARN Error while fetching metadata with correlation id 8 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:14,349] WARN Error while fetching metadata with correlation id 9 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:14,455] WARN Error while fetching metadata with correlation id 10 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:14,561] WARN Error while fetching metadata with correlation id 11 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:14,668] WARN Error while fetching metadata with correlation id 12 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:14,773] WARN Error while fetching metadata with correlation id 13 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:14,879] WARN Error while fetching metadata with correlation id 14 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:14,986] WARN Error while fetching metadata with correlation id 15 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:15,090] WARN Error while fetching metadata with correlation id 16 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:15,196] WARN Error while fetching metadata with correlation id 17 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:15,304] WARN Error while fetching metadata with correlation id 18 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:15,411] WARN Error while fetching metadata with correlation id 19 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:15,519] WARN Error while fetching metadata with correlation id 20 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2017-11-06 17:01:15,625] WARN Error while fetching metadata with correlation id 21 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
^Cbash-4.2$
@xiaods
Copy link
Author

xiaods commented Nov 6, 2017

any clues?

@mattf
Copy link
Owner

mattf commented Nov 6, 2017 via email

@jkremser
Copy link
Contributor

I see the same issue on 3.6. I can create and retrieve a topic, but kafka-console-producer.sh fails the same way

@mattf
Copy link
Owner

mattf commented Nov 20, 2017

how did you create your cluster?
are there any pods waiting to start? possible lack of resources?

@jkremser
Copy link
Contributor

how did you create your cluster?

oc cluster up, but I'll try also w/ minishift tomorrow

oc get all
NAME              REVISION   DESIRED   CURRENT   TRIGGERED BY
dc/apache-kafka   1          1         1         config
dc/kafka-debug    1          1         1         config

NAME                DESIRED   CURRENT   READY     AGE
rc/apache-kafka-1   1         1         1         21m
rc/kafka-debug-1    1         1         1         21m

NAME               CLUSTER-IP     EXTERNAL-IP   PORT(S)             AGE
svc/apache-kafka   172.30.82.94   <none>        9092/TCP,2181/TCP   21m

NAME                      READY     STATUS    RESTARTS   AGE
po/apache-kafka-1-9l2mh   2/2       Running   0          21m
po/kafka-debug-1-fsrsq    1/1       Running   0          21m

kafka log:
https://pastebin.com/NYWQC7mx

zookeeper log:
https://pastebin.com/jyfddxFs

btw. this https://github.com/debianmaster/openshift-kafka worked for me. Couldn't this be the difference?

@jkremser
Copy link
Contributor

he also overrides the broker.id here https://github.com/debianmaster/openshift-kafka/blob/master/50kafka.yml#L26

@jjylik
Copy link

jjylik commented Aug 16, 2018 •
edited
Loading

The problem is that advertised.host.name is not the same as the pod IP

To fix this, one option is to set the service as headless.

   kind: Service
   metadata:
     name: ${NAME}
   spec:
     clusterIP: None
     ports:
     - name: kafka
       port: 9092
     - name: zookeeper
       port: 2181
     selector:
       deploymentconfig: ${NAME}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@xiaods @mattf @jkremser @jjylik