Skip to content
This repository has been archived by the owner on Nov 28, 2023. It is now read-only.

Connector Error Workaround

Jump to bottom
Jason Dugdale edited this page Nov 13, 2015 · 4 revisions

It is possible that your connector configuration could face reader errors, writer errors or both. These errors will appear in either the /var/log/datasift/reader.log file or the /var/log/datasift/writer.log file.

Below is a sample of what you might see from the /var/log/datasift/reader.log file.

(2015,11,02,14,53,25,(276)) WARN org.apa.kaf.com.net.Selector:276 - Error in I/O with localhost/127.0.0.1 
java.net.ConnectException: Connection refused 
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) 
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) 
at org.apache.kafka.common.network.Selector.poll(Selector.java:238) 
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192) 
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191) 
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122) 
at java.lang.Thread.run(Thread.java:745) 
(2015,11,02,14,53,40,(184)) ERROR com.dat.con.GnipReader:343 - Exception sending message to Kafka: Failed to update metadata after 60000 ms. 
(2015,11,02,14,54,40,(184)) ERROR com.dat.con.GnipReader:343 - Exception sending message to Kafka: Failed to update metadata after 60000 ms.

Below is a sample of the /var/log/datasift/writer.log error.

(2015,11,02,15,24,57,(347)) DEBUG kaf.con.SimpleConsumer:52 - Disconnecting from 127.0.0.1:6667 
(2015,11,02,15,24,57,(347)) INFO com.dat.con.wri.SimpleConsumerManager:415 - Waiting on Kafka broker connection... 
(2015,11,02,15,25,00,(347)) DEBUG kaf.con.SimpleConsumer:52 - Disconnecting from 127.0.0.1:6667

These errors are caused by the hostname being incorrectly written which means that the EC2 instance is unable to resolve it.

Below is a possible workaround which has worked in previous cases

  1. Add line to /opt/kafka/config/server.properties:
host.name=127.0.0.1
  1. Restart Kafka, Reader and Writer:
supervisorctl restart kafka
supervisorctl restart gnip-reader
supervisorctl restart datasift-writer

If the above doesn't work:

  1. Add line to /etc/hosts:
127.0.0.1 ip-172-30-1-157.ec2.internal
  1. Restart Kafka, Reader and Writer:
supervisorctl restart kafka
supervisorctl restart gnip-reader
supervisorctl restart datasift-writer

Note that a vagrant provision once these modifications have been made will reset the state of either file, so may cause an issue again.

If you continue to face issues, please don't hesitate to get in contact with the support team.

Clone this wiki locally