All Questions
Tagged with apache-zookeeper python
58
questions
0
votes
0
answers
29
views
Custom Dockerfile and docker-compose.yml file for Kafka and Zookeeper containers doesn't produce and consume messages
I'm trying to create a custom Dockerfile and docker-compose.yml files for Kafka and Zookeeper services, in order to understand how to configure it, and how those services are communicated between each ...
1
vote
0
answers
58
views
Spark job stuck at running process
I use docker-compose to make a cluster of Spark and Kafka. When I tried to consume a topic by submitting it to this cluster, the job was always stuck and not executed.
Here is my docker-compose file:
...
0
votes
1
answer
70
views
How do I connect to Accumulo Thrift Proxy?
My main goal is to use pyaccumulo, which from the description is a python client library for Apache Accumulo that uses the Accumulo Thrift Proxy.
But I can't seem to connect Accumulo Thrift Proxy ...
- 1
0
votes
1
answer
106
views
How does Kafka store messages offsets on a local computer?
How does Kafka store messages on a local server or laptop?
I'm new to Kafka and just playing around with the tech for now but I'm curious to the answer because I started by looking at the Kafka ...
- 5,950
0
votes
1
answer
256
views
Python to create zookeeper local session
I wants to know if i could create a localSession using kazoo, i see since zookeeper 3.5.0 there's localSessions use to decrease global cluster connections, and the local session creation is at ...
- 51
1
vote
0
answers
1k
views
Connect Hive with keytab using python
I am looking for an option to connect Hive with keytab using python. Tried the option Jaydebeapi to establish the connection, but no success yet
import jaydebeapi
DIR = root_path
jarFile = [ DIR + '...
- 101
2
votes
2
answers
7k
views
Python Kafka consumer with offset management
I am a newbie to Kafka and I am trying to set up a consumer in Kafka such that it reads messages published by Kafka Producer.
Correct me if I am wrong, the way I understood if Kafka consumer stores ...
- 2,293
-2
votes
1
answer
4k
views
How solve kafka.errors.UnrecognizedBrokerVersion: UnrecognizedBrokerVersion?
Create docker containers from
https://github.com/wurstmeister/kafka-docker.git
docker-compose.yml
version: '3'
services:
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181:2181"...
- 1,634
4
votes
2
answers
173
views
How to use Python regular expression to process zookeeper logfiles?
Ive got zookeeper logs like the following:
2019-09-25 11:16:39,253 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@357] - caught end of stream exception
EndOfStreamException: ...
- 43
0
votes
1
answer
232
views
Which nodes do you need to specify for bootstrap_servers (Kafka Python) and zookeeper.connect (ZooKeeper)?
The docker-compose.yml file referenced here allows for easy setup of ZooKeeper and Kafka in 3 machines.
Let's have the IP addresses of the 3 machines as 1.1.1.1, 1.1.1.2, and 1.1.1.3.
Machine 1: zk1,...
- 31
3
votes
1
answer
3k
views
Handling a producer and consumer using Kafka-python
First I would like to say that I am a newbie to Kafka and also stackoverflow, So I am sorry if I am not asking this in the right way.
I am trying to implement the producer-consumer using kafka-python.
...
- 284
0
votes
1
answer
500
views
'WARNING:: Cannot resolve hostname: [Errno -2] Name or service not known' when trying to connect Zookeeper with Pysolr
I am trying to run a docker-compose that will let me have a Zookeeper ensemble that manages my SolrCloud. Everything runs and from every way I've checked inside the container, my Zookeeper ensemble ...
1
vote
0
answers
640
views
Register services with kazoo python and use for service discovery
I am trying to register some services and use it as a service discovery using Apache Zookeeper and implementing it in python using the Kazoo library. Having a hard time to figure out how to trigger ...
- 883
0
votes
1
answer
238
views
How to invalidate a resource released via a context manager in python?
My question relates to Kazoo/Zookeeper, but more generally it's about allocating a resource and then letting the caller of that resource no they should pause. From Kazoo docs:
It’s highly ...
- 4,046
1
vote
0
answers
956
views
How to fix the zookeeper timeout issue in spark streaming job
I am trying to do some analysis on twitter streaming data. The tweets are consumed successfully through kafkaconsumer. Now I am trying to stream those tweets through spark streaming job to do some ...
- 15