Camel RabbitMQ consumer: what's the interaction between concurrentConsumers and threadPoolSize options?

Question

Camel RabbitMQ component allows setting both the option concurrentConsumers and threadPoolSize. Their description and defaults is as follows:

concurrentConsumers - default 1 - Number of concurrent consumers when consuming from broker. (eg similar as to the same option for the JMS component).

threadPoolSize - default 10 - The consumer uses a Thread Pool Executor with a fixed number of threads. This setting allows you to set that number of threads.

Could someone explain how these two will interact, particularly from the performance standpoint?

In particular, going into nuance a bit:

1. Are they roughly interchangeable? I.e. is it roughly true that 2 consumers, 5 threads ~ 5 consumers, 2 threads?
2. Does each concurrent consumer get as many threads as specified in threadPoolSize or are these threads shared between all concurrent consumers?

Many thanks!

Answer 1

I´ve made some tests on my machine and this is what I got (I did not go through the docs so I may be wrong):

1. I noticed that the number of consumers is the number of "listeners" that you will have attached to your queue (they can receive the message but delegate the processing to the worker threads). The number of threads is the number of workers that will actually process the message.

Test 1: With 1 thread, 10 consumers and 10 messages, you will have 10 messages simultaneously "delivered" (but not acked) with 1 being processed at a time by the single worker thread.

Test 2: With 10 threads, 1 consumer and 10 messages, you will also have 1 message being processed at a time but while a message is being processed the others are available in the queue (only 1 at a time is delivered), so if another listener attaches, it will be able to process the remaining messages (not the case in the first example).

2. I think the threads are shared because if this is not the case, on Test 1 the 10 messages would be consumed in parallel (1 thread for each consumer totalling 10 thread and not only one) wich is not what happenned.

I hope this helps!

Answer 2

Here:

• concurrentConsumers is the number of competing native RabbitMQ consumers that Camel will create for you. Camel invokes [channel.basicConsume()](https://www.rabbitmq.com/releases/rabbitmq-java-client/v2.7.1/rabbitmq-java-client-javadoc-2.7.1/com/rabbitmq/client/Channel.html#basicConsume(java.lang.String, boolean, com.rabbitmq.client.Consumer)) to create each consumer.

• threadPoolSize is the number of threads in the ExecutorService that RabbitMQ requires when creating a Connection from the ConnectionFactory. Camel will create this threadpool for you.

The RabbitMQ component uses a single RabbitMQ Connection per Camel (not RabbitMQ) consumer. This means that if you have 5 camel-rabbitmq consumers, Camel will open 5 connections, each with a threadpool of size threadPoolSize, regardless of the concurrentConsumers parameter.