Agents Continuously Go Offline

[pwnengine]

I installed mythic c2 on two different fresh Ubuntu installs and continue to have an issue of agents (apollo, poseidon, kharon, etc) going offline. From logs it seems to be an issue of rabbitmq refusing a connection. I have search the web and haven't found a fix. I've seen some issues some what related and have tried the solutions, lowering cpu in .env as an example. Restarting keeps them up for a minute or so, but it continues to go offline/online. The status command shows all container up and healthy...

The testing system is 2cpu/4GB memory.

[its-a-feature]

Interesting. Can you share the output of sudo ./mythic-cli logs mythic_rabbitmq and the same for one of the agent containers that's going offline?

[pwnengine]

[+] updated config, echoing it out
mqtt.allow_anonymous = false
log.file = false
log.default.level = error
log.exchange = false
log.connection.level = error
log.channel.level = critical
log.upgrade.level = none
log.federation.level = none
log.mirroring.level = none
log.queue.level = critical

default_user = mythic_user
default_pass = password
default_vhost = mythic_vhost
listeners.tcp.default = 5672
2025-09-03 04:11:52.951111+00:00 [notice] <0.44.0> Application syslog exited with reason: stopped
2025-09-03 04:11:53.048221+00:00 [notice] <0.258.0> Logging: switching to configured handler(s); following messages may not be visible in this log output

## RabbitMQ 3.13.7

########## Copyright (c) 2007-2024 Broadcom Inc and/or its subsidiaries

########## Licensed under the MPL 2.0. Website: https://rabbitmq.com

Erlang: 26.2.5.14 [jit]
TLS Library: OpenSSL - OpenSSL 3.1.8 11 Feb 2025
Release series support status: see https://www.rabbitmq.com/release-information

Doc guides: https://www.rabbitmq.com/docs
Support: https://www.rabbitmq.com/docs/contact
Tutorials: https://www.rabbitmq.com/tutorials
Monitoring: https://www.rabbitmq.com/docs/monitoring
Upgrading: https://www.rabbitmq.com/docs/upgrade

Logs:

Config file(s): /etc/rabbitmq/rabbitmq.conf
/etc/rabbitmq/conf.d/10-defaults.conf

Starting broker... completed with 5 plugins.
2025-09-03 04:49:33.790572+00:00 [error] <0.697.0> closing AMQP connection <0.697.0> (172.18.0.6:33720 -> 172.18.0.5:5672):
2025-09-03 04:49:33.790572+00:00 [error] <0.697.0> {writer,send_failed,closed}
2025-09-03 05:57:20.554725+00:00 [error] <0.9868.0> closing AMQP connection <0.9868.0> (172.18.0.6:34990 -> 172.18.0.5:5672):
2025-09-03 05:57:20.554725+00:00 [error] <0.9868.0> {writer,send_failed,closed}
2025-09-03 07:45:51.165154+00:00 [error] <0.52333.0> closing AMQP connection <0.52333.0> (130.131.162.246:60322 -> 172.18.0.5:5672):
2025-09-03 07:45:51.165154+00:00 [error] <0.52333.0> {bad_header,<<"MGLNDD_1">>}
2025-09-03 12:20:21.354701+00:00 [error] <0.1635.0> closing AMQP connection <0.1635.0> (172.18.0.1:58486 -> 172.18.0.5:5672):
2025-09-03 12:20:21.354701+00:00 [error] <0.1635.0> missed heartbeats from client, timeout: 30s
2025-09-03 15:38:50.982561+00:00 [error] <0.156503.0> closing AMQP connection <0.156503.0> (195.184.76.237:34853 -> 172.18.0.5:5672):
2025-09-03 15:38:50.982561+00:00 [error] <0.156503.0> {bad_header,<<22,3,3,1,166,1,0,1>>}
2025-09-03 15:38:54.347638+00:00 [error] <0.156508.0> closing AMQP connection <0.156508.0> (195.184.76.234:57177 -> 172.18.0.5:5672):
2025-09-03 15:38:54.347638+00:00 [error] <0.156508.0> {bad_header,<<"GET / HT">>}
2025-09-03 22:48:23.529266+00:00 [error] <0.257260.0> closing AMQP connection <0.257260.0> (3.131.215.38:37764 -> 172.18.0.5:5672):
2025-09-03 22:48:23.529266+00:00 [error] <0.257260.0> {bad_header,<<"GET / HT">>}
2025-09-03 22:49:26.265086+00:00 [error] <0.257598.0> closing AMQP connection <0.257598.0> (3.131.215.38:49878 -> 172.18.0.5:5672):
2025-09-03 22:49:26.265086+00:00 [error] <0.257598.0> {bad_header,<<"GET / HT">>}
2025-09-03 22:51:13.410658+00:00 [error] <0.258042.0> closing AMQP connection <0.258042.0> (3.131.215.38:40296 -> 172.18.0.5:5672):
2025-09-03 22:51:13.410658+00:00 [error] <0.258042.0> {bad_header,<<22,3,1,0,123,1,0,0>>}
2025-09-03 22:52:48.544366+00:00 [error] <0.258372.0> closing AMQP connection <0.258372.0> (3.131.215.38:40916 -> 172.18.0.5:5672):
2025-09-03 22:52:48.544366+00:00 [error] <0.258372.0> {bad_header,<<"SSH-2.0-">>}
2025-09-03 22:54:36.430816+00:00 [error] <0.258765.0> closing AMQP connection <0.258765.0> (3.131.215.38:44784 -> 172.18.0.5:5672):
2025-09-03 22:54:36.430816+00:00 [error] <0.258765.0> {bad_header,<<22,3,1,0,123,1,0,0>>}
2025-09-04 00:15:53.944114+00:00 [error] <0.1618.0> closing AMQP connection <0.1618.0> (172.18.0.1:58472 -> 172.18.0.5:5672):
2025-09-04 00:15:53.944114+00:00 [error] <0.1618.0> missed heartbeats from client, timeout: 30s
2025-09-04 04:22:10.950747+00:00 [error] <0.336492.0> closing AMQP connection <0.336492.0> (172.18.0.6:40788 -> 172.18.0.5:5672):
2025-09-04 04:22:10.950747+00:00 [error] <0.336492.0> {handshake_timeout,frame_header}
2025-09-04 04:47:55.140578+00:00 [error] <0.112299.0> closing AMQP connection <0.112299.0> (172.18.0.1:52498 -> 172.18.0.5:5672):
2025-09-04 04:47:55.140578+00:00 [error] <0.112299.0> missed heartbeats from client, timeout: 30s
2025-09-04 04:48:35.472512+00:00 [error] <0.343528.0> closing AMQP connection <0.343528.0> (172.18.0.1:37258 -> 172.18.0.5:5672):
2025-09-04 04:48:35.472512+00:00 [error] <0.343528.0> {handshake_timeout,frame_header}
2025-09-04 07:43:01.683562+00:00 [error] <0.384885.0> closing AMQP connection <0.384885.0> (115.231.78.11:30000 -> 172.18.0.5:5672):
2025-09-04 07:43:01.683562+00:00 [error] <0.384885.0> {handshake_timeout,frame_header}
2025-09-04 08:39:32.766475+00:00 [error] <0.398463.0> closing AMQP connection <0.398463.0> (206.168.34.65:38812 -> 172.18.0.5:5672):
2025-09-04 08:39:32.766475+00:00 [error] <0.398463.0> amqp1_0_plugin_not_enabled
2025-09-04 09:33:56.252047+00:00 [error] <0.411762.0> closing AMQP connection <0.411762.0> (3.85.106.228:48284 -> 172.18.0.5:5672):
2025-09-04 09:33:56.252047+00:00 [error] <0.411762.0> {bad_header,<<"GET / HT">>}
2025-09-04 09:33:57.125289+00:00 [error] <0.411774.0> closing AMQP connection <0.411774.0> (3.85.106.228:48292 -> 172.18.0.5:5672):
2025-09-04 09:33:57.125289+00:00 [error] <0.411774.0> {bad_header,<<22,3,1,2,0,1,0,1>>}
2025-09-04 09:40:05.562767+00:00 [error] <0.413146.0> closing AMQP connection <0.413146.0> (18.209.5.244:56424 -> 172.18.0.5:5672):
2025-09-04 09:40:05.562767+00:00 [error] <0.413146.0> {bad_header,<<22,3,1,2,0,1,0,1>>}
2025-09-04 14:14:24.348558+00:00 [error] <0.478923.0> closing AMQP connection <0.478923.0> (20.169.104.27:48076 -> 172.18.0.5:5672):
2025-09-04 14:14:24.348558+00:00 [error] <0.478923.0> {bad_header,<<"MGLNDD_1">>}

[pwnengine]

ERROR 2025-09-04 16:34:49,149 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 4800af0c-b7d7-46c1-90f4-7b4c481a3f4c, retrying...
ERROR 2025-09-04 16:34:59,173 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 9eba1fed-a01f-4eaf-bfa6-7420e2b8a651, retrying...
ERROR 2025-09-04 16:35:09,187 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 7b5931ab-15ea-47c3-872b-22e6e5b3435e, retrying...
ERROR 2025-09-04 16:35:19,217 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 2e4c03fe-9d6b-43f9-9b5e-c632a8e39219, retrying...
ERROR 2025-09-04 16:35:29,238 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 9a58b95c-ec39-4f4a-927b-d5243949a62b, retrying...
ERROR 2025-09-04 16:35:39,260 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id f11473ee-6ef5-458b-bcb2-9dd276f615a7, retrying...
ERROR 2025-09-04 16:35:49,285 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 22529d30-f666-4f8f-9d2b-b27e70e762fc, retrying...
ERROR 2025-09-04 16:35:59,312 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id c5d6c85b-f8dc-4983-95a8-feba913213ad, retrying...
ERROR 2025-09-04 16:36:09,336 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id fba7f5fb-6719-4db9-bd11-f19e42b97fc7, retrying...
ERROR 2025-09-04 16:36:19,363 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 9bef4ac8-a281-442a-b6d8-5356b77e9eae, retrying...
ERROR 2025-09-04 16:36:29,396 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 453e4595-d6fc-4611-97a6-7a731c91a9df, retrying...
ERROR 2025-09-04 16:36:39,419 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 7c824447-6ff7-49ee-864d-501176236c1b, retrying...
ERROR 2025-09-04 16:36:49,447 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id f209aaa6-8c41-4460-8d7e-758c9339551c, retrying...
ERROR 2025-09-04 16:36:59,477 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 87003bd6-fec8-4aba-8597-b97d0b9b970e, retrying...
ERROR 2025-09-04 16:37:09,500 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 1feed59d-1e6e-4567-928a-d052720a2576, retrying...
ERROR 2025-09-04 16:37:19,545 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 2ee63cfa-0604-45a5-b8f5-df2faf807509, retrying...
ERROR 2025-09-04 16:37:29,577 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 7441d8c6-aae3-46e0-b0ef-be03dcc5c80b, retrying...
ERROR 2025-09-04 16:37:39,600 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id cc43eb48-f366-49e1-8618-e2ee3b3648d6, retrying...
ERROR 2025-09-04 16:37:49,626 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 9c29b8de-5ebf-4684-bcca-a088f07d8a60, retrying...
ERROR 2025-09-04 16:37:59,646 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 69c1aa81-c920-4d4b-9107-91ec45c9f78f, retrying...
ERROR 2025-09-04 16:38:09,670 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id dad0624c-4172-4b9a-bc5f-4b99df60acf1, retrying...
ERROR 2025-09-04 16:38:19,697 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 896da6b3-b098-4637-9bba-65eb4061f3b0, retrying...
ERROR 2025-09-04 16:38:29,725 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 5fb71741-c8f4-41c5-b004-42850ed5a627, retrying...
ERROR 2025-09-04 16:38:39,753 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 917a26e4-594b-4d47-969e-8bd39f6e392c, retrying...
ERROR 2025-09-04 16:38:49,777 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 7b75102f-8d04-4d2c-8428-79ca3ad88b7d, retrying...
ERROR 2025-09-04 16:38:59,799 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id d32697f8-a595-415b-9dda-448a9e8dc157, retrying...
ERROR 2025-09-04 16:39:09,821 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 984a9544-bfe9-4df5-9ca8-abfdea1d2394, retrying...
ERROR 2025-09-04 16:39:19,846 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id a884593d-6079-41c9-a7e7-e1d060d98790, retrying...
ERROR 2025-09-04 16:39:29,874 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 3c09ce69-f593-41d8-bdea-0c8dc2b7f5f9, retrying...
ERROR 2025-09-04 16:39:39,896 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 55d2ef42-cdcb-4eeb-9b53-13361382e036, retrying...
ERROR 2025-09-04 16:39:49,919 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id c4485c83-5f67-41ff-9038-5697103db877, retrying...
ERROR 2025-09-04 16:39:59,946 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 13c042bb-90f0-4363-89e2-6927b4737e6b, retrying...
ERROR 2025-09-04 16:40:09,977 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id b6ed96ac-b752-4323-9064-df10daf68ca9, retrying...
ERROR 2025-09-04 16:40:20,005 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 31499ee3-6b74-4319-a997-7c3b7696c1fb, retrying...
ERROR 2025-09-04 16:40:30,034 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 51ecf4da-0413-49af-8f3c-1da311d49047, retrying...
ERROR 2025-09-04 16:40:40,059 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 71d7ec4b-86b3-4ab1-a694-988acebdbddc, retrying...
ERROR 2025-09-04 16:40:50,086 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 91a86703-c629-4b32-a7f8-fb027bd437e2, retrying...
ERROR 2025-09-04 16:41:00,116 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 48478ab9-c336-477d-ba64-6f6ebec05a7a, retrying...
ERROR 2025-09-04 16:41:10,138 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id c07104bc-4eca-42fc-9b21-d3e968c7f4e5, retrying...
ERROR 2025-09-04 16:41:20,165 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 1d713fa7-e2e3-4296-9d8a-8abda29abeb7, retrying...
ERROR 2025-09-04 16:41:30,188 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id f0a334df-5b08-4d0e-b5c4-ae6da553d381, retrying...
ERROR 2025-09-04 16:41:40,211 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 52ad5e59-72c6-4072-b698-38e621e358d0, retrying...
ERROR 2025-09-04 16:41:50,238 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 83fc8aec-936b-4805-be50-53c4231e780d, retrying...
ERROR 2025-09-04 16:42:00,267 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 3d0a190a-e1a2-4edc-966a-75e75fd87b4a, retrying...
ERROR 2025-09-04 16:42:10,299 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 213cfa67-9421-4f6e-9289-ddcdb1dd133f, retrying...
ERROR 2025-09-04 16:42:20,331 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 2d6e9f47-8be7-48e4-b7eb-97826b34b4e1, retrying...
ERROR 2025-09-04 16:42:30,358 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id cedd0a32-bb98-4702-b8cd-1cfaecd563e8, retrying...
ERROR 2025-09-04 16:42:40,377 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 904e6e8a-7229-427e-bc59-fa5ac784d025, retrying...
ERROR 2025-09-04 16:42:50,405 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id c3c61900-7ca7-418a-b585-ca39240537d0, retrying...
ERROR 2025-09-04 16:43:00,435 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 44b90e45-689b-4f55-b4dc-e833bddf74a7, retrying...
ERROR 2025-09-04 16:43:10,456 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id f132d6f0-d783-4ddc-a47b-c47456bf802e, retrying...
ERROR 2025-09-04 16:43:20,477 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id e077e614-2c97-4402-93ad-d7062c69baa6, retrying...
ERROR 2025-09-04 16:43:30,506 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 8a343142-857e-4ba2-8730-246fc19860db, retrying...
ERROR 2025-09-04 16:43:40,526 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 6a629207-33b5-45d4-b622-a292c5fb4029, retrying...
ERROR 2025-09-04 16:43:50,557 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 66353c25-8570-4143-ab35-22d14abcf290, retrying...
ERROR 2025-09-04 16:44:00,576 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id b84dcaef-245e-437b-b82f-7585daafffcc, retrying...
ERROR 2025-09-04 16:44:10,597 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id e10aa4c9-88ac-44be-9b1e-1f34dd514d24, retrying...
ERROR 2025-09-04 16:44:20,616 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 71022382-ca99-4761-a886-543cd787beca, retrying...
ERROR 2025-09-04 16:44:30,644 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 17138ff2-8516-4b74-b6c4-39fc0a0f8889, retrying...
ERROR 2025-09-04 16:44:40,663 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 6444f7ca-de6d-4bf5-a79d-bf5b1c99a178, retrying...
ERROR 2025-09-04 16:44:50,692 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 6e1854da-7b38-43bd-a5a6-382251df537d, retrying...
ERROR 2025-09-04 16:45:00,719 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 2bd65427-3938-4693-a0b3-ed6582f60554, retrying...
ERROR 2025-09-04 16:45:10,741 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id b7e3c001-6cef-4528-81c2-78c665168660, retrying...
ERROR 2025-09-04 16:45:20,773 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 9618ebe4-07ad-4130-83e2-87c09f76dabf, retrying...
ERROR 2025-09-04 16:45:30,805 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 2858c1a7-9c33-4c27-b0d8-379e8eb843b0, retrying...
ERROR 2025-09-04 16:45:40,828 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id e0c4cbb8-a118-4e0f-a55f-f679f02ce01e, retrying...
ERROR 2025-09-04 16:45:50,857 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 698a8155-31a1-4551-9085-876d92a0dff1, retrying...
ERROR 2025-09-04 16:46:00,889 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 78984e18-0807-4cf1-9639-55f12535c2ad, retrying...
ERROR 2025-09-04 16:46:10,915 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id e6487617-8164-4c62-be40-df961e88166b, retrying...
ERROR 2025-09-04 16:46:20,937 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 1cdf7525-0985-4b3f-a67e-d5f3d9613ad5, retrying...
ERROR 2025-09-04 16:46:30,966 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 945a8e91-f43b-48cf-88fd-369d17cd80ad, retrying...
ERROR 2025-09-04 16:46:40,986 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 4156c909-a16a-40df-8e38-66a25ea707b5, retrying...
ERROR 2025-09-04 16:46:51,011 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 255cd84a-0032-4055-9f48-612cc5870340, retrying...
ERROR 2025-09-04 16:47:01,031 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 3c26ac5b-9dca-4ef0-896a-ac2731658362, retrying...
ERROR 2025-09-04 16:47:11,057 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 9faf77a2-14b3-4637-ab93-fac65e82cd05, retrying...
ERROR 2025-09-04 16:47:21,089 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 75bd7271-c6ee-43d3-a31c-0c4ef9eb578d, retrying...
ERROR 2025-09-04 16:47:31,120 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id b956d93c-0bfd-4ff5-9908-50f9de59ae23, retrying...
ERROR 2025-09-04 16:47:41,147 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id ee8935e0-b567-4d21-b4d7-28fdbad34926, retrying...
ERROR 2025-09-04 16:47:51,172 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id bf065163-1005-481b-a7eb-78d1dcdd12e3, retrying...
ERROR 2025-09-04 16:48:01,199 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 00af784c-27b6-4feb-8782-75fd7e8a016f, retrying...
ERROR 2025-09-04 16:48:11,224 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id bd88ced8-bad0-4559-8b97-659c1eef1304, retrying...
ERROR 2025-09-04 16:48:21,244 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 3ae0b2ce-be67-4595-94aa-7f2fc70d0304, retrying...
ERROR 2025-09-04 16:48:31,270 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 6e8e5066-a2b1-4728-8381-395f9d7f3f9f, retrying...
ERROR 2025-09-04 16:48:41,292 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 9c72748a-51bf-4fab-8ace-903108b53ea2, retrying...
ERROR 2025-09-04 16:48:51,313 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id bae5c666-6fe8-4340-92c9-ee2f5b9cb21a, retrying...
ERROR 2025-09-04 16:49:01,345 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 8a2f9549-b475-4f8d-9e71-d9d0961cb1ab, retrying...
ERROR 2025-09-04 16:49:11,366 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id eb4efb20-ec38-4f3d-9568-09de498049fc, retrying...
ERROR 2025-09-04 16:49:21,387 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 1f737968-6e03-4bea-aaef-3c30d10a46c5, retrying...
ERROR 2025-09-04 16:49:31,412 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id e99e2ebf-0306-4992-99a3-e4067fa28ad6, retrying...
ERROR 2025-09-04 16:49:41,436 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 0a9886b9-5e3a-401e-b5dd-8aaf07b373b3, retrying...
ERROR 2025-09-04 16:49:51,455 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 49cc8c4f-869c-446e-9c89-f7d7ead62eab, retrying...
ERROR 2025-09-04 16:50:01,487 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id eb74df96-513c-4d21-806e-8a7efb0252bc, retrying...
ERROR 2025-09-04 16:50:11,512 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id aa391842-add9-4d5c-abc6-e0b0ec02ba6e, retrying...
ERROR 2025-09-04 16:50:21,535 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 9a3dbd7d-36b2-48f6-a72c-ab3995ae2314, retrying...
ERROR 2025-09-04 16:50:31,564 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 615270c0-a6d2-4b28-b454-4ffff13c3676, retrying...
ERROR 2025-09-04 16:50:41,576 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 3eb36bd3-b5ba-42cf-86d2-1ace24ced6ea, retrying...
ERROR 2025-09-04 16:50:51,599 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 02b9ec3b-c8c6-4985-a3d7-38528c95ff62, retrying...
ERROR 2025-09-04 16:51:01,627 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 979f689b-a37e-43c9-a510-ba5a7f903a44, retrying...
ERROR 2025-09-04 16:51:11,652 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 6877b0d9-3cfd-4081-8429-9e1acb43dd04, retrying...
ERROR 2025-09-04 16:51:21,668 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 4be030ef-2d20-472b-b970-34111f8fc8fb, retrying...
ERROR 2025-09-04 16:51:31,789 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 63749012-eb36-4751-9165-0572e3e7ce1c, retrying...
ERROR 2025-09-04 16:51:41,813 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 527ab192-840c-429d-826d-9c96a97edbf5, retrying...
ERROR 2025-09-04 16:51:51,831 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 145e9433-841f-488a-b376-2ba9dc7d7c35, retrying...
ERROR 2025-09-04 16:52:01,858 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id f69b8b00-ea1a-4701-819d-6dd5056b1539, retrying...
ERROR 2025-09-04 16:52:11,879 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id dca6ee89-9232-4aeb-8541-259e08a6e7c2, retrying...
ERROR 2025-09-04 16:52:21,901 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id ed26cbc6-303a-4561-a601-75d2ebf453ae, retrying...
ERROR 2025-09-04 16:52:31,926 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 16fba1dd-1cdf-446d-b2a2-37af39b6e63c, retrying...
ERROR 2025-09-04 16:52:41,946 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 895a4fbd-b949-4f14-bfde-49626d2e5542, retrying...
ERROR 2025-09-04 16:52:51,965 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 065a5e75-cfb5-4865-b1be-5fd8d28f3340, retrying...
ERROR 2025-09-04 16:53:01,996 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id b956cc10-c9d8-4084-a5fb-f9d00dec2bf7, retrying...
ERROR 2025-09-04 16:53:12,031 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id bb445387-49e5-4f8b-8f95-bebe45aef635, retrying...
ERROR 2025-09-04 16:53:22,053 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 029c2405-ebf6-4397-bbee-9a37025e34f6, retrying...
ERROR 2025-09-04 16:53:32,081 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 40841172-6b06-4f82-8635-b4a03a030362, retrying...
ERROR 2025-09-04 16:53:42,098 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id b60fb65c-c757-43ee-81f9-6207369cedcc, retrying...
ERROR 2025-09-04 16:53:52,116 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id aa6a9c6c-ee32-4310-a53f-9fc999ef109f, retrying...
ERROR 2025-09-04 16:54:02,141 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id fc265d77-9c77-4b2d-b536-5074a9970592, retrying...
ERROR 2025-09-04 16:54:12,166 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 8bd15ae4-585e-4c8c-b5b9-9f69057e7d96, retrying...
ERROR 2025-09-04 16:54:22,186 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id e963770f-b926-456c-9726-1afabea6985d, retrying...
ERROR 2025-09-04 16:54:32,206 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 017f738a-07a0-43c0-b28b-0302b46de967, retrying...
ERROR 2025-09-04 16:54:42,233 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 5b25f32f-bede-4455-87c8-0fbb6273a90f, retrying...
ERROR 2025-09-04 16:54:52,253 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 3f4d993d-4b0e-4f65-adba-44cf295b4f48, retrying...
ERROR 2025-09-04 16:55:02,281 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 01beb20c-34c5-4ad8-a60e-d87e6d03945d, retrying...
ERROR 2025-09-04 16:55:12,308 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 8e04a417-691b-4ea4-9725-8a9de7285a74, retrying...
ERROR 2025-09-04 16:55:22,333 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 4d1d6345-8150-490e-96c7-6c6a63677381, retrying...
ERROR 2025-09-04 16:55:32,356 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id d60e0fff-7841-4079-a9cf-589dcb0cf1b6, retrying...
ERROR 2025-09-04 16:55:42,376 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 510f4d75-f999-408e-8055-4f77f285c1bc, retrying...
ERROR 2025-09-04 16:55:52,396 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 6bd6f91b-3e9f-4df3-88b0-fabc0f406c21, retrying...
ERROR 2025-09-04 16:56:02,419 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id bb71a608-e4a3-4fc0-b1e9-bc66112c1bac, retrying...
ERROR 2025-09-04 16:56:12,443 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 611cec8a-def8-436b-928a-ae4b87546432, retrying...
ERROR 2025-09-04 16:56:22,469 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id d7c7cef1-1cc4-49dd-a1bc-0e9b9eaa8a7b, retrying...
ERROR 2025-09-04 16:56:32,494 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id e19589fb-2d6d-4ba7-9c3c-5c6c89a3cc6a, retrying...
ERROR 2025-09-04 16:56:42,516 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id a388ee36-0b2a-4cea-bc45-4839dc449e1d, retrying...
ERROR 2025-09-04 16:56:52,540 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id f8721b74-c1a7-4957-865d-bdfd9eacd54a, retrying...
ERROR 2025-09-04 16:57:02,562 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 9d735d21-c83c-4084-92e7-36cc91886114, retrying...
ERROR 2025-09-04 16:57:12,588 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id a14e9b8c-b15e-49af-b46f-6c4ce9c9f226, retrying...
ERROR 2025-09-04 16:57:22,608 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 5e127ebe-c5d2-429b-84cd-08e54c52ba77, retrying...
ERROR 2025-09-04 16:57:32,637 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id b8754318-da57-4480-aad6-ba24fe63bdae, retrying...
ERROR 2025-09-04 16:57:42,663 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id a34ebfda-96b4-4f88-bf14-a810a6d33063, retrying...
ERROR 2025-09-04 16:57:52,683 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id fcaf53b9-b290-4939-9dac-d9bc22478c08, retrying...
ERROR 2025-09-04 16:58:02,713 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id a7f502a7-85aa-4277-9cd0-f7cda32b9b7a, retrying...
ERROR 2025-09-04 16:58:12,738 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id a54a64e7-b53f-43f9-bf60-7c9ae3df66e2, retrying...
ERROR 2025-09-04 16:58:22,759 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 4e56c378-4d57-4f3e-9888-7e7fc4b7fdcf, retrying...
ERROR 2025-09-04 16:58:32,778 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 15a3879f-547d-4205-b726-209c9afc592f, retrying...
ERROR 2025-09-04 16:58:42,805 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id f3b0d449-53e4-4452-ae76-002106951b0c, retrying...
ERROR 2025-09-04 16:58:52,826 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 44cb3f60-1c3e-4694-bdbf-830aab4e5c0b, retrying...
ERROR 2025-09-04 16:59:02,855 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 2e75e053-cc78-436c-b7eb-4438695487a9, retrying...
ERROR 2025-09-04 16:59:12,888 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id b8395fd0-8a69-4f5f-b3ec-0845e5415ba6, retrying...
ERROR 2025-09-04 16:59:22,909 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 376d0dc6-016a-47ec-9172-ec3a27a08244, retrying...
ERROR 2025-09-04 16:59:32,937 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 22428723-4e75-4461-8226-65731c0690c1, retrying...
ERROR 2025-09-04 16:59:42,997 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 3d0a294c-cfea-47fa-ade9-f5aed155acdc, retrying...
ERROR 2025-09-04 16:59:53,016 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 97b2836d-b59b-4c4c-9886-efc6bb105da3, retrying...
ERROR 2025-09-04 17:00:03,045 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 4c87c089-a6c7-4ad4-b3d1-b217399f8bd1, retrying...
ERROR 2025-09-04 17:00:13,067 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 246e2a9e-8704-4c2a-b40e-32b99be0dfd3, retrying...
ERROR 2025-09-04 17:00:23,088 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id b72fa699-dc57-413b-ae31-638123e78e6b, retrying...
ERROR 2025-09-04 17:00:33,121 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 42d40265-944b-43f5-8d0a-1c30d1a8b9d3, retrying...
ERROR 2025-09-04 17:00:43,152 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 9453596a-ec5d-4403-8c85-9df6c67a5a38, retrying...
ERROR 2025-09-04 17:00:53,173 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 015efa36-d421-47aa-be88-5ca32d0136d0, retrying...
ERROR 2025-09-04 17:01:03,201 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 1a37d27d-6350-4556-89f7-c9a8d28d0cf3, retrying...
ERROR 2025-09-04 17:01:13,226 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 4b6e2103-2600-49e5-b263-2700793bb6b7, retrying...
ERROR 2025-09-04 17:01:23,246 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id e554f6bf-58a7-4e38-b116-ed08e140aaa9, retrying...
ERROR 2025-09-04 17:01:33,266 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 56fa926a-e12e-4b01-9fcf-88d6d7816f79, retrying...
ERROR 2025-09-04 17:01:43,293 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id e43b41ed-d711-4107-93c7-c4e4745fbac7, retrying...
ERROR 2025-09-04 17:01:53,317 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 4cac3198-8b8a-48c9-9c75-424c80d5908a, retrying...
ERROR 2025-09-04 17:02:03,343 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 7bbb992d-f139-4b9a-b470-669cbcdc38c4, retrying...
ERROR 2025-09-04 17:02:13,366 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 3ed5a5c5-1439-44ed-b804-a2ee1e21c310, retrying...
ERROR 2025-09-04 17:02:23,386 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id b37bb35c-209b-4d2b-8ae0-6787bf332bc8, retrying...
ERROR 2025-09-04 17:02:33,411 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id c50726fa-0137-4edd-b5f7-808bfaa6ec90, retrying...
ERROR 2025-09-04 17:02:43,436 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 186fe95e-14e1-40ce-9fb8-47c2a68431b7, retrying...
ERROR 2025-09-04 17:02:53,460 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 18a10c5c-55c4-4a64-9b3b-8eaad19d807d, retrying...
ERROR 2025-09-04 17:03:03,491 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id d29520d3-d04f-4759-8347-fba9f4940b45, retrying...
ERROR 2025-09-04 17:03:13,515 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 6c8b8b02-5fa2-4dfe-88a2-7a219a02f019, retrying...
ERROR 2025-09-04 17:03:23,553 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 4289d121-f71e-4b68-bf5e-bd1e9630daa4, retrying...
ERROR 2025-09-04 17:03:33,577 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 14b731c2-9ac3-4a2e-baf7-c7eb7d9d40aa, retrying...
ERROR 2025-09-04 17:03:43,604 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 9e5760a3-5cd8-4638-a8e8-8bd2ce3a1278, retrying...
ERROR 2025-09-04 17:03:53,627 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 2dcce5c1-103a-4c5b-a9a9-d3e182977bd8, retrying...
ERROR 2025-09-04 17:04:03,657 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 3094f766-335a-45d1-9719-4cd0374ec16c, retrying...
ERROR 2025-09-04 17:04:13,686 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id f8c4820a-69e8-411b-a173-e9700a3fc101, retrying...
ERROR 2025-09-04 17:04:23,712 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 39e57605-44b1-46de-af9c-7f44f03aad13, retrying...
ERROR 2025-09-04 17:04:33,736 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 8de16921-8e58-4319-95ba-e7002def2861, retrying...
ERROR 2025-09-04 17:04:43,763 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id b06839c8-d69b-4bc3-b39f-2d5fa762d069, retrying...
ERROR 2025-09-04 17:04:53,784 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 83519a94-7ca5-477f-badf-ca51e71fe447, retrying...
ERROR 2025-09-04 17:05:03,811 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 4186e203-842c-4f2e-b0d8-03bbae7691c4, retrying...
ERROR 2025-09-04 17:05:13,852 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 0ecf11f8-0c61-4b6c-a2f7-1a8298564b74, retrying...
ERROR 2025-09-04 17:05:24,709 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id e3fc4864-5901-468e-8a35-f23b4a16d0f1, retrying...
ERROR 2025-09-04 17:05:37,443 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 19b94b17-61b9-4958-abc2-6e8a7126b0b6, retrying...
ERROR 2025-09-04 17:05:38,909 SendRPCMessage 179 : hit error trying to send RPC message in pt_sync for correlation_id: b07cef42-6c37-4a01-bc22-a7a6377028bf, retrying...:
('NO_ROUTE', 'pt_sync')
ERROR 2025-09-04 17:05:43,933 SendRPCMessage 179 : hit error trying to send RPC message in pt_sync for correlation_id: 0bb008e9-1751-451d-b323-97b2bda85afc, retrying...:
('NO_ROUTE', 'pt_sync')
ERROR 2025-09-04 17:05:48,961 SendRPCMessage 179 : hit error trying to send RPC message in pt_sync for correlation_id: 50f3f2db-7f4d-4793-a77f-f5cfc1efb86a, retrying...:
('NO_ROUTE', 'pt_sync')
ERROR 2025-09-04 17:05:53,989 SendRPCMessage 179 : hit error trying to send RPC message in pt_sync for correlation_id: a7f05e70-dc6e-4368-9c18-91f2553b6167, retrying...:
('NO_ROUTE', 'pt_sync')
ERROR 2025-09-04 17:05:59,016 SendRPCMessage 179 : hit error trying to send RPC message in pt_sync for correlation_id: 84c78a7b-abfc-4c21-96ae-6475668173a3, retrying...:
('NO_ROUTE', 'pt_sync')
ERROR 2025-09-04 17:06:14,043 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 3b8e0ade-3b39-4d82-836d-d90641df36d7, retrying...
ERROR 2025-09-04 17:06:24,071 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id a54ee20c-547a-4b93-88de-89fb336b8c53, retrying...
ERROR 2025-09-04 17:06:34,093 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id cfb22f1f-5067-4367-ab5d-ed55888071af, retrying...
ERROR 2025-09-04 17:06:44,121 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id 021dc7c0-46d5-455f-a98c-3f77a10c8a15, retrying...
ERROR 2025-09-04 17:06:54,146 SendRPCMessage 190 : hit timeout waiting for RPC response on pt_sync for correlation_id c7bd39de-8258-43d6-b1a7-70348422baae, retrying...

[its-a-feature]

2025-09-04 07:43:01.683562+00:00 [error] <0.384885.0> {handshake_timeout,frame_header}
2025-09-04 08:39:32.766475+00:00 [error] <0.398463.0> closing AMQP connection <0.398463.0> (206.168.34.65:38812 -> 172.18.0.5:5672):
2025-09-04 08:39:32.766475+00:00 [error] <0.398463.0> amqp1_0_plugin_not_enabled
2025-09-04 09:33:56.252047+00:00 [error] <0.411762.0> closing AMQP connection <0.411762.0> (3.85.106.228:48284 -> 172.18.0.5:5672):
2025-09-04 09:33:56.252047+00:00 [error] <0.411762.0> {bad_header,<<"GET / HT">>}
2025-09-04 09:33:57.125289+00:00 [error] <0.411774.0> closing AMQP connection <0.411774.0> (3.85.106.228:48292 -> 172.18.0.5:5672):
2025-09-04 09:33:57.125289+00:00 [error] <0.411774.0> {bad_header,<<22,3,1,2,0,1,0,1>>}
2025-09-04 09:40:05.562767+00:00 [error] <0.413146.0> closing AMQP connection <0.413146.0> (18.209.5.244:56424 -> 172.18.0.5:5672):
2025-09-04 09:40:05.562767+00:00 [error] <0.413146.0> {bad_header,<<22,3,1,2,0,1,0,1>>}
2025-09-04 14:14:24.348558+00:00 [error] <0.478923.0> closing AMQP connection <0.478923.0> (20.169.104.27:48076 -> 172.18.0.5:5672):
2025-09-04 14:14:24.348558+00:00 [error] <0.478923.0> {bad_header,<<"MGLNDD_1">>}

I've never seen those sorts of errors before from rabbitmq. What version of ubuntu are you using? I just tried a fresh install and everything worked just fine

[pwnengine]

I'm on ubuntu 22.04 lts and 24.04. Also, the two systems I've tested on were completely different hosts.

[pwnengine]

Could it be a memory limitation?

[its-a-feature]

I just downloaded Ubuntu 24.04.3, installed Mythic + apollo + http and I don't seem to be having any issues. Can you walk me through the steps you're doing for install and getting everything set up?

[pwnengine]

After installing the OS (just normal installation and updates):

git clone https://github.com/its-a-feature/Mythic --depth 1

cd Mythic

./install_docker_ubuntu.sh

apt upgrade (again)

apt install docker-compose-plugin

apt install make

make

./mythic-cli install github (http-url)
./mythic-cli install github (apollo-url)

./mythic-cli start

(look if containers go up)
(they do and status is healthy)

set RABBITMQ_BIND_LOCALHOST_ONLY to false
set MYTHIC_SERVER_BIND_LOCALHOST_ONLY to false

./mythic-cli restart

Then I check the web ui and see the agent(s) are failing be contacted and appear online/offline.
Also, I'm doing all of this as root.
The only thing that's different from system to system that I've installed on is I changed the nginx port on one of the installs (did it while reading through the environment variables)

[its-a-feature]

out of curiosity, if you set those two services back to localhost only and restart mythic, does that change anything?

[pwnengine]

Nope, didn't fix it. Strange...

[its-a-feature]

That's so strange, I was thinking maybe something else on the host or network is trying to connect to that rabbitmq port and is causing all the errors