Rabotnik crash
Second time Rabotnik crashed with the same error. It's some HTTP time-out, that we should look into later.
First there is a rabotnik-obm error (channel has been closed) and afterwards rabotnik-messagebus has an error.
ERROR:
rabotnik-obm_1 | [2022-08-10 15:52:11,100: ERROR/ForkPoolWorker-20] Task rabotnikobm.assembly.run_task[40528a5a-af11-41db-91f5-c7677506b624] raised unexpected: ChannelInvalidStateError('Channel has been closed')
rabotnik-obm_1 | Traceback (most recent call last):
rabotnik-obm_1 | File "/usr/local/lib/python3.9/dist-packages/celery/app/trace.py", line 451, in trace_task
rabotnik-obm_1 | R = retval = fun(*args, **kwargs)
rabotnik-obm_1 | File "/usr/local/lib/python3.9/dist-packages/celery/app/trace.py", line 734, in __protected_call__
rabotnik-obm_1 | return self.run(*args, **kwargs)
rabotnik-obm_1 | File "/srv/rabotnik-instance/rabotnikobm/assembly.py", line 30, in run_task
rabotnik-obm_1 | rule.evaluate(building_id)
rabotnik-obm_1 | File "/srv/rabotnik-instance/rabotnikobm/rules/get_quadkey.py", line 65, in evaluate
rabotnik-obm_1 | self.loop.run_until_complete(send_to_messagebus(self.messagebus, quadkey))
rabotnik-obm_1 | File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
rabotnik-obm_1 | return future.result()
rabotnik-obm_1 | File "/srv/rabotnik-instance/rabotnikobm/rules/get_quadkey.py", line 19, in send_to_messagebus
rabotnik-obm_1 | await messagebus.send(message="quadkey", payload={"building_id": str(quadkey)})
rabotnik-obm_1 | File "/usr/local/lib/python3.9/dist-packages/rabotnik/bus.py", line 125, in send
rabotnik-obm_1 | exchange = await self.channel.declare_exchange(message, ExchangeType.FANOUT)
rabotnik-obm_1 | File "/usr/local/lib/python3.9/dist-packages/aio_pika/channel.py", line 253, in declare_exchange
rabotnik-obm_1 | channel=self.channel,
rabotnik-obm_1 | File "/usr/local/lib/python3.9/dist-packages/aio_pika/channel.py", line 118, in channel
rabotnik-obm_1 | raise aiormq.exceptions.ChannelInvalidStateError(
rabotnik-obm_1 | aiormq.exceptions.ChannelInvalidStateError: Channel has been closed
rabotnik-message-bus_1 | 2022-08-10 15:52:36.332675+00:00 [info] <0.3150.0> closing AMQP connection <0.3150.0> (172.20.0.4:50578 -> 172.20.0.2:5672, vhost: '/', user: 'user')
^Z
[12]+ Stopped docker-compose up rabotnik-obm rabotnik-tiles rabotnik-message-bus
laurens@rs59:~/deploy/rabotnik-messagebus$ docker-compose stop rabotnik-obm
Stopping rabotnik-messagebus_rabotnik-obm_1 ...
ERROR: for rabotnik-messagebus_rabotnik-obm_1 UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=70)
ERROR: An HTTP request took too long to complete. Retry with --verbose to obtain debug information.
If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 60).
laurens@rs59:~/deploy/rabotnik-messagebus$