perf(monitoring): stop blocking event loop with synchronous Docker calls

Customer search and detail loads were intermittently slow because every
customer-table render (including each search keystroke) triggered
/monitoring/customers/local-update-status, which looped synchronously over
all customers doing blocking `docker inspect` subprocess calls on the event
loop — stalling all other in-flight requests, including search itself.

- Offload per-service image/container inspection to the thread pool and run
  checks concurrently instead of sequentially (image_service, docker_service)
- Reuse a single Docker SDK client instead of reconnecting per customer
- Cache local-update-status results for 20s since the underlying data only
  changes after an image pull, not on every keystroke
- Parallelize /monitoring/customers container status lookups

Co-Authored-By: Claude Sonnet 5 <[email protected]>
This commit is contained in:
2026-07-23 14:44:56 +02:00
co-authored by Claude Sonnet 5
parent f6b7eb2dae
commit ac843da4ca
4 changed files with 94 additions and 12 deletions
+23 -2
View File
@@ -27,13 +27,24 @@ async def _run_cmd(cmd: list[str], timeout: int = 120) -> subprocess.CompletedPr
)
_client: Optional[docker.DockerClient] = None
def _get_client() -> docker.DockerClient:
"""Return a Docker client connected via the Unix socket.
"""Return a shared Docker client connected via the Unix socket.
The client is created once and reused — creating a new client per call
(as `docker.from_env()` does) re-negotiates the API version and opens a
fresh connection every time, which is wasteful when called once per
customer in a loop.
Returns:
docker.DockerClient instance.
"""
return docker.from_env()
global _client
if _client is None:
_client = docker.from_env()
return _client
async def compose_up(
@@ -212,6 +223,16 @@ def get_container_status(container_prefix: str) -> list[dict[str, Any]]:
return results
async def get_container_status_async(container_prefix: str) -> list[dict[str, Any]]:
"""Thread-offloaded wrapper around get_container_status().
Use this when checking status for multiple customers so the Docker SDK
calls run in the thread pool instead of blocking the event loop.
"""
loop = asyncio.get_event_loop()
return await loop.run_in_executor(None, get_container_status, container_prefix)
def get_container_logs(container_name: str, tail: int = 200) -> str:
"""Retrieve recent logs from a container.