Monitoring complex stacks, detecting when things go haywire, is hard. Big Data Europe offers a component which monitors Docker’s network traffic in order to debug erroneous states.
The HTTP logger monitors network traffic, transforms its HTTP contents into a JSON format, stores it in Elastic Search and visualizes it in Kibana.
In this post, we sketch the context and indicate how to use the component. Further posts will explain how this is technically implemented on top of Docker networks.
A word about monitoring
Looking at the problem of monitoring from a high level, anything structured can be logged and introspected, as long as you can make sense of said structure.