How to Collect docker daemon’s Logs through logstash

  docker, question

Docker daemon’s log is output to standard output and standard error.
For the convenience of management, it was decided to use systemd as the start and stop of dockerd, so the logs were collected by systemd’s journal module.
viajournalctl -u docker -f -n 0Can follow in real time, similar totail -f.
There is no problem with the pipe docking with the logstash.

But it is not perfect.
Once the logstash is hung up or restarted, the next time you follow the log again, you will not be able to connect it and will lose some of it.

After several days of watching systemd, no good method was found.
Now we are considering collecting all syslog logs directly and see if there is any way to distinguish them later.

We added a layer of fluentd and inverted the log of journald/syslog/container to s3 and ELK.

Fluentd has a configurable buffer, which pauses when there is an error in the logstash and resumes writing after reconnecting.