Using Multitail with Docker Compose to group interspersed logs

October 05, 2017

Background

Docker combined with Docker Compose has proved itself to be valuable tooling for heavy lifting a lot of container creation and management boilerplate. The unified logging available via docker-compose logs -f, is nice as one can see all the logs from various services. One can also filter and view logs only relevant to a particular service with additional parameters docker-compose logs -f <service-name>.

However, it would be superb to be able to tail all the logs for various services, but grouped together in their own window/pane/splits etc. This is where multitail comes in. multitail is a generic tool for viewing multiple logs at once, and is not necessarily tied to docker ecosystem, but it makes a great combo with services run via docker-compose. It is terminal friendly and uses ncurses interface, which allows for nifty interactive things right in the terminal.

Needless to mention, sharing it here so that I can look it up later.

Goal

The end goal is to be able to tail logs from various docker-compose services, all of them grouped in their own panes. A screenshot should make things more clear.

Requirements

Implementation

I’ll be using a Makefile and docker-compose.yml to demo this. The core idea is to run multitail and ask it to tail logs for each of the services (in docker-compose.yml) separately.

Here’s the docker-compose.yml that we will be running.

version: '3'

services:

  db:
    image: library/postgres:9.5.2

  cache:
    image: library/redis:4.0

  search:
    image: library/elasticsearch:5.5.2

and, here’s the Makefile. Note the multitail target.

.DEFAULT_GOAL:=start

DOCKER_COMPOSE:=docker-compose -f docker-compose.yml

.PHONY: start stop multitail

start:
	$(DOCKER_COMPOSE) up -d

stop:
	$(DOCKER_COMPOSE) down

multitail:
	multitail -o beep_method:popup                                        \
	          -cT ansi -l '$(DOCKER_COMPOSE) logs -f db'     \
	          -cT ansi -l '$(DOCKER_COMPOSE) logs -f cache'  \
	          -cT ansi -l '$(DOCKER_COMPOSE) logs -f search'

logs:
	$(if $(SERVICE_NAME), $(info -- Tailing logs for $(SERVICE_NAME)), $(info -- Tailing all logs, SERVICE_NAME not set.))
	$(DOCKER_COMPOSE) logs -f $(SERVICE_NAME)

We configure multitail to use a popup when beeping (instead of flash+beep) which can be annoying. Further, we also enable ansi terminalmode, so that ANSI terminal sequences (color etc.) are interpreted properly.

More goodies

While the above should get you a long way, I prefer to have more configuration for multitail. We can ask multitail to not wrap long lines (-p l), and to truncate a few characters at the beginning (-p o 10). Here’s the modified Makefile.

multitail:
	multitail -o beep_method:popup                                        \
	          -p o 10 -p l -cT ansi -l '$(DOCKER_COMPOSE) logs -f db'     \
	          -p o 10 -p l -cT ansi -l '$(DOCKER_COMPOSE) logs -f cache'  \
	          -p o 10 -p l -cT ansi -l '$(DOCKER_COMPOSE) logs -f search'

Basic usage

multitail has an in-built interactive help that you should be able to learn a lot from. h is your friend. I won’t mention all the shorcuts here, but here’s a list of quick one that I use fairly often.

Wrap up

Thanks to a colleague of mine, who demo-ed multitail, I have started using it on all my projects. Many thanks to the author of multitail, without which this simple would not have been possible.

Do check out the manual for multitail to find out all the cool stuff you can do with it. And, let me know in the comments if there’s further tricks that I’ve been missing out on.


comments powered by Disqus