Docker logs are valuable for diagnosing and resolving issues that may arise within your containerized applications and environment. Understanding where these logs are stored and how they can be accessed can help you quickly identify the root cause of errors, track performance trends, and ensure the overall stability of your Docker environment.
Docker container logs are stored in subdirectories under /var/lib/docker/containers/ on Linux hosts and in C:\ProgramData\docker\containers on Windows hosts.
In this article, you will briefly explain about Docker logs and their types, explain where they are located, how to clear them, and how they can be monitored.
What are Docker Logs?
Docker logs refer to the logs generated by Docker containers and the Docker daemon. These logs are essential for monitoring, debugging, and understanding the behavior of individual containers and the Docker system.
Types of Docker Logs
There are two different types of logs Docker generates, they include:
- Container Logs: These logs capture the standard output (stdout) and standard error (stderr) streams produced by the applications running inside a Docker container. They provide valuable insights into the application's behavior, including messages, errors, warnings, and debugging information.
- Daemon Logs: These logs capture the events and messages generated by the Docker daemon itself, which manages and orchestrates Docker containers. Daemon logs can help diagnose issues with Docker's core functionality, resource usage, and interactions with the host system.
Where are the Docker Logs Located?
Generally, Docker logs are stored in the below directory:
/var/lib/docker/
Within this directory, the exact location of logs can vary depending on the type of logs and the Docker logging driver in use. Since Docker generates two types of logs, the exact location for each log type will differ:
Where are Docker Container Logs Stored?
The location where Docker container logs are stored depends on the host operating system.
On Linux Systems
Docker container logs are stored in subdirectories under /var/lib/docker/containers/ on Linux hosts.
Each container's directory is named after its unique identifier (<container_id>) and they each have their own log file; the name of the file is {container_id}.log. Docker, by default, uses the JSON File logging driver, to store container logs so each log file is in a JSON log format.
Each container has a log specific to their ID (the full ID, not the shortened one that's usually displayed), and you can access it like so:
To access the logs of a container, you will need the complete ID of that container, not the shortened one that's usually displayed when you run docker ps. The logs are stored in the below syntax format:
/var/lib/docker/containers/ID/ID-json.log
To get the full container ID of all running containers, run:
docker ps -q --no-trunc
To get the container ID of a specific container:
docker inspect --format '{{.Id}}' <container-name>
Once you have the container ID, you can get the logs from the directory using this command:
sudo cat /var/lib/docker/containers/<container-ID>/<container-ID>-json.log
For example:
sudo cat /var/lib/docker/containers/f1d1369976e6407f6ea1735173fc5044499c22f505145f0ccc46aa8080884d1a/f1d1369976e6407f6ea1735173fc5044499c22f505145f0ccc46aa8080884d1a-json.log
The output should look like:
{"log":"/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration\n","stream":"stdout","time":"2024-06-30T14:57:36.300165665Z"}
{"log":"/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/\n","stream":"stdout","time":"2024-06-30T14:57:36.300893743Z"}
{"log":"/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh\n","stream":"stdout","time":"2024-06-30T14:57:36.304796874Z"}
{"log":"10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf\n","stream":"stdout","time":"2024-06-30T14:57:36.315498124Z"}
{"log":"10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf\n","stream":"stdout","time":"2024-06-30T14:57:36.327361401Z"}
{"log":"/docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh\n","stream":"stdout","time":"2024-06-30T14:57:36.327623451Z"}
{"log":"/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh\n","stream":"stdout","time":"2024-06-30T14:57:36.327866763Z"}
{"log":"/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh\n","stream":"stdout","time":"2024-06-30T14:57:36.333079719Z"}
{"log":"/docker-entrypoint.sh: Configuration complete; ready for start up\n","stream":"stdout","time":"2024-06-30T14:57:36.334676835Z"}
{"log":"2024/06/30 14:57:36 [notice] 1#1: using the \"epoll\" event method\n","stream":"stderr","time":"2024-06-30T14:57:36.343120703Z"}
{"log":"2024/06/30 14:57:36 [notice] 1#1: nginx/1.27.0\n","stream":"stderr","time":"2024-06-30T14:57:36.343243992Z"}
{"log":"2024/06/30 14:57:36 [notice] 1#1: built by gcc 12.2.0 (Debian 12.2.0-14) \n","stream":"stderr","time":"2024-06-30T14:57:36.343613602Z"}
{"log":"2024/06/30 14:57:36 [notice] 1#1: OS: Linux 6.8.0-1008-aws\n","stream":"stderr","time":"2024-06-30T14:57:36.343622759Z"}
{"log":"2024/06/30 14:57:36 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576\n","stream":"stderr","time":"2024-06-30T14:57:36.343626903Z"}
{"log":"2024/06/30 14:57:36 [notice] 1#1: start worker processes\n","stream":"stderr","time":"2024-06-30T14:57:36.343630676Z"}
{"log":"2024/06/30 14:57:36 [notice] 1#1: start worker process 28\n","stream":"stderr","time":"2024-06-30T14:57:36.34363423Z"}
On Mac Systems
Direct Docker Engine installation is not supported on MacOS. However, you can use Docker on macOS through Docker Desktop.
To access individual Docker container logs on Docker Desktop, from the Docker Desktop UI, select the container of choice and click on the Logs section:
On Windows Systems
Docker Engine is currently not supported natively on Windows. However, you can use Docker on Windows through Docker Desktop. Depending on your configuration, Docker Desktop leverages either the Windows Subsystem for Linux (WSL 2) or Hyper-V and sets up a lightweight VM that runs a Linux kernel.
Docker container logs are stored within the file system of this VM and can be accessed via WSL 2 or through the Docker Desktop interface.
The Docker container logs within the VM are typically stored at:
\\wsl$\docker-desktop-data\version-pack-data\community\docker\containers\
Individual container logs can be accessed at:
\\wsl$\docker-desktop-data\version-pack-data\community\docker\containers\[containerID]\[containerID]-json.logs
Note: The log path may differ based on your Docker version. Be sure to check your Docker version when accessing the container logs.
Where are Docker Daemon Logs Stored?
The location where Docker daemon logs are stored depends on the host operating system.
On Linux Systems
On older Docker versions and Linux hosts, Docker daemon logs are located at the /var/log/syslog or /var/log/messages directories. For modern Linux distributions and recent Docker versions, Docker logs are stored in the journal files under /var/log/journal. This is because, on most modern Linux distributions using systemd, Docker uses the default logging settings, which typically log to the system journal managed by systemd (i.e., journald).
To identify the location where Docker logs are stored on your Linux system, you need to check the Docker service file. The Docker service file contains configuration details about how the Docker daemon is started and managed. On systems using systemd, this file is typically located at /lib/systemd/system/docker.service.
To see the Docker service file content:
cat / lib / systemd / system / docker.service
Look for the ExecStart line to see how the Docker daemon is being started.
From the above screenshot, the ExecStart line in the Docker service unit file, specifically /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock, indicates that Docker is set up to use socket activation with systemd. In this setup, the Docker daemon communicates with systemd via the file descriptor passed (fd://), and systemd handles the activation and monitoring of the Docker service.
As a result, logs generated by the Docker daemon are directed to systemd's journald. Journal files are typically binary and not designed for direct human readability. To effectively view the Docker daemon logs captured by journald, use the command:
journalctl -xu docker.service
journalctl is a command-line utility for querying and displaying log entries from the systemd journal. The journal is a structured log that stores information about system events, service starts/stops, and errors.
The above command displays and filters the systemd journal logs for messages related to the Docker service, including errors.
On Mac Systems
On macOS, the Docker Engine itself is not directly supported. However, you can use Docker Engine on macOS through Docker Desktop.
The Docker daemon logs, tagged as dockerd on macOS, are stored at:
~/Library/Containers/com.docker.docker/Data/log/vm/dockerd.log
This log file captures events and messages related to the Docker daemon (dockerd) running inside the Docker virtual machine (VM) managed by Docker Desktop. It includes information about Docker engine operations, such as container lifecycle events (like creation, start, stop, removal), network configuration, storage management, and general Docker daemon activities.
For logs specific to the containerd runtime, which is responsible for managing container lifecycle operations and their runtime environments, they are stored at:
~/Library/Containers/com.docker.docker/Data/log/vm/containerd.log
This log file includes detailed information about container runtime events, execution events (like starts and exits), image management, and lower-level container operations.
On Windows Systems
Docker daemon (dockerd) logs are stored within the filesystem of the WSL2 VM that Docker Desktop uses:
%LOCALAPPDATA%\Docker\log\vm\dockerd.log
If you are interested in the container runtime, containerd logs, they are also stored within the WSL2 VM filesystem at:
%LOCALAPPDATA%\Docker\log\vm\containerd.log
How to Clear the Docker Log File
Docker logs files can be cleared based on how Docker was installed. To clear the Docker log file on Linux systems, first stop Docker:
sudo systemctl stop docker
Navigate to the directory where the Docker log file is located and use journalctl to clear the logs related to the Docker service:
sudo journalctl --vacuum-time=1s --unit=docker.service
This command deletes any archived systemd journal logs related to the Docker service that are older than one second. It ensures that only the logs from the Docker service are cleared, leaving other system logs intact.
Note: Docker continuously generates new logs as containers are started, stopped, or interacted with. Clearing logs with journalctl only removes archived logs, not the active ones currently in memory. Once the logs are cleared, restart the Docker service to resume normal operation:
sudo systemctl start docker
Monitor and Analyze Docker Logs in SigNoz
Monitoring and analyzing Docker logs directly in the terminal is not very efficient or scalable, especially as your application grows in complexity and the volume of logs increases. For advanced log management and analysis, it is best to forward the logs to a log management tool like SigNoz.
SigNoz is an open-source observability platform that offers comprehensive log management capabilities.
SigNoz uses a columnar database, ClickHouse, to store logs. This database is very efficient at ingesting and storing log data and making it available for analysis. It also uses OpenTelemetry for instrumenting applications. OpenTelemetry, backed by CNCF, is quickly becoming the world standard for instrumenting cloud-native applications.
The logs tab in SigNoz has advanced features like a log query builder, search across multiple fields, structured table view, JSON view, etc.
You can also view logs in real time with live tail logging.
With the advanced Log Query Builder, you can filter out logs quickly by mixing and matching fields.
You can visit the documentation for steps on monitoring and analyzing your Docker container logs in SigNoz.
Getting Started with Signoz
SigNoz cloud is the easiest way to run SigNoz. Sign up for a free account and get 30 days of unlimited access to all features.
You can also install and self-host SigNoz yourself since it is open-source. With 19,000+ GitHub stars, open-source SigNoz is loved by developers. Find the instructions to self-host SigNoz.
Conclusion
Understanding where Docker logs are stored and how to access them is useful for troubleshooting and monitoring your containerized applications.
Key takeaways:
- Docker logs provide insights into the health, performance, and issues of both your Docker containers (application logs) and the Docker system itself (daemon logs).
- Docker log location varies based on the host system configuration.
- To access Docker logs, you can use the
docker logscommand in the terminal to retrieve logs directly. - For advanced log analysis, utilize log management tools like SigNoz.
- For efficient analysis and troubleshooting, consider using a log management tool like SigNoz. that offers advanced features for searching, filtering, and visualizing your logs.
FAQs
Where can I find docker logs?
Docker logs are typically found in /var/lib/docker/containers on Linux and C:\ProgramData\docker\containers on Windows.
Where does a container store logs?
Containers don't store logs persistently; they output to stdout and stderr, which are managed by the Docker daemon.
Where are old docker logs?
Old logs may be rotated or removed depending on the logging driver configuration.
How to check docker logs live?
Use docker logs -f <container-name> or docker logs -f <container-id>.
Where is the docker login stored?
Docker login credentials are stored in ~/.docker/config.json file on Linux and %USERPROFILE%\.docker\config.json on Windows.
How to check docker-compose logs?
Use docker-compose logs to check logs of all services defined in docker-compose.yml.
Where is a docker container stored?
Containers aren't "stored" as single files; their data exists in layers within the Docker image and runtime environment.
What are logs in storage?
Logs in storage refer to log files generated by Docker containers stored locally on the host filesystem.
How to reduce docker logs?
To reduce Docker logs, configure log rotation in Docker settings or with logging drivers like json-file using max-size and max-file options.
Are docker logs persistent?
Docker logs are persistent as long as the containers producing them exist. They are stored on the host filesystem until manually deleted or managed by log rotation.
How do I list all docker containers?
Use docker ps -a.
What is the command for docker logs tail?
Use docker logs --tail <number_of_lines> <container_id> to tail the last specified number of lines from a container's logs.
How to check docker running status?
- Check Docker running status with
systemctl status dockeron Linux orGet-Service dockeron Windows PowerShell.
Where are docker images stored?
Docker images are stored in /var/lib/docker/ on Linux and C:\ProgramData\docker\ on Windows by default.
Where are container logs stored in Kubernetes?
On Linux nodes that use systemd, the kubelet and container runtime write to journald by default. On Windows, the kubelet by default writes logs to files within the directory C:\var\logs.
