Secure config not working in swarm mode with userns-remap #58
Description
Hi all,
I have been trying for the last 2 weeks to find the perfect security config to run docker swarm on production and I can't manage to find it. I am trying to follow the CIS checklist (https://github.com/docker/docker-bench-security) but I have the following errors when I use the docker-socket-proxy :
error in docker-socket-proxy
10.0.4.4:43234 [09/Sep/2021:09:52:34.244] dockerfrontend dockerbackend/dockersocket 0/0/-1/-1/0 503 213 - - SC-- 1/1/0/0/3 0/0 "GET /v1.24/version HTTP/1.1"
10.0.4.4:43236 [09/Sep/2021:09:52:36.154] dockerfrontend dockerbackend/dockersocket 0/0/-1/-1/0 503 213 - - SC-- 1/1/0/0/3 0/0 "GET /v1.24/version HTTP/1.1"
10.0.4.4:43238 [09/Sep/2021:09:52:40.870] dockerfrontend dockerbackend/dockersocket 0/0/-1/-1/0 503 213 - - SC-- 1/1/0/0/3 0/0 "GET /v1.24/version HTTP/1.1"
error in traefik
time="2021-09-09T08:56:52Z" level=error msg="Failed to retrieve information of the docker client and server host: Error response from daemon: <html><body><h1>503 Service Unavailable</h1>\nNo server is available to handle this request.\n</body></html>" providerName=docker
time="2021-09-09T08:56:52Z" level=error msg="Provider connection error Error response from daemon: <html><body><h1>503 Service Unavailable</h1>\nNo server is available to handle this request.\n</body></html>, retrying in 15.355607007s" providerName=docker
Here is my config
I have created the following network
sudo docker network create --driver overlay --opt encrypted web-servers
/etc/docker/daemon.json
{
"icc": false, #I don't want automatic network discovery of the containers
"live-restore": false, #not used in swarm mode
"userland-proxy": false, #(I have tried with true and it doesn't work either)
"iptables": true,
"no-new-privileges": true, #I don't want privileged containers
"log-driver" : "syslog",
"userns-remap": "default" #I want containers to run as non root users
}
docker-compose-traefik.yaml
version: "3.3"
services:
traefik:
image: "traefik:v2.4"
ports:
- "80:80"
- "443:443"
volumes:
# traefik static configuration
- ./traefik-config.yaml:/etc/traefik/traefik.yml:ro
# custom folder with SSL certs
- ./domainssl:/etc/traefik/domainssl:ro
# custom folder with dynamic configuration
- ./custom:/etc/traefik/custom:ro
# ssl volumes to store acme.json
- certs:/letsencrypt
networks:
- web-servers
- socket-proxy
deploy:
placement:
constraints:
- node.role == manager
labels:
- "traefik.enable=true"
- "traefik.docker.network=web-servers"
- "traefik.http.routers.dashboard.rule=Host(`XXXXXXXX`)"
- "traefik.http.routers.dashboard.entrypoints=websecure"
- "traefik.http.routers.dashboard.service=api@internal"
- "traefik.http.routers.dashboard.middlewares=traefik-auth"
- "traefik.http.middlewares.traefik-auth.basicauth.users=XXXX:XXXXXXX"
- "traefik.http.services.dashboard.loadbalancer.server.port=8080"
socket-proxy:
image: tecnativa/docker-socket-proxy
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
CONTAINERS: 1
SERVICES: 1
NODES: 1
NETWORKS: 1
TASKS: 1
VERSION: 1
networks:
- socket-proxy
deploy:
placement:
constraints:
- node.role == manager
networks:
web-servers:
external: true
socket-proxy:
internal: true
volumes:
certs:
traefik-config.yaml
entryPoints:
web:
address: :80
http:
redirections:
entryPoint:
to: websecure
scheme: https
websecure:
address: :443
http:
tls:
certResolver: myresolver
middlewares:
- SecHeaders@file
accessLog: {}
api:
dashboard: true
insecure: true
providers:
docker:
exposedByDefault: false
endpoint: "tcp://socket-proxy:2375"
swarmMode: true
network: "web-servers"
watch: true
file:
directory: /etc/traefik/custom/
watch: true
certificatesResolvers:
myresolver:
acme:
email: "XXXXX"
storage: "/letsencrypt/acme.json"
tlsChallenge: {}
The problem is coming from the userns-remap option in the daemon config. When I switch if off (reinstall without), it is working but both containers have root permissions... Is there a way to make it work with the userns-remap option on?
Many thanks in advance for your help