New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
new security-opt: privileged-without-host-devices (for safe DinD with Kata) #39702
base: master
Are you sure you want to change the base?
Conversation
@bergwolf PTAL? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A query on where we are restricting the /dev
mount?
@AkihiroSuda One more question. When a new device is attached to the host and an entry is created in I can see that already there is a bug in docker which does not update the |
For Kata, new device wont be enabled because there is no mount for the device. For non-kata, the flag should not be considered as a security boundary. |
Ohkay. Got it. But can these same functionality be extended to non-kata? Or is it used only to get past the VM isolation issue in kata. |
It doesn't make sense. Privileged non-kata container can execute arbitrary command on the host anyway to access any device |
Ohkay!. 👍 Thanks for explaining. |
LGTM! Thanks @AkihiroSuda ! |
1 similar comment
cc @tianon WDYT? |
The option seems ok, but I'm not sure why someone would use privileged and also expect it to be secure? |
dind with kata needs --privileged and is expected to be secure |
Isn't this combination possible with #36644, or is that tweaking something else? (haven't looked in depth). |
Unrelated, this one aims at preventing Kata from mounting host /dev completely |
I get conceptually that this solves a problem Kata has, but I don't think I understand why this particular solution was chosen? We've long regarded the |
Yes, I was thinking; what privileges does If those missing options can be added, then instead of using --cap-add=all
--security-opt apparmor=unconfined
--security-opt seccomp=unconfined
--security-opt systempaths=unconfined
--security-opt host-devices=unconfined (start with "default", and add what's needed) |
$ docker run -d --name dind --runtime=kata --security-opt apparmor=unconfined --security-opt seccomp=unconfined --security-opt systempaths=unconfined --cap-add all docker:19.03-dind
$ docker exec -it dind docker run -it --rm alpine
...
docker: Error response from daemon: OCI runtime create failed: container_linux.go:345: starting container process caused "process_linux.go:281: applying
cgroup configuration for process caused \"mkdir /sys/fs/cgroup/cpuset/docker: read-only file system\"": unknown.
... (Kata 1.8.0 with Moby e20b732) after remounting cgroup as read-write before starting dockerd-entrypoint: $ docker exec -it dind docker run -it --rm alpine
docker: Error response from daemon: cgroups: cannot find cgroup mount destination: unknown.
ERRO[0002] error waiting for container: context canceled |
…leged) Docker-in-Kata had required `--privileged` but it ruins the benefit of Kata because it mounts `/dev` from the host. Now Docker-in-Kata can be launched without `--privileged`: $ docker run --runtime kata -e DOCKER_REMOUNT_SYS_RW=1 --cap-add all --security-opt seccomp=unconfined --security-opt systempaths=unconfined docker:dind Tested with Kata Containers 1.8.0 (1.8.1 is broken: kata-containers/runtime#2047) Alternative to moby/moby#39702 Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
For DIND usecase, this DIND PR seems to work: docker-library/docker#191
I can close this PR unless there is still demand from Kata maintainers. |
…leged) Docker-in-Kata can be launched with `--privileged` but it ruins the benefit of Kata because it mounts `/dev` from the host. Now Docker-in-Kata can be launched without `--privileged`: $ docker run --runtime kata -e DOCKER_REMOUNT_SYS_RW=1 --cap-add all --security-opt seccomp=unconfined --security-opt systempaths=unconfined docker:dind Tested with Kata Containers 1.8.0 (1.8.1 is broken: kata-containers/runtime#2047) Alternative to moby/moby#39702 Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
…leged) Docker-in-Kata can be launched with `--privileged` but it ruins the benefit of Kata because it mounts `/dev` from the host. Now Docker-in-Kata can be launched without `--privileged`: $ docker run --runtime kata -e DOCKER_REMOUNT_SYS_RW=1 --cap-add all --security-opt seccomp=unconfined --security-opt systempaths=unconfined docker:dind Tested with Kata Containers 1.8.0 (1.8.1 is broken: kata-containers/runtime#2047) Alternative to moby/moby#39702 Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
I'm keeping this PR open, as discussed in Kata ML: http://lists.katacontainers.io/pipermail/kata-dev/2019-September/001029.html |
@dmcgowan @justincormack ptal |
rebased |
CRI-O adopted equivalent of this as well as containerd/CRI: cri-o/cri-o#2730 |
Aside from Kata, this PR turned out to be also useful for protecting the host console from cc @cpuguy83 |
For the (mostly privileged) systemd case, instead of doing I do think it's slightly unfortunate that But you can gradually add what you need instead of relying on |
`docker run --runtime=kata --privileged` is insecure despite of Kata's VM isolation because host devices are visible to the container. This commit adds a new security-opt `privileged-without-host-devices` to allow privileged mode without mounting host devices. The daemon returns an error if the opt is specified but privileged is not specified. A common use-case of this is to run Docker-in-Docker securely. Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
rebased |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
I still think this is probably not the right UX here -- how long before someone asks for another I agree with @thaJeztah that it would be really useful if we could make a table of features that |
Kata v2 no longer supports runc-style CLI, and Moby does not support non-runc runtimes, so closing. |
@AkihiroSuda Could you please reopen this PR? |
Re #39702 (comment), do you have a sense of which things Kata needs that are enabled by |
Signed-off-by: Akihiro Suda akihiro.suda.cz@hco.ntt.co.jp
- What I did
docker run --runtime=kata --privileged
is insecure despite of Kata'sVM isolation because host devices are visible to the container. kata-containers/runtime#1568
This commit adds a new security-opt
privileged-without-host-devices
toallow privileged mode without mounting host devices.
The daemon returns an error if the opt is specified but privileged is
not specified.
A common use-case of this is to run Docker-in-Docker securely with Kata.
Fixes #39697
Relates to containerd/cri#1225 cri-o/cri-o#2730
- How I did it
Added a new security-opt
- How to verify it
CLI: docker/cli#2037
Without
privileged-without-host-devices
With
privileged-without-host-devices
:Verified with Kata 1.8.0
- Description for the changelog
new security-opt: privileged-without-host-devices
- A picture of a cute animal (not mandatory but encouraged)