Skip to content
This repository has been archived by the owner on Mar 9, 2022. It is now read-only.

Add flag to overload default privileged host device behaviour #1225

Merged

Conversation

awprice
Copy link
Contributor

@awprice awprice commented Aug 7, 2019

This commit adds a flag to the runtime config that allows overloading of the default
privileged behaviour. When the flag is enabled on a runtime, host devices won't
be appended to the runtime spec if the container is run as privileged.

By default the flag is false to maintain the current behaviour of privileged.

Fixes #1213

Signed-off-by: Alex Price aprice@atlassian.com

@k8s-ci-robot
Copy link

Hi @awprice. Thanks for your PR.

I'm waiting for a containerd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@Random-Liu Random-Liu self-assigned this Aug 7, 2019
@Random-Liu Random-Liu added this to the v1.3 milestone Aug 7, 2019
@Random-Liu
Copy link
Member

/ok-to-test

Copy link
Member

@mikebrow mikebrow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/LGTM

@Random-Liu
Copy link
Member

Update docs/config.md

@@ -168,7 +168,8 @@ func (c *criService) CreateContainer(ctx context.Context, r *runtime.CreateConta
logrus.Debugf("Use OCI runtime %+v for sandbox %q and container %q", ociRuntime, sandboxID, id)

spec, err := c.generateContainerSpec(id, sandboxID, sandboxPid, config, sandboxConfig,
&image.ImageSpec.Config, append(mounts, volumeMounts...), ociRuntime.PodAnnotations)
&image.ImageSpec.Config, append(mounts, volumeMounts...), ociRuntime.PodAnnotations,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Let's pass in the ociRuntime instead to shorten the arg list

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 fixed

@awprice awprice force-pushed the privileged_without_host_devices_flag branch from 9d2c433 to 3186170 Compare August 8, 2019 02:09
@k8s-ci-robot k8s-ci-robot removed the lgtm label Aug 8, 2019
@k8s-ci-robot
Copy link

New changes are detected. LGTM label has been removed.

@k8s-ci-robot k8s-ci-robot added size/L and removed size/M labels Aug 8, 2019
@awprice
Copy link
Contributor Author

awprice commented Aug 8, 2019

👍 docs/config.md updated

This commit adds a flag to the runtime config that allows overloading of the default
privileged behaviour. When the flag is enabled on a runtime, host devices won't
be appended to the runtime spec if the container is run as privileged.

By default the flag is false to maintain the current behaviour of privileged.

Fixes containerd#1213

Signed-off-by: Alex Price <aprice@atlassian.com>
@awprice awprice force-pushed the privileged_without_host_devices_flag branch from 3186170 to 3353ab7 Compare August 8, 2019 02:17
@Random-Liu Random-Liu merged commit aaf15b6 into containerd:master Aug 8, 2019
@Random-Liu
Copy link
Member

/lgtm

@AkihiroSuda
Copy link
Member

Proposal for porting over this to Moby/Docker: moby/moby#39697

@jongwu
Copy link

jongwu commented Sep 4, 2019

hi @awprice, this feature sounds great. but how to use this feature. I open privileged_without_host_devices in /etc/containerd/config.toml and set this file as the default config for containerd. then I start a container use docker with --privileged. when I type the command "ls /dev" in container, I saw that the output is the same with host.
I think I may take the wrong step. can someone tell me how to test this feature? @Random-Liu @mikebrow @awprice

@awprice
Copy link
Contributor Author

awprice commented Sep 4, 2019

@jongwu privileged_without_host_devices will only work when starting containers with the CRI plugin/interface in containerd, which is usually done through Kubernetes or with crictl.

There is a PR to add this functionality to docker, it might be worth trying out: moby/moby#39702

@jongwu
Copy link

jongwu commented Sep 4, 2019

thanks @awprice, I am not familiar with k8s, I think crictl is simple to test this feature, can you give me some info about how to test it using crictl. thanks.

@awprice
Copy link
Contributor Author

awprice commented Sep 4, 2019

No worries @jongwu. Here is the documentation for crictl - https://github.com/kubernetes-sigs/cri-tools/blob/master/docs/crictl.md

@jongwu
Copy link

jongwu commented Sep 5, 2019

thanks @awprice

@jongwu
Copy link

jongwu commented Sep 5, 2019

hi @awprice, sorry to trouble you again, I have tried for a full day to start a container with --privileged, but failed.
my pod-config is
{
"metadata": {
"name": "true222222222222",
"namespace": "default",
"attempt": 1,
"uid": "hdishd83djaidwnduwk28bcsb"
},
"log_directory": "/tmp",
"linux": {
"securityContext": true
}
}

my container-config.json is
{
"metadata": {
"name": "busybox1111111111111"
},
"image":{
"image": "busybox"
},
"command": [
"top"
],
"log_path":"busybox/0.log",
"linux": {
"SecurityContext": {
"privileged": true
}
}
}
the privileged_without_host_devices is false.
after I start a container, the picture under /dev is totally different with host.
I don't know what's wrong with it. can you help me? thanks!

electrocucaracha added a commit to electrocucaracha/kubespray that referenced this pull request Mar 3, 2021
When privileged is enabled for a container, all the `/dev/*` block
devices from the host are mounted into the guest. The
`privileged_without_host_devices` flag prevents host devices from
being passed to privileged containers.

More information:
* containerd/cri#1225
* cri-o/cri-o@1d0f681
k8s-ci-robot pushed a commit to kubernetes-sigs/kubespray that referenced this pull request Mar 8, 2021
When privileged is enabled for a container, all the `/dev/*` block
devices from the host are mounted into the guest. The
`privileged_without_host_devices` flag prevents host devices from
being passed to privileged containers.

More information:
* containerd/cri#1225
* cri-o/cri-o@1d0f681
champtar pushed a commit to champtar/kubespray that referenced this pull request Mar 11, 2021
When privileged is enabled for a container, all the `/dev/*` block
devices from the host are mounted into the guest. The
`privileged_without_host_devices` flag prevents host devices from
being passed to privileged containers.

More information:
* containerd/cri#1225
* cri-o/cri-o@1d0f681

(cherry picked from commit dc5df57)
k8s-ci-robot pushed a commit to kubernetes-sigs/kubespray that referenced this pull request Mar 15, 2021
When privileged is enabled for a container, all the `/dev/*` block
devices from the host are mounted into the guest. The
`privileged_without_host_devices` flag prevents host devices from
being passed to privileged containers.

More information:
* containerd/cri#1225
* cri-o/cri-o@1d0f681

(cherry picked from commit dc5df57)
LuckySB pushed a commit to southbridgeio/kubespray that referenced this pull request Apr 6, 2021
When privileged is enabled for a container, all the `/dev/*` block
devices from the host are mounted into the guest. The
`privileged_without_host_devices` flag prevents host devices from
being passed to privileged containers.

More information:
* containerd/cri#1225
* cri-o/cri-o@1d0f681
r3dsm0k3 added a commit to reynencourt/kubespray that referenced this pull request May 3, 2021
* Only use stat get_checksum: yes when needed (kubernetes-sigs#7270)

By default Ansible stat module compute checksum, list extended attributes and find mime type
To find all stat invocations that really use one of those:
git grep -F stat. | grep -vE 'stat.(islnk|exists|lnk_source|writeable)'

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
(cherry picked from commit de1d9df)

Conflicts:
	roles/etcd/tasks/check_certs.yml

* Add kube-ipvs0/nodelocaldns to NetworkManager unmanaged-devices (kubernetes-sigs#7315)

On CentOS 8 they seem to be ignored by default, but better be extra safe
This also make it easy to exclude other network plugin interfaces

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
(cherry picked from commit e442b1d)

* Stop using kubeadm to update server in kubeconfigs (kubernetes-sigs#7338)

Using `kubeadm init phase kubeconfig all` breaks kubelet client certificate rotation
as we are missing `kubeadm init phase kubelet-finalize all` to point to `kubelet-client-current.pem`

kubeconfig format is stable so let's just use lineinfile,
this will avoid other future breakage

This revert to the logic before 6fe2248

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
(cherry picked from commit c9c0c01)

* kubeadm-config.v1beta2.yaml.j2: etcd log level arg (kubernetes-sigs#7339)

According to [etcd's docs](https://etcd.io/docs/v3.4.0/op-guide/configuration/#--log-package-levels), argument 'log-package-levels' should not contain underscores.

(cherry picked from commit b7c2265)

* Remove pre kubeadm cert migration tasks

apiserver.pem is not used since ddffdb6

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
(cherry picked from commit fedd671)

Conflicts:
	roles/kubernetes/master/tasks/kubeadm-cleanup-old-certs.yml
	roles/kubernetes/master/tasks/kubeadm-migrate-certs.yml

* Remove useless call to 'kubeadm version'

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
(cherry picked from commit a6e1f5e)

* Remove admin.conf removal

kubeadm is the default for a long time now,
and admin.conf is created by it, so let kubeadm handle it

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
(cherry picked from commit 280036f)

* Remove rotate_tokens logic

kubeadm never rotates sa.key/sa.pub, so there is no need to delete tokens/restart pods

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
(cherry picked from commit 8800b5c)

* Always backup both certs and kubeconfig

There are no reasons not to backup during upgrade

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
(cherry picked from commit 53e5ef6)

Conflicts:
	roles/kubernetes/master/tasks/kubeadm-backup.yml
	roles/kubernetes/master/tasks/kubeadm-certificate.yml

* Delete misnammed kubeadm-version.yml

The important action in kubeadm-version.yml is the templating of the configuration,
not finding / setting the version

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
(cherry picked from commit a9c97e5)

Conflicts:
	roles/kubernetes/master/tasks/kubeadm-version.yml

* Add privileged_without_host_devices support (kubernetes-sigs#7343)

When privileged is enabled for a container, all the `/dev/*` block
devices from the host are mounted into the guest. The
`privileged_without_host_devices` flag prevents host devices from
being passed to privileged containers.

More information:
* containerd/cri#1225
* cri-o/cri-o@1d0f681

(cherry picked from commit dc5df57)

* ansible and jinja2 updates (kubernetes-sigs#7357)

* Update ansible to v2.9.18

Signed-off-by: Maciej Wereski <m.wereski@partner.samsung.com>

* Update jinja2 to v2.11.3

Signed-off-by: Maciej Wereski <m.wereski@partner.samsung.com>
(cherry picked from commit b07c596)

* Fixup kubelet.conf to point to kubelet-client-current.pem (kubernetes-sigs#7347)

c9c0c01 only fix the problem for new clusters

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
(cherry picked from commit 14b63ed)

Conflicts:
	roles/kubernetes/master/tasks/kubelet-fix-client-cert-rotation.yml

* Check for dummy kernel module (kubernetes-sigs#7348)

The dummy module is needed for nodelocaldns.

(cherry picked from commit 5a54db2)

* Fixup one more missing kubespray-defaults (kubernetes-sigs#7375)

"The error was: 'proxy_disable_env' is undefined\n\nThe error appears to
be in '<censored>scale.yml': line 72, column 7"

Fixes 067db68

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
(cherry picked from commit 057e8b4)

* Upgrade openSUSE Leap to 15.2 (kubernetes-sigs#7331)

15.1 has reached EOL on 2021-02-02.

Signed-off-by: Maciej Wereski <m.wereski@partner.samsung.com>
(cherry picked from commit 69d11da)

* Update kube-ovn to 1.6.0 (kubernetes-sigs#7240)

(cherry picked from commit edc4bb4)

* Minor update to cilium and calico

(cherry picked from commit de46f86)

* Update nodelocaldns to 1.17.1

(cherry picked from commit 5f2c8ac)

* Download Calico KDD CRDs (kubernetes-sigs#7372)

* Download Calico KDD CRDs

* Replace kustomize with lineinfile and use ansible assemble module

* Replace find+lineinfile by sed in shell module to avoid nested loop

* add condition on sed

* use block for kdd tasks + remove supernumerary kdd manifest apply in start "Start Calico resources"

(cherry picked from commit 1c62af0)

Conflicts:
        roles/network_plugin/calico/tasks/install.yml

* Update CNI (calico, kubeovn, multus) and Helm

(cherry picked from commit 05f132c)

* Fix calico crds missing 3.16.9 (kubernetes-sigs#7386)

(cherry picked from commit ead8a4e)

* Update hashes for 1.20.5/1.19.9/1.18.17

(cherry picked from commit 6d3dbb4)

* Set K8S default to v1.19.9

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>

* Auto renew control plane certificates (kubernetes-sigs#7358)

While at it remove force_certificate_regeneration
This boolean only forced the renewal of the apiserver certs
Either manually use k8s-certs-renew.sh or set auto_renew_certificates

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
(cherry picked from commit efa1803)

Conflicts:
	roles/kubernetes/master/templates/k8s-certs-renew.service.j2
	roles/kubernetes/master/templates/k8s-certs-renew.sh.j2
	roles/kubernetes/master/templates/k8s-certs-renew.timer.j2

* Add cryptography installation (kubernetes-sigs#7404)

To avoid ModuleNotFoundError due to no module named 'setuptools_rust',
this adds cryptography installation to requirements.txt.

Created by jfc-evs originally as kubernetes-sigs#7264

(cherry picked from commit 49abf60)

* Allow connecting to bastion via non-standard SSH port (kubernetes-sigs#7396)

* Allow connecting to bastion via non-standard port

* Fix bastion connection when ansible_port is not provided

(cherry picked from commit 6fa3565)

* Correct Jinja Syntax for etcd-unsupported-arch (kubernetes-sigs#6919)

`-%` causes `etcd-unsupported-arch: arm64` to print on COL 1 instead of
COL 6.

Signed-off-by: anthr76 <hello@anthonyrabbito.com>
(cherry picked from commit edfa3e9)

* Fix k8s-certs-renew for k8s < 1.20 (kubernetes-sigs#7410)

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
(cherry picked from commit 2d1597b)

* Remove ignore_errors from drain tasks and enable retires (kubernetes-sigs#7151)

* Remove ignore_errors from drain tasks and enable retires

* Fix lint error by checking if stdout length is not 0, ie string is not empty.

(cherry picked from commit ccd3aee)

* Fix remove-node by removing jq usage (kubernetes-sigs#7405)

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
(cherry picked from commit 36a3a78)

* Remove left over nodes_to_drain

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>

* remove local lb privileged (kubernetes-sigs#7437) (kubernetes-sigs#7454)

Co-authored-by: Samuel Liu <liupeng0518@gmail.com>

* Add new kubernetes hashes (1.19.10, 1.20.6)

* Default to latest kubernetes patch version (1.19.10)

* Update k8s-certs-renew.sh.j2 (kubernetes-sigs#7422)

fix undefinedElse

(cherry picked from commit cce9d31)

* reset roles need flush iptables:raw (kubernetes-sigs#7426)

(cherry picked from commit 7f52c1d)

* Remove calico-rr from local inventory hosts file (kubernetes-sigs#7439)

(cherry picked from commit 596d028)

Conflicts:
	inventory/local/hosts.ini

* Replace deprecated 'with_dict' with 'loop' (kubernetes-sigs#7442)

(cherry picked from commit 6479e26)

* local provisioner 'useNodeNameOnly' option can be configured (kubernetes-sigs#7421)

(cherry picked from commit 7e75d48)

* fix scale (kubernetes-sigs#7449)

(cherry picked from commit 7340a16)

* remove-node roles: fix kubectl absolute path (kubernetes-sigs#7469)

* kubelet absolute path

* kubelet absolute path

(cherry picked from commit e2a7f3e)

* add CI test for auto_renew_certificates (kubernetes-sigs#7472)

* add CI test for auto_renew_certificates

* change timer value

fix typo error in rotate cert script

(cherry picked from commit cce0940)

Conflicts:
	roles/kubernetes/master/templates/k8s-certs-renew.timer.j2

* Remove dead code from kubeadm-etcd (kubernetes-sigs#7470)

(cherry picked from commit aa086e5)

* format ansible output (kubernetes-sigs#7482)

(cherry picked from commit 90c643f)

* Regenerate apiserver.crt on all control-plane nodes (kubernetes-sigs#7463)

We were regenerating only the cert of the first node
While at it speed up the check step

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
(cherry picked from commit e444b3c)

Conflicts:
	roles/kubernetes/master/tasks/kubeadm-setup.yml

* Add auto_renew_certificates_systemd_calendar (kubernetes-sigs#7490)

This allow to configure when K8S certificates renewal runs

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
(cherry picked from commit bf6a39e)

Conflicts:
        inventory/sample/group_vars/k8s-cluster/k8s-cluster.yml
        roles/kubernetes/master/defaults/main/main.yml
        roles/kubernetes/master/templates/k8s-certs-renew.timer.j2

* Check if python netaddr and recent enough jinja are installed (kubernetes-sigs#7486)

CentOS 7 provides up to date Ansible with really old jinja version

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
(cherry picked from commit 332cc1c)

* Add missing proxy environment in crio_repo.yml (kubernetes-sigs#7492)

(cherry picked from commit 2a2fb68)

Co-authored-by: Etienne Champetier <e.champetier@ateme.com>
Co-authored-by: Du9L.com <dujiulun@users.noreply.github.com>
Co-authored-by: Victor Morales <chipahuac@hotmail.com>
Co-authored-by: Maciej <m.wereski@partner.samsung.com>
Co-authored-by: Lennart Jern <lennart.jern@elastisys.com>
Co-authored-by: Florian Ruynat <16313165+floryut@users.noreply.github.com>
Co-authored-by: Erwan Miran <mirwan@users.noreply.github.com>
Co-authored-by: Kenichi Omichi <ken1ohmichi@gmail.com>
Co-authored-by: Kaleb Elwert <kaleb@coded.io>
Co-authored-by: Anthony Rabbito <hello@anthonyrabbito.com>
Co-authored-by: David Louks <2402775+dlouks@users.noreply.github.com>
Co-authored-by: bleech1 <15leechb@gmail.com>
Co-authored-by: Samuel Liu <liupeng0518@gmail.com>
Co-authored-by: Fredrik Liv <fredrik.liv@elastisys.com>
Co-authored-by: Helmut Januschka <helmut@januschka.com>
Co-authored-by: Maxime Lavandier <maxime.l@live.fr>
Co-authored-by: orange-llajeanne <71634751+orange-llajeanne@users.noreply.github.com>
Co-authored-by: Sergey <s.bondarev@southbridge.ru>
Co-authored-by: Krystian Młynek <75165213+krystianmlynek@users.noreply.github.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

runtime based privileged container translation
6 participants