Getting root on a Kubernetes node with gitRepo and CVE-2018-11235

Shortly after the recent Git vulnerability was announced, a security advisory for Kubernetes was published.

A recent vulnerability in git, CVE 2018-11235, is exposed in Kubernetes through the GitRepo volume source. If you are running a multitenant environment in which users can create pods but shouldn’t be able to gain root privileges on the node, then you need to take remedial action.

https://groups.google.com/forum/#!msg/kubernetes-security-announce/ayqL4LiUcV4/09HL6e11AgAJ

This was really interesting, firstly, I did not know about the gitRepo volume type and secondly, the idea of using a Git vulnerability to compromise a Kubernetes cluster was intriguing.

gitRepo Volume

Kubernetes provides numerous volume types that can be used to provide long lived storage to containers. The gitRepo volume mounts an empty directory and clones a Git repository into the Pod.

Note: This volume type has been marked for deprecation. The reasons are pretty obvious, seeing how this easily allowed for root access to Node

The interesting thing to note here is that the clone operation happens directly in the Node, rather than in an isolated container, meaning it runs as root. To make use of a gitRepo volume you can supply the repository name, revision and the directory into which the repository should be cloned.

A sample YAML for this:

apiVersion: v1
kind: Pod
metadata:
  name: server
spec:
  containers:
  - image: nginx
    name: nginx
    volumeMounts:
    - mountPath: /mypath
      name: git-volume
  volumes:
  - name: git-volume
    gitRepo:
      repository: "git@somewhere:me/my-git-repository.git"
      revision: "22f1d8406d464b0c0874075539c1f2e96c253775"
      directory: "myrepo"

The supplied values will be passed to the git clone command and this is where we hope to trigger the vulnerability. One prerequisite for the Git vulnerability is that the repository is cloned with --recurse-submodules, we need to verify that this is the case with gitRepo volumes. The relevant code for this can be found in the Kubernetes source for pkg/volumes/git_repo.go at https://github.com/kubernetes/kubernetes/blob/master/pkg/volume/git_repo/git_repo.go#L184.

The clone command and arguments are found on lines 198-206:

args := []string{"clone", "--", b.source}


if len(b.target) != 0 {
    args = append(args, b.target)
}
if output, err := b.execCommand("git", args, dir); err != nil {
    return fmt.Errorf("failed to exec 'git %s': %s: %v",
        strings.Join(args, " "), output, err)
}

It looks like we might be out of luck, the clone does not include --recursive-submodules. Furthermore, the code is written in Go which prevents command injection, so changing our repository to include --recurse-submodules would not work:

repository: "--recurse-submodules git@somewhere:me/my-git-repository.git"

Because of the way Go escapes the command arguments, Git will take "--recurse-submodules git@somewhere:me/my-git-repository.git" as a single argument and complain that it is an unknown argument.

“Injection Vulnerability”

Maybe not all is lost, after-all, we have three arguments that we control, these translates as follows between the YAML config and the Go code:

  • Repository == b.source
  • Revision == b.revision
  • Directory == b.target

When doing a git clone the target directory is optional, this gives us an opportunity to replace the directory with another Git argument without breaking the command.

If we set directory to --recurse-submodule our final Git command would be the following:

git clone git@somewhere:me/my-git-repository.git --recurse-submodule

Meaning we’ve met the prerequisites for the vulnerability to be triggered (given that the Git version on the Kubernetes Node is outdated).

PoC

To try this I setup minikube and in it’s default configuration started a Pod to trigger the vulnerability.

__

Fix

Granted that exploiting this vulnerability requires a user to already have the ability to define and create Pods in the cluster, meaning the user already has a fair degree of privilege. However in some multi-tenant environments you may wish to allow this, without allowing the user to actually have root on the Node. In this case you can use a PodSecurityPolicy to block the use of gitRepo volumes (the recommended policy excludes gitRepo by default).

Oh, the most obvious solution here should also be to apply the patch to Git on the Kubernetes Node.