When running tools like like docker, podman, kubernetes, etc. internal networks are set up that could collide with routed networks within the University of Waterloo’s infrastructure. If this collision happens, then any traffic that is supposed to route to those will stay on the given system. An example for this, if you are on VPN with the ip address 172.25.10.1, and you start a container or compose project, if docker allocates 172.25.0.0/20, your connection to the server will drop, as any traffic that was destined for you now gets directed to the docker network.

To prevent these issues from happening, campus Network Operations Centre (NOC) has reserved a subnet that will not be routed. This can be used for containers. The network block that has been allocated is:

10.255.0.0/16

which contains 65,535 addresses that can be used for container networks. The various tools below use different default subnets and are configured differently. Known configurations are listed below. If there are any others that should be added, FAST members can edit this page, or a request can be made to document the specific tooling.

Configurations

1 Docker
2 Docker Swarm
3 Podman
4 Kubernetes
- 4.1 RKE2
5 LXC

Docker

/etc/docker/daemon.json

{
....
  "default-address-pools": [
    {"base":"10.255.0.0/16","size":25}
  ]
}

size specifies the default network size for each docker network. /25 contains 128 addresses, so adjust based on your needs.

If you are setting this on a host that has existing containers, networks will need to be recreated.

TODO: Fill in instructions for doing this

Docker Swarm

In addition to the above docker configuration, you must also do the following

Default address pool must be set on creation and cannot be changed

docker swarm init --default-addr-pool 10.255.0.0/16 --default-addr-pool-mask-length 25

default-addr-pool-mask-length specifies the default network size for each docker network. /25 contains 128 addresses, so adjust based on your needs.

Podman

See https://github.com/containers/common/blob/main/docs/containers.conf.5.md for configuration locations

[network]
default_subnet="10.255.0.0/24"
default_subnet_pools = [
  {"base" = "10.255.1.0/24", "size": 25},
  {"base" = "10.255.2.0/23", "size": 25},
  {"base" = "10.255.4.0/22", "size": 25},
  {"base" = "10.255.8.0/21", "size": 25},
  {"base" = "10.255.16.0/20", "size": 25},
  {"base" = "10.255.32.0/19", "size": 25},
  {"base" = "10.255.64.0/18", "size": 25},
  {"base" = "10.255.128.0/17", "size": 25},
]

size specifies the default network size for each docker network. /25 contains 128 addresses, so adjust based on your needs.

Kubernetes

This is highly dependent on which provider you are using. k3s uses docker, so use the Docker instructions.

RKE2

This should be done when creating the cluster, it isn’t well supported to change this after creation

/etc/rancher/rke2/config.yaml

cluster-cidr: 10.255.0.0/17
service-cidr: 10.255.128.0/17

LXC

/etc/default/lxc-net

You could do a smaller range if you wanted to as well

FAST Knowledgebase

Preventing Network Collisions Between Campus Infrastructure and Docker, Podman, Kubernetes, LXC, etc