@dajudge can you squash this to single commit and rebase with master branch?

I did quick testing with this and it looks to be effective for all new networks, including ingress and all custom overlay networks as long it is set to manager node which means that it would help also with this case moby/libnetwork#2661 Will do testing with Windows also after rebase is done (now I was not able to build on Windows).

@thaJeztah @cpuguy83 is this something which can be considered to be included to Moby and maybe also backported to 22.06 ? It is kind of breaking change but because most of the environments are anyway using only default MTU and those who changes it will do it on purpose it would make sense to simplify their life with this.

Hi @olljanat - thanks for your feedback!

If the potentially breaking nature of the patch is a problem it'd also be possible to introduce a new cmdline arg instead of reusing --mtu - just let me know what makes more sense.

I'll gladly take care of rebasing in the next few days if that helps getting this PR merged! 👍

can you squash this to single commit and rebase with master branch?

@olljanat done 👍

I'd love to get some feedback on this PR from you folks (@olljanat, @cpuguy83, @thaJeztah) to learn what could help bringing it forward.

Do you maybe have some ideas regarding tests that could be added? What do you think about the overall concept of introducing a new cmdline flag for this?

I can also help backporting if it is considered valuable to have it in 22.06 as @olljanat suggested.

Your input is greatly appreciated!

@dajudge purpose of tests is proof that feature works as expected and more importantly make sure that it does not go broken as part of future code changes. Because there have been already that earlier MTU setting there most probably is some tests which can be extended or alternatively you can write new one(s). Looks folder integration about how to build those and maybe create something which run dockerd with this settings enabled, create custom bridge network, start container and check inside of it that MTU value is correct.

New flag seems ok to me.
Marking this for code review.

Just need a test and I think we can move forward.

Added test case to verify behavior in the following cases:

• Defaults to MTU=1500 when no cmdline args are passed
• Applies MTU when provided via cmdline

Each case is verified by creating a new network, starting a container attached to that network and then checking the MTU of the eth0 interface in the container.

Not to be annoying but is there any plans of integrating this feature yet? Id love to use it!

Resolved some merge conflicts.

uuuuhm

Thanks for your feedback, @cpuguy83! Please let me know if there's something I can do to help move this forward!

+1 on this. On our cloud provider when using Ubuntu 22.04 something has changed that caused the default mtu to break overlay networks. For now we can live with setting the mtu in our stack files but a default would help newcomers.

in my setup i have to put

networks:
  default:
    driver_opts:
      com.docker.network.driver.mtu: 1420

into every single service im hosting.
It's not only incredibly annoying but also not portable at all.

can we please have this feature merged?
ill even take it as a late christmas gift so you dont have to get me anything

Thinking about this a little bit:

1. If we pass a default value to drivers, I think the option should be driver specific (so something like --network-default-mtu=<driver>=<value>.
2. It might be nice to introduce a new standard option for max MTU which the driver itself can then go and subtract it's own overhead from. This allows the driver to do its own default rather than pushing a default config down from docker.

This is said with the assumption that drivers are responsible for setting MTU and not libnetwork... I haven't taken the time to dig into the code and am currently on PTO.

So yes, as suspected this should all be handled by the driver so we either need to namespace the options for each driver or pass down an option to drivers for max MTU.

Max MTU is of course a pretty big change since every driver would need to be changed to support it.

Hi @cpuguy83,

many thanks for your input!

Disclaimer: I'm far from being an expert on the nitty gritty networking layer details that are involved here, so please bear with me if this idea would not work for some reason I'm not taking into account.

What about leaving the parameter as-is but taking a driver-specific overhead into account before applying it via

genericData[netlabel.DriverMTU] = fmt.Sprintf("%d", c.cfg.DefaultMTU)

My thinking here is:

genericData[netlabel.DriverMTU] = fmt.Sprintf("%d", c.cfg.DefaultMTU - cap.PacketOverhead)

WDYT? Would it make sense to go down such a road?

cc @corhere

@dajudge So you are saying to keep a global default and then a new driver capability to return packet overhead?
Perhaps that could work. We could just take this change in as is.

The drivers are responsible for setting MTU. And the only drivers today which do so are bridge and overlay. I much prefer the idea of introducing a new "Max MTU" option for drivers to subtract their overhead from over the capability idea. The packet overhead might not necessarily be a per-driver constant: per-network options could influence the overhead, for instance.

As for this PR, I have to wonder whether we should be adding a special case daemon config option just for one particular network config option. What about com.docker.network.enable_ipv6, com.docker.network.container_iface_prefix, etc.? Should we have a bespoke daemon flag for every one? I'd be more inclined to have the daemon configuration be more open-ended by allowing defaults to be set for any per-network option. E.g.:

dockerd \
  --default-network-opt=overlay=com.docker.network.driver.mtu=1420 \
  --default-network-opt=*=com.docker.network.container_iface_prefix=foo-

{
  "default-network-opts": {
    "*": {"com.docker.network.container_iface_prefix": "foo-"},
    "overlay": {"com.docker.network.driver.mtu": "1420"},
    "bridge": {"com.docker.network.driver.mtu": "1450"}
  }
}

Is there any provision for using this option through daemon.json? "default-network-opt": "bridge=com.docker.network.driver.mtu=1234" doesn't do anything, and "default-network-opt": {"bridge": {"com.docker.network.driver.mtu": "1234"}} results in an unknown configuration option error.

@richardg867 what version of docker are you running?

@richardg867 the daemon.json option is spelled "default-network-opts" – note the plural.

Good call; also just realized we need to update the docs in https://github.com/docker/cli/blob/master/docs/reference/commandline/dockerd.md

@richardg867 what version of docker are you running?

24.0.0-beta.1

@richardg867 the daemon.json option is spelled "default-network-opts" – note the plural.

After correcting the name, it now works for bridge networks, but overlay networks cause an unknown option error...

# cat daemon.json
{"default-network-opts": {"bridge": {"com.docker.network.driver.mtu": "1434"}}}

# journalctl -xe
[...] Started Docker Application Container Engine. [...]

# cat daemon.json
{"default-network-opts": {"overlay": {"com.docker.network.driver.mtu": "1434"}}}

# journalctl -xe
[...] unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives don't match any configuration option: overlay [...]

...even though they do work if I specify the option on the dockerd command line instead of daemon.json.

# cat daemon.json
{}

# grep ExecStart /lib/systemd/system/docker.service
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --default-network-opt=overlay=com.docker.network.driver.mtu=1434

# docker network create --driver overlay testnet
3652a8e237da365c74df56e7f60a79b1aff4415216dd886b5bd9e61842d01fa1

# docker network inspect 3652a8e237da365c74df56e7f60a79b1aff4415216dd886b5bd9e61842d01fa1
[...]
        "Options": {
            "com.docker.network.driver.mtu": "1434"
        },
[...]

Just found another interesting issue: default options are not applied to networks created in swarm mode through the command line. They are applied to the ingress network on swarm init (EDIT:) and any networks created through docker stack deploy, though.

# docker network create --driver overlay test_net_please_ignore
bfbba07887feb61b818c67a0ad7f3c481a0991ed8bf722f0cfea33da6976c6dc

# docker network inspect bfbba07887feb61b818c67a0ad7f3c481a0991ed8bf722f0cfea33da6976c6dc
[...]
        "Options": {
            "com.docker.network.driver.mtu": "1434"
        },
[...]

# docker network rm bfbba07887feb61b818c67a0ad7f3c481a0991ed8bf722f0cfea33da6976c6dc
bfbba07887feb61b818c67a0ad7f3c481a0991ed8bf722f0cfea33da6976c6dc

# docker swarm init
Swarm initialized: current node [...]

# docker network inspect ingress
[...]
        "Options": {
            "com.docker.network.driver.mtu": "1434",
            "com.docker.network.driver.overlay.vxlanid_list": "4096"
        },
[...]

# docker network create --driver overlay test_net_please_ignore
hjch5eyuxfspk1drvtgnnkz9r

# docker network inspect hjch5eyuxfspk1drvtgnnkz9r
[...]
        "Options": {
            "com.docker.network.driver.overlay.vxlanid_list": "4097"
        },
[...]

@richardg867 try inspecting the network while a container is attached to it.

Thanks very much for this patch - we use OpenStack so it's very useful.

Question: can it be expressed as a daemon.json config? I tried, but whilst this works:

/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --default-network-opt=bridge=com.docker.network.driver.mtu=1442

This is accepted as valid syntax in daemon.json, but does not change the MTU (stays at default 1500 on creating user bridges):

"default-network-opt": 
{
  "bridge": 
    {
      "com.docker.network.driver.mtu": 1442
    }
},

Yes, I can patch a config change into systemd for the docker service, but it would be nice if all I have is an error in my daemon.json which I could fix :)

Many thanks, Tim

If the bridge already exists it doesn't change anything.
You'll have to stop docker, delete the bridge, then docker will recreate it.

If the bridge already exists it doesn't change anything. You'll have to stop docker, delete the bridge, then docker will recreate it.

Hi, Thanks for taking the time to reply - I already did that. I've tried it both ways:

When I put --default-network-opt=bridge=com.docker.network.driver.mtu=1442 into the systemd docker.service file, reload systemd and restart docker, bringing up containers does the right thing - new bridges are configured with MTU 1442.

When I remove that option from systemd, reload systemd, add the json config above into /etc/docker/daemon.json and restart docker: The daemon restarts without error (which it doesn't do if I make a typo in that particular bit of the config) - But it seems to ignore the setting.

I assumed that all dockerd command line options will map to daemon.json settings?

• So either they don't, in which case I will have to patch the systemd service config;
• Or they do, but I have not expressed the map correctly???

OK - Got it!

There is a disparity in the daemon.json and the command line options:

This works:

"default-network-opts": { "bridge": { "com.docker.network.driver.mtu": "1442" } },

Notice the extra s on -opts.

I found this out with a dive into the source code:

config.go: flags.Var(opts.NewNamedMapMapOpts("default-network-opts", conf.DefaultNetworkOpts, nil), "default-network-opt", "Default network options")

and

config_linux_test.go:

"default-network-opts": {
    "overlay": {
      "com.docker.network.driver.mtu": "1337"
    }
  }

It's working (not using the expected config, but working as I wanted).

The plural has always been how these types of options (that can be specified multiple times on the command line) have worked; cc @dvdksn to see if we can call this out louder for the daemon.json docs.

The plural has always been how these types of options (that can be specified multiple times on the command line) have worked; cc @dvdksn to see if we can call this out louder for the daemon.json docs.

Maybe it's me, but I really struggled finding good daemon.json docs and examples.

It was actually the example in the config_linux_test.go that alerted me.

Looks like docs were added in this PR;

• docs: add default-network-opt daemon option docker/cli#4635

(published at https://docs.docker.com/reference/cli/dockerd/#default-network-options)

so removing the docs/revisit label