Skip to content

Too many open files since containerd 2.0 #847

@cziebuhr

Description

@cziebuhr

Containerd 2.0 removed LimitNOFILE=infinity from its service file, resulting in allowing only 1024 open files in the docker container. See containerd/containerd#8924

This leeds to msg="Error writing snapshot from compactor" engine=tsm1 op_name=tsm1_cache_snapshot error="compaction in progress: open /var/lib/influxdb/data/db/retention/333/002671920-000000001.tsm.tmp: too many open files". This then triggers influxdata/influxdb#26073, when using a not-uptodate version, resulting in filling up the disk.

The non-docker versions set LimitNOFILE=65536 in its service file (respectively ulimit -n 65536 in init.sh). I think this is also needed in Dockerfile.

The issue has been seen with influxdb:1.8.10-alpine, but probably applies to all versions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions