Promtail leaves too many files open

I've run into the following problem:

# kubectl logs -n oas-apps nc-nextcloud-7d4b68744b-vkcpc -f
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 10.42.0.72. Set the 'ServerName' directive globally to suppress this message
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 10.42.0.72. Set the 'ServerName' directive globally to suppress this message
[Wed May 06 12:52:40.618714 2020] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.38 (Debian) PHP/7.3.11 configured -- resuming normal operations
[Wed May 06 12:52:40.618734 2020] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'
213.108.108.186 - - [06/May/2020:12:52:40 +0000] "GET /status.php HTTP/1.1" 400 1449 "-" "kube-probe/1.15"
failed to create fsnotify watcher: too many open filesroot

I think this is because of promtail, because when I run lsof, I get 213262 lines of this:

promtail   2134  2324 promtail              root  125u  a_inode               0,10         0       1027 [eventpoll]
promtail   2134  2324 promtail              root  126r     FIFO                0,9       0t0    1945396 pipe
promtail   2134  2324 promtail              root  127w     FIFO                0,9       0t0    1945396 pipe
promtail   2134  2324 promtail              root  128r      REG              202,1       934   44400671 /var/lib/docker/containers/dc0aaf72f5b1f5741392c2944c3da97f00088694c3c215801b59c8b1f7a7e8b4/dc0aaf72f5b1f5741392c2944c3da97f00088694c3c215801b59c8b1f7a7e8b4-json.log
promtail   2134  2324 promtail              root  129r  a_inode               0,10         0       1027 inotify
promtail   2134  2324 promtail              root  130u  a_inode               0,10         0       1027 [eventpoll]
promtail   2134  2324 promtail              root  131r     FIFO                0,9       0t0    1945398 pipe
promtail   2134  2324 promtail              root  132w     FIFO                0,9       0t0    1945398 pipe
promtail   2134  2324 promtail              root  133r  a_inode               0,10         0       1027 inotify
promtail   2134  2324 promtail              root  134u  a_inode               0,10         0       1027 [eventpoll]
promtail   2134  2324 promtail              root  135r     FIFO                0,9       0t0    1945399 pipe
promtail   2134  2324 promtail              root  136w     FIFO                0,9       0t0    1945399 pipe
promtail   2134  2324 promtail              root  137r  a_inode               0,10         0       1027 inotify
promtail   2134  2324 promtail              root  138u  a_inode               0,10         0       1027 [eventpoll]
promtail   2134  2324 promtail              root  139r     FIFO                0,9       0t0    1945400 pipe
promtail   2134  2324 promtail              root  140w     FIFO                0,9       0t0    1945400 pipe
promtail   2134  2324 promtail              root  141r  a_inode               0,10         0       1027 inotify
promtail   2134  2324 promtail              root  142u  a_inode               0,10         0       1027 [eventpoll]
promtail   2134  2324 promtail              root  143r     FIFO                0,9       0t0    1945401 pipe
promtail   2134  2324 promtail              root  144w     FIFO                0,9       0t0    1945401 pipe
promtail   2134  2324 promtail              root  145r  a_inode               0,10         0       1027 inotify
promtail   2134  2324 promtail              root  146r      REG              202,1     39976   55287837 /var/lib/docker/containers/6ac90e37df3f3fff1818ceb2b2b0d22195e327832848dcf8068a65bc55a99481/6ac90e37df3f3fff1818ceb2b2b0d22195e327832848dcf8068a65bc55a99481-json.log
promtail   2134  2324 promtail              root  147u  a_inode               0,10         0       1027 [eventpoll]
promtail   2134  2324 promtail              root  148r     FIFO                0,9       0t0    1945402 pipe
promtail   2134  2324 promtail              root  149w     FIFO                0,9       0t0    1945402 pipe
promtail   2134  2324 promtail              root  150r      REG              202,1      5793   34152535 /var/lib/docker/containers/7b9d9d5585e3f90e4a21353a515f3ec62bb7fddebe5dae5a0cfe5ad312984123/7b9d9d5585e3f90e4a21353a515f3ec62bb7fddebe5dae5a0cfe5ad312984123-json.log
promtail   2134  2324 promtail              root  151r      REG              202,1     11065   31015063 /var/lib/docker/containers/8580838d439c66f99c65d6086b6220e2e82bd3033bc002d1716b9e55fd63f7ad/8580838d439c66f99c65d6086b6220e2e82bd3033bc002d1716b9e55fd63f7ad-json.log
promtail   2134  2324 promtail              root  152r      REG              202,1      3364   38215733 /var/lib/docker/containers/9fcf5355d8b3fbda098117191070cb85bf389e79979ab8507eaa80d1494494c3/9fcf5355d8b3fbda098117191070cb85bf389e79979ab8507eaa80d1494494c3-json.log
promtail   2134  2324 promtail              root  153r      REG              202,1      3131   51249174 /var/lib/docker/containers/7080569737b66342a38e68983d450af4f00d22c2b43376876c1a04d9d1d97d23/7080569737b66342a38e68983d450af4f00d22c2b43376876c1a04d9d1d97d23-json.log
promtail   2134  2324 promtail              root  154r  a_inode               0,10         0       1027 inotify
promtail   2134  2324 promtail              root  155u  a_inode               0,10         0       1027 [eventpoll]
promtail   2134  2324 promtail              root  156r     FIFO                0,9       0t0    1945403 pipe
promtail   2134  2324 promtail              root  157w     FIFO                0,9       0t0    1945403 pipe
promtail   2134  2324 promtail              root  158r      REG              202,1      1039   34136072 /var/lib/docker/containers/8001c881f4ac438ab861a602a77680a36d6a68aee72a8e023903c4afb4f3a14b/8001c881f4ac438ab861a602a77680a36d6a68aee72a8e023903c4afb4f3a14b-json.log

I have a hunch this might worsen if you reinstall pods a few times. I also have a hunch that this can cause other processes than kubectl logs -f to fail as well, because my cluster isn't working well at the moment...