Uncontrolled filling of logfiles

Raspberry Pi Network HATs 3G/4G&LTE Base HAT
#1

Running everything just fine on a Raspberry Pi v4, sometimes when the cell service becomes weak, the QMI process spams the /var/log/syslog.log and the /var/log/daemon.log with the following message:

Jun 13 06:24:37 raspberrypi sh[576]: [06-13_06:24:37:081] QmiWwanThread poll err/hup/inval
Jun 13 06:24:37 raspberrypi sh[576]: [06-13_06:24:37:081] poll fd = 7, events = 0x0008

This happens endlessly until the log files have filled the disk space (10’s of GB), which then cause all sorts of other problems including limited connectivity.

  1. Is there any way to fix this?
  2. If not, is there a way to disable logging?
  3. If not, is there a proper way to configure logrotate so it catches this quickly and dumps the logs?
#2

I’m still looking for an answer to this problem. I’ve disabled all logging in order to prevent disk filling and installed watchdog to reboot the system when the network goes down.

Would prefer to be able to have some logging. :thinking:

#3

  1. You could tweak your logrotate settings to rotate more often and delete files older than specified amount of days…
    Have a look at /etc/logrotate.conf and /etc/logrotate.d/rsyslog

  2. You could also modify the service file

/etc/systemd/system/qmi_reconnect.service

And send standard error and/or standard output to where nobody cares about it:

StandardOutput=null
StandardError=null

The spamming probably comes from StandardError so just sending that to null might suffice.

Note: make sure you do a…

systemctl daemon-reload

After you have modified service files.

#4

I am having the exact same issue and cannot find /etc/systemd/qmi_reconnect.service. Was stephen.smith able to fix this?

#5

@harri.jokipera I disabled a lot of logging to make it work. I haven’t spent the time yet to actually fix this problem so that I can have logging back. @sebstyle.nl’s solution works. Make sure you edit

/etc/systemd/system/qmi_reconnect.service

You left out a /system in your question.

#6

@harri.jokipera did you manage to find the root reason for this issue ?
i got the same error message as you discribed and i have no clue what is causing it !?

#7

Unfortunately I didn’t find the root cause. I circumvented the problem with not logging that line. That helped but I have no idea what the underlying issue is.

#8

@k.omar At the very least, here’s all of my observations about the problem:

  • It seems independent of cell carrier as I’ve seen it on ATT, Verizon, and T-Mobile in the US
  • It usually only occurs when the cell reception is weak
  • It occurs independent of platform as I’ve seen this on NVIDIA’s jetson products, RPI 3, RPI 4, and Hardkernel’s Odroid N2 (however, RPI is the only system that freezes up, the rest somehow keep working with some operations limited due to disk space)
  • Quectel doesn’t know what this is either as I’ve posted in their forums, which is surprising given that it’s their code I believe
#9

@stephen.smith thanks for your reply ,
to be honest i dont think its linked to the carrier , i have the same situation on diffrent carriers at the same location (Telekom.de Vodafone O2) , and the Quectel is able to reconnect after a power cycle ,
i tried smalll scripts to catch this error and try to force to regist to the network again , but this didnt work as well .
the question that i dont have answer for is that happened during a power cycle that fix this issue and make it work again !?

#10

I have had this same problem. Just replying to comment that I tried the solution from @sebstyle.nl and I found that when only setting StandardError to null that the problem still occurred. So looks like it is necessary to set StandardOutput to null as well.