Issue
Clamd start-up times out
Since Nov 29th clamd does not startup anymore, it takes too long to read the virus database and is therefore terminated by systemd:
In clamd.log I see that the time needed for loading the virus database incresed from 40 seconds on Nov 28th to 1min 40 seconds (when I start clamd from command line).
Systemd timeout is 90s, thats why the service is terminated and does not come up anymore. I deleted and freshly updated the virus databases, but this didn't change anything.
Any ideas what has changed on Nov. 29th causing the problem? I have two systems showing the same behaviour (both are Zotac ZBOX CI323 nano running ClearOS 7.4)
Nov 29 01:35:19 gateway systemd: Starting ClamAV daemon...
Nov 29 01:36:49 gateway systemd: clamd.service start operation timed out. Terminating.
Nov 29 01:36:50 gateway systemd: Failed to start ClamAV daemon.
Nov 29 01:36:50 gateway systemd: Unit clamd.service entered failed state.
Nov 29 01:36:50 gateway systemd: clamd.service failed.
In clamd.log I see that the time needed for loading the virus database incresed from 40 seconds on Nov 28th to 1min 40 seconds (when I start clamd from command line).
Systemd timeout is 90s, thats why the service is terminated and does not come up anymore. I deleted and freshly updated the virus databases, but this didn't change anything.
Any ideas what has changed on Nov. 29th causing the problem? I have two systems showing the same behaviour (both are Zotac ZBOX CI323 nano running ClearOS 7.4)
Share this post:
Accepted Answer
So it is looking like any clearsdn-antimalware since 29/11 is possibly causing the issue. I'll ask the dev's but I may not get to talk to them until the end of the week. I have no idea what the update contains.
Have you tried something like adding:
Have you tried something like adding:
TimeoutSec=5min
to the [Service] section in /usr/lib/systemd/system/clamd.service? I don't think the 5min needs to be quoted and produced no errors when I tried it. Then run:systemctl daemon-reload
systemctl restart clamd.service
Responses (9)
-
Accepted Answer
I filed a tracker item, 18081 and the devs have responded. It looks like the workround you implemented is the best for the moment. -
Accepted Answer
-
Accepted Answer
-
Accepted Answer
I uninstalled a bunch of apps: app-file-scan, app-antimalware, app-antiphishing, app-antivirus, app-content-filter,app-mail-antivirus, clamav*, clearsdn-antimalware, clearsdn-antispam, dansguardian*, and web-proxy I think. I also removed all remaining clam* and dansguardian* folders and config files.
Unfortunately the problem is back again, after the update of clearsdn-antimalware today night at 05:15:39.
There have been two updates tonight, clearsdn-antispam at 3:55am and clearsdn-antimalware at 05:15. After clearsdn-antispam update everything was still ok, but after antimalware clamd service continously timed out. top shows clamd running with 100%, but I think this is because systemd tries to start it up after the service startup timed out.
clamd.log from tonight:
Sat Dec 2 18:32:27 2017 -> +++ Started at Sat Dec 2 18:32:27 2017
Sat Dec 2 18:32:27 2017 -> Received 0 file descriptor(s) from systemd.
Sat Dec 2 18:32:27 2017 -> clamd daemon 0.99.2 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64)
Sat Dec 2 18:32:27 2017 -> Running as user clam (UID 990, GID 988)
Sat Dec 2 18:32:27 2017 -> Log file size limited to 4294967295 bytes.
Sat Dec 2 18:32:27 2017 -> Reading databases from /var/lib/clamav
Sat Dec 2 18:32:27 2017 -> Not loading PUA signatures.
Sat Dec 2 18:32:27 2017 -> Bytecode: Security mode set to "TrustSigned".
Sat Dec 2 18:32:51 2017 -> Loaded 6355499 signatures.
Sat Dec 2 18:32:55 2017 -> TCP: Bound to [127.0.0.1]:3310
Sat Dec 2 18:32:55 2017 -> TCP: Setting connection queue length to 30
Sat Dec 2 18:32:55 2017 -> LOCAL: Unix socket file /var/run/clamav/clamd.sock
Sat Dec 2 18:32:55 2017 -> LOCAL: Setting connection queue length to 30
Sat Dec 2 18:32:55 2017 -> Limits: Global size limit set to 104857600 bytes.
Sat Dec 2 18:32:55 2017 -> Limits: File size limit set to 104857600 bytes.
Sat Dec 2 18:32:55 2017 -> Limits: Recursion level limit set to 10.
Sat Dec 2 18:32:55 2017 -> Limits: Files limit set to 10000.
Sat Dec 2 18:32:55 2017 -> Limits: MaxEmbeddedPE limit set to 10485760 bytes.
Sat Dec 2 18:32:55 2017 -> Limits: MaxHTMLNormalize limit set to 10485760 bytes.
Sat Dec 2 18:32:55 2017 -> Limits: MaxHTMLNoTags limit set to 2097152 bytes.
Sat Dec 2 18:32:55 2017 -> Limits: MaxScriptNormalize limit set to 5242880 bytes.
Sat Dec 2 18:32:55 2017 -> Limits: MaxZipTypeRcg limit set to 1048576 bytes.
Sat Dec 2 18:32:55 2017 -> Limits: MaxPartitions limit set to 50.
Sat Dec 2 18:32:55 2017 -> Limits: MaxIconsPE limit set to 100.
Sat Dec 2 18:32:55 2017 -> Limits: MaxRecHWP3 limit set to 16.
Sat Dec 2 18:32:55 2017 -> Limits: PCREMatchLimit limit set to 10000.
Sat Dec 2 18:32:55 2017 -> Limits: PCRERecMatchLimit limit set to 5000.
Sat Dec 2 18:32:55 2017 -> Limits: PCREMaxFileSize limit set to 26214400.
Sat Dec 2 18:32:55 2017 -> Archive support enabled.
Sat Dec 2 18:32:55 2017 -> Archive: Blocking encrypted archives.
Sat Dec 2 18:32:55 2017 -> Algorithmic detection enabled.
Sat Dec 2 18:32:55 2017 -> Portable Executable support enabled.
Sat Dec 2 18:32:55 2017 -> ELF support enabled.
Sat Dec 2 18:32:55 2017 -> Detection of broken executables enabled.
Sat Dec 2 18:32:55 2017 -> Mail files support enabled.
Sat Dec 2 18:32:55 2017 -> OLE2 support enabled.
Sat Dec 2 18:32:55 2017 -> PDF support enabled.
Sat Dec 2 18:32:55 2017 -> SWF support enabled.
Sat Dec 2 18:32:55 2017 -> HTML support enabled.
Sat Dec 2 18:32:55 2017 -> XMLDOCS support enabled.
Sat Dec 2 18:32:55 2017 -> HWP3 support enabled.
Sat Dec 2 18:32:55 2017 -> Phishing: Always checking for cloaked urls
Sat Dec 2 18:32:55 2017 -> Phishing: Always checking for ssl mismatches
Sat Dec 2 18:32:55 2017 -> Self checking every 600 seconds.
Sat Dec 2 18:42:55 2017 -> SelfCheck: Database status OK.
Sat Dec 2 18:52:55 2017 -> SelfCheck: Database status OK.
Sat Dec 2 19:02:55 2017 -> SelfCheck: Database modification detected. Forcing reload.
Sat Dec 2 19:02:57 2017 -> Reading databases from /var/lib/clamav
Sat Dec 2 19:03:21 2017 -> Database correctly reloaded (6355620 signatures)
Sat Dec 2 19:13:21 2017 -> SelfCheck: Database status OK.
Sat Dec 2 19:23:21 2017 -> SelfCheck: Database status OK.
Sat Dec 2 19:33:21 2017 -> SelfCheck: Database status OK.
Sat Dec 2 19:43:21 2017 -> SelfCheck: Database status OK.
Sat Dec 2 19:53:21 2017 -> SelfCheck: Database status OK.
Sat Dec 2 20:03:21 2017 -> SelfCheck: Database status OK.
Sat Dec 2 20:13:22 2017 -> SelfCheck: Database status OK.
Sat Dec 2 20:23:22 2017 -> SelfCheck: Database status OK.
Sat Dec 2 20:33:22 2017 -> SelfCheck: Database status OK.
Sat Dec 2 20:43:22 2017 -> SelfCheck: Database status OK.
Sat Dec 2 20:53:22 2017 -> SelfCheck: Database status OK.
Sat Dec 2 21:03:22 2017 -> SelfCheck: Database status OK.
Sat Dec 2 21:13:22 2017 -> SelfCheck: Database status OK.
Sat Dec 2 21:23:22 2017 -> SelfCheck: Database status OK.
Sat Dec 2 21:33:22 2017 -> SelfCheck: Database status OK.
Sat Dec 2 21:43:22 2017 -> SelfCheck: Database status OK.
Sat Dec 2 21:53:22 2017 -> SelfCheck: Database status OK.
Sat Dec 2 22:03:22 2017 -> SelfCheck: Database status OK.
Sat Dec 2 22:13:23 2017 -> SelfCheck: Database status OK.
Sat Dec 2 22:23:23 2017 -> SelfCheck: Database status OK.
Sat Dec 2 22:33:23 2017 -> SelfCheck: Database status OK.
Sat Dec 2 22:43:23 2017 -> SelfCheck: Database status OK.
Sat Dec 2 22:53:23 2017 -> SelfCheck: Database status OK.
Sat Dec 2 23:03:23 2017 -> SelfCheck: Database status OK.
Sat Dec 2 23:13:23 2017 -> SelfCheck: Database status OK.
Sat Dec 2 23:23:23 2017 -> SelfCheck: Database status OK.
Sat Dec 2 23:33:23 2017 -> SelfCheck: Database status OK.
Sat Dec 2 23:43:23 2017 -> SelfCheck: Database status OK.
Sat Dec 2 23:53:23 2017 -> SelfCheck: Database status OK.
Sun Dec 3 00:03:23 2017 -> SelfCheck: Database status OK.
Sun Dec 3 00:13:24 2017 -> SelfCheck: Database status OK.
Sun Dec 3 00:23:24 2017 -> SelfCheck: Database status OK.
Sun Dec 3 00:33:24 2017 -> SelfCheck: Database status OK.
Sun Dec 3 00:43:24 2017 -> SelfCheck: Database status OK.
Sun Dec 3 00:53:24 2017 -> SelfCheck: Database status OK.
Sun Dec 3 01:03:24 2017 -> SelfCheck: Database status OK.
Sun Dec 3 01:13:24 2017 -> SelfCheck: Database status OK.
Sun Dec 3 01:23:24 2017 -> SelfCheck: Database status OK.
Sun Dec 3 01:33:24 2017 -> SelfCheck: Database status OK.
Sun Dec 3 01:43:24 2017 -> SelfCheck: Database status OK.
Sun Dec 3 01:53:24 2017 -> SelfCheck: Database status OK.
Sun Dec 3 02:03:24 2017 -> SelfCheck: Database status OK.
Sun Dec 3 02:13:24 2017 -> SelfCheck: Database status OK.
Sun Dec 3 02:23:24 2017 -> SelfCheck: Database status OK.
Sun Dec 3 02:33:25 2017 -> SelfCheck: Database status OK.
Sun Dec 3 02:43:25 2017 -> SelfCheck: Database status OK.
Sun Dec 3 02:53:25 2017 -> SelfCheck: Database status OK.
Sun Dec 3 03:03:25 2017 -> SelfCheck: Database modification detected. Forcing reload.
Sun Dec 3 03:03:26 2017 -> Reading databases from /var/lib/clamav
Sun Dec 3 03:03:49 2017 -> Database correctly reloaded (6355730 signatures)
Sun Dec 3 03:13:49 2017 -> SelfCheck: Database status OK.
Sun Dec 3 03:23:49 2017 -> SelfCheck: Database status OK.
Sun Dec 3 03:33:49 2017 -> SelfCheck: Database status OK.
Sun Dec 3 03:43:49 2017 -> SelfCheck: Database status OK.
Sun Dec 3 03:53:49 2017 -> SelfCheck: Database status OK.
Sun Dec 3 03:56:10 2017 -> Pid file removed.
Sun Dec 3 03:56:10 2017 -> --- Stopped at Sun Dec 3 03:56:10 2017
Sun Dec 3 03:56:10 2017 -> Socket file removed.
Sun Dec 3 03:56:10 2017 -> +++ Started at Sun Dec 3 03:56:10 2017
Sun Dec 3 03:56:10 2017 -> Received 0 file descriptor(s) from systemd.
Sun Dec 3 03:56:10 2017 -> clamd daemon 0.99.2 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64)
Sun Dec 3 03:56:10 2017 -> Running as user clam (UID 990, GID 988)
Sun Dec 3 03:56:10 2017 -> Log file size limited to 4294967295 bytes.
Sun Dec 3 03:56:10 2017 -> Reading databases from /var/lib/clamav
Sun Dec 3 03:56:10 2017 -> Not loading PUA signatures.
Sun Dec 3 03:56:10 2017 -> Bytecode: Security mode set to "TrustSigned".
Sun Dec 3 03:56:31 2017 -> Loaded 6440052 signatures.
Sun Dec 3 03:56:34 2017 -> TCP: Bound to [127.0.0.1]:3310
Sun Dec 3 03:56:34 2017 -> TCP: Setting connection queue length to 30
Sun Dec 3 03:56:34 2017 -> LOCAL: Unix socket file /var/run/clamav/clamd.sock
Sun Dec 3 03:56:34 2017 -> LOCAL: Setting connection queue length to 30
Sun Dec 3 03:56:34 2017 -> Limits: Global size limit set to 104857600 bytes.
Sun Dec 3 03:56:34 2017 -> Limits: File size limit set to 104857600 bytes.
Sun Dec 3 03:56:34 2017 -> Limits: Recursion level limit set to 10.
Sun Dec 3 03:56:34 2017 -> Limits: Files limit set to 10000.
Sun Dec 3 03:56:34 2017 -> Limits: MaxEmbeddedPE limit set to 10485760 bytes.
Sun Dec 3 03:56:34 2017 -> Limits: MaxHTMLNormalize limit set to 10485760 bytes.
Sun Dec 3 03:56:34 2017 -> Limits: MaxHTMLNoTags limit set to 2097152 bytes.
Sun Dec 3 03:56:34 2017 -> Limits: MaxScriptNormalize limit set to 5242880 bytes.
Sun Dec 3 03:56:34 2017 -> Limits: MaxZipTypeRcg limit set to 1048576 bytes.
Sun Dec 3 03:56:34 2017 -> Limits: MaxPartitions limit set to 50.
Sun Dec 3 03:56:34 2017 -> Limits: MaxIconsPE limit set to 100.
Sun Dec 3 03:56:34 2017 -> Limits: MaxRecHWP3 limit set to 16.
Sun Dec 3 03:56:34 2017 -> Limits: PCREMatchLimit limit set to 10000.
Sun Dec 3 03:56:34 2017 -> Limits: PCRERecMatchLimit limit set to 5000.
Sun Dec 3 03:56:34 2017 -> Limits: PCREMaxFileSize limit set to 26214400.
Sun Dec 3 03:56:34 2017 -> Archive support enabled.
Sun Dec 3 03:56:34 2017 -> Archive: Blocking encrypted archives.
Sun Dec 3 03:56:34 2017 -> Algorithmic detection enabled.
Sun Dec 3 03:56:34 2017 -> Portable Executable support enabled.
Sun Dec 3 03:56:34 2017 -> ELF support enabled.
Sun Dec 3 03:56:34 2017 -> Detection of broken executables enabled.
Sun Dec 3 03:56:34 2017 -> Mail files support enabled.
Sun Dec 3 03:56:34 2017 -> OLE2 support enabled.
Sun Dec 3 03:56:34 2017 -> PDF support enabled.
Sun Dec 3 03:56:34 2017 -> SWF support enabled.
Sun Dec 3 03:56:34 2017 -> HTML support enabled.
Sun Dec 3 03:56:34 2017 -> XMLDOCS support enabled.
Sun Dec 3 03:56:34 2017 -> HWP3 support enabled.
Sun Dec 3 03:56:34 2017 -> Phishing: Always checking for cloaked urls
Sun Dec 3 03:56:34 2017 -> Phishing: Always checking for ssl mismatches
Sun Dec 3 03:56:34 2017 -> Self checking every 600 seconds.
Sun Dec 3 04:06:34 2017 -> SelfCheck: Database status OK.
Sun Dec 3 04:16:34 2017 -> SelfCheck: Database status OK.
Sun Dec 3 04:28:33 2017 -> SelfCheck: Database status OK.
Sun Dec 3 04:38:33 2017 -> SelfCheck: Database status OK.
Sun Dec 3 04:48:33 2017 -> SelfCheck: Database status OK.
Sun Dec 3 04:58:33 2017 -> SelfCheck: Database status OK.
Sun Dec 3 05:08:34 2017 -> SelfCheck: Database status OK.
Sun Dec 3 05:17:18 2017 -> Pid file removed.
Sun Dec 3 05:17:18 2017 -> --- Stopped at Sun Dec 3 05:17:18 2017
Sun Dec 3 05:17:18 2017 -> Socket file removed.
Sun Dec 3 05:17:18 2017 -> +++ Started at Sun Dec 3 05:17:18 2017
Sun Dec 3 05:17:18 2017 -> Received 0 file descriptor(s) from systemd.
Sun Dec 3 05:17:18 2017 -> clamd daemon 0.99.2 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64)
Sun Dec 3 05:17:18 2017 -> Running as user clam (UID 990, GID 988)
Sun Dec 3 05:17:18 2017 -> Log file size limited to 4294967295 bytes.
Sun Dec 3 05:17:18 2017 -> Reading databases from /var/lib/clamav
Sun Dec 3 05:17:18 2017 -> Not loading PUA signatures.
Sun Dec 3 05:17:18 2017 -> Bytecode: Security mode set to "TrustSigned".
Sun Dec 3 05:18:49 2017 -> +++ Started at Sun Dec 3 05:18:49 2017
Sun Dec 3 05:18:49 2017 -> Received 0 file descriptor(s) from systemd.
Sun Dec 3 05:18:49 2017 -> clamd daemon 0.99.2 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64)
Sun Dec 3 05:18:49 2017 -> Running as user clam (UID 990, GID 988)
Sun Dec 3 05:18:49 2017 -> Log file size limited to 4294967295 bytes.
Sun Dec 3 05:18:49 2017 -> Reading databases from /var/lib/clamav
Sun Dec 3 05:18:49 2017 -> Not loading PUA signatures.
Sun Dec 3 05:18:49 2017 -> Bytecode: Security mode set to "TrustSigned".
Sun Dec 3 05:20:19 2017 -> +++ Started at Sun Dec 3 05:20:19 2017
Sun Dec 3 05:20:19 2017 -> Received 0 file descriptor(s) from systemd.
Sun Dec 3 05:20:19 2017 -> clamd daemon 0.99.2 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64)
Sun Dec 3 05:20:19 2017 -> Running as user clam (UID 990, GID 988)
Sun Dec 3 05:20:19 2017 -> Log file size limited to 4294967295 bytes.
Sun Dec 3 05:20:19 2017 -> Reading databases from /var/lib/clamav
Sun Dec 3 05:20:19 2017 -> Not loading PUA signatures.
Sun Dec 3 05:20:19 2017 -> Bytecode: Security mode set to "TrustSigned".
Sun Dec 3 05:21:50 2017 -> +++ Started at Sun Dec 3 05:21:50 2017
Sun Dec 3 05:21:50 2017 -> Received 0 file descriptor(s) from systemd.
Sun Dec 3 05:21:50 2017 -> clamd daemon 0.99.2 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64)
Sun Dec 3 05:21:50 2017 -> Running as user clam (UID 990, GID 988)
Sun Dec 3 05:21:50 2017 -> Log file size limited to 4294967295 bytes.
Sun Dec 3 05:21:50 2017 -> Reading databases from /var/lib/clamav
Sun Dec 3 05:21:50 2017 -> Not loading PUA signatures.
-
Accepted Answer
I've tried removing clam* and app-antiv*. This also removed app-antimalware*, app-antiphishing*, app-mail-antivirus* and clearsdn-anti* as dependencies. Reinstalling them and restarting clamd made little of no difference - startup time dropped slightly to 18s so not the improvement you've seen. Did you reinstall the clearsdn bits?
I don't run dansguardian and IDS is a completely different program suite. Is there anything else you removed and reinstalled? -
Accepted Answer
-
Accepted Answer
"freshclam" showed everything up to date, also "freshclam" after deleting *.cld and *.cvd didn't help. There were no error messages in any log when directly starting clamd from command line. "service clamd start" shows some "timed out" message.
Reading database took about 30s on Nov 28th (and before), after the antimalware update on Nov 29th the service continously timed out.
"yum history" shows that clearsdn-antimalware had been updated:
Loaded plugins: clearcenter-marketplace, fastestmirror
ClearCenter Marketplace: fetching repositories...
Transaction ID : 319
Begin time : Wed Nov 29 01:33:39 2017
Begin rpmdb : 1154:9330a47426a2ae9d4945f513b20d07703750e0c4
End time : 01:36:52 2017 (193 seconds)
End rpmdb : 1154:bf4ec60c717658705c1a5b25910636d026c81e92
User : root <root>
Return-Code : Success
Transaction performed with:
Installed rpm-4.11.3-25.el7.x86_64 @clearos-centos-verified
Installed yum-3.4.3-154.el7.centos.noarch @clearos-centos-verified
Installed yum-plugin-fastestmirror-1.1.31-42.el7.noarch @clearos-centos-verified
Packages Altered:
Updated clearsdn-antimalware-6.1-20171127.1025.noarch @private-clearcenter-antimalware
Update 6.1-20171128.1025.noarch @private-clearcenter-antimalware
I cannot undo this update because yum cannot download the old version anymore.
I therefore uninstalled everything related to antimalware (gateway-antivirus, clam*, dans*, intrusion*, etc.) and reinstalled step by step. Now everthing is running as before, clamd loads the database again in 30s.
Interestingly the same happed on the other system at the same time, I'll try to repair this one tomorrow. -
Accepted Answer
It looks like you can change systemd timeouts for specific services. Have a look at this thread. The file you'd want to edit is /usr/lib/systemd/system/clamd.service.
By way of comparison, Passmark scores my processor (i3-4130) at 4792 and yours at 1662 so roughly three times faster. -
Accepted Answer
Can you try updating the virus database with the command "freshclam"? Then try starting clamd from the command line ("service clamd start") and look for any error messages there and in the /var/log/clamav/ logs....... except you seem to have pretty much done this.
Mine seems to take just 19s to start.
[edit]
Even odder, but clamav was not updated on Nov 29th.
[/edit]

Please login to post a reply
You will need to be logged in to be able to post a reply. Login using the form on the right or register an account if you are new here.
Register Here »