A an alternative you could run this little program (needs to be executable). As above you may need to alter the words grep searches for if your lm_sensors output varies from mine...

#!/bin/sh

#



         for (( i=1; $i <= 10800; i++ ))

            do

               /bin/sleep 10

               /usr/bin/sensors | /bin/grep "CPU_Fan" | /bin/awk '{print $2}' >>  /tmp/fan_recording

               /bin/date >> /tmp/fan_recording

               /usr/bin/sensors | /bin/grep "CPU Temp:" | /bin/awk '{print $3}' >> /tmp/temp_recording

               /bin/date >> /tmp/temp_recording

            done



    exit 0

This will run for about 30 hours or until the machine crashes/reboots and produce output like this..

[root@r2r1a0d ~]# cat /tmp/fan_recording



611

Mon May 13 10:04:08 EST 2013

1915

Mon May 13 10:04:19 EST 2013

1839

Mon May 13 10:04:29 EST 2013

731

Mon May 13 10:04:39 EST 2013

613

Mon May 13 10:04:49 EST 2013

[root@r2r1a0d ~]#

[root@r2r1a0d ~]# cat /tmp/temp_recording



+54°C

Mon May 13 10:04:08 EST 2013

+54°C

Mon May 13 10:04:19 EST 2013

+54°C

Mon May 13 10:04:29 EST 2013

+55°C

Mon May 13 10:04:39 EST 2013

+54°C

Mon May 13 10:04:49 EST 2013

[root@r2r1a0d ~]#

This is the code I am using for the graphs above. As it is run a different Linux distribution (but still Redhat based) you may need to make minor changes to run under your ClearOS...

This is an example of my sensors output. You may need to make minor adjustments to the code if the CPU temperature and fan labels, for instance, are different to mine.

[root@r2r1a0d ~]# sensors

lm85b-i2c-0-2e

Adapter: SMBus I801 adapter at 18a0

V1.5:        +1.47 V  (min =  +0.00 V, max =  +3.32 V)

VCore:       +1.35 V  (min =  +0.00 V, max =  +2.99 V)

V3.3:        +3.35 V  (min =  +0.00 V, max =  +4.38 V)

V5:          +5.10 V  (min =  +0.00 V, max =  +6.64 V)

V12:         +12.00 V  (min =  +0.00 V, max = +15.94 V)

CPU_Fan:    607 RPM  (min =   93 RPM)

fan2:         0 RPM  (min =    0 RPM)

fan3:         0 RPM  (min =   93 RPM)                     ALARM

fan4:         0 RPM  (min  =  93 RPM)               °C)   ALARM

CPU Temp:      +54°C  (low  =  -127°C, high =  +127°C)

Board Temp:    +38°C  (low  =  -127°C, high =  +127°C)

Remote Temp:  +45°C  (low  =  -127°C, high =  +127°C)

CPU_PWM:    30

Fan2_PWM:   30

Fan3_PWM:   30

vid:      +1.388 V  (VRM Version 10.0)

[root@r2r1a0d ~]#

As the fan speed varies significantly, its speed is measured about every 10 seconds and the average calculated over each 5 minute period for mrtg.
Create a file called /usr/sbin/fanspeed_run with this content and make executable.

#!/bin/sh

#



         tmp_file=/tmp/fanspeed

         old_file=/tmp/fanspeedold

         fanspeed_avg=/tmp/fanspeed_avg



         /usr/bin/sensors | /bin/grep CPU_Fan | /bin/awk '{print $2}' >   /$tmp_file



         for (( i=1; $i <= 29; i++ ))

            do

               /bin/sleep 10

               /usr/bin/sensors | /bin/grep CPU_Fan | /bin/awk '{print $2}' >>  /$tmp_file

            done



         cat $tmp_file | tr "[:cntrl:]" ' ' | /bin/awk '{print($1+$2+$3+$4+$5+$6+$7+$8 \

         +$9+$10+$11+$12+$13+$14+$15+$16+$17+$18+$19+$20+$21+$22+$23+$24+$25+$26+$27+ \

         $28+$29+$30)/30}' > $fanspeed_avg



         /bin/mv -f $tmp_file $old_file



    exit 0

Create a directory called /var/www/html/mrtg - this is where your reports will be created and you will need to have httpd running so you can view the web pages created there...
/etc/mrtg/mrtg.cfg should look like this

# Multi Router Traffic Grapher -- Configuration File

###############################################################################



WorkDir: /var/www/html/mrtg

Options[_]: nopercent,growright,nobanner,nolegend,noinfo,gauge

MaxBytes[_]: 3250000000



###############################################################################

#

# Stats

#

###############################################################################



#-------------------------------

# Fan Speed -

#-------------------------------



Options[fanspeed]: nopercent,growright,nobanner,nolegend,noinfo,gauge,unknaszero

Target[fanspeed]: `/etc/mrtg/system fanspeed`

Title[fanspeed]: CPU Fan Speed

PageTop[fanspeed]: <h3>CPU Fan Speed</h3>

YLegend[fanspeed]: RPM

ShortLegend[fanspeed]: &nbsp;&nbsp;&nbsp;

LegendI[fanspeed]:

LegendO[fanspeed]: CPU Fan&nbsp;



#----------------------------

# Load

#----------------------------



Options[_]: nopercent,growright,nobanner,nolegend,noinfo,gauge,unknaszero

Target[load]: `/etc/mrtg/system load`

Title[load]: Load Averages (x 100)

PageTop[load]: <h3>Load Averages - r2r1a0d</h3>

YLegend[load]: Load Averages

ShortLegend[load]: &nbsp;&nbsp;&nbsp;

LegendI[load]: 5-minute Load&nbsp;

LegendO[load]: 15-minute Load&nbsp;



#-------------------------------

# CPU Temp - 

#-------------------------------



Target[cputemp]: `/etc/mrtg/system cputemp`

Title[cputemp]: CPU &Board Temperatures

PageTop[cputemp]: <h3>CPU and Board Temperatures</h3>

YLegend[cputemp]: Celsius

ShortLegend[cputemp]: &nbsp;&nbsp;&nbsp;

LegendI[cputemp]: Board &nbsp;

LegendO[cputemp]: CPU &nbsp;

This is the contents of /etc/mrtg/system - create it and this needs to be executable

#!/bin/sh



if [ "$TYPE" = "load" ]; then

    INDATA=`/bin/cut -d ' ' -f2 /proc/loadavg | /bin/sed 's/\.//g' | /bin/sed 's/^0//g'`

    OUTDATA=`/bin/cut -d ' ' -f3 /proc/loadavg | /bin/sed 's/\.//g' | /bin/sed  's/^0//g'`

fi



if [ "$TYPE" = "cputemp" ]; then

    INDATA=`/usr/bin/sensors | /bin/grep -A 1 "Board Temp" | tail -n 1 | /bin/awk '{print $1}'`

    OUTDATA=`/usr/bin/sensors | /bin/grep "CPU Temp:" | /bin/awk '{print $3}'`

fi



if [ "$TYPE" = "fanspeed" ]; then

    INDATA=`/bin/cat /tmp/fanspeed_avg`

    OUTDATA=$INDATA

fi



echo $INDATA

echo $OUTDATA

echo `/usr/bin/uptime | /bin/cut -d"," -f1,2`

echo $TYPE

Hope there are no typos here... sorry for the length...

As you are very interested in temperature - then instead of measuring the temperature every 5 minutes you could do something similar to what I did with the fan and measure every 10 seconds and average.

My experience is the same as Tonys. I ran cpuspeed with Pentium 4's for a little while...and the poor implementation of the cpu driver (p4-clockmod?) meant you get no power saving, the speed changes actually slow down the I/O performance and only reduced temperature

As already mentioned - your CPU should not be overheating under full load, as this means you have a hardware / cooling issue. If you run with the case off, and fan set to a higher value does the system continue OK? It's off that it's the same time every day...does it coincide with high load?

My Pentium 4s are NOT running cpuspeed...

It's years since I investigated this and my findings, I think, went on the Clark Connect Forums that are no longer available.

My recollection (which could be wrong) was that only the Pentium M (Mobile) was capable of changing frequency to run cooler when idle. Now, I seem to recall that the only way the Pentium 4 could run cooler was by inserting 'null' instructions that used very little power - hence it then ran cooler, but was slower running your programs. However, the effect was minimal and I ditched the program controlling this. What I cannot remember was whether cpuspeed did this, or whether it was another similar program.

Something also for you to investigate - I thought those Pentium 4s would throttle when they started to get too hot?

When I get time I will post some sample mrtg code - it's Mother's Day here in Oz and just about to go to the beach for a picnic with the greater family...

They are both in clearos-core which, since 6.4, is disabled by default:

yum install cpuspeed mrtg --enablerepo=clearos-core

mrtg and its prereqs, and I have also tried cpuspeed

What are you trying to install?

Wow, that is a lot of info, but I am having some issues installing this stuff. As said before, I am new to Linux still trying to find my way around it. When try to install using yum I am getting "no package ... available". I have also tried enablerep with rpmforge & rpmforge-extras. I am probably using the wrong repo, but could not find another one on google. I am hesitant to trial and error some repo as I want to avoid messing up my installation

I did check the BIOS of the IBM machine and this is set to NORMAL too. I do find indeed it increase speed when load occurs, so that is similar.

In the meantime, my server shut down again consistently in the timeslot between 09.00-.09.30 am CET.
Cron log:
May 11 09:01:02 stevenaar CROND[23137]: (root) CMD (run-parts /etc/cron.hourly)
May 11 09:01:02 stevenaar run-parts(/etc/cron.hourly)[23137]: starting 0anacron
May 11 09:01:02 stevenaar run-parts(/etc/cron.hourly)[23146]: finished 0anacron
May 11 09:02:01 stevenaar CROND[23181]: (root) CMD (/usr/sbin/proxy2db >/dev/null 2>&1)
May 11 09:05:01 stevenaar CROND[23333]: (root) CMD (/usr/sbin/network2db >/dev/null 2>&1)
May 11 09:05:01 stevenaar CROND[23334]: (root) CMD (LANG=en_US /usr/clearos/apps/base/deploy/servicewatch >/dev/null)
May 11 09:05:01 stevenaar CROND[23332]: (root) CMD (/usr/sbin/resource2db >/dev/null 2>&1)
May 11 09:10:01 stevenaar CROND[23611]: (root) CMD (LANG=en_US /usr/clearos/apps/base/deploy/servicewatch >/dev/null)
May 11 09:10:01 stevenaar CROND[23612]: (root) CMD (/usr/sbin/resource2db >/dev/null 2>&1)
May 11 09:10:01 stevenaar CROND[23610]: (root) CMD (/usr/sbin/network2db >/dev/null 2>&1)

I checked the BIOS of the IBM ThinkCenter P4 3 GHz machine which generated the graphs above...
The fan has three options :-
QUIET - reduces fan speed and may even reduce processor speed
NORMAL - provides best processor performance
ON - never turns FAN off

Mine is set to NORMAL and you can just hear the fan change speed with processor load...

I also have IBM NetVista P4 2.0 GHz machines with similar Fan Controls

and just in case anybody was thinking the fans in these machines are unreliable - the last one for me to replace was in April just gone for an IBM desktop machine built in March of 1999. That fan lasted just over 14 years of 24 x 7 operation - that's pretty good in my book...

Might as well complete the set with the fan speed...

For the temps.. CPU is blue green is system

OK - since you have lm_sensors running you should be able to install mrtg to graph your CPU temp continuously and see how it varies... You should also be able to graph the CPU fan speed...

Most IBM's of that vintage have a BIOS setting that gives you two or three options as to how the fan speed is controlled - what is yours set at? I forget the name mine is set at - will check at work tomorrow on mine. I can actually hear the fan spin faster as the CPU work load increases - hence the minimal increase in CPU temp I experience... then it slows down again as the work load diminishes.

57 degrees seems a bit high for a machine in that ambient temperature. I have an IBM workstation which is a Pentium 4 3.0 GHz in an ambient of about 20 - 22 running at about 55-56 and doesn't peak that much higher. During the night there is a sustained load over 160 and up to 320 for about a one hour 30 minute period, temperature during this time peaks at only 61 - 63.

This machine has a fan blowing air through the CPU heatsink. If yours is like that did you ensure that the gaps between the fins of the heatsink are not filled with dust or dirt. I have seen some caked solid so no air actually blows between the fins. You should also ensure the heatsink hasn't come loose. Also what are the condition of the fan bearings? I have had to replace a fans on three of these IBM machines (out of 6) now as the fan bearing would get tight and the fan stall, then after a few minutes start spinning again. On machines of this age this is not surprising...

As Tim mentioned, most of the housekeeping is done during the night. Look in /etc/cron.daily and /etc/cron.d to see some of these. You could try changing the time or disabling to see which, if any, are producing the problem load and hence heat... though you log extract seems to indicate a 9.00 am problem...

As a thought, is cpuspeed installed? Earlier in the release of the 6.3 I noticed it was not installed by default.. I have not checked 6.4. Without it your CPU will run at 100% all the time. If that is the case, it is not really a software problem as it indicates the server does have inadequate cooling to run at full load, but perhaps it has never been consistanty run at full load before. Cpuspeed will throttle the speed whenever all the CPU power is not needed.

Please can you post back with what you find as I feel quite strongly about the installation of cpuspeed.

If you are not feeling any hot casing or hot air from the fan, it could be that the CPU cooler has become slightly dislodged from the CPU so proper heat transfer is not taking place. This means the CPU would get hot but ot the heatsink. You could try removing the cooler, replacing the thermal paste then re-seating the cooler.

[edit]
If you do look at or remove the cooler, it is worth getting rid of as much dust as possible from the heatsink and the fan.
[/edit]

[edit2]
Having said that about cpuspeed, I am now not sure that the Pentiun4 will respond to cpuspeed.
[/edit2]

It is an IBM desktop with Intel(R) Pentium(R) 4 CPU 2.80GHz that I am using as a little standalone home server to learn how this Linux dist works and use it for backup and webserver purpose. It has very little traffic and is mainly used by the family.
It is physically located in my garage with an ambient temperature of 16 degrees.

It has a regular fan that runs fine, and before I installed the Centos server, I have cleaned all dust etc.

The 90 C temp is something that pops up in putty just before the machine shuts down with this message. I have installed lmsensors but I am not sure if it measures it correctly. The time I was home when it shut down, I could not feel any hot casing or air from the fan. This tells me that it is prob a software setting as I see a lot of messages about Linux and the kernel error on temp. This issue started after I had installed the 6.4 release, I did not have this problem on the previous release.

This is the output at this moment of the lmsensor
acpitz-virtual-0
Adapter: Virtual device
temp1: +57.0°C (crit = +90.0°C)

Can you list the precise details of your hardware please? make model etc of all components including case, and size, number and placement of fans.

I am not convinced reading this thread that it is necessarily a software problem. There are other hardware issues other than fans being dirty of not spinning, and general cleanliness that can cause a hardware triggered shutdown...

Depending on your hardware there also may be software tools that can be installed to monitor the hardware and give a clue as to what may be going on...

Any decent desktop should be able to run the CPU at 90 to 95% or so continuously, and at less than 90 degrees C, and not overheat (Assuming it is in a room with a reasonable ambient temp)..

Also you didn't mention in your first post how you knew the CPU had gone over 90 degrees. What is your proof of this please?

login as: root
root@192.168.2.111's password:
Last login: Mon May 6 19:29:40 2013 from 192.168.2.5
[root@stevenaar ~]# ls -la /var/log/'
> ls -la /var/log/
>
>


When I look at the cron log, this is an extract of a lot of lines, minutes before the teh server shuts down on a temp issue.

May 4 09:01:01 stevenaar run-parts(/etc/cron.hourly)[1597]: starting 0anacron ...
May 4 09:01:01 stevenaar run-parts(/etc/cron.hourly)[1606]: finished 0anacron ...
May 4 09:02:01 stevenaar CROND[1617]: (root) CMD (/usr/sbin/proxy2db >/dev/null 2>&1) ...
May 4 09:05:01 stevenaar CROND[1690]: (root) CMD (/usr/sbin/network2db >/dev/null 2>&1) ...
May 4 09:05:01 stevenaar CROND[1691]: (root) CMD (LANG=en_US /usr/clearos/apps/base/deploy/servicewatch >/dev/null) ...
May 4 09:05:01 stevenaar CROND[1694]: (root) CMD (/usr/sbin/resource2db >/dev/null 2>&1) ...
May 4 09:10:01 stevenaar CROND[1830]: (root) CMD (/usr/sbin/resource2db >/dev/null 2>&1) ...
May 4 09:10:01 stevenaar CROND[1831]: (root) CMD (/usr/sbin/network2db >/dev/null 2>&1) ...
May 4 09:10:01 stevenaar CROND[1832]: (root) CMD (LANG=en_US /usr/clearos/apps/base/deploy/servicewatch >/dev/null) ...

May 5 09:01:01 stevenaar CROND[19066]: (root) CMD (run-parts /etc/cron.hourly) ...
May 5 09:01:01 stevenaar run-parts(/etc/cron.hourly)[19066]: starting 0anacron ...
May 5 09:01:01 stevenaar run-parts(/etc/cron.hourly)[19075]: finished 0anacron ...
May 5 09:02:01 stevenaar CROND[19086]: (root) CMD (/usr/sbin/proxy2db >/dev/null 2>&1) ...
May 5 09:05:01 stevenaar CROND[19137]: (root) CMD (LANG=en_US /usr/clearos/apps/base/deploy/servicewatch >/dev/null) ...
May 5 09:05:01 stevenaar CROND[19138]: (root) CMD (/usr/sbin/resource2db >/dev/null 2>&1) ...
May 5 09:05:01 stevenaar CROND[19139]: (root) CMD (/usr/sbin/network2db >/dev/null 2>&1) ...
May 5 09:10:01 stevenaar CROND[19293]: (root) CMD (LANG=en_US /usr/clearos/apps/base/deploy/servicewatch >/dev/null) ...
May 5 09:10:01 stevenaar CROND[19294]: (root) CMD (/usr/sbin/network2db >/dev/null 2>&1) ...
May 5 09:10:01 stevenaar CROND[19295]: (root) CMD (/usr/sbin/resource2db >/dev/null 2>&1) ...

May 6 09:01:01 stevenaar CROND[24466]: (root) CMD (run-parts /etc/cron.hourly) ...
May 6 09:01:01 stevenaar run-parts(/etc/cron.hourly)[24466]: starting 0anacron ...
May 6 09:01:01 stevenaar run-parts(/etc/cron.hourly)[24475]: finished 0anacron ...
May 6 09:02:01 stevenaar CROND[24509]: (root) CMD (/usr/sbin/proxy2db >/dev/null 2>&1) ...
May 6 09:05:01 stevenaar CROND[24646]: (root) CMD (LANG=en_US /usr/clearos/apps/base/deploy/servicewatch >/dev/null) ...
May 6 09:05:01 stevenaar CROND[24645]: (root) CMD (/usr/sbin/resource2db >/dev/null 2>&1) ...
May 6 09:05:01 stevenaar CROND[24649]: (root) CMD (/usr/sbin/network2db >/dev/null 2>&1) ...

A lot of house keeping occurs around 4am, and is called by the cron scripts - you can find clues in /var/log/cron (the scripts live at /etc/cron.d)

Do you have particularly large log files for processing? what's the output of 'ls -la /var/log/'

A little late, but as the topic is still open and I am still suffering regular shutdowns, I have another question.

It looks like my system shuts down on a temperature issue at or around a set time in the morning. My guess is that something kicks in that starts a process that causes this alleged temparture issue, but when going through the various logs I have not been able to identify the root cause process. Probably I am overlooking something. Can someone please advise what log I should use to determine the issue, assuming it is a process or services that starts.. That will hopefully take me further into the analysis.
Tx

Ronald

It is a quiet system, as it currently only serves as a homeserver.

I think the Java is driven by PCmonitor that I am running from my Ipad to check the system.

Thanks for the top output. It's a quiet system at that point in time:

- System load: 0.00
- CPU idle time: 98% (i.e. the CPU isn't doing much)

What's the java process? It's a busy one.

Thanks Ben, this is the output

top - 20:04:41 up 8:06, 2 users, load average: 0.00, 0.00, 0.00
Tasks: 205 total, 1 running, 204 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.7%us, 1.0%sy, 0.0%ni, 98.0%id, 0.3%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 1030224k total, 1002184k used, 28040k free, 23804k buffers
Swap: 2064376k total, 153732k used, 1910644k free, 101072k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1005 root 20 0 2856 1112 824 R 0.3 0.1 0:00.27 top
2100 clearsyn 20 0 10408 1616 996 S 0.3 0.2 0:22.80 clearsyncd
3654 root 20 0 28252 1660 576 S 0.3 0.2 0:05.73 l7-filter
7580 root 20 0 163m 30m 3340 S 0.3 3.0 1:14.04 java
1 root 20 0 2948 916 768 S 0.0 0.1 0:01.67 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.01 kthreadd
3 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
4 root 20 0 0 0 0 S 0.0 0.0 0:00.40 ksoftirqd/0
5 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
6 root RT 0 0 0 0 S 0.0 0.0 0:00.14 watchdog/0
7 root 20 0 0 0 0 S 0.0 0.0 0:02.17 events/0
8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 cgroup
9 root 20 0 0 0 0 S 0.0 0.0 0:00.00 khelper
10 root 20 0 0 0 0 S 0.0 0.0 0:00.00 netns
11 root 20 0 0 0 0 S 0.0 0.0 0:00.00 async/mgr
12 root 20 0 0 0 0 S 0.0 0.0 0:00.00 pm
13 root 20 0 0 0 0 S 0.0 0.0 0:00.07 sync_supers

Ronald,

Can you login via shell and type:

top

You'll get an idea for which processes are consuming CPU. Post the results back here and we'll see if we can help you resolve the issue.

B.