Thanks.
Trying it out. It was fun while it lasted though

To be honest, the script is pretty much obsoleted. As soon as a link to the open-nogpl was posted there was little point in using my script as there is no longer a rule clash with the open-gpl rules which were included in the ClearOS rules and the ET rules. If you use the script to add blocks, this can be done directly by creating and editing a /etc/sid-block.map file. I now have a much simpler script to update the ET rules.

Thanks for the heads up about the RBN rules. I knew they were going but not when. FWIW snort is not a good tool to block IP's. They (ET) pointed me to a program called ipset. You can see my implementation of it in this thread. It effectively replaces the DSHIELD, DROP and BOTCC block rules in a far more efficient manner. When I was pointed to it the IP list also contained the RBN IP's but they were dropped before I even wrote my script.

I was thinking of extending the ipset script to parse the compromised and tor rules as they are just IP lists as well. This would be more CPU efficient than using snort to do the blocking.

It would also be more efficient to implement country blocks this way rather than a myriad of firewall rules although I don't do any country blocking.

For all those people who are using the script to update Emerging Threats rules, please note that the RBN rules are being fazed out on Mar. 13th 2014.

Apparently ET is going to distribute blank RBN scripts but it probably makes sense to update the script and config file to exclude these rules.

I will probably just update the script to exclude downloading RBN and leave the existing RBN file for another while as I find it still fires off every couple of weeks.

Date: Thu, 6 Mar 2014 12:09:01 -0600
From: Will Metcalf <wmetcalf@emergingthreatspro.com>
To: Emerging Sigs
Subject: [Emerging-updates] Announcement: EOL for the RBN signatures
March 13 2014

We will no longer be shipping a set of RBN signatures as of March 13, 2014.
The files themselves will still be distributed but will be blank (this
should keep your IDS engines happy without you having to modify your
configs). This ruleset has been stale for quite some time, and much the
address space has been reallocated.

Regards,

Will

I use the rule set which matches with the version of snort I am running. COS 6.x uses Snort 2.9.x.

From recent observations, 2.9.x does not worry about duplicate SID's. If you use the nogpl rules and the /etc/sid-block.map file if you want to add any blocks then my script is redundant.

When you add the extra "include" lines, add them in a section above the current rules section rather than below. For ipvar, COS should set it on a network to your internal subnets and your WAN IP.

Nick Howitt wrote:

Thanks Tim,

Looks like it is the same for reference.config. It is better to disable the default line for that as the ET file has all the default entries plus more. For the classification.config the default file is the better one to use.

so just a quick recap, since i havent been updating my snort rules for some time..
the old script can be scrapped to use the nogpl rules together with the existing COS rules, yes??

as for the

ipvar HOME_NET [192.168.10.0/24,172.17.2.0/24]

i should just use my internal IP range right?

Nick Howitt wrote:

Thanks Tim,

Looks like it is the same for reference.config. It is better to disable the default line for that as the ET file has all the default entries plus more. For the classification.config the default file is the better one to use.

and as for this, comment the necessary files out??

i noticed you are using the 2.9.0 rule set, i am assuming this wont work on COS 5.2 ?

Thanks Tim,

Looks like it is the same for reference.config. It is better to disable the default line for that as the ET file has all the default entries plus more. For the classification.config the default file is the better one to use.

it means classifcation.config is referenced twice in /etc/snort.conf, they actually appear the same (both the ET version, and the default ClearOS version) so you can remove the include line which refers to it

I've no idea - it is exactly the same as I get. I also get a massive start up log which makes it pretty much useless!

You'll also get a warning on flowbits. There is some information in /usr/share/doc/snort-2.9.0.4/README.flowbits but I need to work out exactly what I need to to to clear the warning.

I also tried to use emerging-botcc.rules rules but they gave errors which caused snort to fail because thresholds were not set anywhere. More reading some time ... There is some more documentation about it.

[edit]
... and if you can find out what to do with the rules.emergingthreats.net/open-nogpl/snort-2.9.0/rules/sid-msg.map file, I'd love to know. I could not work out what it did or where to put it to make it do anything.
[/edit]

I get this as well so I dont think we messed up. As its complaining its a duplicate I just ignored it. I assume its something not quite right in classification.config which we downloaded so is beyond our control

Hey! I used your latest script (THANK YOU!!!!!!) and noticed this in the logs after restarting Snort:

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(7) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring th$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(8) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring th$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(9) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring th$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(10) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(11) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(12) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(13) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(14) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(15) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(16) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(17) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(18) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(19) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(20) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(21) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(22) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(23) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(24) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(25) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(26) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(27) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(28) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(29) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(30) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(31) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(32) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(33) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(34) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(35) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

Sep  2 12:45:02 orion snort[3646]: WARNING /etc/snort.d/rules/emerging_threats/classification.config(36) Duplicate classification "/etc/snort.d/rules/emerging_threats/classification.config"found, ignoring t$

What did I mess up?

Wow thanks for the quick reply. I made the changes to rsyslog.conf restarted rsyslog then restarted snort and all seems good. I do have smtp running but I think I will run the server as is for a while and check everything is working before I try playing around with rulesets and break stuff

I've added:

# Sort out rate limiting problem with snort

$ModLoad imuxsock # provides support for local system logging

# $SystemLogRateLimitInterval 0 # turn off rate limiting

$SystemLogRateLimitInterval 10

$SystemLogRateLimitBurst 5000

to the end of /etc/rsyslog.conf and restarted rsyslog. If you google around you will see different values suggested, but this works for me.

You may also want to look at different rule sets. If you expose different services such as smtp you may want more rules.

Excellent Thanks Nick I have got it installed and Snort starts and is running . I defined my ipvar Home_NET to be my LAN and openvpn network it was previously set to "any"

I think you are right that I need to do something with rsyslog as when I start snort in /var/log/messages I see

Sep  2 19:38:22 fs1 snort[921909]:         Check for Telnet Cmds: YES alert: YES

Sep  2 19:38:22 fs1 rsyslogd-2177: imuxsock begins to drop messages from pid 921909 due to rate-limiting

Sep  2 19:38:29 fs1 rsyslogd-2177: imuxsock lost 91 messages from pid 921909 due to rate-limiting

Sep  2 19:38:29 fs1 snort[921909]: 7280 Snort rules read

I also get

Sep  2 19:38:01 fs1 snort[921719]: Could not remove pid file /var/run//snort_eth0.pid: No such file or directory

Sep  2 19:38:02 fs1 snort[921719]: Snort exiting

Sep  2 19:38:21 fs1 snort[921909]: Running in IDS mode

But I dont think that is something to worry about?
*** Edit ****
later on I get

Sep  2 19:38:33 fs1 kernel: [529771.489163] device eth0 entered promiscuous mode

Sep  2 19:38:33 fs1 snort[921937]: Decoding Ethernet

Sep  2 19:38:33 fs1 snort[921937]: Checking PID path...

Sep  2 19:38:33 fs1 snort[921937]: PID path stat checked out ok, PID path set to /var/run/

Sep  2 19:38:33 fs1 snort[921937]: Writing PID "921937" to file "/var/run//snort_eth0.pid"

Sep  2 19:38:33 fs1 snort[921937]: Set gid to 490

Sep  2 19:38:33 fs1 snort[921937]: Set uid to 490

Sep  2 19:38:33 fs1 snort[921937]: 

Sep  2 19:38:33 fs1 snort[921937]:         --== Initialization Complete ==--

So it really is nothing to worry about

Add the following to /etc/snort.conf:

include $RULE_PATH/emerging_threats/classification.config

include $RULE_PATH/emerging_threats/reference.config

include $RULE_PATH/emerging_threats/emerging-attack_response.rules

include $RULE_PATH/emerging_threats/emerging-botcc.rules

include $RULE_PATH/emerging_threats/emerging-current_events.rules

include $RULE_PATH/emerging_threats/emerging-exploit.rules

include $RULE_PATH/emerging_threats/emerging-malware.rules

include $RULE_PATH/emerging_threats/emerging-trojan.rules

include $RULE_PATH/emerging_threats/emerging-worm.rules

include $RULE_PATH/emerging_threats/emerging-web_server.rules

include $RULE_PATH/emerging_threats/emerging-compromised-BLOCK.rules

include $RULE_PATH/emerging_threats/emerging-drop-BLOCK.rules

include $RULE_PATH/emerging_threats/emerging-dshield-BLOCK.rules

include $RULE_PATH/emerging_threats/emerging-rbn-BLOCK.rules

include $RULE_PATH/emerging_threats/emerging-rbn-malvertisers-BLOCK.rules

include $RULE_PATH/emerging_threats/emerging-tor-BLOCK.rules

You may also need to define ipvar explicitly e.g.:

ipvar HOME_NET [192.168.10.0/24,172.17.2.0/24]

Create the folders /etc/snort.d/rules/emerging_threats/ and /etc/snort.d/rules/emerging_threats/temp/.

Lastly remember to make your script executable.

If snort fails to start please post back with the error at the end of /var/log/messages. There may be a tweak needed fof rsyslog as well.

So quick question if I use your script Nick what changes if any do I need to make to snort.conf ?Or how can I tell my downloaded rules are being processed.

I now use this script which has a slightly more limited rule set:

cd /etc/snort.d/rules/emerging_threats/temp



# Execute some rules on Saturday only

if [[ $(date +%u) = 6 ]] ; then

	wget -q http://rules.emergingthreats.net/open-nogpl/snort-2.9.0/rules/emerging-attack_response.rules

	wget -q http://rules.emergingthreats.net/open-nogpl/snort-2.9.0/rules/emerging-exploit.rules

	wget -q http://rules.emergingthreats.net/open-nogpl/snort-2.9.0/rules/emerging-web_server.rules

	wget -q http://rules.emergingthreats.net/blockrules/emerging-rbn-BLOCK.rules

	wget -q http://rules.emergingthreats.net/blockrules/emerging-rbn-malvertisers-BLOCK.rules

	wget -q http://rules.emergingthreats.net/blockrules/emerging-tor-BLOCK.rules

	wget -q http://rules.emergingthreats.net/open-nogpl/snort-2.9.0/classification.config

	wget -q http://rules.emergingthreats.net/open-nogpl/snort-2.9.0/rules/reference.config

fi

wget -q http://rules.emergingthreats.net/open-nogpl/snort-2.9.0/rules/emerging-botcc.rules

wget -q http://rules.emergingthreats.net/open-nogpl/snort-2.9.0/rules/emerging-current_events.rules

wget -q http://rules.emergingthreats.net/open-nogpl/snort-2.9.0/rules/emerging-malware.rules

wget -q http://rules.emergingthreats.net/open-nogpl/snort-2.9.0/rules/emerging-trojan.rules

wget -q http://rules.emergingthreats.net/open-nogpl/snort-2.9.0/rules/emerging-worm.rules

wget -q http://rules.emergingthreats.net/blockrules/emerging-dshield-BLOCK.rules

wget -q http://rules.emergingthreats.net/blockrules/emerging-compromised-BLOCK.rules

wget -q http://rules.emergingthreats.net/blockrules/emerging-drop-BLOCK.rules





mv -f * ..



service snort restart > /dev/null

Yes, they don't clash so all I've been doing for some time is download the rules, copy to the right directory & restart snort & snortsam.

I bet it would be easier to do with pulledpork but I never got around to figuring out how to install & run in on COS.

To be honest the script is pretty much pointless once someone pointed me to a link with the nogpl rules. These will never clash with the ClearOS one so you don't risk duplicates. That means you can then do a much more simple script to get your rules.

The service command missing probably means the script is not running as root so not inheriting the correct environment. This can happen if you use a cron job via the crontab command. I simply place the script in /etc/cron.weekly (or daily) to get round this.

I've had to make some changes to the original script because of an error that occurred when I enabled the emergingthreats.sh in the crontab. I set the root crontab to run the script every few days rather than once a week as the rules have started to have pretty important changes more frequently.

This is still for COS 5.2!! I started to get "service: command not found" in reference to lines 24 & 25 of the script. I needed to change the "service snort restart" & "service snortsam restart" commands to "/etc/init.d/snort restart > /dev/null" and "/etc/init.d/snortsam restart > /dev/null".

Over the past months I've also added a few more of the rules to my set. So please check against your script and add the new rules to your snort.conf file.

Here is the updated script in its entirety.

#!/bin/bash



#

# The purpose of this script is to update Snort & Snortsam to use the most recent Emerging

# Threats rules.

# Updated Aug 28 2013 by Nuke

#



# move to temp folder for downloading rules

cd /tmp/et/



# download the rules you want to use, extract & install

wget http://rules.emergingthreats.net/open-nogpl/snort-2.8.4/emerging.rules.tar.gz http://rules.emergingthreats.net/open-nogpl/snort-2.8.4/rules-md5.txt http://rules.emergingthreats.net/blockrules/emerging-compromised-BLOCK.rules http://rules.emergingthreats.net/blockrules/emerging-drop-BLOCK.rules http://rules.emergingthreats.net/blockrules/emerging-dshield-BLOCK.rules http://rules.emergingthreats.net/blockrules/emerging-rbn-BLOCK.rules http://rules.emergingthreats.net/blockrules/emerging-tor-BLOCK.rules http://rules.emergingthreats.net/blockrules/emerging-rbn-malvertisers-BLOCK.rules

tar -xzf emerging.rules.tar.gz 

mv emerging.rules.tar.gz /root/`date +%Y%m%d`_emerging.rules.tar.gz

cp /tmp/et/rules/sid-msg.map /tmp/et/rules/emerging-sid-msg.map

cp -pf *-BLOCK.rules /etc/snort/

cp -pf /tmp/et/rules/emerging-botcc.rules /tmp/et/rules/emerging-exploit.rules /tmp/et/rules/emerging-current_events.rules /tmp/et/rules/emerging-attack_response.rules /tmp/et/rules/emerging-malware.rules /tmp/et/rules/emerging-trojan.rules /tmp/et/rules/emerging-p2p.rules /tmp/et/rules/emerging-web_client.rules /tmp/et/rules/emerging-web_specific_apps.rules /tmp/et/rules/emerging-worm.rules /tmp/et/rules/emerging-smtp.rules /tmp/et/rules/emerging-pop3.rules /tmp/et/rules/emerging-imap.rules /tmp/et/rules/emerging-web_server.rules /tmp/et/rules/emerging-mobile_malware.rules /tmp/et/rules/emerging-sid-msg.map /etc/snort/



# remove temp files rules

rm -Rf /tmp/et/rules/ /tmp/et/emerging.rules.tar.gz /tmp/et/rules-md5.txt /tmp/et/*-BLOCK.rules



# restart snort and snortsam

/etc/init.d/snort restart > /dev/null

/etc/init.d/snortsam restart > /dev/null

The error is really in ClearOS. The start up script in 6.4 is slightly broken as it refers to "var" as opposed to "ipvar". If this is fixed (with one other fix) then the init script will automatically set HOME_NET correctly for you. There is a bug which I filed to this effect with the bits of script which need changing but ClearOS are going down a different and probably better route for 6.5. They are stripping out that part of the script entirely and replacing it with a clearsync event (bug 1225). There is nothing to stop you modifying the current init script until 6.5 or just manually changing snort.conf.

Hate to Necro post but are the following rules in ClearOS out of the box?

Multiple Inbound SMTP

alert tcp !$HOME_NET any -> $HOME_NET 25 (msg:"LOCAL Multiple Inbound SMTP Connections -- BLOCKING SOURCE"; \

flags: S,12; threshold: type both, track by_src, count 9, seconds 300; classtype:misc-activity; sid:100000x; rev:1; fwsam:src, 15 minutes

DNS Requests from Other than our DNS

alert udp !$DNS_SERVERS any -> $EXTERNAL_NET 53 (msg:"LOCAL Outbound Non-DNS Server DNS Traffic"; content:"|01|"; offset:2; \

depth:1; threshold:type limit,track by_src,count 1, seconds 60; classtype:misc-activity; sid:100000x; rev:1

Not 100% sure if thats correct though. (The rules)

The problem is that some ET rules have ! $HOME_NET and !any is not allowed and snort will fail to start.

Nick Howitt wrote:

From what I've seen today, I think my change to DNS_SERVERS was wrong. Leave DNS_SERVERS set to $HOME_NET, but change HOME_NET to your LAN or as I have done for the moment:

ipvar HOME_NET [10.0.0.0/8,172.16.0.0/12,192.168.0.0/16]

Then it covers all private LAN ranges.

What's the difference between having the private ranges and simply "ipvar HOME_NET any" (that's the default setting)?

From what I've seen today, I think my change to DNS_SERVERS was wrong. Leave DNS_SERVERS set to $HOME_NET, but change HOME_NET to your LAN or as I have done for the moment:

ipvar HOME_NET [10.0.0.0/8,172.16.0.0/12,192.168.0.0/16]

Then it covers all private LAN ranges.

@Eric Carson,
I've now done something similar, using slightly different rule sets. If you want to use the emerging-current_events.rules, change the line in /etc/snort.conf:

var DNS_SERVERS $HOME_NET

to

ipvar DNS_SERVERS [208.67.222.222,208.67.220.220]

Change your DNS servers to suite your configuration. I use OpenDNS.

I also had a problem with rsyslogd stopping snort from starting because of rate limiting. See this article for a fix.

What do you mean by "add the BLOCK rules directly to snortsam"? The emerging-drop-BLOCK rules automatically block for 30 days if triggered - see the "fwsam: src, 30 days;" bit at the end of the rules. If you want to add the firewall rules directly, have a look at http://rules.emergingthreats.net/fwrules/. You would need to call the file up from /etc/clearos/firewall.d/local and you may need to define the variable $IPTABLES to point to /sbin/iptables.

# Path to your rules files (this can be a relative path)

# Note for Windows users:  You are advised to make this an absolute path,

# such as:  c:\snort\rules

var RULE_PATH /etc/snort.d/rules

var SO_RULE_PATH /etc/snort.d/rules

var PREPROC_RULE_PATH /etc/snort.d/rules

You were right Nick. Luckily for me common sense was correct as well!

My only concern now is how to add the BLOCK rules directly to snortsam, and eventually how to get the webconfig to wake up and show the additional rule sets.

Sorry for the typo. It is not /etc/snort.ini, but /etc/snort.conf where you will find $RULE_PATH. The file /etc/rc.d/init.d/snort is used just to start and stop snort with the service command (or called directly).

Nick Howitt wrote:

What is your RULE_PATH in /etc/snort.ini. Mine is /etc/snort.d/rules which suggests you are copying your rules to the wrong place in your script (/etc/snort.d).

I don't have time to try it at the moment.

I'm moderately sure everything is operational on my 6.3 box at this point. Read on!
ClearOS 6.3 must have changed it up a little;
Could not locate the snort.ini file where you specified, so after a small amount of digging I found it /etc/rc.d/init.d and the file name is merely snort with no file extension. However there is no information within that file as to the location of the RULE_PATH

Operating on the principle that the gpl rules are found in /etc/snort.d/rules/gpl/ and snort.config file points like thus include $RULE_PATH/gpl/dns.rules the logical conclusion is that $RULES_PATH is equal to /etc/snort.d/rules/

That said, the rules need a new and separate home, I chose /etc/snort.d/rules/emerging/ as the logical place

mkdir /etc/snort.d/rules/emerging/

Now the script needs adjustment to place the updated rules files in the new directory. I removed the pipe to null at the end of the script for the service restart so I could have a better indication if things were broken or not.

#!/bin/bash



#

# The purpose of this script is to update Snort & Snortsam to use the most recent Emerging

# Threats rules.

# Threat rules are unzipped in a folder you have to create beforehand ie /tmp/et



# move to temp folder for downloading rules

cd /tmp/et/



# download the rules you want to use, extract & install

wget http://rules.emergingthreats.net/open-nogpl/snort-2.9.0/emerging.rules.tar.gz http://rules.emergingthreats.net/open-nogpl/snort-2.9.0/rules-md5.txt http://rules.emergingthreats.net/blockrules/emerging-botcc-BLOCK.rules http://rules.emergingthreats.net/blockrules/emerging-compromised-BLOCK.rules http://rules.emergingthreats.net/blockrules/emerging-drop-BLOCK.rules http://rules.emergingthreats.net/blockrules/emerging-dshield-BLOCK.rules http://rules.emergingthreats.net/blockrules/emerging-rbn-BLOCK.rules http://rules.emergingthreats.net/blockrules/emerging-tor-BLOCK.rules http://rules.emergingthreats.net/blockrules/emerging-rbn-malvertisers-BLOCK.rules



tar -xzf emerging.rules.tar.gz



# Move a copy of the downloaded rules to /root for backup

# You will have to manually delete the rules in /root occasionally

mv emerging.rules.tar.gz /root/`date +%Y%m%d`_emerging.rules.tar.gz



# Create a sid-map for the emerging rules

cp /tmp/et/rules/sid-msg.map /tmp/et/rules/emerging-sid-msg.map



# Copy/update all the Block rules to /etc/snort.d/rules/emerging/

# Note that you have to add each of the rules added to the bottom of the snort.conf in order for them to run

cp -pf *-BLOCK.rules /etc/snort.d/rules/emerging/



# Copy other rules that you want to use to /etc/snort.d/rules/emerging/

# Note that you have to add each of the rules added to the bottom of the snort.conf in order for them to run

cp -pf /tmp/et/rules/emerging-exploit.rules /tmp/et/rules/emerging-current_events.rules /tmp/et/rules/emerging-attack_response.rules /tmp/et/rules/emerging-malware.rules /tmp/et/rules/emerging-trojan.rules /tmp/et/rules/emerging-p2p.rules /tmp/et/rules/emerging-web_client.rules /tmp/et/rules/emerging-web_specific_apps.rules /tmp/et/rules/emerging-worm.rules /tmp/et/rules/emerging-smtp.rules /tmp/et/rules/emerging-pop3.rules /tmp/et/emerging-imap.rules /tmp/et/rules/emerging-sid-msg.map /etc/snort.d/rules/emerging/



# remove temp files rules

rm -Rf /tmp/et/rules/ /tmp/et/emerging.rules.tar.gz /tmp/et/rules-md5.txt /tmp/et/*-BLOCK.rules



# restart snort and snortsam

service snort restart

service snortsam restart

Finally the /etc/snort.conf needs corrected to reflect the location change. I commented out the current-events.rules because it was causing an error due to rule 93 having !ANY as URL.

include $RULE_PATH/emerging/emerging-attack_response.rules

include $RULE_PATH/emerging/emerging-compromised-BLOCK.rules

#include $RULE_PATH/emerging/emerging-current_events.rules

include $RULE_PATH/emerging/emerging-drop-BLOCK.rules

include $RULE_PATH/emerging/emerging-dshield-BLOCK.rules

include $RULE_PATH/emerging/emerging-exploit.rules

include $RULE_PATH/emerging/emerging-malware.rules

include $RULE_PATH/emerging/emerging-p2p.rules

include $RULE_PATH/emerging/emerging-pop3.rules

include $RULE_PATH/emerging/emerging-rbn-BLOCK.rules

include $RULE_PATH/emerging/emerging-rbn-malvertisers-BLOCK.rules

include $RULE_PATH/emerging/emerging-smtp.rules

include $RULE_PATH/emerging/emerging-tor-BLOCK.rules

include $RULE_PATH/emerging/emerging-trojan.rules

include $RULE_PATH/emerging/emerging-web_client.rules

include $RULE_PATH/emerging/emerging-web_specific_apps.rules

include $RULE_PATH/emerging/emerging-worm.rules

now run the script to see if everything behaves

./etc/cron.weekly/emergingthreats.sh

What is your RULE_PATH in /etc/snort.ini. Mine is /etc/snort.d/rules which suggests you are copying your rules to the wrong place in your script (/etc/snort.d).

I don't have time to try it at the moment.

Well I ran into an issue. Snort will not start. I did some digging and here is what I found.

service snort start

Starting snort:                                            [FAILED]

I wanted to know why

# /usr/sbin/snort -c /etc/snort.conf

Running in IDS mode



        --== Initializing Snort ==--

Initializing Output Plugins!

Initializing Preprocessors!

Initializing Plug-ins!

Parsing Rules file "/etc/snort.conf"

PortVar 'HTTP_PORTS' defined :  [ 80 3128 8000 8080 ]

PortVar 'SHELLCODE_PORTS' defined :  [ 0:79 81:65535 ]

PortVar 'ORACLE_PORTS' defined :  [ 1024:65535 ]

PortVar 'SSH_PORTS' defined :  [ 22 ]

Detection:

   Search-Method = AC-Full-Q

    Split Any/Any group = enabled

    Search-Method-Optimizations = enabled

    Maximum pattern length = 20

ERROR: Unable to open rules file "/etc//etc/snort.d/rules/emerging-attack_response.rules": No such file or directory.

Fatal Error, Quitting..

Now why is the path /etc//etc/ ? I'm at a loss... As you can see from the post above I followed normal convention for snort.conf

EDIT: Well I copied all .rules to /etc/snort.d/rules and now the last error is

ERROR: /etc/snort.d/rules/emerging-current_events.rules(916) !any is not allowed                                                      : !$DNS_SERVERS.

Fatal Error, Quitting..

I commented out that ruleset in snort.conf and everything began smoothly. This means to me the script needs modified again to change the destination directory to /etc/snort.d/rules or some other way of adding rules in snort.conf to reflect the different directory.

Now I am concerned I am missing the entire current_events.rules and what is the .map file for? I left it in /etc/snort.d/ but should I?

This is quite a change from my script and is probably the best way to go. The key thing to making it work is to use the nogpl rules so you will not get a clash with any of the ClearOS rules. This means you don't need my huge script which checks if the the rule exists more than once.

I think it is well worth reading this when deciding which rule sets to use - although it was last update in 2008.

If you want to ad block rules for snortsam (the IPS bit), add them to /etc/sid-block.map which I believe needs to be created. I'll post back if I have the location wrong as I'm at work at the moment. See here for usage.

For my clearOS 6.3 install I made the following recommended changes to the script. Basically removing the emerging-virus.rules, changing the directory to reflect the move to /etc/snort.d/ and changed the wget URL's to snort 2.9.0.3.

A surprise to me was that wget was not installed on ClearOS 6.3 out of the box so a quick

yum install wget

took care of that. Be forewarned.



Here is the modified script:

#!/bin/bash



#

# The purpose of this script is to update Snort & Snortsam to use the most recent Emerging

# Threats rules.

# Threat rules are unzipped in a folder you have to create beforehand ie /tmp/et



# move to temp folder for downloading rules

cd /tmp/et/



# download the rules you want to use, extract & install

wget http://rules.emergingthreats.net/open-nogpl/snort-2.9.0/emerging.rules.tar.gz http://rules.emergingthreats.net/open-nogpl/snort-2.9.0/rules-md5.txt http://rules.emergingthreats.net/blockrules/emerging-botcc-BLOCK.rules http://rules.emergingthreats.net/blockrules/emerging-compromised-BLOCK.rules http://rules.emergingthreats.net/blockrules/emerging-drop-BLOCK.rules http://rules.emergingthreats.net/blockrules/emerging-dshield-BLOCK.rules http://rules.emergingthreats.net/blockrules/emerging-rbn-BLOCK.rules http://rules.emergingthreats.net/blockrules/emerging-tor-BLOCK.rules http://rules.emergingthreats.net/blockrules/emerging-rbn-malvertisers-BLOCK.rules



tar -xzf emerging.rules.tar.gz



# Move a copy of the downloaded rules to /root for backup

# You will have to manually delete the rules in /root occasionally

mv emerging.rules.tar.gz /root/`date +%Y%m%d`_emerging.rules.tar.gz



# Create a sid-map for the emerging rules

cp /tmp/et/rules/sid-msg.map /tmp/et/rules/emerging-sid-msg.map



# Copy/update all the Block rules to /etc/snort.d/

# Note that you have to add each of the rules added to the bottom of the snort.conf in order for them to run

cp -pf *-BLOCK.rules /etc/snort.d/



# Copy other rules that you want to use to /etc/snort.d/

# Note that you have to add each of the rules added to the bottom of the snort.conf in order for them to run

cp -pf /tmp/et/rules/emerging-exploit.rules /tmp/et/rules/emerging-current_events.rules /tmp/et/rules/emerging-attack_response.rules /tmp/et/rules/emerging-malware.rules /tmp/et/rules/emerging-trojan.rules /tmp/et/rules/emerging-p2p.rules /tmp/et/rules/emerging-web_client.rules /tmp/et/rules/emerging-web_specific_apps.rules /tmp/et/rules/emerging-worm.rules /tmp/et/rules/emerging-smtp.rules /tmp/et/rules/emerging-pop3.rules /tmp/et/emerging-imap.rules /tmp/et/rules/emerging-sid-msg.map /etc/snort.d/



# remove temp files rules

rm -Rf /tmp/et/rules/ /tmp/et/emerging.rules.tar.gz /tmp/et/rules-md5.txt /tmp/et/*-BLOCK.rules



# restart snort and snortsam

service snort restart > /dev/null

service snortsam restart > /dev/null

I went and added the following lines to the end of /etc/snort.conf

include $RULE_PATH/emerging-attack_response.rules

include $RULE_PATH/emerging-compromised-BLOCK.rules

include $RULE_PATH/emerging-current_events.rules

include $RULE_PATH/emerging-drop-BLOCK.rules

include $RULE_PATH/emerging-dshield-BLOCK.rules

include $RULE_PATH/emerging-exploit.rules

include $RULE_PATH/emerging-malware.rules

include $RULE_PATH/emerging-p2p.rules

include $RULE_PATH/emerging-pop3.rules

include $RULE_PATH/emerging-rbn-BLOCK.rules

include $RULE_PATH/emerging-rbn-malvertisers-BLOCK.rules

include $RULE_PATH/emerging-smtp.rules

include $RULE_PATH/emerging-tor-BLOCK.rules

include $RULE_PATH/emerging-trojan.rules

include $RULE_PATH/emerging-web-client.rules

include $RULE_PATH/emerging-web_specific_apps.rules

include $RULE_PATH/emerging-worm.rules

Hi Rob.

Does anyone know if there is a new filename that has replaced emerging-virus.rules that I should use rather than just commenting it out?

The emerging-virus.rules were discontinued. The relevant definitions and new rules are put into the malware and trojan rules. The old rules were deleted.

I only found out over the past few weeks.

Hope this helps.

Nuke

Sorry. I'm lazy and go for the emerging-all rules. It means I get a lot of log entries for services I don't run or expose to the internet, but it saves picking and choosing the rule sets.

I noticed last week that I was getting a failure message mailed to me:

/etc/cron.weekly/update_snort_rules.sh:

emerging-virus.rules not downloaded
Exiting

I checked at http://rules.emergingthreats.net/open/snort-2.8.4/rules/ and see that there is no longer a emerging-virus.rules file in the directory, so I just commented it out in the script...

The script completed without error after that....

Does anyone know if there is a new filename that has replaced emerging-virus.rules that I should use rather than just commenting it out?

-Rob A>

What you are describing to change the script sounds about right as the ClearOS file locations have changed and as the snort version has changed as well you need to pull the rules from a different bit of the Emerging Threats site.

The script did not specifically back up the files, it just did not get round to deleting the downloaded rules. I was in two minds about that when I wrote the script. The best thing to do is to back up your snort.conf before you make the changes. If anything goes wrong you can just restore your old .conf file and you'll be OK. You may leave a few files (downloaded, combined and possibly working) lying around but it won't do any damage.

[edit]
I see nuke beat me by about a minute. I believe his method of using the nogpl gets round the issue of duplicate rules and to a large extent makes my script redundant. The only thing my script would add is the ability to add block rules.

If you don't want to use vi or nano and you have a Windoze box, use the test editor in WinSCP. I never use vi and have used nano only a couple of times.
[/edit]

I'm running COS as a home server too. I've got everything pretty locked down. Recently started getting email bombed so it doesn't matter to the a-holes out there if you are a corporate, educational or home network. If you have ports open someone will try to exploit them.

So I'll try to tackle you're questions in order. I'm not an expert. But we have a wonderful community here and they'll surely correct or explain why something should/shouldn't be done.

Yes, you should change to use the 2.9 version.

I can't really answer you on where the rules go in 6.2 since I don't run that version yet. You could check into the more recent version of Redhat that COS 6.x is running on to see if that is the right directory for the snort rules. But, it sounds like you found it.

In addition to copying the ET rules into the directory, you will have to enable the snort.conf to actually use the rule. You can probably find out if the /etc/snort.d/ directory is the correct place for the rules from the snort.conf file.

I save the backup file because if the new rules somehow break something, I have a copy of the last rule set I used. I don't always have the time to figure out what went wrong, so this is my way of having a Plan B until I can figure out the problem. Having said that, I don't think I had a problem since I got the rules running in the first place.

As a suggestion, add only a few rules at a time and check there is no conflict. Theoretically there shouldn't be since the Snort provided rules should have rule numbers < 250000 (or something like that) and ET are larger numbers.

Yes you will need to use Nano. I ended up using vi since Nano sometimes didn't copy/paste long lines properly for me. But it doesn't matter. You need to use a editor as root.

Pleasure to help. I like to give back in return for all the help I have received.

I was quite shocked when I realized that port 81 was open by default. My memory of Clarkconnect was getting my school IP banned from my own network for trying to OpenVPN in with my laptop. I also remember watching portscan attempts get zapped like flies - I usually had 3 or 4 banned for portscanning. I liked having an overzealous shoot-first-ask-questions-later firewall because this is a home network that doesn't host anything beyond game servers.

It appears ClearOS 6.2 uses Snort version 2.9.0.3 (In the config, the line is "Compatible with Snort Versions:... Versions: 2.9.0.3).

The folder /etc/snort doesn't exist. However, /etc/snort.d does exist and it has a /rules/ directory in it.

Looking at your script a few posts above it looks like it could be modified to just pull from .../open-nogpl/snort-2.9.0/... and extract to /etc/snort.d/... I'm not certain though and this is a production system (on a home network with family members who aren't afraid to bludgeon the resident geek for an Internet outage), so any changes I make, if it breaks something, it needs to theoretically be something I can roll back quickly.

I did notice that it automatically backs up the rules it's downloading. I'm not on a high risk network and I'm fine with manually fetching the rules files manually if something goes wrong, is there any other reason to have the backups?

Am I right to assume that I won't be able to turn the rules on and off through the ClearOS webconfig and will instead have to get my hands dirty with KiTTY and Nano?

Thanks for your time!

The open port issue you can control in the webconfig if you haven't done so already. I'm actually surprised also that there are a bunch of ports open. This hasn't been the case in past releases. You had to open every port specifically. In webconfig you should have something like -> Network:Firewall:Incoming where you can specifically open ports. I suspect that there are some rules there that open ports for apps that you've installed???

As to the IDS/IPS scripts, you will need to check which version of snort is running in COS6.2. I've not upgraded yet so I haven't checked which version of snort is used in COS6.2. When you know the version of snort, then you just have to update the script to download the rules for that version from Emerging Threats. You'll find the download location if you go to the Emerging Threats download site.

And, of course you will need to decide which of the rules you want to run and update the config file for snort so that those rules run.

I realize this is a bit vague but hope you can figure it out. If not, post another question and we''ll try to help.