I have the answer about the X-Spam headers. When spamassassin (SA) is integrated with amavisd-new, the X-Spam headers are inserted by amavisd-new and not SA. It is in the "No spam-related headers inserted?" section of the amavisd-new spam FAQ's and it is controlled by the parameter $sa_tag_level_deflt. On my default installation this comes set to -99. If something is whitelisted the spam scoring starts at -100, so if the rest of the scores add up to less than 1 the overall score is less than -99 and no headers are inserted. I've confirmed this by setting it to -199 and suddenly I get all the spam headers.

I have come to the conclusion that using free uribl is like a lottery... Like most things you get what you pay for - and if you subscribe to uribl - it is probably reliable.

No matter how you setup a DNS or what program you use - you don't access the internet's root servers directly - but progress towards them with a DNS lookup until you get an answer from a higher level server... and if uribl has banned that... no go. Also some free DNS providers will automatically route to the nearest DNS server they operate - so you might not be using the DNS server address you thought you were. This is what mystified me at first with DNS servers I knew nothing about that uribl had banned. At least that is the way I understand it - but happy to be corrected. Not done an in depth analysis... not important enough for me.

and a bit OT but while talking about DNS servers - some 'free' ones are not there as a public service, but to make money e.g. 8.8.8.8/8.8.4.4 and Google's all-knowing policy and single login + capturing your details isn't for fun. It's about total tracking and monetization of you and/or what you search for and/or sites you access as the product.

Mike Stafford wrote:
If it makes you feel any better, Reindl Harald has a reputation. There has been more than one request to suspend him from the Fedora dev mailing list.

Someone also said this to me off-list about his attitude. I just wish the experience was more pleasant.

@Tony,
I'd remembered you ran BIND when I posted so I did not think you were going to have this issue but it appears you do a bit as well.

According to Reindl Harald, spamassassin is perfect and it must be amavis. I'm going to struggle to get a test case as it looks like the amavis website likes you to put amavis into debug mode before you post for help. The last e-mail from peacocks-mail.com went through spamassassin so that is no good. I've just tried whitelisting ebay like you to see if I get the same results as you. If I do, that would be good as my wife is an avid user so we get lots of e-mails from them. If not, are you able to get a test case to post to the amavis mailing list?

For the moment I've slightly back-tracked with my unbound set up, reinstating dnsmasq for all normal DNS queries, but just pointing spamassassin to unbound on port 1053 (after which the daily peacocks-mail.com worked but that may be a coincidence).

Nick Howitt wrote:
So far the only results are that I've been flamed for not running my own DNS resolver

If it makes you feel any better, Reindl Harald has a reputation. There has been more than one request to suspend him from the Fedora dev mailing list.

I cannot edit my append because of a "Unable to load composer." error :-(

Ignore the last section of my most recent append and use this instead...

Going to http://uribl.com/refused.shtml and running the test shown there, sometimes the query is refused and sometimes not. In all cases of refusal it is one or other of my two internet addresses that is refused. However, the DNS address in the reply when there is a refusal is one I know nothing about e.g. 74.125.41.69 and
173.194.93.7. No idea where these are coming from...
I can only assume that the quantity of emails I handle must be hitting their free limit some of the time... or something wierd with DNS or???

Interesting - thanks for the continuing investigation Nick...

Well... I run bind, not dnsmasq, and sometimes I get "URIBL_BLOCKED=0.001" in headers and sometimes not, when I checked a few just now....

Going to http://uribl.com/refused.shtml and running the test shown there, sometimes the query is refused and sometimes not. In all cases of refusal it is one or other of my two internet addresses that is refused.

I can only assume that the quantity of emails I handle must be hitting their free limit some of the time...

Well I received another e-mail from peacocks-mail.com address yesterday so I posted to the SpamAssassin mailing list here. So far the only results are that I've been flamed for not running my own DNS resolver causing some of the spam checks to fail. Virtually everyone who has replied has said that dnsmasq is no good for running your own mail server with spamassassin as many or the RBL look ups will appear from your chosen DNS servers (OpenDNS for me) which is pretty much guaranteed to have used more than its free quota of lookups at the various RBL providers so the checks will fail (URIBL_BLOCKED=0.001 in the header).

No constructive comments about the issue. Best being "Dunno" and "perhaps amavis". Clearly SpamAssassin is perfect.

Now I've set up unbound to keep them happy I hope for something more constructive. Also amavisd-new could be a lead.

I can't do much today. I've always been dropping localhost->my_domain lines from the headers using postfix header_checks so some of the evidence is being destroyed (e.g. it removes the received: header which hands over to amavisd-new). I've reset my header_checks to preserve the lines. Hopefully later in the week I may be able to post to the spamassassin mailing list.

In your headers you can see the handover to amavisd-new so we know that is working. Even on my filtered header we can assume it is working as I get the X-Virus-Scanned header, but I'd rather have a full header before I post to a new list.

Nick - you are not alone... Found this in the morning's email...
I have *@reply1.ebay.com whitelisted - extract from the headers...

Return-Path: <e3us-1866060775958-2e54b5ii12b968ii2@reply1.ebay.com>

Received: from localhost (localhost [127.0.0.1])

	 by danda.sraellis.com (Cyrus v2.3.16-Fedora-RPM-2.3.16-13.v6) with LMTPA;

	 Mon, 23 May 2016 08:49:01 +1000

X-Sieve: CMU Sieve 2.3

Received: from localhost (localhost [127.0.0.1])

	by danda.sraellis.com (Postfix) with ESMTP id DFBD26A239

	for <sraellis@sraellis.com>; Mon, 23 May 2016 08:49:01 +1000 (AEST)

X-Virus-Scanned: amavisd-new at sraellis.com

Received: from danda.sraellis.com ([127.0.0.1])

	by localhost (danda.sraellis.com [127.0.0.1]) (amavisd-new, port 10024)

	with ESMTP id p9aGsYucG3zF for <sraellis@sraellis.com>;

	Mon, 23 May 2016 08:48:47 +1000 (AEST)

Received: from localhost (localhost [127.0.0.1])

	by danda.sraellis.com (Postfix) with ESMTP id B9D1B6A23C

	for <sraellis@sraellis.com>; Mon, 23 May 2016 08:48:47 +1000 (AEST)

Something very fishy is going on. This morning another e-mail from peacocks-mail.com shows no evidence in the header of having gone through spamassassin with a whitelisted entry of *@peacocks-mail.com, but an e-mail from clearos.com had all the correct headers when whitelisted with *@clearos.com. I'm now trying resetting the whitelist and manually typing it in again.

Totally puzzled :S

From single whitelisted address:

Return-Path: <ANO@rocketmail.com>

Received: from localhost (localhost [127.0.0.1])

	 by server.mydomain.com (Cyrus v2.3.16-Fedora-RPM-2.3.16-13.v6) with LMTPA;

	 Thu, 19 May 2016 18:37:56 +0100

X-Sieve: CMU Sieve 2.3

X-Virus-Scanned: amavisd-new at mydomain.com

X-Spam-Flag: NO

X-Spam-Score: -98.697

X-Spam-Level: 

X-Spam-Status: No, score=-98.697 tagged_above=-99 required=5

	tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,

	FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SORBS_SPAM=0.5,

	URIBL_BLOCKED=0.001, URIBL_SC_SWINOG=0.9, USER_IN_WHITELIST=-100]

	autolearn=no

Received: from nm4-vm8.bullet.mail.ir2.yahoo.com (nm4-vm8.bullet.mail.ir2.yahoo.com [212.82.96.106])

	by mailserver.mydomain.com (Postfix) with ESMTPS id 218D7E60E9

	for <me@mydomain.com>; Thu, 19 May 2016 18:37:54 +0100 (BST)

From un-whitelisted address:

Return-Path: <ANO@googlemail.com>

Received: from localhost (localhost [127.0.0.1])

	 by server.mydomain.com (Cyrus v2.3.16-Fedora-RPM-2.3.16-13.v6) with LMTPA;

	 Sat, 21 May 2016 06:14:44 +0100

X-Sieve: CMU Sieve 2.3

X-Virus-Scanned: amavisd-new at mydomain.com

X-Spam-Flag: NO

X-Spam-Score: 0.403

X-Spam-Level: 

X-Spam-Status: No, score=0.403 tagged_above=-99 required=5

	tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,

	FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SORBS_SPAM=0.5,

	URIBL_BLOCKED=0.001] autolearn=no

Received: from mail-wm0-f43.google.com (mail-wm0-f43.google.com [74.125.82.43])

	by mailserver.mydomain.com (Postfix) with ESMTPS id 0C96AE69FA

	for <me@mydomain.com>; Sat, 21 May 2016 06:14:42 +0100 (BST)

From whitelisted domain (*@peacocks-mail.com)

Return-Path: <g-22286502917-22331-2200267037-1463811224122@bounce.peacocks-mail.com>

Received: from localhost (localhost [127.0.0.1])

	 by server.mydomain.com (Cyrus v2.3.16-Fedora-RPM-2.3.16-13.v6) with LMTPA;

	 Sat, 21 May 2016 07:13:46 +0100

X-Sieve: CMU Sieve 2.3

X-Virus-Scanned: amavisd-new at mydomain.com

Received: from unusquattuorseptem.theta.eccluster.com (unusquattuorseptem.theta.eccluster.com [91.192.41.147])

	by mailserver.mydomain.com (Postfix) with ESMTP id CE7FFE2255

	for <me@mydomain.com>; Sat, 21 May 2016 07:13:44 +0100 (BST)

It looks like the last one completely bypassed spamassassin, but the previous one, less than an hour earlier, passed through it as have later e-mails - and the ClearOS one notifying me of your post got spammed because of URIBL_JP_SURBL=7! I'll see what happens when the next peacocks e-mail comes in (or clearos.com which I've now whitelisted).

Another example -part of the headers of an email which is also has a whitelisted address...

Return-Path: <bo-b80xr1wbfef58cauzd1etbymrfqh86@b.e.liteneasy.com.au>

Received: from localhost (localhost [127.0.0.1])

	 by danda.sraellis.com (Cyrus v2.3.16-Fedora-RPM-2.3.16-13.v6) with LMTPA;

	 Tue, 17 May 2016 14:45:29 +1000

X-Sieve: CMU Sieve 2.3

Received: from localhost (localhost [127.0.0.1])

	by danda.sraellis.com (Postfix) with ESMTP id 122E46ACCE

	for <aellis@sraellis.com>; Tue, 17 May 2016 14:45:29 +1000 (AEST)

X-Virus-Scanned: amavisd-new at sraellis.com

X-Spam-Flag: NO

X-Spam-Score: -98.768

X-Spam-Level: 

X-Spam-Status: No, score=-98.768 tagged_above=-99 required=5

	tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,

	DKIM_VALID_AU=-0.1, FSL_HELO_NON_FQDN_1=0.001,

	HTML_IMAGE_RATIO_02=0.437, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7,

	RDNS_NONE=0.793, USER_IN_WHITELIST=-100] autolearn=no

Nick - here is the full header from the spam I whitelisted, don't go to their web-site :-)

Return-Path: <bounce@new.coinletters1.com>

Received: from localhost (localhost [127.0.0.1])

	 by danda.sraellis.com (Cyrus v2.3.16-Fedora-RPM-2.3.16-13.v6) with LMTPA;

	 Sat, 21 May 2016 07:34:58 +1000

X-Sieve: CMU Sieve 2.3

Received: from localhost (localhost [127.0.0.1])

	by danda.sraellis.com (Postfix) with ESMTP id D69AB6B836

	for <aellis@sraellis.com>; Sat, 21 May 2016 07:34:58 +1000 (AEST)

X-Virus-Scanned: amavisd-new at sraellis.com

X-Spam-Flag: NO

X-Spam-Score: -93.097

X-Spam-Level: 

X-Spam-Status: No, score=-93.097 tagged_above=-99 required=5

	tests=[BAYES_99=3.5, BAYES_999=0.2, FSL_HELO_NON_FQDN_1=0.001,

	HTML_MESSAGE=0.001, PRICES_ARE_AFFORDABLE=0.551,

	RCVD_IN_DNSWL_LOW=-0.7, RDNS_NONE=0.793, TRACKER_ID=1.306,

	URIBL_ABUSE_SURBL=1.25, URIBL_BLOCKED=0.001, USER_IN_WHITELIST=-100]

	autolearn=no

Received: from danda.sraellis.com ([127.0.0.1])

	by localhost (danda.sraellis.com [127.0.0.1]) (amavisd-new, port 10024)

	with ESMTP id GkHUIp0slYBk for <aellis@sraellis.com>;

	Sat, 21 May 2016 07:34:57 +1000 (AEST)

Received: from localhost (localhost [127.0.0.1])

	by danda.sraellis.com (Postfix) with ESMTP id 2847F6B83A

	for <aellis@sraellis.com>; Sat, 21 May 2016 07:34:57 +1000 (AEST)

Received: from danda.sraellis.com (localhost [127.0.0.1])

	by danda.sraellis.com (Postfix) with ESMTP id 120756B836

	for <aellis@localhost.>; Sat, 21 May 2016 07:34:57 +1000 (AEST)

Received: from mail.bigpond.com [61.9.189.249]

	by danda.sraellis.com with POP3 (fetchmail-6.3.17)

	for <aellis@localhost.> (single-drop); Sat, 21 May 2016 07:34:57 +1000 (AEST)

Received: from nschwcmgw04p ([61.9.190.164]) by nschwmtas03p.mx.bigpond.com

          with ESMTP

          id <20160520213338.EZDR2036.nschwmtas03p.mx.bigpond.com@nschwcmgw04p>

          for <aellis@bigpond.net.au>; Fri, 20 May 2016 21:33:38 +0000

Received: from relay678.mysmtp.mobi ([93.90.123.52])

	by nschwcmgw04p with BigPond Inbound

	id wlZM1s01g17wP8D01lZNRV; Fri, 20 May 2016 21:33:33 +0000

X-Authority-Analysis: v=2.0 cv=Op3NOlDt c=1 sm=1 b=1 p=_Co6XzKuwfkA:10

 p=gSt2w7CqAAAA:8 p=PJqjg17qwVoS8VFneAEA:9 a=ZZnuYtJkoWoA:10 a=yrkiwgmsf1kA:10

 a=i7L2eM_4Xz4grGw359gA:9 a=QEXdDO2ut3YA:10 a=2hlaKCniJtwA:10

 a=05QaKCh-KrAA:10 a=afyMZWh9bLAA:10 a=SSmOFEACAAAA:8 a=P4HG6qqaAAAA:8

 a=frz4AuCg-hUA:10 a=6zGKsNmq6mMA:10 a=mYjIQDWp9HsA:10 a=FLFSC_Nf5vAA:10

 a=GF28B-u7t_YA:10 a=RKS9kxj4ezgA:10 a=ORwa81s4RVoA:10 a=LS73N7N7V50A:10

 a=-WhZ_FieKgsA:10 a=L8w3y3esxnsA:10 a=Yr-BkEv6mRUA:10 a=Wk6o2s_tGOcA:10

 a=MBRWn8yRYF0A:10 a=p403mkujtbAA:10 a=LmZ0Pe8dLuZuL8q6:21

 a=jAOle1nqshde_Pxc:21 a=bOrnOhHowNB5xXY1:21 a=x5LRralTdJy7ZLmChtFrAw==:117

Received: from relay678.mysmtp.mobi (relay678.mysmtp.mobi [93.90.123.52]) 

	by relay678.mysmtp.mobi (Postfix) with ESMTPA id F04EB788738

	for <aellis@bigpond.net.au>; Fri, 20 May 2016 22:58:04 +0200 (CEST)

To: aellis@bigpond.net.au

Subject: Re: Dear , your last chance to confirm your Woolworths package. Dont miss it!

Message-ID: <807aba58cbd745086fbdc663ff305e40@new.coinletters1.com>

Date: Fri, 20 May 2016 22:58:38 +0200

From: "IT" <info@new.coinletters1.com>

Reply-To: info@new.coinletters1.com

MIME-Version: 1.0

X-Mailer-LID: 167,255,156,157,148

List-Unsubscribe: <http://new.coinletters1.com/JG5bU30428897_1TIp3374ddcbec53a7f7c1217b22b21419c5/2KO0156/Lpo21450.html>;

X-Mailer-RecptId: 30428897

X-Mailer-SID: 1450

X-Mailer-Sent-By: 6

x-job: publicmailer2_dk-1794

Content-Type: multipart/alternative; charset="UTF-8"; boundary="b1_1ae129717607d59f26395cdbb340a06f"

Content-Transfer-Encoding: 8bit

Actually looking at it I wonder whether it would have been tagged if not whitelisted...

Hi Tony,
Thanks for that. Yesterday I was researching spamassassin and came to the conclusion that I probably needed the leading "*". Checking other e-mails, whitelisting seemed to be working where I specified the full e-mail address, just not the "domain" ones I was using. I am now waiting for a relevant message - at the moment it looks like spamassassin has not even checked the relevant message this morning but has checked another non-whitelisted message OK.

If you have in your whitelist *@something.com do you see any X-Spam entries in your header? The whitelisted e-mail is missing them this morning, but another genuine e-mail has the normal X-Spam entries in the header.
Nick

Hi Nick - whitelisting is working for me on ClearOS 6.7. Yesterday when I saw your email I whitelisted an address that consistently sends spam. This morning I had two emails in my inbox not tagged as spam from them and none in Trash. I subscribed to the Peacock Newsletter - will see what happens...
Not sure if it makes a difference, but I always put a "*" in the beginning of an address to whitelist e.g. *@ancestry.com.au