Community Forum

Share this post:
Responses (16)
  • Accepted Answer

    Monday, June 18 2018, 07:41 AM - #Permalink
    Resolved
    0 votes
    I have not used gmail for years so it may have changed. In the past gmail used to allow you to relay through a valid account up to about 8 or 12 permitted other e-mail addresses but they would rewrite either the "from" or "reply-to" header. Is this no longer the case?

    In my testing I did not see postfix being bypassed, but I believe it has to be set up in the e-mail settings except because of a installation failed setting. Once that was fixed, I saw my mails as relaying to the docker LAN IP which is picked up by postfix which binds to all local IP's and being sent from the ClearGLASS subnet (the br-????? interface). Postfix then relayed via my ISP's SMTP sever as normal but my ISP does not use authenticated SMTP and allows me to relay on port 25.
    The reply is currently minimized Show
  • Accepted Answer

    Todd
    Todd
    Offline
    Monday, June 18 2018, 05:57 AM - #Permalink
    Resolved
    0 votes
    There are two issues. The email is getting sent as team@clear.glass, and the email mails being sent by the docker container are bypassing the relay configured by the clearos app.
    Here is the fix

    1st Create a Canonical map that converts team@clear.glass to username@yourdomain.com
    2nd forward all the email to your smtp relay server.
    The example below is for gmail. Gmail smtp relay will only work if you "enable less secure apps" in gmail.


    cp /etc/postfix/main.cf /etc/postfix/main.cf_backup

    #Create a canonical map
    touch /etc/postfix/sender_canonical
    chmod 600 /etc/postfix/sender_canonical
    echo '/team@clear.glass/ username@gmail.com' >> /etc/postfix/sender_canonical

    #Test the sender connical mask
    postmap -q "team@clear.glass" regexp:/etc/postfix/sender_canonical

    #Enter email address and password
    touch /etc/postfix/sasl_passwd
    chmod 600 /etc/postfix/sasl_passwd
    echo '[smtp.gmail.com]:587 username@gmail.com:password' >> /etc/postfix/sasl_passwd

    #Create a db file
    postmap /etc/postfix/sasl_passwd

    # Add the following to the bottom of /etc/postfix/main.cf
    # I left it as a script
    echo 'relayhost = [smtp.gmail.com]:587' >> /etc/postfix/main.cf
    echo 'echo "smtp_use_tls = yes ' >> /etc/postfix/main.cf
    echo 'smtp_sasl_auth_enable = yes ' >> /etc/postfix/main.cf
    echo 'smtp_sasl_security_options = ' >> /etc/postfix/main.cf
    echo 'smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd' >> /etc/postfix/main.cf
    echo 'smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt' >> /etc/postfix/main.cf
    echo 'sender_canonical_maps = regexp:/etc/postfix/sender_canonical' >> /etc/postfix/main.cf


    systemctl restart postfix.service
    The reply is currently minimized Show
  • Accepted Answer

    Sunday, June 17 2018, 04:13 PM - #Permalink
    Resolved
    0 votes
    More issues:

    1. Is ClearGlass hijacking my server FQDN and forcing an http -> https rewrite as I can no longer access the transmission webconfig on port 9090 (I had to switch it because of Openfire), but it is still available on http if I use another FQDN which resolves to the same IP. If so it is a bit antisocial and should only be rewriting on port 9443.
    2. In the settings section of the webconfig, you appear to be able to choose the ClearGlass SSL certificate, but the "Go to ClearGLASS" button link stays pointing to the server name. Shouldn't the link follow the certificate? If it does not, you get an Insecure Connection warning in the browser. I am trying to get round the problem in 1 above, but if I create a new certificate for clearglass.howitts.co.uk and tell the webconfig to use it and to all the DNS mapping, the Go To button still points to server.howitts.co.uk. and http to server.howitts.co.uk still gets rewritten to https. /var/lib/clearglass/settings/settings.py is being updated to the new CORE_URI, just not the button.
    3. The log files are horrendous between docker and ClearGlass. 13MB in under 3 days and I have not yet done anything in ClearGlass. Half the lines have " INFO " or "level=info" in them but this would still only cut the log in by half. Most of these are ClearGLASS logs and a few are docker. Can anything be done to cut this down without resorting too filtering in rsyslog?
    4. I have a recurring error:
    5. Jun 14 22:18:48 server journal: [httpd] 172.19.0.16 - - [14/Jun/2018:21:18:48 +0000] "POST /query?q=CREATE+DATABASE+metering HTTP/1.1" 200 57 "-" "python-requests/2.18.4" 876a9a6a-7018-11e8-8128-000000000000 283
      Jun 14 22:18:48 server journal: #033[1;31m2018-06-14 21:18:48,824 ERROR Dummy-1 methods - get_usage: Failed to execute: SELECT MAX(cores) AS cores, NON_NEGATIVE_DERIVATIVE(MAX(checks)) AS checks FROM usage WHERE time >= '2018-06-12 00:00:00'GROUP BY time(1d),owner#033[0m#015
      Jun 14 22:18:48 server journal: 2018-06-14 21:18:48,825 INFO Dummy-1 job - on_success: Task mist.api.monitoring.tasks.reset_traefik_config[5a12b931-3bee-4d5c-96c7-0c2da9f46b12] succeeded in 0.0050125800044s: None#015
      Jun 14 22:18:48 server journal: #033[1;31m2018-06-14 21:18:48,826 ERROR Dummy-1 log - log: Task mist.api.portal.tasks.check_new_versions[e0d5b276-6628-46aa-95ee-12d1592bf2ea] raised unexpected: BadRequestError("Bad Request: Bad Request: Failed to parse results: {u'results': [{u'statement_id': 0}]}",)#015
      Jun 14 22:18:48 server journal: Traceback (most recent call last):#015
      Jun 14 22:18:48 server journal: File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task#015
      Jun 14 22:18:48 server journal: R = retval = fun(*args, **kwargs)#015
      Jun 14 22:18:48 server journal: File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 438, in __protected_call__#015
      Jun 14 22:18:48 server journal: return self.run(*args, **kwargs)#015
      Jun 14 22:18:48 server journal: File "/mist.api/src/mist/api/portal/tasks.py", line 33, in check_new_versions#015
      Jun 14 22:18:48 server journal: params = get_version_params(portal)#015
      Jun 14 22:18:48 server journal: File "/mist.api/src/mist/api/portal/tasks.py", line 25, in get_version_params#015
      Jun 14 22:18:48 server journal: for key, value in get_current_portal_usage().items():#015
      Jun 14 22:18:48 server journal: File "/mist.api/src/mist/api/metering/methods.py", line 79, in get_current_portal_usage#015
      Jun 14 22:18:48 server journal: usage = get_usage(owner_id='', full_days=2)#015
      Jun 14 22:18:48 server journal: File "/mist.api/src/mist/api/metering/methods.py", line 57, in get_usage#015
      Jun 14 22:18:48 server journal: raise BadRequestError('Failed to parse results: %s' % results)#015
      Jun 14 22:18:48 server journal: BadRequestError: Bad Request: Bad Request: Failed to parse results: {u'results': [{u'statement_id': 0}]}#033[0m#015
      er journal: BadRequestError: Bad Request: Bad Request: Failed to parse results: {u'results': [{u'statement_id': 0}]}#033[0m#015
      It looks like it may just be update checking for a new version of ClearGlass, but the error repeats every 2 seconds generating over 31,000 lines of logs in under 3 days.
    The reply is currently minimized Show
  • Accepted Answer

    Friday, June 15 2018, 08:28 PM - #Permalink
    Resolved
    0 votes
    I have a suggestion for the mail server Trusted Networks. I am very uncomfortable about adding a 172.16/12 subnet to Trusted Networks as it adds the whole 172.16/12 address space when it really only needs a /16 subnet within it. All ClearOS recommendations to date are not to use Trusted Networks, but to use authentication, then suddenly we are trusting a huge address space.

    My suggestion is to add another parameter to /etc/postfix/main.cf, lets say "clearglassnetwork" and set it to the ClearGlass network. This can be set from the br-????????? interface. Then change the "mynetworks" to something like:
    mynetworks = 127.0.0.0/8, [::1]/128, 172.17.2.0/23, $clearglassnetwork
    (172.17.2.0/23 is my own network)

    The SMTP Server Webconfig still works and you can change, add and delete networks, but you now also have a separate parameter which can be maintained programatically and is clearly distinguishable in the webconfig so it won't be mistaken for a user added subnet.
    The reply is currently minimized Show
  • Accepted Answer

    Friday, June 15 2018, 04:34 PM - #Permalink
    Resolved
    0 votes
    Hmm. It looks like a possible solution to the docker DNS issue is to add a line to /etc/docker/daemon.json:
    "dns": ["172.18.0.1"]
    Where the IP is that of the docker0 interface. Is it possible to do that automatically and maintain it in ClearOS or does the IP not exist if docker is not running? Having stopped docker I can see the still interface exists, but does it always?

    [edit]
    Some sort of solution like this is going to be needed to manage on-premises devices by FQDN, otherwise you are stuck with managing them by IP address.
    [/edit]


    I am also seeing in the logs:
    level=warning msg="could not change group /var/run/docker.sock to docker: group docker not found"
    Should the group exist ans is there any harm creating it?
    The reply is currently minimized Show
  • Accepted Answer

    Friday, June 15 2018, 01:34 PM - #Permalink
    Resolved
    0 votes
    I'm seeing this in my logs:
    Jun 14 18:45:58 server dockerd-current: time="2018-06-14T18:45:58.083955517+01:00" level=info msg="No non-localhost DNS nameservers are left in resolv.conf. Using default external servers: [nameserver 8.8.8.8 nameserver 8.8.4.4]"
    Jun 1
    But in ClearOS, resolvers are kept in /etc/resolv-peerdns.conf. Is there anything that can be done to make docker either read that, or, better, use dnsmasq as its resolver do it can use the full ClearOS set up?
    The reply is currently minimized Show
  • Accepted Answer

    Thursday, June 14 2018, 07:36 PM - #Permalink
    Resolved
    0 votes
    I've had some help from the devs and my e-mail problems were a combination of installation and self-inflicted issues.

    The self-inflicted issues were:
    1 - I'd noticed a trusted networks in the SMTP server set to 172.16.0.0/12 which I didn't recognise so I'd deleted it. Something is needed to cover the subnets of the docker0 and the br-????????? interfaces. I've asked if the devs can narrow down the range they add to the trusted networks as you really want to trust as little as possible.
    2 - As an anti-spam measure, in the SMTP server I'd blocked all mail from IP's without a reverse DNS record and the docker0 IP's don't have a reverse DNS record.

    The installation issue:
    The installer creates a file /var/lib/clearglass/settings/settings.py and in it should be the IP address of the docker0 interface and for some reason it got it wrong. Rerunning the installation script /var/clearos/events/network_configuration/clearglass fixed that.
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, June 13 2018, 10:48 AM - #Permalink
    Resolved
    0 votes
    I've had a number of issues getting this going.

    1 - docker was complaining about filesystem support:
    Jun  7 13:42:47 server dockerd-current: time="2018-06-07T13:42:47.424497984+01:00" level=warning msg="overlay2: the backing xfs filesystem is formatted without d_type support, which leads to incorrect behavior. Reformat the filesystem with ftype=1 to enable d_type support. Running without d_type support will no longer be supported in Docker 1.16."
    I fixed this by moving /var/lib/docker onto a compliant file system (xfs with ftype=1 - check with the command "xfs_info /dev/sdaX" where /dev/sdaX is your hard drive partition). The LVM/ext4 file system seems to be compliant.

    2 - docker could not send the registration e-mail for a number of reasons. Looking at the message log, docker was operating on the 172.19.0.0/16 subnet so I set up a firewall log to find out what was happening:
    iptables -I FORWARD -s 172.19.0.0/16 -p tcp --dport 25 -j LOG
    and I was seeing:
    Jun 13 11:05:56 server kernel: IN=br-8984974d75f3 OUT=enp2s0 MAC=02:42:a3:f3:21:56:02:42:ac:13:00:16:08:00 SRC=172.19.0.22 DST=172.17.0.1 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=27553 DF PROTO=TCP SPT=45430 DPT=25 WINDOW=29200 RES=0x00 SYN URGP=0 
    Jun 13 11:06:00 server kernel: IN=br-8984974d75f3 OUT=enp2s0 MAC=02:42:a3:f3:21:56:02:42:ac:13:00:16:08:00 SRC=172.19.0.22 DST=172.17.0.1 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=27554 DF PROTO=TCP SPT=45430 DPT=25 WINDOW=29200 RES=0x00 SYN URGP=0
    Jun 13 11:06:08 server kernel: IN=br-8984974d75f3 OUT=enp2s0 MAC=02:42:a3:f3:21:56:02:42:ac:13:00:16:08:00 SRC=172.19.0.22 DST=172.17.0.1 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=27555 DF PROTO=TCP SPT=45430 DPT=25 WINDOW=29200 RES=0x00 SYN URGP=0
    Jun 13 11:06:24 server kernel: IN=br-8984974d75f3 OUT=enp2s0 MAC=02:42:a3:f3:21:56:02:42:ac:13:00:16:08:00 SRC=172.19.0.22 DST=172.17.0.1 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=27556 DF PROTO=TCP SPT=45430 DPT=25 WINDOW=29200 RES=0x00 SYN URGP=0
    Jun 13 11:06:56 server kernel: IN=br-8984974d75f3 OUT=enp2s0 MAC=02:42:a3:f3:21:56:02:42:ac:13:00:16:08:00 SRC=172.19.0.22 DST=172.17.0.1 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=27557 DF PROTO=TCP SPT=45430 DPT=25 WINDOW=29200 RES=0x00 SYN URGP=0
    It looks like docker is working out my mail server incorrectly. It is on 172.17.2.1 not 172.17.0.1. To get round this I've set up a firewall rule to redirect the traffic to the mail server:
    iptables -t nat -I PREROUTING -s 172.19.0.0/16 -d 172.17.0.1 -p tcp --dport 25 -j DNAT --to-destination 172.17.2.1
    At this point the mail server was rejecting it. From /var/log/maillog:
    Jun 13 11:11:22 server postfix/smtpd[32311]: NOQUEUE: reject: RCPT from unknown[172.19.0.22]: 450 4.2.0 <nick@howitts.co.uk>: Recipient address rejected: Greylisted for 289 seconds; from=<team@clear.glass> to=<nick@howitts.co.uk> proto=ESMTP helo=<[172.19.0.22]>
    Jun 13 11:11:22 server postfix/smtpd[32360]: lost connection after RSET from unknown[172.19.0.22]
    Jun 13 11:11:22 server postfix/smtpd[32360]: disconnect from unknown[172.19.0.22]
    Jun 13 11:11:22 server postfix/smtpd[32311]: lost connection after RSET from unknown[172.19.0.22]
    Jun 13 11:11:22 server postfix/smtpd[32311]: disconnect from unknown[172.19.0.22]
    In the SMTP server I set up 172.19.0.0/16 as a trusted network and it all worked.

    Note don't just set up the one IP address you see the e-mail coming from. I restarted docker/ClearGLASS once during this and it switched sending e-mails from 172.19.0.18 to 172.19.0.22 so potentially it could be switching IP's on each start. Looking at other firewall rules it is reserving a whole /16 for itself so that is the subnet you may need to trust in the SMTP server.
    Also note the subnet docker is running on may be different subnets in different installations so have a look in /var/log/messages for the "journal" process to see what your docker subnet is.
    The reply is currently minimized Show
  • Accepted Answer

    Friday, May 04 2018, 03:13 PM - #Permalink
    Resolved
    0 votes
    Well , I reinstalled SMTP server and configured PostFix with SMTP Authentication, now I get NO EMAILS what so ever from ClearOS or ClearGlass, looking at the logs on our mails server it appears ClearOS/ClearGlass does succeed in authenticating just fine, however it does not send anything out port 465, 587 or port 25, it just sits its in queue on SMTP on ClearOS/ClearGlass with the error "Connection Timed out." All other mails on our MAIL server processes just fine, so it is not the in-house mail server, it definitely something with ClearOS/ClearGlass and it's SMTP setup. Once i remove the ClearOS SMTP Server, all goes back to working again, except for emails from ClearGlass.

    I am going to follow-up with ClearOS support, and I let them know about this thread as well as I see others are having issues with the sign up emails as well.
    The reply is currently minimized Show
  • Accepted Answer

    Friday, May 04 2018, 10:14 AM - #Permalink
    Resolved
    0 votes
    Same issue here.

    I am trying to use our inhouse mail server as next hop.

    When trying to sign up all I get is the message "Service Unavailable"
    The reply is currently minimized Show
  • Accepted Answer

    Friday, May 04 2018, 08:51 AM - #Permalink
    Resolved
    0 votes
    To run the SMTP server in ClearOS, in your case, you'd need to configure postfix to use SMTP Authentication, assuming your ISP allows you to relay on port 587.

    However, I do tend to agree that it is a bug. Unfortunately I can't run ClearGlass to test as I don't quite have 8GB RAM - I have 8GB but a portion is shared with the onboard video which trips the installation check
    The reply is currently minimized Show
  • Accepted Answer

    Thursday, May 03 2018, 08:52 PM - #Permalink
    Resolved
    0 votes
    Tried running the SMTP server for ClearOS, this does mess up the ClearOS test emails now. Also I still have the original issue, so I have removed the SMTP server for ClearOS, to get it back to the configuration I had at the time it was build and clearglass was installed.

    ClearGlass still sends everything over Port 25, and bypasses the local SMTP server and external mail relay host completely, It appears to run it's own type of SMTP server on port 25. Though I can't see any SMTP service other than the ClearOS server. I am not sure if there is a SMTP server running in one the docker containers for ClearGlass or not, but that is my suspicion as to the cause of this issue.
    The reply is currently minimized Show
  • Accepted Answer

    Thursday, May 03 2018, 08:42 PM - #Permalink
    Resolved
    0 votes
    If you run the SMTP server in ClearOS, can you relay via that? It can take you mail on port 25 and relay it out on 587. It can do it on 465 but the set up is more complicated and you need a helper program (stunnel).
    The reply is currently minimized Show
  • Accepted Answer

    Thursday, May 03 2018, 08:25 PM - #Permalink
    Resolved
    0 votes
    I think I discovered a bug.....I opened a case with ClearOS support about it.

    It appears that ClearGlass is not using the ClearOS Mail Settings for sending out emails, in our case, We use SSL/Port 465 to send emails to an external server we have this set in ClearOS>System>Settings>Mail Settings, because port 25 is blocked by our ISP, however ClearGlass is sending it's emails out port 25 and not using SSL, so it get rejected. Our test emails from ClearOS which DOES use the Mail Settings, gets sent just fine, however emails from ClearGlass do not.

    Waiting to see if ClearOS confirms this is a bug or by design (which is going to be an issue for us if this is by design.)
    The reply is currently minimized Show
  • Accepted Answer

    luan
    luan
    Offline
    Thursday, May 03 2018, 05:42 PM - #Permalink
    Resolved
    0 votes
    I have also installed ClearGlass Community and have not received the email yet.
    The reply is currently minimized Show
  • Accepted Answer

    Thursday, May 03 2018, 04:34 PM - #Permalink
    Resolved
    1 votes
    I just installed ClearGlass Community Edition, however I am not getting the sign up emails. They are not in my junk folder either.
    The reply is currently minimized Show
Your Reply