Clean server install fails SSL initialization error for 443

[EKjellquist]

Support,

I'm running a longtime installation of AMS 4.2.4 on Windows Server 2012 R2 with no issues, and I have it set up so on the LAN the domain points directly at the AMS server. This server is behind NAT, and the edge router points all traffic on the relevant ports to the AMS server, though ports 80 and 443 are forwarded to my Apache server (on a different box). the AMS domain is routed via proxypass to the AMS server.

I'm trying to replicate this setup on another domain/LAN but running into SSL issues. I can route to the AMS domain from the WAN (certificate comes up in browser ok, but I get ERR_SPDY_PROTOCOL_ERROR. If I try to do so on the LAN or on the AMS server itself, I get ERR_CONNECTION_RESET. This occurs for both Webmail and remote admin, which are both enabled on ports 8000/443 and 9000/9100 respectively.

In the logs when connection is attempted:

[Port 9100: Implicit SSL initialisation error or client closed connection unexpectedly.]
[Port 443: Implicit SSL initialisation error or client closed connection unexpectedly.]

Rebooting, disabling the Windows firewall seem to have no effect. I can verify the server is listening on the relevant ports, and this is a clean server install (no other applications glomming onto ports).

The only difference is, on the working server, it's using a PEM certificate in which the subject CN is the same as the domain URL; in the not-yet-working example, the CN is for the top-level domain and the certificate is a UCC in which the AMS subdomain is one of the Subject Alternative Names.

Now I CAN log in absolutely fine to either Webmail or Remote Admin via the non-ssl ports fine on the server or the LAN using ports 8000 or 9000, so it's strictly an SSL problem as far as I can tell. HOWEVER, if I try to do that using the LAN IP of the server and port (which should just give me a certificate / security warning), it comes up with the same ERR_CONNECTION_RESET.

Any ideas?

[Code Crafters]

It is critical that the common name of the certificate matches the domain that you're connecting to in your URL. If not, the connection will generally be rejected.

If the certificate is self-signed you usually need to install the certificate on the client machine too.

[EKjellquist]

No, this is a purchased certificate from a legit CA. In this case, it's a UCC in which the mail server is on a subdomain and the base CN of the certificate is the top-level domain for the certificate. So the CN is domain.com and the SANs are domain2.com domain3.com domain4.com and mail.domain.com. Should that not work?

[Code Crafters]

The common name such as domain.com needs to have an A record to your mail server IP. If not then it probably won't work. Generally, you have a subdomain like mail.domain.com and set this as your certificate's common name for it to work correctly.

[EKjellquist]

It does have the right DNS entries, but my issue is more local than even that; I can't even get to https://mail.domain.com or https://mail.domain.com:9100 on the server itself, but I CAN get to http://mail.domain.com:8000 and http://mail.domain.com:9000 on the server fine, and the server settings are identical from my 'working' AMS setup and the 'non-working' AMS setup as far as I can tell...

The 'working' server is also a Windows Server 2012 R2 machine and the 'non-working' server is Windows Server 2016

[EKjellquist]

I recall that you guys were considering updating the certificate functionality to allow for a separate certificate for each service, unique to each AMS domain - are you saying that at least currently any cert used with AMS, the CN specifically has to match the domain and Subject Alternative Names are a no go?

Any status for when that limitation might be lifted? Essentially I'd like to use a UCC for, say, 5 domains and either just specify it once in AMS, or at least use the same file/location and specify it for each AMS domain discreetly...

[Code Crafters]

SSL certificates are per service but not per domain. Doing this would be a huge change to Ability Mail Server and isn't planned for any update at the moment.

You can host emails for many domains but still access your server via a common domain (e.g. mail.domain.com) for WebMail / IMAP4 mail server etc.

You may be able to purchase certificates with multiple domains or wildcards in some way but you'd have to research into this further.

We replied to your email but it was blocked. Can you please email us from another email address and we'll reply to that instead.

[EKjellquist]

Wildcards (so far as I can tell) do with in AMS currently because the CN on those always qualifies with subdomains <whatever subdomain>.domain.com. I already have a UCC cert for 5 domains; for all intents and purposes it's the equivalent to 5 discrete certificates; UCC certs are just set up so you have one CN and however many SANs that are all certified under the same CA, using a single private key to provide SSL, rather than 5 individual certs with 5 keys. I *believe* this is what's causing the aforementioned SSL initialization errors because the CN of the cert is not the same as the AMS server name, which is one of the SANs.

I'm ok waiting for the 'one certificate per service' issue to be resolved, if the behavior can at least be updated to allow UCC certs to work. If I have a cert with a CN and 5 SANs, I should be able to use one instance of AMS for all 6 domains (if I wanted to) because it'd be a single certificate file. AMS would have to be updated to recognize those SANs, which may be less work than what it would take to do one cert per domain per service...

[Code Crafters]

I'm not sure what you mean by UCC but I guess you mean some kind of multiple domain certificate? What do you mean by using 5 SANs?

Generally, you have a mail server hosting many domains but have all domains set to the same domain name as their MX record. This domain is the one that should be connected to and be the common name of the certificate. If you need to connect on more than one domain, you'd need a certificate that supports this.

[EKjellquist]

Yes, when I say UCC I'm talking about a multiple domain certificate, commonly known as a Unified Communications Certificate (UCC). The idea is you still have a single Common Name (CN) and up to some number of Subject Alternative Names (SANs). These have worked in Apache for years, with the ability to use the same cert regardless of what the CN is and what the SANs are; if you have 5 domains on a single cert, it doesn't matter which domain is the CN and which are the SANs; you just set up your SSL parameters for the domains / virtualhosts the same way you would if you had a singlestandard single-domain certificate and it works because the SANs are validated using the same private / public key info the CN is.

I'm guessing that updating AMS to accommodate that would be simpler than allowing multiple discrete cert files, which is why I'm suggesting exploring that as an option. I already have such a certificate and it's the one I've been referring to that's involvd with the SSL initialization issues...

https://www.godaddy.com/help/what-is-a- ... icate-3908

[Code Crafters]

I think that sort of certificate has been used by customers before and we'd recommend trying that option.