SSL Cert Requirements..

[EKjellquist]

We're using 3.0.0 of AFS, and we were just using the self-signed cert for the last several years. I'm trying to switch over to a LE cert (same as we're using with AMS) and I keep getting 'connection refused' and 'SSL_PROTOCOL_ERROR' errors when trying to log into it via port 7200, on localhost or remote. Firewall is configured correctly. self-signed cert seems to be sha1/RSA2048, LE cert is sha256/RSA4096.

I don't need to log in to the remote via a browser (I can just use the application mode), but that would be nice. My issue is that connections that work ok over the self-signed cert no longer work with the LE cert, and I'm thinking maybe b/c this version of AFS can't handle 4096-bit RSA? AMS seems to be fine with it, thought maybe AFS just hadn't caught up yet?

[Code Crafters]

Both AMS and AFS can create up to 2048 bit self signed certificates. Ability Mail Server 4.2.5 is a newer version of OpenSSL but otherwise should have the same code for SSL. You can try copying the SSL DLLs from your Ability Mail Server installation folder to your Ability FTP Server installation folder to see if the newer OpenSSL fixes the issue. We will also look into upgrading both to be able to create 4096 bit keys with SHA256 too.

[EKjellquist]

I did go ahead and try updating to the current 1.0.2t version of openSSL, replacing libeay32.dll and ssleay32.dll in both AMS 4.2.4 and AFS 3.0.0; Things work ok in AFS but definitely not in AMS (so I reverted those).
downloaded from https://indy.fulgan.com/SSL/openssl-1.0 ... -win32.zip if it helps anyone else.

I still can only get TLS 1.0 to work with AFS, 1.1 or 1.2 always fail to connect; the issue for me is PCI compliance is flagging anything prior to TLS 1.2 as unacceptable.

I normally use FileZilla, and my AFS is set up to enable SSL, encrypt by default, enable implicit / explicit, using a cert generated from Let's Encrypt. If I attempt to require TLS 1.1 or 1.2 on the server, I get this upon connection attempts:

2019-10-25 10:05:23 2912 1 Status: Connecting to <Server IP>:21...
2019-10-25 10:05:23 2912 1 Status: Connection established, waiting for welcome message...
2019-10-25 10:05:23 2912 1 Response: 220-Welcome to the <company> FTP Server. Please consult the server rules and
2019-10-25 10:05:23 2912 1 Response: 220 tutorial at http://<domain>/tutorials/FAQ14.htm!
2019-10-25 10:05:23 2912 1 Command: AUTH TLS
2019-10-25 10:05:23 2912 1 Response: 234 Starting TLS...
2019-10-25 10:05:23 2912 1 Status: Initializing TLS...
2019-10-25 10:05:23 2912 1 Error: GnuTLS error -110: The TLS connection was non-properly terminated.
2019-10-25 10:05:23 2912 1 Status: Server did not properly shut down TLS connection
2019-10-25 10:05:23 2912 1 Status: Connection attempt failed with "ECONNABORTED - Connection aborted".
2019-10-25 10:05:23 2912 1 Error: Could not connect to server
2019-10-25 10:05:23 2912 1 Status: Waiting to retry...
2019-10-25 10:05:28 2912 1 Status: Connecting to <Server IP>:21...
2019-10-25 10:05:28 2912 1 Status: Connection established, waiting for welcome message...
2019-10-25 10:05:28 2912 1 Response: 220-Welcome to the <company> FTP Server. Please consult the server rules and
2019-10-25 10:05:28 2912 1 Response: 220 tutorial at http://<domain>/tutorials/FAQ14.htm!
2019-10-25 10:05:28 2912 1 Command: AUTH TLS
2019-10-25 10:05:28 2912 1 Response: 234 Starting TLS...
2019-10-25 10:05:28 2912 1 Status: Initializing TLS...
2019-10-25 10:05:28 2912 1 Error: Could not connect to server

If I switch to TLS 1.0 it connects fine. There's something wrong with the TLS handshake that I've never been able to figure out, other than it's not a firewall or installation issue. The certificate is trusted fine, there are no AFS permission issues; Can anyone else get AFS with TLS 1.2 to work?