RSA Identity Governance and Lifecycle

There are many different scenarios that can lead to an SSL handshake failing between the application server and another server or even local application.
This debug could be used to troubleshoot connectivity from:
The Access Fulfillment Express (AFX) instance to the application server.
1.The application server to a collector.
A browser connecting to the application server.
2. Some of the different details you can derive from the debug would be failures like:
Using an unsupported TLS version.
No common SSL ciphers between the client and server.
An unsupported or invalid certificate attribute.
Deprecated certificate signing algorithm.
A keystore referenced in the debug is different than what was expected.

SSL Debug Trace for IBM WebSphere

These traces should be removed as soon as you have reproduced the problem and collected the trace. 1.This debug trace generates a significant amount of events in the WebSphere SystemOut.log file.
In the WebSphere Application Server (WAS) Admin Console, navigate to Servers > Server Types > WebSphere application servers, then select the server name.
2.Under Server Infrastructure, expand Java and Process Management > Process definition > Java Virtual Machine.
3.Add the following to the end of the Generic JVM Arguments box: 
-Djavax.net.debug=ssl,handshake,data,trustmanager
4.Save to the master config, and restart the server for it to take hold.
5.This will add debug trace of the SSL handshake to the <Websphere installation>/<AppServer>/profiles/<profile name>/logs/<server name>/SystemOut.log

NOTE: To get useful/verbose messages, the IBM Trust manager may need to be changed from IbmPKIX to IbmX509.  This setting is in the WebSphere Admin GUI under Security > SSL Certificate and Key Management > SSL configurations > Select Resource > Trust and Key Managers.  The default trust manager for that resource can be changed using the pull-down menu.

Steps to disable SSLv3 protocol on WebSphere:

Login to ibm admin console

1.  Go to Security > SSL certificate and key management > SSL configurations

2.  The collection of all SSL configurations is listed. For each SSL configuration in the list the SSL protocol will need to be modified to use TLS.

3.  Select an SSL Configuration then click Quality of protection (QoP) settings under Additional Properties on the right.

4.  On the Quality of protection (QoP) settings panel, select TLS form the pull down list in the box labeled Protocol.

5.  Apply/Save.

6.  Restart application server

NOTE: The Protocol label SSL_TLS will not disable SSLv3. This means protocol supports SSLv3, TLS 1.0, TLS 1.1 and TLS 1.2. So select TLS, TLSv1, TLSv1.1 or TLSv1.2 only. 


How to check if SSLv3 is disabled:

1.  Install Openssl on windows machine (http://gnuwin32.sourceforge.net/packages/openssl.htm)

2.  In command prompt run the below commands

     openssl s_client -connect <machine_name>:<ssl_port> -ssl3
 
You will see some error something like below

Loading 'screen' into random state - done

CONNECTED(00000170)

7468:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake

failure:./ssl/s3_pkt.c:530:

3.Loading 'screen' into random state - done

CONNECTED(00000170)

7468:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake

failure:./ssl/s3_pkt.c:530:

4.  If SSLv3 is enabled, and you run the same command. You will see an output something like below

Loading 'screen' into random state - done

CONNECTED(00000170)

Server certificate

-----BEGIN CERTIFICATE-----

DKFLDSJFSDKLJFSDAKLJFDKL

----END CERTIFICATE-----

No client certificate CA names sent

---

SSL handshake has read 628 bytes and written 206 bytes

---

New, TLSv1/SSLv3, Cipher is RC4-MD5

Server public key is 512 bit

Compression: NONE

Expansion: NONE

SSL-Session:

    Protocol  : SSLv3

    Cipher    : RC4-MD5

    Session-ID: <>

    Session-ID-ctx:

    Master-Key: <>

    Key-Arg   : None

    Start Time: <TIME DURATION>

    Timeout   : 7200 (sec)

    Verify return code: 21 (unable to verify the first certificate)

Configuring WebSphere Application Server to support TLS 1.2 for NIST SP 800-131

To comply with the US government SP 800-131 security standard, you can configure the WebSphere® Application Server that hosts Rational® solution for CLM applications to support the Transport Layer Security (TLS) 1.2 protocol.
Procedure
1.Log in to the WebSphere Application Server Integrated Solutions Console.
2.Click Security > SSL certificate and key management, and under Related Items, click SSL configurations.
3.Click the default SSL settings link to open it and, under Additional Properties, click Quality of protection (QoP) settings.
For the protocol, ensure that TLSv1.2 is selected, for the Cipher suite groups, ensure that Strong is selected, and then click Update selected ciphers.
4.Click OK and save directly to the master configuration.
5.Click the SSL certificate and key management link and then click Manage FIPS.
In the Manage FIPS window, click Enable SP800-131 and then select Strict.
Click OK. 
Under Related Items, click Convert certificates.
Ensure that the Algorithm setting is Strict.
For the New certificate key size, select 2048 bits.
6.Click OK and save directly to the master configuration.
7.Go to WAS_Profile_Dir/properties and open the ssl.client.props file for editing.
Search for com.ibm.security.useFIPS and change the property to true.
Search for com.ibm.websphere.security.FIPSLevel and if the line does not exist add it, and then set the property to SP800-131.
Search for com.ibm.ssl.protocol and change the property to TLSv1.2.
8.Click Server > Server Types > WebSphere application servers and then click server1 to open it.
Under Server Infrastructure, click Java and Process Management > Process definition.
Under Additional Properties, click Java Virtual Machine and then click Custom properties.
9.Add the following three custom properties:
com.ibm.team.repository.transport.client.protocol with a value of TLSv1.2
com.ibm.jsse2.sp800-131 with a value of strict
com.ibm.rational.rpe.tls12only with a value of true
10.Restart the application server.

Protocols & Cipher suites related links from IBM

1.https://www.ibm.com/support/knowledgecenter/SSYKE2_6.0.0/com.ibm.java.security.component.60.doc/security-component/jsse2Docs/ciphersuites.html

2.https://www.ibm.com/support/knowledgecenter/SSYKE2_6.0.0/com.ibm.java.security.component.60.doc/security-component/jsse2Docs/protocols.html


Kerberos Cipher Suites:

https://www.ibm.com/support/knowledgecenter/SSYKE2_6.0.0/com.ibm.java.security.component.60.doc/security-component/jsse2Docs/kerberosciphersuites.html

IBMJSSE2 Provider:

https://www.ibm.com/support/knowledgecenter/SSYKE2_6.0.0/com.ibm.java.security.component.60.doc/security-component/jsse2Docs/ibmjsse2.html

Configuring Reverse Proxy Servers For WebSphere

To configure the IBM HTTP Server for use as a reverse proxy server, you use the IHS plug-in.

Before you perform the following steps, you need to install the following items:
IBM HTTP Server.
Web server plug-ins.
See PeopleTools Installation for your database platform, Installing Web Server Products, "Installing IBM HTTP Server and Web Server Plug-ins"
To configure IHS for reverse proxy:
Start WebSphere server and open the ISC window.
Navigate to Environment, Virtual Hosts, pia_host, Host Aliases.
The PeopleSoft application is deployed on a virtual host called "pia_host".
Create new entries for the required ports.
For example:
Hostname = *, Port =80/others ports  (for web server port )
Hostname = *, Port =10002 (for HTTP Administration Server port)
Hostname = *, Port =10043 (for SSL port assigned to IHS)
In a multi server environment, repeat the steps 2. and 3. for the other virtual hosts and "psemhub_host".
Click Apply and save the settings to "master".
This updates <HTTP_HOME>/webserv/profile_name/config/cells/node_name/virtualhosts.xml
From the WebSphere Plug-ins installation, copy the configureWebserverDefinition script from the <Plugin_Install_Root>/bin to the directory <PS_HOME>/webserv/profile_name/bin and run it.
This creates the web server definition in WebSphere server.
Generate the plugin-cfg.xml by selecting the web server definition in Servers, Web servers.
Copy the plugin-cfg.xml from <WAS_HOME>/profile_name/config/cells/cell_name/nodes/node_name/servers/WebserverDefinition to <Plugin_Install_Root>/config/WebserverDefinition so that IHS can communicate with WebSphere directly and access the PeopleSoft application.
Restart the WebSphere server, IBM HTTP Server, and IBM HTTP Administration Server.
Verify accessing the PeopleSoft application using the IHS HTTP port.

Note: In scenarios where the system needs to process large amounts of data, a page may become stuck in the processing status. While the change is reflected on the database, it cannot be viewed on a page until another session is started. This problem can be resolved by increasing the ServerIOTimeout from 60 seconds to 600 seconds, for example, in the Plugin-cfg.xml file located in <PLUGIN_HOME>/config/webserver1.

TLS on WebSphere

1.Enabling TLS-Only on WebSphere:
Transport Layer Security (TLS) protocol is an improvement on the SSL v3 protocol. This section discusses:
Configuring WebSphere for TLS.
Configuring browsers for TLS.
Testing TLS.
Configuring Reverse Proxy Servers for TLS.

2.Configuring WebSphere for TLS, Press Enter to collapse:
To enable TLS-only on WebSphere:
Login to ISC (http://host:adminport/ibm/console).
Under the Security menu, select SSL certificate and key management, SSL configurations, NodeDefaultSSLSettings, Quality of protection (QoP) settings.
Change the Protocol value to TLS orTLSv1.
This ensures that WebSphere server will accept only TLS connections. That is, when the web server acts as a server (inbound) or as client (outbound) the SSL connections will be established through the TLS protocol. When testing from a browser make sure to check the browser settings to initiate TLS handshakes only.

3.Configuring Browsers for TLS, Press Enter to collapse:
Setting Up TLS on Microsoft Internet Explorer

4.To set up TLS on Internet Explorer:
Launch Internet Explorer.
Select Tools, Internet Options, and select the Advanced tab.
In the Settings box in the Security section, disable Use SSL 3.0 and enableUse TLS 1.0.
Click OK and restart the browser.
Setting Up TLS on Mozilla Firefox

To set up TLS on Firefox:
Launch Firefox.
Select Tools, Options, click the Advanced icon, and select the Encryption tab.
In the Protocols group box, disable Use SSL 3.0 and enableUse TLS 1.0.
Click OK and restart the browser.

5.Testing TLS, Press Enter to collapse:
After setting TLS for WebSphere and browsers, the TLS communication can be verified by logging in to the PeopleSoft application through WebSphere’s default SSL port (HTTPS).
For example:
https://<host_name>:<https_port>/<PIA site>/signon.html
You can find the HTTPS port in the WebSphere Administrative Console, by selecting Servers, Application Server, server1, ports. Find the port corresponding to the entry WC_defaulthost_secure

6.Configuring Reverse Proxy Servers for TLS, Press Enter to collapse:
It is strongly recommended to that you access the vendor's documentation of the web server you are using for a reverse proxy server and use their instructions for setting up TLS.



OR 

SL certificate and key management > SSL configurations > CellDefaultSSLSettings > Quality of protection (QoP) settings

or via a Jython script: -

AdminTask.modifySSLConfig('[-alias CellDefaultSSLSettings -scopeName (cell):BAMCell1 -keyStoreName CellDefaultKeyStore -keyStoreScopeName (cell):BAMCell1 -trustStoreName CellDefaultTrustStore -trustStoreScopeName (cell):BAMCell1 -jsseProvider IBMJSSE2 -sslProtocol TLSv1.2 -clientAuthentication false -clientAuthenticationSupported false -securityLevel HIGH -enabledCiphers ]') 
AdminConfig.save()
AdminNodeManagement.syncActiveNodes()

/opt/IBM/WebSphere/AppServer/profiles/BAMCell1AppSrv01/logs/nodeagent/config/cells/BAMCell1/security.xml
and change: -

sslProtocol="SSL_TLS"

to: -

sslProtocol="TLSv1.2"

Note: To do rest ot nodes as per above to change TLS version.

/opt/IBM/WebSphere/AppServer/profiles/BAMCell1Dmgr01/config/cells/BAMCell1/security.xml


Once I made this change, and manually restarted the Node Agent, all was well, and both Deployment Manager and Node Agent were correctly using TLS 1.2.

Queries on TLS

Which is the oldest version of WebSphere Application Server that supports TLS 1.2?

Java 7 is not required. WebSphere has supported TLS1.2 since version 7.0.0.23. Essentially every in-service release of WebSphere supports TLS1.2 (7.0 will go out of service next year)

Does Websphere support multiple SSL versions?

There is an alternative option, SSL_TLSv2, which will enable support for TLSv1.0, TLSv1.1, and TLSv1.2 in the environment. Please use this setting SSL_TLSv2 in environments where support for multiple TLS protocols is required, or if you are not sure whether your WAS environment interacts with other servers or clients using non-TLSv1.2 protocols then, you can configure WAS to use SSL_TLSv2 using same steps as given in the above.

Note:

Without poddle fix and configured WAS to use SSL_TLSv2

SSL_TLSv2 ==> Enables all SSL v3.0 and TLS v1.0, v1.1 and v1.2 protocols. Accepts SSLv3 or TLSv1 hello encapsulated in an SSLv2 format hello.

If you installed Poddle fix (will disable SSLv3 ) and configured WAS to use SSL_TLSv2

SSL_TLSv2 ==> Enables these three TLS v1.0, v1.1 and v1.2 protocols.

So, changing the QoS settings to SSL_TLSv2 allows SSL Handshakes to multiple TLS versions when required.

x