So you need to update your Dell OpenManage Server Administrator’s SSL certificate it runs on because maybe Tenable sees an untrusted SSL cert or your organization requires everything to be signed by their servers. In the past there was a crazy set of command lines to run to make this happen but now you can just export a PFX file from Windows MMC and import it directly with the latest versions of OMSA! I know it works with OMSA 10 at least but unsure about later versions of OMSA 9.
Export a PFX
This will be a brief overview of the special settings needed and will assume you know how to use the mmc certificates plugin and how to request certificates within your organization. You’ll need a Computer certificate with the FQDN of your server that is running OMSA that allows exporting the private key.
Right click your certificate in the MMC -> All Tasks -> Export -> Yes, export the private key -> Choose PFX if it’s not auto chosen -> uncheck Include all certificates, uncheck Enable certificate privacy, check Export all extended privacy.
Then on the next page check Password -> enter a 20 character maximum password, set Encryption to TripleDES-SHA1
Save the PFX files and then transfer it someplace safe you can access from your computer to upload in the next step.
Import the PFX
Login to Dell OMSA -> Preferences in the top right -> General Settings on the far left -> X.509 Certificate in the center. Then click Import a PKCS#12 keystore -> Next and you’re finally at the import page. Select Choose FIle and browse for your PFX file and then enter your 20 character or lower password -> Import. The system will give you the option to restart the web interface which can take a minute or two then Ctrl-R the website to force a fresh check and the browser should now show a good certificate.
I didn’t think this worked at first because Dell OMSA would error saying it was an invalid PKCS#12 file or a bad password. Well it’s clearly not the password because I’m copy pasting it from a password manager into every field. So I went down a chase to turn it into a Java Keystore file because the Dell OMSA documentation states the file is a JKS. After going through all that it still didn’t work. Come to find out as you noticed above, the password was the issue. I noticed something after several attempts, the dots in the password field didn’t look as long as it should have. Come to find out the Dell OMSA password field only allows 20 characters to be entered and I was using a 30 character password! So frustrating! This is an old issue from older Java 8 where its keystore tool only supported maximum 20 characters but the new OMSA runs on Java 11 which doesn’t have this restriction.