A recent customer experience
Recently a customer reported that HA1 had became unresponsive. Fortunately we were able to bring it back online with minimal configuration: a single CPU (in slot 1) and one DIMM. This setup allowed us to check the serial number on the motherboard, revealing that this host had previously been replaced by IBM, as only they could add the licensed RAID 6 feature. During that replacement, a different, more powerful CPU was installed, which has a higher thermal design power (TDP). This appears to be the main root cause of the host’s issues, as the SAS card near CPU2 (which was non-functional on the old motherboard) also failed.
We were able to rescue this host because it was CPU slot 2 that was broken, not slot 1. If slot 1 had failed, we would have had no way to verify the original serial number provided by the IBM technician or recover the IMM licenses given that the FoD keys are shipped by IBM and are no longer available.
The danger/risk here is that if the failed host cannot be successfully booted, it will not be possible to backup the FoD keys and transfer these to the replacement component, which will make it correspondingly difficult (if not impossible) to bring the host back up with replacement components and to recognise the existing disk arrays.
Failed motherboards may not have a serial number that matches the external label on the chassis, and exported keys cannot be imported and activated into replacement motherboards with a different serial number. We can change the serial number of the replacement to match the original failed component, but only if we know what the original serial number was.
So the upshot of all the above is:
- it’s vital to backup existing FoD keys ASAP and keep the files somewhere safe, using the procedure described below
- Equally, it’s essential to make sure that the serial number of the motherboard from which the keys are extracted is also stored somewhere safe
As precaution, we would also advise recording the self-encrypting drive (SED) key(s) and keeping them off-site..
Procedure for identifying activation keys
1. Obtain IMM IP for both hosts. This may be stored in /etc/hosts, or use the following command (on each host):
/opt/nz-hwsupport/install_tools/asu/asu64 show | grep -i ip2
2. In web browser navigate to both hosts IPs. Default credentials are:
user: USERID
password: PASSW0RD
3. Navigate to: IMM Managegement -> Activation Key Management
4. Export all keys present there and save to safe storage
On the last step we advise you to store this within shared DRBD partitions on the active host (for example /nz/licenses directory). In case any host requires a motherboard replacement it will be available to the support team.
