Why it's crucial to keep your Netezza software up to date
When contracting customers to our Netezza Afterlife Support Service we first do a comprehensive health check of their Netezza system. The health check often reveals that the customer is running an old version of Netezza. In fact, a recently onboarded customer was running NPS version 7.2.1.0 which was released in 2015.
Running older versions of the software may not result in any specific issues whilst the appliance is being supported by IBM, but that ceases to be the case after the IBM End of Support (EOS) date because the customer won’t be able to download any new versions and that which they are running won’t be getting any patches. That is the reason we stress that before their IBM support expires, Netezza customers should download all the upgrades and patches available and keep them in a safe place.
There are three main components of a Netezza / PureData System for Analytics that can be upgraded: NPS, RHEL, FDT, and HPF.
What is NPS?
NPS is the Netezza Performance Server, the main high performance database server for the system.
What is RHEL?
This is Red Hat Enterprise Linux, the Operating System.
What is HPF?
HPF is the Host Platform. The HPF software contains kernel updates, firmware, and other special software used by the Netezza host servers.
What is FDT?
FDT is the Firmware Diagnostics and Tools and contains the firmware updates for the hardware components of the Netezza system.
Reasons why Netezza software is often not upgraded
Upgrading to the latest software can be time consuming and often requires an outage. Many Netezza DBAs will only upgrade NPS and will not apply the highest versions of FDT and HPF because they are not considered necessary and/or it would add a few more hours to what is already going to take 6-8 hours. Moreover, a Red Hat upgrade is only possible if a specific version of FDT is installed, adding an additional 3-4 hours to the upgrade time. Consequently, Red Hat is often not upgraded either.
To illustrate how problematic running older version of the software is, here are a couple of recent support cases we resolved.
Case 1: A system in afterlife support was experiencing blade failures resulting in continuous system restarts. The blade had recently been replaced and was not showing any errors or issues except that sysrevcheck flagged a higher firmware version requirement in FDT.
When you have system in afterlife support that has a hardware failure, you will usually need to source refurbished spare parts, as the originals are no longer manufactured. To work, these parts will require a specific firmware version and it is very unlikely that the field engineer will have the IBM certified tools to downgrade the firmware on these components. The built-in Netezza utilities (replacespu and firmwareupgrader) prevent firmware downgrades from being done and so in most cases a firmware downgrade will fail and as a result the replaced hardware component will also fail.
Case 2: A system in afterlife support had already had multiple disks replacements but then went down after almost 40 drives failed in single rack.
Performing data disk replacements in a Netezza system is different to those of regular storage systems. Data disks in Netezza systems are not controlled by a RAID controller but connected with blades through SAS connections and their topology is controlled by an NPS database. In addition to replacing the data drives you must install the correct firmware version and activate the drives.
We have observed that in some older versions of NPS disk replacements often fail. The solution is usually to simply restart the database and activate the drives so that they are recognized by blades. However, a simple upgrade of the NPS system is all it would take to avoid such disk failures altogether, as the issue was addressed in a later patch that many Netezza customers haven’t installed.
We have examples of where, in some FDT versions, the firmwareupdater (a Netezza tool to patch firmware components in system) doesn’t recognise the NPS state, so without the IBM workaround/patch you’ll run into problems if you have a hardware failure.
Case 3: A customer experienced a blade failure. They were running old versions of every component in the system - FDT 3.0, NPS 7.2.0.1, HPF 5.2, and RHEL 6.4. After the blade failure we found we were unable to replace it as the Advanced Management Module (AMM) would not recognize the hardware.
To solve this problem, we had to come up with a workaround, which in this case was to find a replacement blade with the lowest version possible and then upgrade its firmware until it became compatible, quite a task as the blade replacement we found was quite old.
This is a good example of when downloading and keeping new versions of the software will make future hardware replacements a lot easier.
Which versions of the software should you install?
To mitigate future instability of your Netezza system we recommend that you have the following installed or at the very least downloaded whilst you are still under IBM support:
HPF: 5.6.0.1 RHEL: 5.10 (TwinFin/N1001) / 6.10 (Striper/N200X and Mako/N3001) FDT: 4.3.1.7 NPS: 7.2.1.8-P2 for TwinFin/N1001 or higher for Striper/N200X and Mako/N3001
Support tools: HW Support: 8.3.0.3
Optional software: INZA: 3.3 or higher SQL extension toolkit: 7.2.1.8 or higher
If you are approaching EOS for your Netezza system and you intend continue to run your system with an afterlife support contract, we can help you ensure that you are running the latest software, contact us for more information.