Netezza Database Migration

Migrating your existing data warehouse to a new platform can potentially be a risky, expensive, time consuming, manual, and error-prone exercise, which can have a dramatic effect on achieving time to value, business user satisfaction, and return on investment (ROI) objectives.

Most data warehouse migrations these days will be into the Cloud, either to a fully cloud hosted solution, or as part of a hybrid cloud upgrade. There are still scenarios, however, that require a migration to another on-premises system. This may occur, for example, if a business elects to upgrade their legacy Netezza system (Skimmer, TwinFin or Mako) to Cloud Pak for Data System 1.0.7.8, aka Hammerhead, or if they choose to move away from Netezza altogether and install an alternative system that still sells physical hardware, such as Yellowbrick.

Rehosting, aka Lift and Shift, is a migration strategy whereby an application or workload, which includes it's data and the operating system, is moved from one IT environment to another - usually from on-premises to public or private cloud.

This strategy is considered to be a quicker and cheaper option compared to other strategies because there is no requirement to make significant changes to the architecture or code. It’s also a convenient way for an organisation to begin shifting its IT budget from capital expense (CapEx) to operational expense (OpEx) when initiating a hybrid cloud strategy and start leveraging the more economical and extendible computing power, storage, and networking infrastructure of the cloud.

Nowadays, lift and shift is used more for migrating workloads that are already cloud-ready to some extent. This is because cloud architectures have evolved considerably and allow for much improved developer productivity and more favourable cloud pricing models, meaning the refactoring costs to leverage the cloud environment would outweigh the initial savings of lift and shift.

IBM Cloud, AWS and Azure all provide Infrastructure as a Service (IaaS) for Netezza rehosting.

For workloads that are already cloud ready to some extent, Lift and Shift offers:

• A quick, cost-effective, migration with minimal disruption. You don't need to dedicate a large team to the task and the existing system can remain in place during the migration so there is no interruption of service.
  
• An immediate path to the cloud - many of the benefits of cloud can be realised immediately, such as improved performance, expanded capacity (compute, storage and additional network bandwidth), scalability on demand, enhanced security and reduced on-premises data centre costs, all without having without having to purchase the hardware yourself.

A Platform-as-a-Service (PaaS) migration involves revising your data warehouse applications to take greater advantage of the cloud provider’s PaaS stack. This can be done prior to the migration itself if the existing system has been upgraded to be cloud-ready, or as a refactoring exercise after migration, making small changes to optimize its performance for cloud or to leverage specific cloud capabilities. With this approach, developers can reuse languages, frameworks, and containers leveraging code that’s strategic to the company.

Some businesses take advantage of the anticipated cloud benefits to completely redesign the application using the cloud provider’s development tools and enhanced platform capabilities that improve developer productivity.

For businesses with a mature data warehouse, PaaS migration is more costly, labor-intensive, and time-consuming up-front than lift-and-shift. It does, however, enable you to take greater advantage of cloud native operations automation, security, developer productivity, resiliency, and pay-as-you-go cost models, which in combination can quickly recover your initial investment.

PaaS works well for small businesses and startup companies with much smaller and/or less complex data stores and customisations to migrate. Being cost effective, it provides smaller organizations access to state-of-the-art resources without the big price tag. Most small companies don't have established development environments on premises, so PaaS provides a path for accelerating software development. It also allows companies to focus on what they specialize in without worrying about maintaining basic infrastructure.

Other advantages of PaaS include:

Cost Effectiveness - There is no requirement to purchase and maintain hardware and the pay-as-you-go model is attractive to businesses that experience periodic down time
Speed to Market - PaaS providers offer sophisticated development tools that help speed up the creation of apps
Future-Proofing - Users always have access to state-of-the-art technologies
Increased Security - PaaS providers invest heavily in security technology and expertise
Dynamic Scalability - Users can add capacity in peak times and scale down as needed
Flexibility - PaaS allows employees to log in and work on applications from anywhere

A Software-as-a-Service (SaaS) migration means replacing your on-premises system with a ready-made, cloud-based alternative that provides the similar functionality and leverages more of the benefits of your cloud provider’s infrastructure.

Many SaaS products are now engrained in business culture, such as email and messaging, marketing automation tools and customer relationship management (CRM) solutions. Data warehousing as a service (DWaaS) is an example of SaaS except on a potentially much larger scale and taking advantage of huge advancements in processing power and affordable storage capabilities that were only available on-premises until fairly recently.

With DWaaS, an organization does not have to spend money up-front to build an on-premises data warehouse or worry about setting up and managing the hardware and software.

The benefits of DWaaS are similar to any software-as-a-service (SaaS) offering, such as:

• Improved accessibility - Ability to run via an internet browser 24/7 from any device
• Reduced management overhead - No installation, hardware updates or traditional licensing management
• Cost Effectiveness - No upfront hardware costs and flexible payment methods such as pay-as-you-go models
• Scalability and Elasticity - Easily scale a solution up or down to accommodate changing needs
• Improved functionality - Access to state of the art data reporting and intelligence tools and developer productivity tools
• Increased Security - DWaaS providers invest heavily in security technology and expertise

DWaaS is a good choice for organizations such as startups with small or limited IT departments. There is no requirement to pay for capacity that may never be used as DWaaS contracts allow for scaling up or down according to need and even for free startup options for small quantities of data. Some larger organizations have the option to use DWaaS for selected purposes such as proofs of concept if they don't want to move entirely away from their existing hybrid cloud environment.

As with any cloud-based offering, performance and availability are primary considerations so DWaaS is only an option for a businesses with a reliable high speed internet connection. Some large enterprises may also be averse to adopting DWaaS because of concerns about security, compliance and data sovereignty but as DWaaS matures these concerns are expected to subside.

Popular DWaaS providers include IBM Netezza Perfomance Server as a Service on Microsoft Azure (launched in November 2021), Amazon Redshift, Microsoft Azure SQL Data Warehouse and Snowflake.

When migrating between older generation PureData for Analytics/Netezza systems and the new Netezza Performance Server, Smart Associates has a tool to assist you with process.

Smart Database Replication is a feature of our Smart Management Frameworks product that allows you to fully automate the process of migrating between platforms without any system downtime. Whereas the IBM recommended method for migrating between the two is to use nz_migrate, this procedure does involve significant manual intervention and can lead to a prolonged data migration. As an alternative, Smart Database Replication allows you to fully replicate all of your Netezza databases and to simply cut-over to your new platform when it suits you.

Smart Associates' Lift 'n' Shift Data Warehouse Migration Service utilises a library of reusable methodologies, tools and techniques to automate as much of the process as possible in a repeatable way. This benefits the customer by dramatically reducing the overall cost of the migration project.

This mature capability along with the breadth and depth of our database experience is what differentiates us from the competition, and what enabled us recently to migrate 30K+ objects and 120TB of SQL Server data to a PostgreSQL-based database in a matter of weeks; or run existing Teradata SQL scripts containing non-standard functions on DB2 without having to rewrite them first.

Furthermore, to give you peace of mind, our fully automated reconciliation and validation process that is a feature of Smart Data Frameworks will highlight any source and target discrepancies at the row or column level post-migration.

Supported platforms

Reusable migration templates and optimisations exist for the following specific source and/or target database platforms:

• PureData for Analytics/Netezza Performance Server
• Yellowbrick
• Snowflake
• Microsoft SQL Server
• Oracle
• Db2 (including IIAS, Big Query, Db2 Warehouse on Cloud)
• Teradata
• Pivotal Greenplum
• PostgreSQL
• Google Big Table

To download the Lift 'n' Shift Data Warehouse Migration Services brochure, click here.

For more information please contact us here and we'll get in touch.