Migrating your existing data warehouse to a new platform can potentially be a risky, expensive, time consuming, manual, and error-prone exercise, which can have a dramatic effect on achieving time to value, business user satisfaction, and return on investment (ROI) objectives.
Most data warehouse migrations these days will be into the Cloud, either to a fully cloud hosted solution, or as part of a hybrid cloud upgrade. There are still scenarios, however, that require a migration to another on-premises system. This may occur, for example, if a business elects to upgrade their legacy Netezza system (Skimmer, TwinFin or Mako) to Cloud Pak for Data System 18.104.22.168, aka Hammerhead, or if they choose to move away from Netezza altogether and install an alternative system that still sells physical hardware, such as Yellowbrick.
Rehosting, aka Lift and Shift, is a migration strategy whereby an application or workload, which includes it's data and the operating system, is moved from one IT environment to another - usually from on-premises to public or private cloud.
This strategy is considered to be a quicker and cheaper option compared to other strategies because there is no requirement to make significant changes to the architecture or code. It’s also a convenient way for an organisation to begin shifting its IT budget from capital expense (CapEx) to operational expense (OpEx) when initiating a hybrid cloud strategy and start leveraging the more economical and extendible computing power, storage, and networking infrastructure of the cloud.
Nowadays, lift and shift is used more for migrating workloads that are already cloud-ready to some extent. This is because cloud architectures have evolved considerably and allow for much improved developer productivity and more favourable cloud pricing models, meaning the refactoring costs to leverage the cloud environment would outweigh the initial savings of lift and shift.
IBM Cloud, AWS and Azure all provide Infrastructure as a Service (IaaS) for Netezza rehosting.
For workloads that are already cloud ready to some extent, Lift and Shift offers:
A Platform-as-a-Service (PaaS) migration involves revising your data warehouse applications to take greater advantage of the cloud provider’s PaaS stack. This can be done prior to the migration itself if the existing system has been upgraded to be cloud-ready, or as a refactoring exercise after migration, making small changes to optimize its performance for cloud or to leverage specific cloud capabilities. With this approach, developers can reuse languages, frameworks, and containers leveraging code that’s strategic to the company.
Some businesses take advantage of the anticipated cloud benefits to completely redesign the application using the cloud provider’s development tools and enhanced platform capabilities that improve developer productivity.
For businesses with a mature data warehouse, PaaS migration is more costly, labor-intensive, and time-consuming up-front than lift-and-shift. It does, however, enable you to take greater advantage of cloud native operations automation, security, developer productivity, resiliency, and pay-as-you-go cost models, which in combination can quickly recover your initial investment.
PaaS works well for small businesses and startup companies with much smaller and/or less complex data stores and customisations to migrate. Being cost effective, it provides smaller organizations access to state-of-the-art resources without the big price tag. Most small companies don't have established development environments on premises, so PaaS provides a path for accelerating software development. It also allows companies to focus on what they specialize in without worrying about maintaining basic infrastructure.
Other advantages of PaaS include:
Cost Effectiveness - There is no requirement to purchase and maintain hardware and the pay-as-you-go model is attractive to businesses that experience periodic down time
Speed to Market - PaaS providers offer sophisticated development tools that help speed up the creation of apps
Future-Proofing - Users always have access to state-of-the-art technologies
Increased Security - PaaS providers invest heavily in security technology and expertise
Dynamic Scalability - Users can add capacity in peak times and scale down as needed
Flexibility - PaaS allows employees to log in and work on applications from anywhere
A Software-as-a-Service (SaaS) migration means replacing your on-premises system with a ready-made, cloud-based alternative that provides the similar functionality and leverages more of the benefits of your cloud provider’s infrastructure.
Many SaaS products are now engrained in business culture, such as email and messaging, marketing automation tools and customer relationship management (CRM) solutions. Data warehousing as a service (DWaaS) is an example of SaaS except on a potentially much larger scale and taking advantage of huge advancements in processing power and affordable storage capabilities that were only available on-premises until fairly recently.
With DWaaS, an organization does not have to spend money up-front to build an on-premises data warehouse or worry about setting up and managing the hardware and software.
The benefits of DWaaS are similar to any software-as-a-service (SaaS) offering, such as:
When migrating between older generation PureData for Analytics/Netezza systems and the new Netezza Performance Server, Smart Associates has a tool to assist you with process.
Netezza Database Replication as a Service (DbRaaS), or SmartSafe, is a feature of our Smart Management Frameworks product that allows you to fully automate the process of migrating between platforms without any system downtime. Whereas the IBM recommended method for migrating between the two is to use nz_migrate, this procedure does involve significant manual intervention and can lead to a prolonged data migration. As an alternative, Smartsafe allows you to fully replicate all of your Netezza databases and to simply cut-over to your new platform when it suits you.
Smart Associates' Lift 'n' Shift Data Warehouse Migration Service utilises a library of reusable methodologies, tools and techniques to automate as much of the process as possible in a repeatable way. This benefits the customer by dramatically reducing the overall cost of the migration project.
This mature capability along with the breadth and depth of our database experience is what differentiates us from the competition, and what enabled us recently to migrate 30K+ objects and 120TB of SQL Server data to a PostgreSQL-based database in a matter of weeks; or run existing Teradata SQL scripts containing non-standard functions on DB2 without having to rewrite them first.
Furthermore, to give you peace of mind, our fully automated reconciliation and validation process that is a feature of Smart Data Frameworks will highlight any source and target discrepancies at the row or column level post-migration.
Reusable migration templates and optimisations exist for the following specific source and/or target database platforms:
Data migration is the process of permanently transferring your data from one computer storage system to another. Activities in the data migration project include selecting, preparing, extracting, and transforming the data on the source system; the physical transfer to the target system; and validation of the migrated data for completeness.
Data migration is an important stage in any system implementation, upgrade, or consolidation. Given the volumes of data involved, especially with data warehouse migrations, they are performed in such a way as to be as automated as possible, freeing up human resources from mundane tasks. Nowadays, data warehouse migrations frequently occur as businesses move from on-premises infrastructure and applications to cloud-based storage and applications to optimise or transform their company.
According to a 2019 Gartner report, more than 50% of data migration projects will exceed budget and timeline and/or harm the business, due their flawed strategy and execution.
Your data is extremely important, and if it is not migrated properly, it can cause a devastating loss of data and disrupt your business. As such you need a data migration plan to help ensure success for your data migration.
The selected data migration strategy should take into consideration the needs of the business/end users, the type/volume of data to be migrated, and the company’s resources. It should also be sponsored at board-level to ensure that adequate funding and resources are provided.
There are three broad strategies that can be adopted:
A big bang approach refers to all the data being transferred from its source to a target system in a single operation and point of time. This is the highest risk strategy that requires a lot of planning and careful execution to ensure success. A downside of this approach is a that some downtime will be needed to complete the migration. Business units that require real-time access to their data 24x7 may not find this acceptable.
A trickle migration divides the data into smaller chunks and is a more in-depth method of database migration. If one of these sub-migrations fails, it can usually be rolled back and re-run without interfering with the rest of the migrations. A downside of this approach is that it is time-consuming and requires more resources, as the old system and new system must run simultaneously.
A zero-downtime migration replicates data from the original database to the target database and allows the use of the source database during the migration process. This lessens the disruption to the business and facilitates a faster migration time. Our product SmartSafe Database Replication as a Service facilitates this approach.
Here are some examples of types of data migration:
Storage data migration. This will occur when a business chooses to rationalise their data to take advantage of more efficient storage technologies, often using virtualisation techniques. It involves involves transferring data between storage subsystems or server systems. The content and format of the data itself will not usually be changed in the process and it can usually be achieved with minimal or no impact to the layers above.
There can be several reasons to do a storage data migration, such as: