In this blog we share some information about the Modern Hybrid Cloud Data Warehouse from the evolution of business intelligence, through the data warehousing revolution to the modern day where a new generation of data warehouses such as Yellowbrick are located on-site, in the cloud or as a hybrid of the two.
In this blog we’d like to share some information about the Modern Hybrid Cloud Data Warehouse, based on a presentation by Ed Bernier, the Senior Systems Engineer at Yellowbrick.
The Evolution of Business Intelligence
Nowadays there is a need for users to look at data more interactively, compared to the early days of Business Intelligence where reports were generated in batches. Before more advanced tools became readily available, users would often download their data and manipulate it in Excel spreadsheets. Today they have access to holistic, drill-able dashboards and advanced AI-driven analytics.
Many businesses recognise that their customer data is unique and extremely valuable and so customer 360 is now king. As a consequence, data volumes have grown exponentially from terabytes a few years ago to petabytes today.
Handling such volumes of data and meeting the growing needs of users is problematic. Legacy systems pose a lot of challenges. They are hard to scale up and their technology is often outdated and inflexible. The costs of expanding existing capacity have skyrocketed.
Cloud bases systems offer a lot of flexibility but they too have limitations, such as the complexity of storing data from disparate sources and the fact that most of them are not designed to handle large datasets.
The Data Warehouse Revolution
The industry has come a long way since the early days of Data Warehousing and Business Intelligence. Data Warehousing began with traditional RDBMS systems such as Oracle, Sql Server, MySQL and PostgreSQL but user demand and increased data volumes drove the introduction of appliances such as Netezza and Teradata. These appliances offered high performance and simplicity and you could deploy them very quickly.
As user demands increased and the cost of storage and commodity hardware fell, the concept of Big Data arrived, with Hadoop allowing you not to throw any data away. At the same time we saw cloud environments rise up such as Redshift, Azure and BigQuery, and data warehouse software designed for the cloud such as Snowflake and Yellowbrick. At this stage these cloud systems, whilst offering a great deal of flexibility and functionality, were not designed for the volumes of data that we see today.
The Attributes of a Modern Hybrid Cloud Data Warehouse
As we see more businesses moving to the Cloud, we have learned that not all workloads belong in the cloud and now there is a drive towards hybrid cloud where some clients have brought back some of their processing power on premise. They like the flexibility to deploy in either location and move between the two. To that end, the modern hybrid cloud solution should address the following:
- Location agnostic - the ability to deploy on premise or in the cloud, or a combination of the two, to match workloads
- The need for predictable performance - most solutions deployed within cloud are on VMs on commodity hardware, and so that currently limits the scaleability and predictability of performance
- Workload management - as more and more diverse workloads need access to the cloud data it poses challenges to keep up with demand and to provide meaningful SLA's to users
- Flexible access to data - not all users are coming from the same platform
- Low latency for business-critical data insights - Users are not satisfied with seeing end of day data, they want real time reporting
- Flexibility to change your mind - you may want to start by deploying on premise and then move into the cloud, and vice versa
- Powerful, interactive analytics - to be able to drill deep into the data with ease
Yellowbrick - A Modern Data Warehouse for Hybrid Cloud
This is how Yellowbrick designed their system:
- It is a solution that can be deployed on-premises, or in the cloud, or as a hybrid of the two
- You can choose between a subscription or CAPEX model on-premise, or as a subscription in the cloud, so it offers a great deal of pricing flexibility
- It has an integrated hardware/software stack with flash architecture that provides extremely high performance against very large data sets
- It has a very small footprint that is extensible
- It supports AWS, Azure and Google Cloud Platform (GCP)
- It has replication based failover and failback across all locations built into the architecture
What's Special about the Yellowbrick Data Warehouse?
- Unparalleled Performance - as data volumes have grown from terabytes to petabytes a lot of systems were not designed to handle such large datasets, but Yellowbrick was built from the ground up for that
- Innovative Simplicity - the system is designed for no tuning, you can load your data and start using it. Migrating from your old system is easy, meaning that deployment is very fast
- Sophisticated workload management and monitoring capability to ensure you can handle the most complex workloads
- Flexible Deployment - On premises, Public Cloud or Hybrid Cloud
If you would like to know more about Yellowbrick, why not contact us for more information.