A Case of Excessive Batch Processing
What should have been an overnight batch process of our client's data warehouse was taking some 22 hours to complete. This left just 2 hours to run queries and reports that were crucial for making patient case load decisions. Any delays in the batch process would lead to completion times greater than 24 hours, and since this could happen several times in succession, the data could become seriously outdated. It was vitally important that reports and query results were up to date, to ensure patient appointments were scheduled according to a set of priority criteria. Since many of the cases were critical, this was becoming a serious issue.
A Data Warehouse Stretched to the Limits
When designing a data warehouse due consideration is given to current and forecast volumes. Businesses, however, are constantly changing and the data warehouse is often stretched beyond its original specification because of growing data volumes, increasing numbers of data sources and a larger number of stakeholders who are demanding more out of it.
Quite often the success of a new data warehouse becomes its downfall and adding disk capacity and processing power only buys a little time, it doesn't reverse a deteriorating situation. In this example, something had to be done to bring the batch processing times back down to an acceptable timeframe.
A New Data Warehousing Solution
To address this problem, Smart Associates worked with the organisation to rapidly and accurately migrate the data warehouse to a modern platform, specifically designed for high performance analytics on large data volumes.
This required expertise and experience, and it also had to be done without excessive consulting fees to satisfy the budgetary restraints of a healthcare client. Smart associates was able to do quickly and at minimum risk by employing Smart Data Frameworks.
Smart Data Frameworks Data Migration Tool
Smart Data Frameworks is an off the shelf solution for enterprises looking to migrate from almost any database platform to an alternative - allowing faster, cheaper and better data integration with shorter time to value.
Reduced Batch Processing Times
On the new platform, the overnight batch update runs in a fraction of the time and with greater reliability. Even if a re-run is necessary, there is still plenty of time to complete without affecting daily operations. With the new platform, reports and queries run faster, allowing time for additional queries when required.
The effect of this is a more effective use of medical consultants’ time, and ultimately a noticeable improvement in health care outcomes as time and priorities have become better ordered.