FairWarning® Overview
FairWarning, Inc. is the world's leading supplier of privacy monitoring solutions for electronic health records, relied on by 1,200 Hospitals and 5,000 Clinics in 47 states across the US and 7 countries worldwide to keep their patients' data safe. FairWarning solutions are compatible with over 250 applications and have audited 120 billion events, each of which was processed and stored in a MySQL database. In January 2014, FairWarning was awarded the highest commendation in healthcare IT as the first ever Category Leader for Patient Privacy Monitoring in the "2013 Best in KLAS: Software & Services" report1.
The Business Challenge
In response to the Health Information Technology for Economic and Clinical Health (HITECH) Act, hospitals and clinics have been moving their patient data from paper to electronic health records (EHRs). This monumental task is made greater by the need to comply with an increasing number of healthcare privacy regulations such as HIPAA, ARRA HITICH privacy Meaningful Use criteria, CMS Meaningful Use audits, and many more. Further complicating matters, most hospitals now have multiple systems containing protected health information (PHI) (one FairWarning customer has over 100), causing workloads to outpace resources and another challenge - high employee turnover.
The FairWarning Solution
FairWarning Patient Privacy Monitoring is an on-premise or SaaS solution that relies on MySQL as it proactively monitors and audits access to patient data, and automates policy enforcement, generating alerts and notifications on any suspicious activities, as well as tracking investigations and providing advanced reporting capabilities whenever needed for full investigations. FairWarning is compatible with every major EHR application and over 250 other applications used in healthcare.
Figure 1: FairWarning Portal (click image to enlarge)
The MySQL Solution
Since its start in 2005, FairWarning has relied on MySQL to power its Patient Privacy Monitoring solutions. MySQL was the right database for FairWarning as a small start-up, and Oracle MySQL continues to meet FairWarning's needs as the market leader.
Selecting MySQL - Again
After Oracle's acquisition of Sun Microsystems, Inc., FairWarning wanted to better understand Oracle's plans for MySQL's future. In a meeting with Oracle MySQL, the questions FairWarning had had about MySQL were answered: "MySQL's product roadmap fully addresses our needs. We like the fact that MySQL Enterprise Edition has everything included; there's no need to purchase separate modules. We've had a lot of good customer support." said Chris Arnold, FairWarning's Vice President of Product Management & Engineering. Based on Oracle's investment in MySQL, and their positive experiences using MySQL, FairWarning made the decision to continue to rely on MySQL as their solution database.
MySQL's Key Capabilities and Benefits
Certain MySQL capabilities, as well as the focus of Oracle MySQL's development efforts, were also key to FairWarning's decision. They include:
Scalability to Handle Terabytes of Data
FairWarning's customers have a lot of data. In fact -
- On average, FairWarning customers receive over 700,000 events to be processed daily.
- Over 25% of their customers receive over 30 million events per day, which equates to over 1 billion events and nearly one terabyte (TB) of new data each month.
- Databases range in size from a few hundred GBs to 10+ TBs for enterprise deployments (data are rolled off after 13 months).
And, MySQL has scaled efficiently to handle it all.
Performance, Reliability, and High Availability Improvements
In need of transactional capabilities, and seeing Oracle's strategic focus on InnoDB for MySQL, Chris and his team decided to migrate their solution from MyISAM to InnoDB and update from MySQL 5.5. to MySQL 5.6.
- Support for Mutli-CPU Threads - MySQL 5.6' support for multiple concurrent CPU threads, and FairWarning's custom data loader now allow multiple files to load into a single table simultaneously vs. one at a time. As a result, data load time has been reduced by 500%.
- Performance Schema - As the size of FairWarning's customers has increased, so have their tables and data volumes. MySQL's new database maintenance and management features have helped FairWarning keep up. Particularly important are the MySQL 5.6 performance schema that expose low-level metrics, providing insight into how the system is performing and why. Chris and his team have also appreciated the improvements to MySQL's reliability in recent releases.
- MySQL Enterprise Hot Backup - Because hospitals and clinics never stop, FairWarning privacy monitoring solutions must also be available 24x7x365, making high availability critical. FairWarning changed from using mysqldump to MySQL Enterprise Hot Backup, which has reduced downtime, restore time, and storage requirements. The biggest improvement has been on restore time - by up to 80% for many of their larger customers.
MySQL Enterprise Monitor and MySQL Workbench
FairWarning provides all administration and support -- including five layers of support within the FairWarning Private Service Cloud, for both SaaS and on-premise solutions. FairWarning uses MySQL Enterprise Monitor and its Query Analyzer to ensure MySQL is performing optimally and to alert them to any slow queries. Their customers also benefit from using MySQL Enterprise Monitor to keep an eye on trends and to changes to database indices.
In addition, Chris and his team use MySQL Workbench for data modeling and administration, saving them significant amounts of time and effort.
Low or Zero Admin = Few DBAs
"MySQL has not required a lot of administration. After it's been tuned, configured, and optimized for size on initial setup, we have very low administrative costs. I can scale and add more customers without adding DBAs. This has had a big, positive impact on our business," said Chris.
Summing Up
In summary, Chris reports that, "We have been very happy with the recent investments in MySQL's performance, scalability and recoverability. MySQL has acted on a lot of feedback that we've given them, which has been really helpful." MySQL has been the right database for FairWarning for over 9 years, from startup, through tremendous growth, to market leadership and expansion worldwide. FairWarning continues to rely on MySQL as they go from success to success.
1 2013 Best in KLAS: Software & Services report, January, 2014. © 2014 KLAS Enterprises, LLC. All rights reserved.