Introduction
The Dragonfly Telephoto Array is a project shared between the University of Toronto Department of Astronomy & Astrophysics, Yale University and Harvard University. It is designed to image large and extremely faint structures in the night sky. The objective is two-fold: improve understanding of dark matter through study of the low surface brightness universe and harness the potential of distributed telescopes combined with advances in information technology.
The Dragonfly Telephoto Array is a mosaic of 48 small telescopes, each of which communicates images simultaneously to a database. The images are then reviewed to eliminate optical distortions before being stacked into a single image for analysis. A final image is derived from the combination of thousands of input images captured over as many as 100 hours of night sky gazing.
Before adopting MySQL Enterprise Edition, the team was struggling to check the images that presented optical distortions caused by double stars. It was experiencing failed queries occurring regularly during batch processing and could not run more than 50 jobs at once. Hundreds of virtual computers were independently accessing data stored in MySQL Community Edition hosted in a Canadian government cloud computing environment. Yet the process was extremely intensive in computing time and manual intervention.
The Dragonfly team contacted Oracle Premier Support for advice on how to optimally configure the database in order to eliminate the bottlenecks preventing smooth stacking of the images.
Business Challenges & Goals
- Organize records of the individual images to facilitate identification of the images to be combined into the final image for analysis.
- Facilitate the batch processing of images by identifying and flagging 'double stars' in order to exclude those images from further analysis.
- Eliminate bottlenecks and failed SQL queries impeding smooth quality control and processing of astronomy images.
- Get guidance on how best to handle the data intensive storage and processing associated with massive volumes of space photographs captured and pulled into the database.
Business Results & Metrics
- Upgraded from MySQL Community Edition to MySQL Enterprise Edition 8.0 for organizing records deep images for identification during the process of combining them into one final image for analysis.
- Found and eliminated the source of failed queries with the assistance of Oracle Premier Support, leading to the ability to run several hundred jobs at once, resulting in time saving and greater efficiency.
- Removed barriers to parallelization batch jobs by drawing on Oracle Premier Support for assistance in correctly configuring and talking to the database.
- Overcame bottlenecks to image analysis by eliminating time-consuming short queries to the database, through increased computing power and query optimization provided by MySQL Enterprise Edition.
- Leveraged MySQL Enterprise Edition's high availability and scalability for managing millions of images destined for release to the public.
Why MySQL Enterprise Edition
"The database had quickly become a limiting factor due to the enormous amount of information that we process. Since we upgraded to MySQL Enterprise Edition and engaged Oracle Premier Support to optimize our database landscape, we are fully focusing on science and research without thinking about the IT technology behind it," said Dr. Roberto Abraham, Professor of Astronomy and Astrophysics, University of Toronto, Department of Astronomy & Astrophysics
About University of Toronto
The University of Toronto, the largest university in Canada, is one of the top-ranked research universities in the world. It is one of the eight global universities to be place in the top 50 in all eleven subject areas by Times Higher Education. The University of Toronto is actively engaged in a wide range of observational and theoretical research.
The Dragonfly project, originating in the David A. Dunlap Department of Astronomy & Astrophysics, has discovered many immense, low surface brightness galaxies in the Coma cluster. These ultra-diffuse galaxies are the sizes of large galaxies, such as the Milky Way, but with up to 1,000 times fewer stars. Follow-up studies are providing tantalizing clues to the distribution and nature of dark matter.
To know more about University of Toronto and the Dragonfly project visit www.utoronto.ca, www.astro.utoronto.ca and www.dragonflytelescope.org.