A quantum leap for molecular simulations

Jun 14, 2021

Developing improved materials for things such as energy storage and drug discovery is of interest to researchers and society alike. Quantum mechanics (QM), a theory that describes the physical properties of nature on the atomic and subatomic scale, is the basis for molecular and materials scientists who develop these useful, futuristic products.

Kennie Merz and Madu Manathunga standing in front of the MSU Chemistry Building — MSU's Kenneth Merz (left) and Madushanka Manathunga were part of a collaborative team that developed software that takes advantage of powerful graphics processing units, or GPUs, to ease computational science on midern computers. Credit: Harley J. Seeley

The challenge is that the QM calculations to describe the many properties of molecules and the materials they make up require a lot of computer power.

To address this issue, a small team of postdoctoral scholars led by Michigan State University’s Kenneth Merz, and University of California, San Diego’s Andreas Goetz have developed software that takes advantage of powerful graphics processing units, or GPUs, for these complex QM calculations of molecules.

The research was recently featured in two peer-reviewed publications: the Journal of Chemical Theory and Computation (JCTC) and the Journal of Chemical Information and Modeling (JCIM).

“The work described in these manuscripts attest to the importance of broad collaborative efforts between academia and industry to create complex yet efficient code,” said Merz, professor and Joseph Zichis Chair in Chemistry at MSU. With the availability of this mixed quantum/classical (QM/MM) capability, both we and other members of the computational science community will be applying it to address chemical and biological problems.”

“The need of freely available open-source software to enable computational molecular sciences on modern computers—from desktops to supercomputers with thousands of GPUs—inspired our efforts,” said Goetz, director of the SDSC Computational Chemistry Laboratory. “This research provides open-source software for efficient QM calculations and for mixed quantum/classical (QM/MM) simulations of molecular systems on graphics processing units.”

A figure showing how QUICK and AMBER work together. — This shows a multi-GPU speedup of B3LYP/6-31G** density functional theory nuclear gradient calculations with the QUantum Interaction Computational Kernel (QUICK) program for a range of organic molecules and proteins. Benchmarks were performed on the SDSC Expanse supercomputer. Credit: M. Manathunga, A. W. Goetz, B. Tolo

The software enables researchers to address problems that would otherwise be computationally intractable in a range of research areas such as computational materials design, catalyst design and drug design.

“Our code can also be used in a high-throughput fashion to generate data for Artificial Intelligence-based molecular design approaches, such as developing improved materials and drugs,” added Merz, who holds joint appointments in the Department of Chemistry and the Department of Biochemistry and Molecular Biology in the MSU College of Natural Science.

Merz’ group at MSU and Goetz’s SDSC group jointly designed the research, developed algorithms and engineered the software. The scientists used data collected on SDSC’s Expanse supercomputer.

In their JCTC paper, “Harnessing the Power of Multi-GPU Acceleration into the Quantum Interaction Computational Kernel Program,” the researchers explain supercomputers in a simplified way as made up of many tightly integrated computers that work in parallel together to solve complex computational problems. So, their software, called QUICK, coordinates computations across multiple GPUs to get solutions more quickly into the hands of researchers.

“This is like coordinating a fleet of trucks to deliver time-sensitive goods quickly from one place to another,” explained Madushanka Manathunga, MSU postdoctoral scholar. “We had to dissect the mathematical equations underlying the QM models to understand how we can efficiently parallelize the computations and effectively distribute the work onto multiple GPUs.”

In their JCIM paper, “Open-Source Multi-GPU-Accelerated QM/MM Simulations with AMBER and QUICK,” the researchers consider QM/MM coupling, which uses their faster GPU code for the complex QM calculations of, in this case, an active site of an enzyme together with a simplified MM model for the remainder of the protein beyond the active site region.

The importance of such computational methods has been recognized with the award of the 2013 Nobel Prize in chemistry for the development of multi-scale models of complex chemical systems.

Illustration of QUICK and AMBER interface — Artistic representation of the novel integration of the QUantum Interaction Computational Kernel (QUICK) program for multi-GPU enabled electronic structure calculations in QM/MM simulations with the AMBER molecular dynamics software package. Credit: V. W. D. Cruzeiro, A. W. Goetz, B. Tolo

“It’s exciting to have this multi-GPU QM code, QUICK, available for QM/MM simulations within AMBER [a suite of biomolecular programs that includes source codes and demos] so the community can take advantage of our code base to add new features and to use it to solve a range of chemical and biological problems,” Merz said.

Vinícius Cruzeiro, postdoctoral scholar at SDSC/UC San Diego, who co-led the software engineering work and co-wrote the papers with Manathunga, describes their technical work.

“This is like using a ‘computational microscope’ to zoom in on very tiny objects, such as important parts of a spike protein on a virus, to further our understanding of what is going on in detail and provide crucial information that might not be accessed experimentally,” he said.

According to the researchers, their software enables so-called ab initio Hartree-Fock and density functional theory (DFT) QM calculations. Due to its favorable ratio of computational cost to accuracy, DFT is widely used in computational molecular sciences.

“The features implemented in QUICK are available in commercial programs, but these require costly license fees, Cruzeiro said. “Therefore, it is rewarding to work in a project that aims to bring these features free-of-charge to researchers all over the world, especially to those impacted by paywalls."

Implementing their software on GPUs written in a special programming language, the researchers used the message passing interface (MPI) to distribute work onto different GPUs both within and across compute nodes. Part of this work was initiated last year at the first digital GPU Hackathon co-hosted by SDSC and GPU inventor Nvidia Corporation.

Other contributors included Kristopher Keipert and Kamesh Arumugam from Nvidia, Metin Aktulga and Chi Jin from MSU, Yipu Miao from Facebook (formerly an MSU graduate student), and Dawei Mu from NCSA (formerly at SDSC).

This work was supported by the National Science Foundation (NSF) (OAC-1835144); San Diego Supercomputer Center (computer time); the Departments of Chemistry and Biochemistry and Molecular Biology and the High-Performance Computer Center (iCER HPCC) at MSU; Nvidia (access to the Nvidia PSG Cluster). This work used the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by the NSF (ACI1053575, TG-CHE130010).

Banner image: This illustration represents the novel integration of the QUICK program as the engine to perform electronic structure calculations in QM/MM simulations with the AMBER software package for molecular dynamics. Figure credit: Vinícius Cruzeiro and Andreas Goetz