AdaptiveFlow
AdaptiveFlow is an in silico platform consisting of three modules (AFLP, AFVS and AFU) and readymade ligand libraries for virtual screens. In this section, general platform-wide features are described that span multiple modules.
Python codebase
AdaptiveFlow has been entirely rewritten in Python, a programming language that enables the implementation of more sophisticated and advanced code while reducing maintenance work. This rewrite has also made it easier for scientists and programmers to join the open-source project and/or extend AdaptiveFlow independently, as Python is the most widely used programming language in scientific communities. In contrast, the previous version of AdaptiveFlow was written in Bash.
Parallelization
In addition to being rewritten in Python, a key goal for AdaptiveFlow was to improve parallelism to decrease the overall time required for screening. AdaptiveFlow has increased the levels of parallelism compared to the previous version. To help manage this increased parallelism, it maintains the use of collections that group around 1,000 ligands together, reducing the overhead of scheduling individual ligands for processing.
AdaptiveFlow also introduces the concept of subjobs, which are atomic units of computational work that include one or more collections for processing. These subjobs are automatically generated by AdaptiveFlow to match the runtime requirements specified in the job configuration. Subjobs are then combined into larger groups called work units, which are designed to allow schedulers to efficiently process the high levels of parallelism generated by AdaptiveFlow. These work units are sent as job arrays to the end schedulers for processing. By optimizing for parallelism, the team was able to achieve 5.6 million concurrent vCPUs running AdaptiveFlow.
Native cloud support
The AFLP and AFVS modules of AdaptiveFlow can run on computer clusters or on the cloud by using a batch system. Currently, two batch systems are supported, Slurm and AWS Batch. Slurm is able to run on many cloud infrastructures, including Google Cloud, Microsoft Azure and AWS. Within AWS, AdaptiveFlow uses AWS Batch and S3 object storage. CloudFormation is used to set up AdaptiveFlow and all necessary components. The AWS-native architecture allows AdaptiveFlow to use millions of vCPUs in parallel, while the previous version was only demonstrated to scale up to 160,000 vCPUs.
AdaptiveFlow is able to use various features from AWS in addition to supporting traditional clusters, making it easier to use and increasing the scale of computations available to researchers. Some of these features are as follows:
-
Scheduling and orchestration. AWS Batch is a cloud-native scheduler and orchestrator for containerized jobs. As part of running a job, AdaptiveFlow can make use of AWS Batch to automatically scale cloud infrastructure up and down as needed and run all of the individual subjobs. AdaptiveFlow can also automatically generate the Docker container used for processing, eliminating the need to understand containerization or other infrastructure.
-
Object storage (Amazon S3). In addition to the usage of standard cluster filesystems, AdaptiveFlow supports the usage of object storage for both ingesting of input datasets and the output of the resulting data from the processing.
-
Spot capacity awareness and spot resilience. Spot capacity is unused computing capacity that cloud providers make available at greatly discounted prices compared to normal on-demand access. In exchange for a lower price, a workload running on a spot instance may be reclaimed (stopped) with a short warning. AdaptiveFlow is designed to take advantage of this capacity model by using shorter running subjobs (approximately 30 min) and automatically rerunning jobs that have prematurely exited through integration with AWS Batch. Additionally, through integration with AWS Batch, AdaptiveFlow is able to select instances that are less likely to be interrupted through the ‘spot-capacity-aware’ scheduling method.
The architecture of AdaptiveFlow when deployed on AWS is shown in Supplementary Fig. 1. The architecture makes use of multiple compute environments, each of which may contain thousands of instances (compute nodes). The instances are automatically allocated and deallocated when the job starts and ends, respectively. Individual subjobs are allocated to the instances by AWS Batch until all subjobs have been completed. These subjobs run as Docker containers on the instances.
ARM
AdaptiveFlow is also able to run on the compute ARM architecture when using ARM-compatible choices in the settings. AFLP is fully ARM compatible and AFVS is ARM compatible when docking programs are used that either make the source code available or provide binaries for ARM. ARM-based computing infrastructure can be more cost-efficient than traditional x86-based architectures.
Enamine REAL Space
The REAL Space (enumerated, version 2022q1-2) was prepared using AFVS. JChem Suite was used for molecule neutralization, stereoisomer enumeration, tautomer generation, protonation state prediction, 3D coordinate calculation and the calculation of multiple molecular properties (JChem Suite 18.20.0; ChemAxon, https://www.chemaxon.com). Open Babel56 (version 2.3.2) was used as a backup program for protonation state prediction, 3D coordinate calculation and target format conversion (all formats except SELFIES). The pip-installable SELFIES package (https://github.com/aspuru-guzik-group/selfies) was used for the conversion of molecules between SMILES and SELFIES. An overview of the processing steps and preparation methods used is shown in Supplementary Table 2.
Multiple formats
AdaptiveFlow provides the REAL Space library in ready-to-dock 3D formats including PDB, PDBQT, MOL2 and SDF. These formats are compatible with a wide range of docking programs for small molecules. The total size of the ready-to-dock library is approximately 50 TB in compressed form for each of these formats, totaling around 200 TB in compressed form (uncompressed, the library is around 2 PB). In addition to these ready-to-dock formats, the REAL Space library is also provided in the SELFIES31, SMILES57 and Parquet formats.
AFLP
AFLP is the module of AdaptiveFlow dedicated to the preparation of ligand libraries into a ready-to-dock format and can handle libraries of any size, including ultralarge libraries. AFLP requires a batch system and supports Slurm and AWS Batch as batch systems. AFLP is configured using a configuration file (example shown in the Supplementary Information). Details on the workflow are available in the Supplementary Information. The new version of AFLP has the below-described features.
Preparation into ready-to-dock format
The primary task of AFLP is to prepare ligands into a ready-to-dock format. To accomplish this, AFLP desalts the original ligands (in case they are salts), neutralizes them, generates stereoisomers and tautomers, predicts the protonation states, computes the minimum-potential energy conformation of the ligand (3D coordinates), optionally validates the generated 3D conformers and converts them into the desired target formats. In total, over 100 output formats are supported. A detailed overview of the processing steps can be seen in Supplementary Fig. 3. The support input formats are SMILES, SELFIES and amino acid sequences. AFLP uses external tools such as ChemAxon’s JChem, Open Babel and RDKit (https://www.rdkit.org/) to carry out the preparation steps. For many of the preparation steps, the user can choose which external tool should be used. Details on which tools are available for each processing step can be found in Supplementary Table 3.
Molecular property calculation
AFLP enables researchers to calculate a variety of physicochemical properties and molecular descriptors for the molecules in the input ligand library, in addition to its primary task of preparing them for molecular docking. By calculating properties such as quantitative estimate of drug likeness, logP, molecular weight and others (full list of supported properties in Supplementary Table 2), researchers can more effectively select and prioritize ligands, which can help reduce the cost and time required for screens using AFVS. AFLP uses mostly external tools to calculate the molecular properties, such as ChemAxon’s JChem (https://chemaxon.com/calculators-and-predictors, Open Babel and RDKit (details in Supplementary Table 3).
Automatic tranche assignments
In AdaptiveFlow, ligand libraries are organized into tranches. Each tranche corresponds to a property of the ligands and is divided into multiple intervals or categories. For example, the molecular weight of the ligands can be used as a tranche type and then partitioned into several categories or intervals such as 0–200, 200–300, 300–400, 400–500 and >500 Da. Multiple tranches allow for the partitioning of a ligand library into a multidimensional grid or table, with one dimension corresponding to each tranche type (ligand property).
While the previous version of AdaptiveFlow required that the input ligand collection was already organized and structured in the tranche format, AdaptiveFlow can automatically reorganize the input ligand library into a new tranche output format on the basis of the computation of specific molecular properties of the ligands, as described above. This feature is useful because most ligand libraries (for example, from chemical compound vendors) are initially in the SMILES format and not in the tranche format. AFLP can then be used to prepare the entire library in the multidimensional grid or tranche format for AFVS.
To characterize how the Enamine REAL Space library populates the 18-dimensional property grid used by AdaptiveFlow, we quantified tranche occupancy (the number of molecules assigned to each grid cell) and summarized it as a histogram (Supplementary Fig. 37). We counted molecules per tranche across the full library. The resulting occupancy counts span several orders of magnitude; thus, we report the distribution using logarithmically spaced occupancy ranges (1, 2–10, 11–100, 101–1,000, 1,000–10,000, 10,000–100,000, 100,000–1 million, 1 million–10 million and >10 million molecules per tranche). The histogram displays the number of tranches in each occupancy range (y axis), highlighting that only a small fraction of the theoretical grid cells are populated and that populated tranches vary widely in size (from single-molecule tranches to highly populated tranches exceeding 10 million molecules), with an average of 5,600 molecules per populated tranche.
Support of SELFIES
SELFIES31,32 has become one of the most popular line notation formats for small molecules within ML communities. The primary reason is that SELFIES are fully robust, meaning that every SELFIES string corresponds to a valid molecule, which is not the case for SMILES. Thus, generative ML models cannot create invalid molecules when using SELFIES. AFLP now supports SELFIES as an input format for the molecules in addition to SMILES. Additionally, the entire REAL Space is free to be downloaded in the SELFIES format, enabling the creation of powerful datasets.
Stereoisomer enumeration
One of the main tasks of AFLP is to prepare ligand libraries from unprepared formats (for example, SMILES) into ready-to-dock formats. The new version of AFLP enables the enumeration of stereoisomers for the ligands, which was not possible in the previous version. Accurate stereochemistry is essential for virtual screenings using structure-based molecular docking but many ligand libraries in unprepared formats (for example, from chemical compound vendors) do not contain stereochemical information. The ability to enumerate stereoisomers allows AFLP to include this information in the prepared ligand library, ensuring that the virtual screening results are accurate.
Extended open-source support
The previous version of AFLP required external tools (from ChemAxon) that are not open source for some of the steps when preparing ligands in a ready-to-dock format. The new version provides the option to choose open-source alternatives for each of the steps to prepare molecules into a ready-to-dock format. In particular, Open Babel and RDKit are supported by AdaptiveFlow as open-source alternatives.
Enhanced speed
AFLP has been improved in terms of speed compared to the original version. Firstly, this has been achieved by AFLP using its own Java code to call the necessary functions of ChemAxon’s JChem package through Nailgun, which greatly reduces the number of calls to Nailgun. Additionally, timeout parameters that can be adjusted by users have been added to the control file, allowing AFLP to skip ligands that require unusually long processing times during preparation. These improvements make AFLP more efficient, robust and able to handle larger and more chemically complex ligand libraries.
Enhanced quality checks
AFLP checks the validity of each ligand that it prepares in the ready-to-dock format by calculating the potential energy of the ligand in 3D format using Open Babel. This allows AFLP to automatically remove ligands that have an excessively high potential energy, which can be an indication that the compound was damaged or corrupted during the preparation process. Additionally, AFLP supports optional plausibility checks on generated 3D conformers using PoseBusters58. This helps to ensure the integrity and accuracy of the prepared ligand library.
Application
AFLP was used to prepare the REAL Space (version 2022q1-2). It was run on AWS using over 5.6 million Intel vCPUs in parallel. Spot instances were used and the used capacity was sustained without notable preemption. In total, <0.1% of the used CPU hours were interrupted because of preemption. The scaling behavior of AFLP was found to be perfectly linear (as shown in Supplementary Fig. 2). This demonstrates the efficiency and effectiveness of AFLP in preparing large ligand libraries in a ready-to-dock format.
AFVS
AFVS is the module of AdaptiveFlow dedicated to ULVSs but can also screen libraries of smaller sizes. It requires a batch system and supports Slurm and AWS Batch. AFVS is configured using a configuration file (example in the Supplementary Information). AFVS and AFU share many virtual screening features, which are described in a later section. AFVS and AFLP share the same parallelization mechanisms when using Slurm or AWS Batch; therefore, the scaling behavior of AFVS is expected to be the same as for AFLP, which was shown to be perfectly linear up to 5.6 million CPUs (Supplementary Fig. 2).
Parquet output format
AFVS has the ability to store output data in the Apache Parquet format, an open-source column-oriented data storage format. When using this format, AFVS can take advantage of any database and query system that can read this format, including Amazon Athena, a serverless service that allows users to efficiently query output score files stored in Amazon S3 (object storage) on AWS. Additionally, AFVS has a feature for automatically postprocessing ATG prescreens using Amazon Athena, enabling researchers to efficiently prepare the input data for the ATG primary screens. Tools are provided with AFVS to facilitate common queries but researchers can also create their own queries using the open-source format of the dataset.
ATG-VSs and benchmarks
Following completion of the ATG prescreen, tranches can be selected for the ATG primary screen using two main strategies. In the first, individual tranches from the 18-dimensional REAL Space matrix can be manually selected, offering fine-grained control over specific chemical subspaces. In the second, selection can be restricted to a hyperrectangular region of the grid. This is achieved by identifying the top-performing intervals (that is, bins) within each of the 18 molecular property dimensions and computing their Cartesian product to form a structured, intersecting subset of tranches. This hyperrectangle approach enables chemically diverse yet targeted selection based on multiple favorable properties. In both selection modes, users define the total number of molecules to include in the ATG primary screen and AdaptiveFlow automatically prioritizes the tranches containing the best-performing representative ligands.
To evaluate the effectiveness of ATG-VS, we performed two types of benchmark studies: (1) full-scale production benchmarks using the entire Enamine REAL Space and (2) smaller-scale ‘test-set-based’ benchmarks designed for rapid prototyping and detailed comparison across different ATG configurations. In the test-set-based benchmarks, we randomly selected synthetic reactions from the Enamine space to create a compound subset totaling approximately 5 million molecules. For each of the ten protein targets tested (listed in Supplementary Table 7), a different random subset was used to ensure unbiased evaluation across targets and screening scenarios.
The test-set-based benchmarks allowed us to systematically assess the benefits of ATG-VS in three configurations: standard ULVS (random selection), ATG without active learning and ATG augmented with the optional ML-based active learning component. In contrast, the full-scale production benchmarks were conducted using the entire 69-billion-compound library, but without the active learning step, to minimize computational overhead while validating the performance of ATG-VS at scale. All docking in these benchmarks was performed using QuickVina 2 with default parameters, unless otherwise specified. For the 100K ATG benchmark configuration presented in Extended Data Fig. 5, tranches for the primary screen were selected using the hyperrectangle method, enabling a balanced, property-informed sampling of chemical space.
ML classifier for tranche prioritization
To improve the efficiency of ATG-VSs, the AdaptiveFlow platform incorporates an optional ML classification step into the primary screening stage. This step prioritizes compounds that are most likely to be high-affinity binders, thereby reducing the number of molecules subjected to full docking. The ML classifier is applied after the ATG prescreen and is used to filter undocked molecules within the selected tranches before docking.
The classifier is implemented as a fully connected feedforward neural network designed to distinguish between promising and less likely binders. Compounds are represented using Morgan fingerprints of length = 1,024 and radius = 2, generated by RDKit (version 2023.03.1). The model is trained on docking scores obtained from all representative molecules used in the prescreen. These scores are converted into binary classification labels using a percentile-based thresholding strategy: the top 25% of compounds with the most negative docking scores are labeled as positive examples (‘high-confidence binders’), while the remaining compounds are labeled as negatives. To mitigate class imbalance, the majority class is undersampled during training.
The neural network architecture comprises four fully connected layers: an input layer of 1,024 nodes, followed by hidden layers of 512, 256 and 128 nodes, each using ReLU activation. The output layer consists of a single sigmoid-activated node that outputs the predicted binding probability. Hyperparameters and layer configurations were selected through limited tuning using cross-validation on the prescreen data. The model is trained with the Adam optimizer (learning rate: 0.001), binary cross-entropy loss and early stopping based on validation loss with a patience of five epochs. A validation split of 10% is used, with training capped at 50 epochs and a batch size of 256, although convergence typically occurs earlier.
Once trained, the model is applied to all undocked compounds within the selected tranches. Compounds with predicted binding probabilities greater than 0.5 are retained for full docking, while the rest are filtered out. This filtering step typically reduces the number of compounds subjected to docking by 30–70%, depending on the target, while maintaining or improving hit enrichment.
The ML classification module is fully integrated into the AFU environment and can be activated through a configuration parameter. The training and inference pipeline is implemented in PyTorch (version 2.0.1) and supports GPU acceleration for scalable and efficient deployment.
AFU
AFU is the all-in-one version of AdaptiveFlow, combining both ligand preparation and virtual screening into a single streamlined workflow. It is the third module alongside AFLP and AFVS. This new module adds many conveniences for workflows where both ligand preparation and virtual screenings have to be carried out in tandem. Notably, AFU is designed to be run as a package, supporting approximately 1,500 docking protocols that can be run independently of job schedulers (Supplementary Table 8). A conceptual overview of the workflow of AFU can be found in Supplementary Fig. 8 and a detailed UML workflow diagram is shown in Supplementary Fig. 14. Specifically, as input, a user specifies the molecule to be docked (as a SMILES, SELFIES or amino acid sequence), the protein file, the docking site and the docking program to be run. On the basis of the user’s choices, error checks in the form of missing docking-choice-specific files are performed and calculations are run. As a result of a successful calculation, the docking score and pose are returned. AFU can be run in standalone mode, where it is run through the command line and specifies the options using a configuration file. This mode is for example useful for scientists in the drug discovery communities who want to carry out molecular dockings in the most convenient and simple way, without requiring computational expertise, while having a large number of docking protocols available. In addition, AFU can be run in API mode to directly interface with other programs and code, which can, for instance, be useful for the ML communities that develop new methods for drug discovery.
AFU comes in two versions. The first is dedicated to single workstations or compute nodes without multinode parallelization. The second, AFUparr, is the parallelized version of VitualFlow Unity. Designed to operate seamlessly on SLURM systems, AFUparr allows users to easily parallelize the workflow on a larger number of CPUs and GPUs. Execution within SLURM environments is highly optimized, with computations distributed in parallel across multiple CPUs and nodes. This design ensures efficient linear scaling relative to the number of molecules provided.
Joint features of AFVS and AFU
The AFU and AFVS modules support the same docking protocols and, therefore, share all docking-specific features.
Supported docking protocols
AdaptiveFlow supports approximately 40 docking protocols, listed in Supplementary Tables 4–6. These include software that performs both pose prediction and scoring (Supplementary Table 4), solely pose prediction (Supplementary Table 6) and only scoring (Supplementary Table 5). Notably, within AdaptiveFlow, the scoring functions can be combined with pose prediction methods, giving rise to approximately 1,500 methods (listed in Supplementary Table 8). We note that many of the newly supported docking software possess special features such as protein–protein or RNA docking, explicit parameterization of solvents or metals and covalent docking. In Supplementary Tables 4–6, we describe these as ‘special features’. Additionally, a key consideration for ULVS is the timing required for carrying out a single calculation. As such, in Extended Data Fig. 9, we provide timing estimates for running a single calculation, averaging over 25 independent runs for all docking programs.
Special types of ligands
In addition to traditional small molecule ligands, AdaptiveFlow supports the use of peptides, protein ligands and covalently bonded ligands in AFVS. Peptides are of increasing interest as therapeutics and several docking programs already support their use. For protein–protein or RNA docking, specialized programs require two protein or RNA targets to generate co-complexes. Covalent docking involves reduced flexibility for certain parts of the input molecule during pose sampling and a few specialized docking programs support this ligand type (Supplementary Table 4).
Enhanced performance
The QuickVina 2 molecular docking tool is used to accelerate AutoDock Vina with no loss of accuracy27. QuickVina 2 execution was profiled in an effort to improve performance even further. To establish baseline performance, the open-source GitHub repository was cloned and the qvina_1buffer branch was compiled using the GNU C++ compiler (version 9.4), the Boost C++ libraries (version 1.79, https://www.boost.org) and the default parameters (build instructions: https://qvina.github.io/compilingQvina2.html). A single docking experiment of avanafil binding to unsolvated PDB 5FNQ was run through the Intel VTune Profiler to look for performance bottlenecks. Unfortunately, the execution profile was flat. The most time-consuming function accounted for less than 20% of the total runtime. Tuning the source code would give rapidly diminishing returns at the risk of affecting accuracy; hence, compiler-level optimization was the best option to improve performance. QuickVina 2 was rebuilt using the Intel oneAPI DPC++/C++ Compiler (version 2022.2.0.20220730) and aggressive optimization: automatic vectorization, interprocedural optimization and relaxed floating-point constraints. Profile-guided optimization was also tried but it did not improve performance. The final executable was 18% faster than the default builds with no loss of accuracy. The new binaries are provided on the QuickVina and AF homepages.
GPU support
Several of the docking programs support GPUs, which can strongly reduce computation time and cost. For pose prediction, these include AutoDock-GPU52, QVina2-GPU53, QVina2-W-GPU53 and Vina=GPU53. Gnina59, NNScore (version 2.0)60, DeepBindRG61 and DeepAffinity62 use GPU-compatible deep learning models for scoring protein–ligand complexes. Additionally, deep learning methods such as TANKBind12, DiffDock13 and EquiBind28 use GPU-compatible neural networks for direct pose prediction of several ligands in parallel. AdaptiveFlow supports GPU compatibility of all these software. The relative speedups of these methods for single calculations can be compared within Extended Data Fig. 9.
Molecular-dynamics-based methods
The combination of molecular docking and molecular dynamics can yield results near experimentally determined crystal structures63,64. AFVS includes methods for selecting and scoring poses on the basis of molecular dynamics simulations, such as MM/PBSA and MM/GBSA65, for evaluating docking poses and predicting binding affinities. MM/PBSA and MM/GBSA are popular for predicting binding free energy because they are more accurate than molecular docking and less computationally intensive than alchemical methods. AdaptiveFlow also supports binding pose metadynamics (BPMD), which uses a stability score based on the resistance of the ligand to perturbation away from the initial pose to rerank predicted binding poses from docking. The OpenBPMD algorithm66 can accurately predict binding poses within 2 Å (root-mean-square deviation) of the crystallographic complex structure for many systems and efficiently rerank docked poses. The MM/PBSA, MM/GBSA and BPMD methods can be used within AdaptiveFlow as rescoring and pose refinement methods.
Enhanced quality checks
AFVS supports optional plausibility checks on generated docking poses of Vina-based docking protocols using PoseBusters, an open-source tool, which checks for chemical as well as intramolecular ligand and intermolecular validity of ligand–protein complexes58.
Reporting summary
Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.
