REAL-TIME DNA DIAGNOSTIC APPLICATION

What is real-time DNA sequence analysis application?

This application analyzes DNA nanoparticles in real time. It detects taxonomic rates, potential viruses and pathologies, antibiotic resistance genes, etc. The application then visualizes the results received through the interactive Sunburst tool. Thanks to convenient and detailed data visualization, medical professionals can make informed decisions about their patients’ treatment plans.

Nanopore DNA sequencing has many applications in the Health and Agriculture industries:

• Diagnosis of infection from blood samples and taxonomic counting
• Monitor, profile and track the evolution of antibiotic resistance genes
• Detection of pathogens in wastewater or public health
• Identifying viruses in cassava

How it works?

Blood and other DNA samples were collected from a portable nanopore sequencer. We used the MinION device developed by Oxford Nanopore Technologies. The received data is then sent to the platform and goes through the following workflow:

• Ingest   – files are uploaded to Google Cloud Platform and streamed into the processing pipeline
• Base call phase   – machine learning model infers DNA sequence from electrical signals
• Sequencing phase   – through the DNA database, samples are analyzed for pathogen sequences, taxonomic enrichment and other abnormalities
• Summary phase   – calculates the percentage of each pathogen in the specific sample
• Storage and visualization   – results are saved to Google Firestore DB and then visualized in real time with D3.js.

Project objectives

To develop a DNA nanoparticle sequence analysis application, we need to perform the following steps.

• Investigate Genomics domain data formats
• Develop raw string data simulator
• Empower applications by reading source data
• Develop logic for the alignment phase
• Develop a process for the briefing phase
• Find the right solutions and tools to store and visualize analyzed data
• Write documentation on how to use the application and make it available on GitHub

Our challenge

During the development of the DNA nanoparticle sequencing application, we faced the following challenges:

• Build scalable and reliable system architecture  . Nanopore Sequencer DNA Analysis is a resource-demanding process. Due to the large volume of data and time constraints, the system needs to scale accordingly. We eliminated this challenge by leveraging the data processing, storage, and compute tools of Google Cloud Platform. Thanks to such an approach, we have achieved smooth, streamlined and reliable scalability of data processing operations.
• Data processing logic  . The project required fine-tuned data processing operations to provide a wide range of results in minimal time. Therefore, we need to develop data processing logic that connects analytics applications to the cloud platform and ensures efficient information flow. To achieve this goal, we used the Apache Beam library that runs on Google Cloud Dataflow and supports integration with other Google Cloud services. We have also integrated the Compute Engine to build auto-scaling Alignment Clusters in the application.
• Integrated DNA Analysis Tool  . The DNA Analysis tools for the Nanopore sequencer are desktop utilities only, so we need to adapt them for cloud platforms. We also needed to integrate desktop DNA analysis tools into a unified, scalable system. We reinterpreted our desktop-based DNA analysis tools for the HTTP format and delivered them as web services so they can process large amounts of data in less time.

How did we do it?

System architecture

For the system architecture, we applied a series of Google Cloud Platform data processing engines as well as computing capabilities. Data collected from the nanopore DNA sequencer will be available in a new DNA read file and uploaded to the Cloud Storage bucket. Once all the files are uploaded, they go through a workflow that converts the input files into actionable reports.

Machine learning engine

The application must include a machine learning algorithm that will analyze the genetic engineering database to produce relevant DNA test results. To achieve this goal, we applied a machine learning model on TensorFlow, previously trained with several genomic databases.

Server side

We needed to create a server-side for web and mobile developers. Therefore, to store the database and files, we leveraged Firebase, a document storage system that can represent hierarchical data, needed to represent biological taxonomies.

Client side

Our main concern is to shorten the time it takes for data to be uploaded and visualized from the sequencer.

To keep things as fast as possible from the client side, we implemented a dynamic dashboard with D3.js, which periodically polls the database for new data and updates the visualization corresponding Sunburst chart.