Annotation-free deep learning for quantitative microscopy
Abstract
Quantitative microscopy is an essential tool for studying and understanding microscopic structures. However, analyzing the large and complex datasets generated by modern microscopes presents significant challenges. Manual analysis is time-intensive and subjective, rendering it impractical for large datasets. While automated algorithms offer faster and more consistent results, they often require careful parameter tuning to achieve acceptable performance, and struggle to interpret the more complex data produced by modern microscopes. As such, there is a pressing need to develop new, scalable analysis methods for quantitative microscopy.
In recent years, deep learning has transformed the field of computer vision, achieving superhuman performance in tasks ranging from image classification to object detection. However, this success depends on large, annotated datasets, which are often unavailable in microscopy. As such, to successfully and efficiently apply deep learning to microscopy, new strategies that bypass the dependency on extensive annotations are required.
In this dissertation, I aim to lower the barrier for applying deep learning in microscopy by developing methods that do not rely on manual annotations and by providing resources to assist researchers in using deep learning to analyze their own microscopy data. First, I present two cases where training annotations are generated through alternative means that bypass the need for human effort. Second, I introduce a deep learning method that leverages symmetries in both the data and the task structure to train a statistically optimal model for object detection without any annotations. Third, I propose a method based on contrastive learning to estimate nanoparticle sizes in diffraction-limited microscopy images, without requiring annotations or prior knowledge of the optical system. Finally, I deliver a suite of resources that empower researchers in applying deep learning to microscopy.
Through these developments, I aim to demonstrate that deep learning is not merely a "black box" tool. Instead, effective deep learning models should be designed with careful consideration of the data, assumptions, task structure, and model architecture, encoding as much prior knowledge as possible. By structuring these interactions with care, we can develop models that are more efficient, interpretable, and generalizable, enabling them to tackle a wider range of microscopy tasks.
Parts of work
1. Quantitative Digital Microscopy with Deep Learning
Benjamin Midtvedt, Saga Helgadottir, Aykut Argun, Jes´us Pineda, Daniel
Midtvedt, Giovanni Volpe
Applied Physics Reviews , Volume 8, Issue 1, 011310, (2021)
https://doi.org/10.1063/5.0034891 2. Fast and Accurate Nanoparticle Characterization Using Deep-Learning-
Enhanced off-Axis Holography
Benjamin Midtvedt, Erik Olsén, Fredrik Eklund, Fredrik Höök, Caroline
Beck Adiels, Giovanni Volpe, Daniel Midtvedt
ACS nano, Volume 15, Issue 2, pp. 2240-2250, (2021).
https://doi.org/10.1021/acsnano.0c06902 3. Extracting quantitative biological information from bright-field cell images using deep learning
Saga Helgadottir*, Benjamin Midtvedt*, Jes´us Pineda*, Alan Sabirsh, Caroline B. Adiels, Stefano Romeo, Daniel Midtvedt, Giovanni Volpe Biophysics Reviews, Volume 2, Issue 3, 031401, (2021) https://doi.org/10.1063/5.0044782 4. Single-shot self-supervised object detection in microscopy
Benjamin Midtvedt, Jesús Pineda, Fredrik Skärberg, Erik Olsén, Harshith
Bachimanchi, Emelie Wesén, Elin K Esbjörner, Erik Selander, Fredrik Höök,
Daniel Midtvedt, Giovanni Volpe
Nature communications, Volume 13, 7492, (2022).
https://doi.org/10.1038/s41467-022-35004-y
Degree
Doctor of Philosophy
University
University of Gothenburg. Faculty of Science
Institution
Department of Physics ; Institutionen för fysik
Disputation
Torsdagen den 9 january 2024, kl 13:00, PJ-salen, Fysikgården 1
Date of defence
2025-01-09
Date
2024-12-05Author
Midtvedt, Benjamin
Keywords
microscopy
deep learning
image analysis
supervised learning
self-supervised learning
contrastive learning
synthetic data
virtual staining
object detection
particle sizing
artificial intelligence
Publication type
Doctoral thesis
Language
eng
Metadata
Show full item recordRelated items
Showing items related by title, author, creator and subject.
-
Flexibel utbildning i praktiken. En fallstudie av pedagogiska processer i en distansutbildning med en öppen design för samarbetslärande
Mattsson, Anita (2009-01-15)The aim of this study is to examine the pedagogical processes that evolve in an "open" design for online learning realized in relation to a specific setting. The study describes and analyzes pedagogical activities in a ... -
Learning aspects of out-of-hospital cardiac arrest and learning activities in basic life support - a study among laypersons at workplaces in Sweden
Bylow, Helene (2021-01-29)Abstract Background: Out-of-hospital cardiac arrest (OHCA) is one of the leading causes of death worldwide. Despite healthcare improvements, prevention for cardiovascular health, training in adult basic life support (BLS) ... -
Learning animal welfare and conservation at the zoo:
Ayalkibet Zeberga, Fikru (2020-11-17)Purpose: The aim of this research is to explore theories of learning animal welfare and conservation at the zoo using digital media. Hence, theories mainly focusing on animal welfare and conservation and learning will ...