Finished! Looks like this project is out of data at the moment!
We have just updated the Field Guide and FAQs with how to deal with imaging artefacts. Plus, check out our first results update.
We are really excited to share a first look at the full results from Synapse Safari. If you missed our livestream last November on Twitch, read on to find out how all your hard work is being used by the team:
We are interested in looking at the changes in structure within the hippocampus during development. In this period, there is a huge amount of change in the brain: Neurons begin to extend massive cable-like structures, dendrites and axons, that contact each other and form connections. We are also interested in studying neurodevelopmental disorders that happen during this key period of development and dynamic structural change – and we want to understand structurally where things go wrong.
To study these structural changes, we use a high-resolution technique called serial block-face scanning electron microscopy (SBF-SEM) to generate a series of 2D images, which when stacked together in sequence, can be reconstructed to form a detailed 3D volume of the original sample.
Before we launched Synapse Safari, Juan, Guilherme, and the rest of the team at King’s College London had been analysing their electron microscopy datasets manually: Going through each 2D slice of the 3D data volume and segmenting out between 10 and 20 dendrites and axons, mapping how they are connected to each other. Analysing the morphologies of the segmented dendrites and axons at this “zoomed-out” level allowed them to begin to create large-scale reconstructions of areas of the brain across development, to try to understand how the wiring process is happening. Juan explains: “We found some really interesting distributions and properties in the way that synapses distribute along these dendrites, along the length of these dendrites. But we were lacking detail.”
The team wanted to look closer, within each synapse, to understand the structure, number, and arrangement of synaptic vesicles and mitochondria – organelles that are fundamental to how the synapse functions. But when each presynaptic terminal contains ~100 vesicles, and a given neuron will receive in the order of 1000 or more synapses, mapping just a handful of the billions of neurons in the brain is a huge task.
Here, we first turned to computational methods: we used a deep learning model called MitoNet 1 to segment the mitochondria, and a technique called template matching to segment the synaptic vesicles. Both of these techniques performed relatively well, which was great! But neither were perfect, so we turned to the crowd to inspect and correct the output from these two techniques: a process called proofreading.
Here’s a look at the numbers – your efforts resulted in over 350,000 classifications, with over 30,000 mitochondria and ~150,000 synaptic vesicles identified.
With the help of our volunteers, we were able to use the corrected segmentations of mitochondria to finetune the MitoNet model, and make it work better for our datasets, meaning fewer missed, fragmented, or poorly outlined mitochondria:
The finetuning resulted in an increase in F1 score (also known as the dice coefficient) and an increased IoU (intersection over union): two metrics we use to assess how good our segmentations are against the “perfect case”. Here’s an article on evaluation metrics if you want to read more.
By clustering the marks made by volunteers on missed or mistaken synaptic vesicles (SV), we were able to get a huge number of correctly segmented vesicles:
Here’s what that clustered result looked like in a few more subjects:
When taking the classifications from 2D slices to the full 3D volume, we had some new challenges. We noticed that some synaptic vesicles seemed to appear in 3 or 4 consecutive slices of our data. This was unusual because synaptic vesicles are typically 40-50 nm in diameter, and the z-resolution of our data (the distance between adjacent slices) was also 40 nm, so we would expect the same vesicles to only appear in 1 or 2 consecutive images.
We found that this was the result of an imaging artifact associated with the SBF-SEM technique. At small section thickness (<50nm), beam-induced resin softening leads to compression during cutting, resulting in uneven material removal (This article has some examples). Essentially, when slicing off each layer of material, some areas of the surface softened and were squished down, rather than being cleanly sliced off.
So, how did we deal with this artifact to get our final 3D reconstructions? Well, it involved a lot of hard work and computational effort from Chloe. We had the full volume imaged, but the z-position of the vesicles in the regions affected by the artifact was ambiguous. To correctly locate and count the synaptic vesicles in the affected regions, Chloe had to track individual vesicles across slices so that they weren’t incorrectly counted multiple times.
Here’s an example of a region affected by the artifact. Each column shows one slice of data, with and without the aggregated locations from the synaptic vesicles workflow. In this example, the first and last columns are visibly different from columns 2-5, but the middle columns have regions that look nearly identical.
For each pair of adjacent slices, Chloe had to measure the similarity between the regions, and if they appeared too similar, track which vesicles were present in both images, and match each vesicle with itself into cross-slice pairs. Then, these cross-slice pairs would be merged (to not count the vesicles twice), whilst vesicles without pairs, or regions that weren’t similar enough, would be counted as normal.
Finally, after all this processing, we were able to get the 3D visualisations of mitochondria and vesicles in the synaptic boutons:
In total, after the aggregation, merging, and 3D reconstruction, you have helped us to find ~5000 mitochondria and ~70,000 synaptic vesicles in all our datasets. Just the three examples above show the heterogeneity of synaptic boutons segmented: some with no mitochondria, some with multiple.
We also uncovered some interesting mitochondria morphologies. The datasets you annotated come from proximal (near) and distal (far) regions compared to the cell body. In both p22 and p100 datasets, we found that proximal mitochondria exhibit the expected elongated morphology, but distal mitochondria have an unexpected “beads-on-a-string” morphology:
Chloe explains: “This has been observed in in some other tissues, like heart muscle. It has been observed in the brain as well in Alzheimer’s patients. But I think we were the first to discover that there's actually a distance related characteristic for these beads-on-a-string morphology.”
The team are looking forward to combining their prior analysis of the distribution of dendrites and synapses with the new information gleaned from the segmentations of vesicles and mitochondria. Having this level of structural detail is key for understanding function as well, Juan says: “We have this structural description of these vesicles and mitochondria within a presynaptic terminal. But what does it mean for the function, for the process of synaptic transmission, for the process of releasing neurotransmitter onto another neuron?”
Guilherme adds: “Having automated methods allows us to go back and analyse and mine the data that we already obtained, but also it has the added incentive of making us acquire more data […] which is something that we had almost given up on, because we thought that we couldn't analyse even the data that we've acquired already”
Watch the full conversation on our YouTube Channel.
[1] Conrad R, Narayan K. Instance segmentation of mitochondria in electron microscopy images with a generalist deep learning model trained on a diverse dataset. Cell Syst. 2023 Jan 18;14(1):58-71.e5. doi: 10.1016/j.cels.2022.12.006.
In Science Scribbler: Synapse Safari, we're working together to segment mitochondria and synaptic vesicles in the hippocampus - the brain's memory and learning centre. With every click, you're helping reveal how these small structures in neurons develop, bringing us closer to understanding brain function.
Since the project launch, your enthusiasm and dedication have been overwhelming. We are grateful for every contribution. Let's take a look at what we've achieved together!
Watch how your classifications filled the 3D volume in just a month!
This gif is showing your classifications for the Correct Synaptic Vesicles workflow on one of the datasets. Each point represents a subject that you’ve worked on. We showed the same image to 10 volunteers before combining your answers. The colour from orange to green shows how many volunteers have classified each subject, from just started (orange) to complete with all 10 classifications (green).
We finetuned the deep learning model for segmenting mitochondria using your classifications.
Sometimes mitochondria take on unusual shapes or appear fragmented - these aren't mistakes, they might hold important clues about brain development or neurological conditions. The original model failed to identify them in the two instances below, but thanks to your contribution, our new finetuned model has learned to recognize these not-so-typical mitochondria!
Using the new AI model trained on your classifications, we've begun assembling a complete picture of mitochondria in our datasets. We've expanded beyond the synaptic boutons you worked on to reconstruct all mitochondria throughout the neurons. Here's what they look like in 3D!
This is just the beginning of our journey. As you continue to classify more structures and we process your results, the AI models will become more powerful, and our understanding of the hippocampus will become more complete and detailed. Every classification adds another piece to this complex biological puzzle!
Thanks again for your continued support!
We were amazed to have received over 30,000 classifications in the first 48 hours since the launch of Science Scribbler: Synapse Safari!
As of now, all three mitochondria workflows – Mito Mapper, Mito Inspector and Correct Machine Segmentation for Mitochondria – have been completed, which means you have verified and corrected all the segmentation of mitochondria in boutons in this entire dataset!
Thank you for your incredible work so far!
In Mito Spotter, you looked at the synaptic boutons without machine segmented mitochondria. You confirmed that 3111 of them truly do not contain any mitochondria. What’s more exciting is that you have spotted 260 potential mitochondria that slipped past the machine learning model. Great catch! Here are some examples:
In the Mito Inspector, you looked at the synaptic boutons where the machine learning model found a mitochondrion. You verified that 394 of them are the correct segmentation, and found 40 that aren’t quite right. Here are some examples:
In this workflow, you corrected the machine mitochondria segmentations that don’t look quite right. Multiple people worked on the same images, and we have aggregated the results. We know how difficult it is to accurately draw on your computer, but collectively, you have done an amazing job! Here are some examples:
Correct Synaptic Vesicles is still active and is in need of your help! These small structures play a key role in how neurons communicate. By accurately segmenting them, we can know their numbers and distribution, as well as how they interact with mitochondria in hippocampal neurons.
Your classifications bring us closer to obtaining an accurate full segmentation of mitochondria in synaptic boutons. Here’s what we will do with the results from each workflow:
The dataset you have just finished was taken from the most distal part of the dendritic branches of CA1 pyramidal neurons in the hippocampus of a young mouse (22 days old). We know these neurons are important both for the formation of memories, as well as working as an internal GPS system. We will be uploading three new datasets soon:
Our previous research has shown that the proximal and distal regions of CA1 pyramidal neurone are specialised for different jobs. These new datasets will enable us to better understand if mitochondria and synaptic vesicles help with this separation of functions, and how they change as the neurons mature. We would greatly appreciate your help in segmenting mitochondria and synaptic vesicles in these new datasets!