Hacking neurons in 3D
July 24, 2015
On June 15-17, 2015, the Allen Institute for Brain Science hosted an annotation “hackathon” as part of BigNeuron: a community effort led by the Allen Institute to define and advance the state-of-the-art of digital reconstructions of neurons. The hackathon drew over a dozen scientists from around the world to Seattle for an intensive session of tracing and analyzing images of neurons from several species in diverse areas of the brain.
“The team we created really worked together to generate useful data for evaluating neural reconstruction results, and developed better methods,” says Hanchuan Peng, Associate Investigator at the Allen Institute for Brain Science. “Everybody was engaged and wanted to contribute. It was very inspiring.”
Over the course of the three-day workshop, during which participants also attended a variety of talks on subjects related to neural reconstruction, the annotators generated 160 “gold standard” reconstructions of neurons, meaning at least six and sometimes seven people reviewed the reconstruction to ensure it was the highest possible quality. Additionally, they generated another set of 140 “silver standard” reconstructions which were reviewed by at least three annotators.
Attendees worked in tight knit teams of three or four people per team, typically with at least one person familiar with the BigNeuron tools and two other people with a background in annotation. “The teams formed tight groups that worked together very quickly and efficiently—in large part because they could share results in real time,” says Peng.
“Watching people who traveled from across the globe assemble around this challenge of creating the highest possible quality neural reconstructions was very exciting,” says Mike Hawrylycz, Investigator at the Allen Institute.
Previous BigNeuron hackathons have been held in Beijing, China, Cambridge, UK, and at the Janelia Research Campus of HHMI, in Ashburn,Virginia. These hackathons focused on porting the algorithms that will be used to analyze neurons and automatically generate reconstructions. In contrast, the Seattle hackathon was the first to focus on setting the standard of reconstruction quality by having many reviewers look at the same neurons and determine the best possible reconstruction.
During the upcoming bench testing phase of the project, supercomputers will be used to test those algorithms against the gold and silver standard neurons generated by this and other groups to see which algorithms most closely match the work done by hand.
“Having the scientists work together in person was particularly valuable because each person brought a slightly different perspective to the best way to annotate the neurons,” says Ed Lein, Investigator at the Allen Institute for Brain Science. “This is exciting because it means we will end up with extremely high quality standards against which to test the algorithms during bench testing.”
“We are so pleased with how the hackathon went,” says Peng. “In just three days we accomplished a great deal of reconstruction work, learned a lot about how we can improve the way we capture individual neurons to make these reconstructions easier and more accurate in the future, and demonstrated that the global community can come together to do more than any of us could do alone.”