Algorithm Porting and Bench-Testing
Algorithm Porting onto A Common Platform
The BigNeuron project aims at an objective comparative evaluation of state-of-the-art neuron reconstruction algorithms. To do so, we consider porting many existing neuron reconstruction methods (algorithms) to a common platform and bench-testing them on supercomputers.
The common platform to implement the BigNeuron bench-test infrastructure will be Vaa3D (in Chinese ‘挖三维’), an open source visualization and analysis software suite created and maintained by Janelia Research Campus of HHMI and by the Allen Institute for Brain Science. This software performs 3D, 4D and 5D reconstruction and rendering of very large image data sets, especially those generated using various modern microscopy methods, and associated 3D surface objects. Vaa3D has a rich set of functions and plugins for neuron quantification (see "Data Visualization and Analysis Protocol") and is compatible with well-established neuron analysis tools such as L-Measure. Vaa3D is suitable for manual, semi-automatic, and completely automated digital tracing. This software has been used in several large neuroscience initiatives and a number of applications in other domains.
Vaa3D has a simple and powerful Open Source plugin interface that allows extending the functionalities of the software. Specifically, a developer can take advantage of a tool called Vaa3D Plugin Creator (which is itself a Vaa3D plugin) to generate source code templates for porting algorithms or other functions into Vaa3D simply and effectively (see "Algorithm Porting Protocol").
We have ported to Vaa3D a number of neuron tracing algorithms covering several major known categories of methods, including fitting geometrical elements, ray casting, spanning tree, shortest paths, deformable curves, pruning, etc. Vaa3D porting allows use of shared image reading functions to load neuron data, and neuron exporting functions to produce SWC format reconstructions. More algorithms are being ported or developed using the same infrastructure (see "Algorithm Porting Protocol").
Because Vaa3D itself runs on Mac, Unix/Linux and Windows, all ported algorithms automatically gain such cross-platform capability, facilitating development, testing, and integration. Importantly, given the plan to run the large-scale bench-testing on supercomputers, the Vaa3D command line execution and scripting feature will come in handy to toggle between “development mode” and “batch-testing mode” of any ported algorithm.
NeuronAssembler for Scaled-up Reconstruction
A ported algorithm in BigNeuron will run as a standalone Vaa3D plugin. For the majority of neurons in the bench-test whose data size is relatively small (e.g. for fruit fly neuron reconstruction), it will trace an individual 3D image stack to produce one single SWC format file. However, when the image volume becomes substantially large (e.g. an image stack of a mouse neuron with more than 50 giga-voxels), it will be impractical to require any existing neuron tracing algorithm to reconstruct such large data set effectively. The computational cost in terms of memory use and tracing time will be huge and often unmanageable. Therefore, we have created NeuronAssembler, a special Vaa3D plugin enabling the ported algorithms to reconstruct very large images without the need to alter their source codes. NeuronAssembler reconstructs tiled regions of large image stacks separately, and then assembles the pieces of neurons together to produce a complete large neuron reconstruction. Any ported algorithms, as long as they can run as standalone Vaa3D plugins, can be directly invoked by NeuronAssembler to handle very large neuron images. Therefore, NeuronAssembler guarantees scalability in terms of neuron data set volume.
Meetings for Algorithm Porting
We will organize several hackathons to help contributors and developers port data and algorithms onto the common Vaa3D platform. Each hackathon will be one week long and largely focused on linking existing algorithms as executable Vaa3D plugins. Instructors from the Vaa3D development team will provide hands-on support of the importing process. Preliminary usability tests of the plugins will be carried out in the hackathons as well. Other potential hackathons will focus on quantitative analysis and evaluation protocols.
Interested people who would like to attend a hackathon, or who like to participate but cannot attend such meetings, should email us at email@example.com.
Algorithm Porting Protocol
- Interested people who would like to contribute or port an algorithm should register by sending an email to firstname.lastname@example.org.
- The common platform used for algorithm porting will be the C/C++ and Qt based Vaa3D platform: Nature Protocols, Vol. 9, No. 1, pp. 193-208, 2014.
- For bench-testing, each neuron reconstruction algorithm will receive the same image input and generate the reconstructions in the same format. These algorithms do not need to handle image data I/O or reconstruction file I/O. These I/O operations will employ a common infrastructure within the Vaa3D plugin interface.
- All ported algorithms will be tested on supercomputers requiring implementation of a dofunc function in the Vaa3D plugin interface to invoke the algorithm developers’ own tracing code. See the instructions at Vaa3D Google Code site.
- Typical Vaa3D plugins can be created by using the Vaa3D plugin creator tool, following the paper Nature Protocols, Vol. 9, No. 1, pp. 193-208, 2014, Protocol G. Importantly, to facilitate easy porting of algorithms, developers are strongly encouraged to use the "Create plugin for neuron reconstruction" menu in the plugin creator tool to generate template source code for algorithm porting.
- Source code examples of ported neuron tracing algorithms can be found in the Open Source Vaa3D tool hackathon folder. To find more information, send an email to email@example.com.
- Source codes, binary or libary based methods contributed to this project will follow the BigNeuron Method Contributor Agreement (see "Open Source Algorithms").
Selected Ported Neuron Reconstruction Algorithms
- All-path-pruning 1 (bottom-up pruning) (Peng et al.)
- All-path-pruning 2 (top-down pruning) (Xiao, et al.)
- FarSight-Snake (Roysam, et al.)
- NeuTube (Zhao, et al.)
- MST-Tracing (Zhou, et al.)
- TreMap (Peng, et al.)
- Most-RayCasting (Wu, et al.)
- SimpleTracing-dt (Yang, et al.)
- Simple-RollingBall (Yang, et al.)
- Simple-RayCasting (Yang, et al.)
- NeuronGPSTree (Zeng, et al.)
Algorithm Contributing Organizations
Allen Institute for Brain Science (USA), Janelia Research Campus, HHMI (USA), Univ. Houston (USA), GE (USA), Univ. Georgia (USA), Huazhong Tech Univ. (China), Univ. of Sydney (Australia), A*Star (Singapore), Maebashi Inst. of Tech (Japan), North Illinois Univ. (USA). National Univ. Sci. Tech. (Taiwan), Univ. Tokyo (Japan), Anhui Univ (China), Northeast Univ. (China), Chinese Academy of Science (China), Beijing Univ. Tech. (China), Tsinghua Univ. (China), etc.
Open Source Algorithms
BigNeuron encourages algorithm/method contributors to use an Open Source license (e.g. MIT) in order to make the bench testing as easy as possible while reserving their future freedom to use the source code for other purposes.
BigNeuron uses Vaa3D as the primary platform for algorithm porting and bench testing. Vaa3D uses a slightly revised MIT License with the only additional requirement being to cite the Vaa3D publications appropriately. Modules, extensions and plugins of Vaa3D may use different licenses imposed by the various contributors when such situations are specified and documented. A license file should be included in the respective source code repository folder for any contributors to the BigNeuron project.
While the BigNeuron project encourages bench-testing of neuron reconstruction methods based on ported versions from their source code, in rare cases if the developers of some methods only make their binary form of their methods available, the developers need to make sure the binary form (e.g. library file or final executable file) can be linked to Vaa3D appropriately. Since a binding will be made to Vaa3D as well, the Vaa3D license should be applied in such cases, too.
It is important to stress that ported source code or any binary form of code should not contain any malware or malicious code. Such software should also not violate any other license agreements. The “Method Contributor Agreement” (see below) should be applied.
Method contributors, who use either open source or close source approaches to port their neuron reconstruction methods, have to agree the following terms before they can participate in the BigNeuron project:
BigNeuron Method Contributor Agreement (2014-2015)
Method contributors, who use either open source or close source approaches to port their neuron reconstruction methods, should agree that they would not participate or use the BigNeuron project in any manner that would violate anyone else's rights, such as copyright, trademark, patent, privacy or other rights. This includes removing any copyright, trademark or other proprietary notices. Method contributors may not create media or hyperlinks to the BigNeuron resources (e.g. website, database, documentation site) that portray this project in a false or misleading light. Method contributors agree that they will only make lawful use of the BigNeuron Data and Resources in compliance with all federal, state, and local laws and regulations. Specifically, all Method contributors have to agree the “BigNeuron Terms of Participation and Use” before they can participate the BigNeuron project.