A competition to compare the brains of male vs. female fruit flies led New Jersey Institute of Technology researchers and two high school students to a second-place finish in the latest edition of the FlyWire Data Challenge, but more importantly the team learned new lessons about using artificial intelligence for setting up research with supercomputers. In the challenge, hosted earlier this year by Princeton University’s Neuroscience Institute, teams had to create efficient maps of the 19,000 synapse connections between neurons in tiny insect brains. The maps take the form of graphs, and the information they convey represents part of the fly brain called a ventral nerve cord connectome.
Distinguished professor David Bader, who directs NJIT’s Institute for Data Science, led NJIT Principal Research Scientist Zhihui Du along with Edison Magnet School students Srijith Chinthalapudi and Harinarayan Asoori Sriram. The team decided to create algorithms that could run on NJIT’s Wulver supercomputer. To start, they selected Anthropic Claude as their AI system. Claude is a research-focused alternative to generic systems such as OpenAI’s ChatGPT.
“The massive scale of matching two 19,000-node graphs required efficient parallel processing,” Bader explained. “Using Claude 3.5 Sonnet for rapid prototyping helped us quickly iterate through different matching strategies, but we needed to ensure the generated code would scale effectively on Wulver, including multiple parallel programming paradigms like multicore CPU, GPU acceleration and message passing. The challenge wasn’t just computational — we needed to maintain solution quality, while handling the complexity of weighted directed edges representing synaptic connections.” To read the full story.