---

Around the corner from the North Carolina State Fairgrounds, tucked behind a gated entrance, sits an unassuming building with something incredible inside: 1.5 million fish specimens. Welcome to the Ichthyology Collection at the North Carolina Museum of Natural Sciences (NCMNS).

Try saying it out loud. It’s pronounced ick-thee-ol-uh-jee.

“Ichthyology is the study of fishes, which means all of the biodiversity of fishes,” says Lily Hughes, research curator for the collection. “There’s about 37,000 species of fish that live in our planet today.”

Inside the collections room — sometimes called “the range” by staff — large stainless steel tanks line the walls and narrow hallways cut through rows and rows of metal shelves engulfed with jars of sea life dating back to 1850. Eels, pufferfish, sturgeon, even a 13-foot thresher shark. It’s all there, just waiting to be explored.

“We are a lending institution,” Hughes explains. “We lend specimens out to researchers so they can ask their own questions and learn more about all of the life on our planet.”

On the other side of the building, a carpeted room hosts even more shelves, but these hold something arguably more important than the specimens themselves: field notes.

Each fish in the collection has a corresponding set of handwritten data that includes who collected it, where and when it was captured and a plethora of other information that gets uploaded into a public database. That’s how a scientist in Japan studying lamprey, for example, could end up getting their hands on information needed for a paper they’re writing.

“If we don’t have that information, then we just have a specimen without any data — and we can’t really tell you much about it,” says Gabriela Hogue. “It can be used for teaching. It can be used for dissecting or making models. But it can’t be used for deeper research or for conservation and management.”

Collection managers like Hogue and their staff spend hours uploading handwritten specimen notes into electronic databases. This process is the same for museums around the world, from NCMNS to the Natural History Museum in London.

The Renaissance Computing Institute (RENCI) at UNC-Chapel Hill is working to reduce the time it takes to do this.

“RENCI … specializes in data science and cyber infrastructure, which essentially just means that we use different computational approaches to help other people do their research,” says Chris Bizon, director of analytics and data science at the institute.

Bizon and his team are trying to develop an agent using a multimodal large language model — AI that can process multiple types of data like text, images, audio, and video at once — to transcribe these handwritten notes and infer the latitude and longitude for where each specimen was discovered.

To build something big, Bizon starts small, spending time with Hogue and Hughes to uncover their concerns, their pain points, and how to achieve accuracy within their transcription process. Most recently, RENCI brought a bunch of their teammates together to host a hackathon, where they spent a half-day coding the agent, assessing problems, and sharing ideas for how to fix them.

“A lot of it is really around trying something, looking at what comes out, going back, modifying the approach,” Bizon says. “It’s just like any other project where we have some goal we’re trying to achieve and some different approaches we can take.”

A tool like this could be revolutionary for museums worldwide.

“It is going to be an absolute life-changing thing for collection managers everywhere,” Hogue says. “By minimizing that time, we can maximize our effort handling the specimens and globally providing the data so that researchers, educators, conservation managers, teachers all over the world can utilize [it].”