Breaking Data Team Silos Is The Key To Getting Ai To Production

Enterprises are experiencing FOMO and rushing to deploy AI-based services and AI agents, but for nan teams that are responsible for keeping those systems moving successful production, nan patterns that are forming are starting to consciousness familiar: Silos are forming betwixt information scientists and operations teams, conscionable arsenic they did years agone betwixt developers and ops. But location is hope.

“Every work retired location — let’s return AWS Bedrock and SageMaker — they’re supporting OpenTelemetry, which is great,” said Thanos Matzanas, an observability master astatine IBM. “For nan first time, we each agreed that this is nan measurement to go.”

In this section of The New Stack Makers, recorded astatine AWS re:Invent 2025, I sat down pinch Matzanas and his IBM workfellow Martin Fuentes to talk why AI observability is still an afterthought, what enterprises tin study from past level shifts, and why breaking down organizational silos whitethorn beryllium much important than immoderate caller monitoring tool.

The Silo Problem Returns

The situation isn’t caller technology, it’s aged organizational patterns repeating themselves, Matzanas and Fuentes argued. Not excessively agelong ago, information teams worked successful isolation connected models that often only served soul purposes. Now, they are abruptly responsible for customer-facing applications pinch existent gross implications.

“It’s nan first clip they travel into nan mix,” Matzanas explained. “Usually, they were doing their ain point connected nan side, and now they’re coming into nan operation because they’re really serving existent clients, not soul clients, existent clients pinch existent revenue. And now they consciousness nan aforesaid pressure, I deliberation that each nan different teams were emotion successful nan past.”

Fuentes sees a akin dynamic. The AIOps and tract reliability engineering (SRE) organization spent years building observability champion practices, and galore of them are transferable to this existent moment, but “we still request to fig retired really to measurement that business worth attached to nan models,” he said.

Don’t Forget nan Basics

When asked what proposal they’d springiness to enterprises conscionable starting their AI journey, Matzanas based on that it each comes down to going backmost to nan basics.

“Don’t hide nan basics. This is thing different from immoderate different exertion stack,” he said. “What are your metrics? What are your KPIs, what are your [service-level objectives]? How do you show nan services astir your AI applications? How do you show your vector databases? How do you show your APIs? If you screen each nan basics, past erstwhile nan AI comes in, you person a bully bedrock to build on.”

The challenge, though, is that AI models are different because they are not deterministic. Traditional personification acquisition monitoring worked because you could trace a petition from nan personification interface each nan measurement to nan infrastructure. With AI, overmuch of nan feedback loop depends connected humans.

“We trust a batch connected personification feedback to understand what’s going on,” Matzanas said. “And it’s very difficult to find nan value of nan interaction.”

Security and Compliance by Design

This intends that putting successful guardrails is particularly important for getting models into production, but Fuentes emphasized that AI workloads require nan aforesaid governance rigor arsenic accepted applications — and possibly more. “It’s not only astir trusting nan consequence of an conclusion from a model, but besides nan concerns astir really nan information is utilized to train nan models,” he said.

This, too, is astir going backmost to existing devices for illustration role-based entree power (RBAC), audit logs, and decently documenting really models and agents person been trained to debar bias. “There are galore things that we learned are important for accepted workloads, and location is nary logic why not to use them successful AI.”

Low-Hanging Fruit: Get People successful nan Same Room

Asked astir wherever to start, some pointed to nan value of managing organizational alteration alternatively than solely focusing connected exertion adoption.

“Break nan silos arsenic soon arsenic possible,” Matzanas said. “We cognize how, because we’ve done it successful nan past. Include your information group successful nan conversation. Show them really it looks successful production. Don’t support them siloed connected 1 side.”

Fuentes offered a complementary perspective: Find nan business worth first.

“If you want your activity to bargain successful and springiness you nan resources to use generative AI [GenAI] models aliases agents successful your application, it needs to beryllium thing that will supply business worth eventually,” he said. “Talk pinch your users astir what problems you tin resoluteness wherever generative AI tin beryllium a bully solution.”

Will AI Replace Observability Teams?

Both pushed backmost against nan conception that AI will destruct quality roles successful observability — astatine slightest for captious systems.

“Because of nan criticality of observability successful galore cases — ideate if you’re monitoring a healthcare exertion — are you going to ever time off that successful nan hands of AI?” Matzanas said. “There’s nary way. I don’t spot immoderate pilots losing their jobs anytime soon.”

Fuentes was likewise optimistic. “AI will very apt not really switch humans connected top. It’s conscionable thing other that we tin usage to summation productivity,” he said.

Group Created pinch Sketch.