The Illusion of Trust

“Demons go to church too.” It’s a provocative reminder that appearances can be deceiving—and that not everything within trusted institutions is inherently good or safe.

In the context of finance, this phrase is more relevant than ever. AI systems and automation tools are being rapidly adopted across investment firms, banks, and corporate finance departments—often under the banner of efficiency, objectivity, and innovation. But here’s the uncomfortable truth: bias, manipulation, and error don’t disappear just because your tech wears a suit and tie.

In fact, if we’re not careful, AI can become the new way “demons” enter the sanctuary—through flawed data, unchecked assumptions, or invisible ethical trade-offs.

The Rise of AI in Finance—And Its Hidden Flaws

AI is transforming financial operations:

• Portfolio optimization
• Credit scoring
• Risk modeling
• Fraud detection
• Sentiment analysis

But even with all these advancements, AI is only as good as the data it’s trained on—and the human decisions behind its deployment. And in many cases, that data carries embedded bias, outdated logic, or incomplete context.

Examples include:

• Credit models that punish low-income applicants due to biased historical data
• Fraud systems that disproportionately flag certain demographics
• Predictive models that optimize purely for profit, ignoring broader social impact

These systems are technically advanced—but ethically dangerous.

When “Sanctified” Systems Go Wrong

AI systems in finance are often seen as neutral, objective, or even superior to human judgment. But that perception can lead to dangerous complacency. We assume:

• “It’s in the model, so it must be fair.”
• “It came from a top vendor, so it must be accurate.”
• “Everyone’s using it, so it must be safe.”

This blind trust is precisely what allows flawed systems to take root undetected.

Just like demons in a church, unethical outcomes can hide inside systems built with good intentions but poor oversight.

Why Finance Professionals Must Pay Attention

You don’t need to be an AI engineer to understand the risks. As a finance professional, your role is to:

• Question the assumptions behind the tools
• Evaluate the real-world impact of automation
• Advocate for transparency and accountability

Because ultimately, the consequences of a flawed system don’t fall on the model—they fall on the people affected by its decisions.

How to Keep “Demons” Out of Your AI Systems

Here are five concrete ways finance professionals can help ensure their AI tools live up to both technical and ethical standards:

1. Audit the Data, Not Just the Code

Ask:

• Where is this data from?
• Who is excluded from it?
• Does it reflect outdated policies or hidden biases?

2. Push for Explainability

If a model makes a decision—can you explain it in plain English? If not, how can you defend it to regulators or clients?

Use tools like:

• SHAP, LIME (for feature importance)
• Transparent scoring frameworks
• Model cards and documentation

3. Question the Objective Function

What is the model optimizing for?

• Is it short-term revenue at the expense of fairness?
• Is it efficiency at the expense of transparency?

Reframing objectives can significantly alter model behavior.

4. Build a Cross-Functional Ethics Panel

Include legal, compliance, data science, and business leads. Regularly review:

• High-risk use cases
• Edge cases and anomalies
• Audit logs and flagged incidents

5. Maintain Human-in-the-Loop Controls

Never fully remove human oversight from critical decisions—especially in lending, hiring, risk scoring, or compliance. Augment, don’t abdicate.

Sanctity Comes from Scrutiny

Just because your AI system was built with good intentions doesn’t make it immune to harm. Just because it’s technically brilliant doesn’t make it ethically sound.

In today’s finance world, we must remember:

Even demons go to church. And even flawed AI can wear a badge of trust.

The challenge for finance professionals is to not only adopt AI—but to steward it. That means building systems that are not just smart, but fair. Not just fast, but accountable. Not just powerful, but worthy of the trust we place in them.