subscribe to our newsletter

Data bias, risks inherent in banking AI

While the majority of banking executives believe AI will separate winning banks from “losers”, new research has shown that there are fundamental risks involved.

A new report released by banking software company Temenos has warned that with the coronavirus pandemic intensifying the use of artificial intelligence (AI) by banks, effective governance is more critical than ever.

The independent global review, carried out by the Economist Intelligence Unit (EIU), which consolidates findings from 25 regulatory reports, has identified data bias, “black box” risk and lack of human oversight as key concerns surrounding AI.

The report has posited that AI is expected to play a key role after the COVID-19 pandemic passes, as banks look to new technologies to help them adapt to changing customer needs and compete with new market entrants.

A recent Temenos survey revealed that 77 per cent of 300 banking executives surveyed during the coronavirus pandemic believe that AI will “separate winning banks from losers”.


Temenos chief product architect and head of AI Prema Varadhan said banks are using AI to process more data in real time and learn from customer behaviours, which helps bring operation costs down and personalise their service.

“Banks are using AI to transform their customer experiences and back-office operations, so ensuring that the technology is deployed ethically is more important than ever,” she said.

“As the custodians of customer data and trusted advisers, banks have a responsibility to adopt transparent, explainable AI technology – those that do stand to gain the competitive advantage in the new normal.”

The report suggests that while the nature of the risks involved in banks’ use of AI does not differ significantly from those faced by other industries, it is the outcomes that could be detrimental should the risks take shape – it could cause financial damage to consumers and financial institutions, and threaten the stability of the global financial system.

The report explains that some of the risks of AI include bias in the data that is fed into AI systems.


“This could result in decisions that unfairly disadvantage individuals or groups of people (for example, through discriminatory lending),” the report stated.

Citi Innovation Labs senior vice president Prag Sharma said bias can occur in AI models in any industry.

“But banks are better positioned than most types of organisations to combat it. Maximising algorithms’ explainability helps to reduce bias,” he said.

“Black box” risks could occur when the steps taken by algorithms cannot be traced and the decisions they reach cannot be explained.

Excluding humans from AI processes could weaken their monitoring and could threaten the integrity of the model, according to the research.

Mr Sharma said at the root of the risks is the inherent complexity of AI.

“Some AI models can look at millions or sometimes billions of parameters to reach a decision,” Mr Sharma said.

“Such models have a complexity that many organisations, including banks, have never seen before.”

The EIU report has listed key governance challenges and summarised regulatory guidance for banks using AI, including:

  • Ethics and fairness: banks must develop AI models that are “ethical by design”. AI use cases and decisions should be monitored and reviewed, and data sources should be regularly evaluated to ensure the data remains representative;
  • Explainability and traceability: steps taken to develop AI models must be documented in order to fully explain AI-based decisions to the individuals they impact;
  • Data quality: bank-wide data governance standards must be established and applied to ensure data accuracy and integrity, and avoid bias; and
  • Skills: banks must ensure the appropriate level of AI expertise across the business so they can build and maintain AI models, as well as oversee these models.

Commenting on the research, EIU editorial director Pete Swabey said: “AI is seen as a key competitive differentiator in the sector.”

“Our new study, drawing on the guidance given by regulators around the world, highlights the key governance challenges banks must address if they are to capitalise on the AI opportunity safely and ethically.”

[Related: New AI banking solution rolled out to accelerate SME finance]

Data bias, risks inherent in banking AI
Data bias, risks inherent in banking AI

Are you a new-to-industry broker in the process of growing your business? Then there’s some great news: The Adviser’s New Broker Academy is back in 2021 and will provide you with essential insights into cutting-edge tools, strategies and processes to fast-track to success. Don’t miss your chance to attend. To secure your FREE place, visit newbroker.com.au now!

Malavika Santhebennur

Malavika Santhebennur is the features editor on the mortgages titles at Momentum Media.

Before joining the team in 2019, Malavika held roles with Money Management and Benchmark Media. She has been writing about financial services for the past six years.

Latest News

Banks are too reliant on regulatory oversight to do what is right for borrowers, the big four boss has said. ...

There has been a surge in Melbourne residents migrating to regional Victoria, and an uplift in new housing approvals in Geelong, new researc...

ASIC reported a significant increase in the number of ACL and AFSL applications it received in the past year due to licensing reforms, inclu...

Join Australia's most informed brokers

Do you know which lenders are providing brokers and their customers with the best service?

Use this monthly data to make informed decisions about which lenders to use. Simply contribute to the survey and we'll send you the results directly to your inbox - completely free!

How long do you think it should take to discharge a mortgage?

Website Notifications

Get notifications in real-time for staying up to date with content that matters to you.