Continuous Updates: The story behind the product

April 1, 2021
Continuous Updates: The story behind the product

Working with banks and financial services organisations ranging from Tier 1 incumbents to challengers and pioneering FinTechs, we are exposed to a world of opportunities to help them do things better.

One of the opportunities we encounter most frequently is the chance to accelerate and streamline customer onboarding

This involves a customer entering data into a web-based application form, validating the data and cross-checking it with the service provider, and a decision made to accept or reject the account.

This can be a long, laborious process involving lots of manual effort. 


How we help solve the digital onboarding challenge

Here at DueDil, we aim to offer our clients the best, most comprehensive and timely business information

This enables our clients to automate onboarding processes, ensure they have the information at their fingertips to comply with regulatory requirements, and intelligently manage risk. 

We go beyond querying registries to identify connections and enrich the data - enabling us to map group structures of businesses, drill down into the true nature of businesses and much more.

Via our REST API, we provide easy access to data on every UK & Irish company. This allows our clients to auto-populate data in the customer onboarding flow.



This flow enables our clients to remove the requirement for applicants to look up and manually enter data, ensure the data input is accurate and significantly reduce the time to onboard or reject a new customer based on regulatory requirements or their own risk appetite. 

For example, Santander UK has leveraged the DueDil API to streamline its onboarding process. What used to take 14-21 days now takes only 5 for the vast majority of applications. 


The missing piece of the puzzle: in-life monitoring

While accelerating and enhancing onboarding is a boon for financial organisations, it doesn’t solve the whole challenge because you need to stay on top of those customers for life, not just when they apply to be a customer.

Businesses change all the time… what if there is a change that would affect either the regulatory status or the risk profile of a business in a client’s book of business? 

The only solution here is “In Life Monitoring”. Typically this would take hours of work by relationship managers and other team members to collect information on potentially thousands of customers or requiring customers to update you when things change. 

This approach is costly, time-consuming, and doesn’t really add much value. It prompted us to build a new solution.

The original DueDil offering was very much a point in time. 

This left clients with two options:

  1. Put their fingers in their ears (not actually an option)
  2. Keep hitting DueDil’s APIs, downloading the data for everyone on their books, and check them all for changes

DueDil’s core data is available on an unlimited basis to our subscribers, but there are some checks that have quotas (e.g. PEPS & Sanctions or Credit reports). Some of our clients are massive financial institutions with thousands of businesses on their books.

Clearly, we had to find a better way that worked for both us and our clients.


The technical challenge to solve

DueDil processes A LOT of data. 

Our Business Information Graph (B.I.G.)™ , the powerhouse at the centre of our solutions, has 115 million business entities with 270 million connections.

That’s a lot of data and it is updated daily. Anyone that’s worked in or around data will understand that Big Data is big because of the three Vs - volume, variety or velocity. 

We have all of these and therefore monitoring changes is a non-trivial task, but one we were keen to solve.

DueDil’s tech stack is fully cloud-native and able to scale vertically and horizontally with ease. This enables us to process massive amounts of data in parallel and ensures we can offer it in a timely manner. 

Our engineering team had a few key challenges to solve:

  • Designing a future-proof schema to represent any kind of change - we don’t want to hard code something and then have to rip it all out and start again to support enhancements
  • Building the relevant business logic to track relevant changes across the data points we want to surface
  • Designing scalable data storage that can handle the volume and provide the response times we need 
  • Designing a way to surface this to clients in a useful manner


The product challenge to solve

DueDil is a lean, agile, product-led organisation. Yes, everyone says that, to the point of it being a cliche, but we live and breathe it. 

The core principles of the product team are::

  • Speak to clients to understand what they need, what the “job to be done” is, and propose a solution that is valuable, viable and feasible
  • Build the smallest possible solution to the problem that provides value - don’t try to boil the ocean
  • Work with clients to integrate and iterate

After initial discovery, we identified the Minimum Viable Product (MVP) that would provide useful functionality to our clients and validate the solution.


Identifying and building the MVP for in-life monitoring

The areas discussed with clients included:

  • The fields they needed updates on (not all fields are created equal)
  • The biggest pain points in the current process (having to continually spam the APIs to get data for every company, and maintain logic to check them)
  • Knowing what has actually changed, rather than just any change at all (not all changes are equally important)

Based on this, we focused on a lightweight, MVP solution which is now generally available.

  • Daily, as our data pipelines run to update the B.I.G, we produce metadata on the most important fields identified by our clients and check to see what has changed since the last run
  • We produce an output file containing the IDs of the companies with changes, as well as the categories of change (e.g. change in PSC, shareholder info updated - full list available in our API docs)
  • An API endpoint which can be queried per date up to 30 days into the past, which will return all companies changed and the types of change. This returns in less than a second.

So now the client flow is:


Get the guide: Download our Continuous Updates product guide to see exactly how this new flow is helping our clients automate in-life monitoring.


How this helps our clients to answer their business questions and tackle in-life monitoring

On average, we track 10,000 individual changes to company data every day. This can escalate up to 60,000 on a given day.

Any one of these changes could be critical to a customer relationship.

Our new approach substantially reduces the amount of calls to the API required and helps our clients to concentrate on the customers that need attention. 

This enables our clients to make smart decisions during the life of their customer engagements, including: 

  • Re-evaluating risks based on change of company status or financials
  • Proactively managing regulatory checks around new company officers
  • Updating risk profiles based on changes in the nature of a business
  • Responding to Gazette notices before the change takes effect

And of course, the ability to keep CRMs up-to-date and identify cross-selling opportunities as businesses evolve.


What comes next? The evolution of the Continuous Updates product

We’ve come a long way already but we’re taking a pause; not because we don’t have ideas but because, like any good product organisation, we’re looking to see how the market reacts and which of our ideas will have the biggest impact. 

There’s plenty more to come, but for now if you’d like to speak to one of our product experts, please get in touch for a demo!

We’d love to speak to you about how Continuous Updates will help you, and any enhancements that could help us to help you even more.

Author: Jason Knight, Product Director at DueDil

Sign up to the latest DueDil news!
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.