Exploring Product Reference Data under the Consumer Data Right
With the passing of the October 1 deadline, the majority of Australian banks are now required to share their product data via APIs as a result of Consumer Data Right legislation (usually referred to under the umbrella of Open Banking).
As the CEO of LIXI, I'm interested in these APIs since some of the data that is sourced from these and other Open Banking APIs invariably make their way into data integrations based on the LIXI standards.
These product APIs (unlike account APIs) expose no customer data, so they do not require any particular credentials to access them, making them a great candidate for a quick exploration to see how easy they are to work with.
This blog post is a quick exploration of how easy it is to work with the product APIs from a variety of these banks using Python (a completely arbitrary decision of programming language). Spoiler alert - it's VERY easy, but I can see some caveats to how easy it will be to use the data for automatic product evaluation and suggestion. We've added a sample showing Node.js at the bottom of this page.
A word of caution - this code is for exploration and illustration purposes only, and no error handling is incorporated at all.
October 1 Deadline
Due to COVID-19, the original deadline for financial services providers to share product data was delayed to 1 October 2020. This includes non-major ADIs, including non-major banks, building societies, credit unions and non-primary brands under the major banks. The major banks already share this product reference data.
Product reference data includes rates, fees and features of banking products (specified here).
Where to start?
Each bank publishes their API details somewhere, and in a quick search, the Westpac Open Banking Product API is at the top of the list. This page shows the URL of their Get Products API.
Let's start a new python script, import two libraries (json and requests), and create a variable for the url. Setting up your Python environment is beyond the scope of this blog post, I've assumed you can do this already. Alternatively, use one of the many on-line Python interpreters such as the one here (an online tool like this is ok for the product API as it contains publicly available data only).
Now since the product API is available to everyone with no restrictions on use, it is a simple as calling a GET request and saving the response into a variable.
We can now load the response data into a json object, iterate through it and interrogate the data for some fields that interest us. The structure of the product details in this json object is available in the CDR Specification here. We might want to print the name, brand, and productId of every product in the list to the console.
And the console shows:
Well that was easy, in six lines of code, we can obtain product data, but why are there only 25 records in the response? Surely Westpac has more than 25 products?
The API is only returning the results one page (of 25 results) at a time, but the API conveniently returns the URL for the next page of responses, so we simply need to loop through until there is no next URL provided. You can read about this in the specification here.
More Financial Institutions?
What if I want a selection of products from across a range of financial institutions? It's easy to create a list of a number of APIs and simply loop through each of those URLS. The entire script, with absolutely zero error handling (so buyer beware), now looks like this:
It's very easy to use the Product List APIs, at least at a superficial level in order to look at hundreds of banking products in just a few minutes of coding and a few seconds of CPU time. I noticed that many of the products have additional whitespace around the name of the product, which does make me wonder about an issue that often arises in working with data standards. Over the years, I've seen any number of ways in which a well-defined standard can leave room for interpretation and variability of implementation. This can lead to unnecessary complexity in how consumers actually use the APIs in real life. Trimming whitespace is trivial, but there are many seemingly minor variations that can have consequences that are far from trivial.
My next step (in a future blog post) will be to look at the quality and consistency of the data as an indicator as to how variable the implementations are, particularly in the more detailed data available in the product detail API.
Another point to note is that all the eligibility data (specified here) that a participant would need to evaluate in order to actually suggest a product to a customer is mostly strings or URLs. It would be very hard to use this data to programmatically analyse the eligibility of a specific product for a specific customer at this stage. This concept of eligibility, particularly when it comes to credit products can become extremely complex. The scale and comprehensiveness of the LIXI data standards is evidence of this, given that the LIXI Data Standards now have been used for 20 years to originate credit products in the Australian lending industry.
Addendum - Node.js Sample
Here is the Node.js you can use to accomplish the same thing, and you can use an online environment such as this one and simply copy the code below into the index.js file and select Run.
Shane Rigby, LIXI Limited CEO
First Published: October 1, 2020 | Last Updated: October 2, 2020