We use essential cookies and similar tools that are necessary to provide our site and services. We use performance cookies to collect anonymous statistics, so we can understand how customers use our site and make improvements. Essential cookies cannot be deactivated, but you can choose “Customize” or “Decline” to decline performance cookies.
If you agree, AWS and approved third parties will also use cookies to provide useful site features, remember your preferences, and display relevant content, including relevant advertising. To accept or decline all non-essential cookies, choose “Accept” or “Decline.” To make more detailed choices, choose “Customize.”
Customize cookie preferences
We use cookies and similar tools (collectively, "cookies") for the following purposes.
Essential
Essential cookies are necessary to provide our site and services and cannot be deactivated. They are usually set in response to your actions on the site, such as setting your privacy preferences, signing in, or filling in forms.
Performance
Performance cookies provide anonymous statistics about how customers navigate our site so we can improve site experience and performance. Approved third parties may perform analytics on our behalf, but they cannot use the data for their own purposes.
Allowed
Functional
Functional cookies help us provide useful site features, remember your preferences, and display relevant content. Approved third parties may set these cookies to provide certain site features. If you do not allow these cookies, then some or all of these services may not function properly.
Allowed
Advertising
Advertising cookies may be set through our site by us or our advertising partners and help us deliver relevant marketing content. If you do not allow these cookies, you will experience less relevant advertising.
Allowed
Blocking some types of cookies may impact your experience of our sites. You may review and change your choices at any time by selecting Cookie preferences in the footer of this site. We and selected third-parties use cookies or similar technologies as specified in the AWS Cookie Notice.
Your privacy choices
We display ads relevant to your interests on AWS sites and on other properties, including cross-context behavioral advertising. Cross-context behavioral advertising uses data from one site or app to advertise to you on a different company’s site or app.
To not allow AWS cross-context behavioral advertising based on cookies or similar technologies, select “Don't allow” and “Save privacy choices” below, or visit an AWS site with a legally-recognized decline signal enabled, such as the Global Privacy Control. If you delete your cookies or visit this site from a different browser or device, you will need to make your selection again. For more information about cookies and how we use them, please read our AWS Cookie Notice.
Este conteúdo não está disponível no idioma selecionado. Estamos trabalhando constantemente para disponibilizar nosso conteúdo no idioma selecionado. Agradecemos pela paciência.
CreditorWatch Democratizes Credit Data for 55,000+ Businesses with Lean AWS Infrastructure
2021
Affordable Credit Checks for Small Businesses
Small businesses continue to dominate the Australian economy, with 9 out of 10 companies down under employing four or less people. For small and medium-sized businesses (SMBs), accessing credit risk information on potential trading partners used to be challenging because credit reporting bureaus charged fees that were too high for the average SMB.
CreditorWatch was founded in 2010 by a small-business owner who wanted to create open-source, affordable ways for SMBs to access and share credit risk information. Today, CreditorWatch’s subscription-based online platform enables its 55,000-plus customers—from sole traders to listed enterprises—to perform credit checks and determine risk to their businesses. It also offers additional integrated products and services that help customers make responsible, informed credit decisions.
AWS provides the tooling and code-based infrastructure that allows us to further monetize our product.”
Joseph Vartuli Chief Technology Officer, CreditorWatch
Migrating to an Event-Driven, Serverless Architecture
In 2013, CreditorWatch realized it had outgrown its single on-premises server. It needed a stable, scalable solution to support its rapidly growing client base and the increasing volume of data it was processing. The company decided to migrate its online platform to Amazon Web Services (AWS) Cloud for scalability and stability.
The next phase of expansion came in 2017, when major corporations came knocking. Enterprises, including Australia’s Big Four banks, were looking to leverage CreditorWatch’s rich dataset and granular analytics capabilities. To drive efficiency and boost agility, the company then pivoted to an event-driven serverless architecture, splitting its single-tier platform into 22 microservices.
The ease and affordability of setting up a multi-tenant architecture on AWS allows CreditorWatch to continue scaling the business without limits. “AWS enables a small player like us to tender with big organizations without driving up overhead due to the shared-service nature of the platform,” says Joseph Vartuli, chief technology officer at CreditorWatch.
Processes 400 Billion Data Points in 8 Hours
CreditorWatch uses
Amazon Kinesis to collect real-time streaming data and more than 250
AWS Lambda functions to execute the entire event-driven process without manual intervention.
AWS Database Migration Service is deployed to extract data from 26 sources. Data is then populated in a compressed format to an
Amazon Simple Storage Service (Amazon S3) data lake. The data processing magic happens in the
Amazon EMR big data platform, and reports are generated to assign businesses a risk score.
The scores take events into account as they happen, with real-time updates including factors that could impact a partner’s ability to repay debt. The volume of data that CreditorWatch processes—some 400 billion data points—means their analytics engines can evaluate businesses across multiple geographic and demographic spaces. It can then output an informed assessment that shows how a business fares compared to peers in its industry.
Data dumps for all 400 billion data points are completed in just 8 hours once a month. Smaller, live updates are streamed and automatically performed as changes occur to the data following that, until the next major data migration 30 days later. CreditorWatch spins up huge Amazon EMR clusters for this monthly data transformation, then shuts down the clusters after the scheduled job. It only pays for the resources used, in line with AWS’s pay-as-you-go pricing model. In an on-premises setup, costs would be 30 times higher, as high-capacity servers would need to run for the entire month.
In 2019, CreditorWatch began running machine learning (ML) algorithms in Amazon EMR to dig deeper and find additional factors that could lead to insolvency in the next 12 months. The ML models were then backtested against actual customer data over the past 5 years to improve accuracy and generate payment predictions.
The business employs just four people on its data analytics team to serve its more than 55,000 customers. Vartuli says, “If we didn’t have tools like Amazon S3 or Amazon EMR clusters at our fingertips, there’s no way we could carry out these ML interactions with such a lean team. AWS provides the tooling and code-based infrastructure that allows us to further monetize our product.”
Proactive Support from AWS Experts and Community
Assistance from
AWS Business Support specialists has been critical in helping CreditorWatch build and optimize its data pipeline. Developers consult frequently with AWS solutions architects throughout the product lifecycle, proactively discussing concepts before implementation and troubleshooting when issues arise. The AWS team participated in determining the right amount of RAM, processing power, and storage required to run CreditorWatch’s analytics engine at scale, yet control costs during off-peak processing times.
CreditorWatch has also connected with other AWS customers that have undergone similar data-driven paths. This allows the business to understand common pitfalls and forge the best path from the start. Several developers have also completed AWS certifications to enhance their understanding of efficient cloud architecting.
Vartuli comments, “It would have taken us at least a year and a half to get our ML processes right if we had to tweak settings on a hard drive in a server room. But with AWS Business Support and Amazon EMR, completion took a matter of weeks.”
Obtaining ISO Certifications and Creating Custom Data Sets
CreditorWatch leads with security by design and is using
Amazon GuardDuty for intelligent threat protection along with
AWS Web Application Firewall (AWS WAF). To instill further confidence in its solution among the rising number of enterprise customers it’s now attracting, the business is seeking ISO 27001 accreditation. Using AWS solutions that are already ISO-certified has streamlined this process.
As a next step, the company has begun working more closely with enterprises and the Big Four banks to provide custom risk scores. CreditorWatch sends its data scientists into an enterprise customer’s office to collaborate with the customer’s internal teams and mutually uncover new value-added applications for their combined data sets.
One recent ongoing opportunity—with the potential for further work—is a project initiated by the Australian Treasury to provide insights into industry payment times. “For a bureau of our size, the cost to integrate a high volume of data and spin up machines would be in the hundreds, if not millions, of dollars on premises,” Vartuli says. “We’re able to spin up and perform these actions affordably in almost no time at all on AWS.”
CreditorWatch has been helping businesses assess their partners’ creditworthiness since 2011. It currently enables its 55,000-plus customers to perform a credit check on any Australian business.
Benefits of AWS
Processes 400 billion data points in 8 hours
Designs ML framework in a few weeks instead of 1.5 years
Controls overhead with multi-tenant, event-driven architecture
Serves 55,000+ customers with a data analytics team of 4
Secures infrastructure against threats while preparing for ISO certification
Receives proactive support from solutions architects
AWS Services Used
Amazon Kinesis
Amazon Kinesis makes it easy to collect, process, and analyze real-time, streaming data so you can get timely insights and react quickly to new information.
Amazon EMR is the industry-leading cloud big data platform for processing vast amounts of data using open source tools such as Apache Spark, Apache Hive, Apache HBase, Apache Flink, Apache Hudi, and Presto.
Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance.
Organizations of all sizes across all industries are transforming their businesses and delivering on their missions every day using AWS. Contact our experts and start your own AWS journey today.
/Creditorwatch_logo%402x.6e5d0da490c143b4b882a503f3887e4367d4767f.png)
