ROI: Should you license or build your own API connections for BigQuery?

Updated: Sep 22

Welcome to our data warehouse ROI series where we discuss real world use cases and real world costs so you can get a realistic view of the data warehouse market for agencies.

The problem

A prospective client recently came to us with this common problem: as an agency with 100+ employees, they wanted to build a central hub of PPC data with a BigQuery data warehouse.

Their previous reporting set-up kept breaking because they used to pull data straight into Data Studio and Google Sheets, which can be extremely slow due to the data processing limitations of Data Studio and Sheets. Their set-up simply hadn't kept up with their agency's growth.

Google's BigQuery is an enterprise level data warehouse that is designed to store and process huge amounts of data. So pulling data into BigQuery for processing before pushing it out to data visualization tools like Data Studio certainly solves the performance issues.

They considered using’s BigQuery connector, but Funnel licenses are expensive at USD 3,500 per month. They wanted to know how much it would cost to set up a BigQuery data warehouse the agency owns with bespoke API connections that are also fully owned by the agency so they never have to pay for license fees for a data pipeline again.

Their scope is fairly representative for many agencies so let’s use this use case as an example to give you an idea of what costs you can expect.

The scope

To cover all the agency’s reporting needs they were looking at connecting 10 data sources, with Google and Facebook being the most important ones. Add to that Microsoft, Twitter, CallRail, Snapchat, TikTok, Pinterest, and Amazon Ads. Plus they wanted to do more than report on data and get macro insights about their agency’s operational efficiencies, e.g. by comparing project management data from Asana side by side with account performance data.

The investment you’re looking at

Scenario 1 - license for BigQuery

Funnel was asking for USD 3,499 per month in licensing fees (their "large agency" license according to their website) to pull that data into BigQuery, billed annually for a total of USD 42,000 per year. Assuming the agency’s needs will not change significantly over the next 3 years you are looking at an investment of USD 126,000 over 3 years.

A decent chunk of change considering only licensing costs, data engineering not included. Enough to consider your alternatives carefully.

What’s your best alternative?

All data pipelines like essentially manage API connections for you so you don’t have to set them up and maintain them yourself. Considering other vendors wouldn’t actually change much. In terms of the data warehousing technology BigQuery is the best choice for agencies and is a decent data pipeline and has comparable cost to other products in the market (e.g. Adverity).

Scenario 2 - setting up your own API connections

At 126k over 3 years your best alternative is to actually build this out yourself by setting up your own API connections!

How much would that cost? Here’s a scope for our data engineering services:

API connection Days

Google Ads 3

Facebook Ads 3

Google Analytics 0.5

Microsoft Ads 3

Asana 2

CallRail 0.5

Shopify 2

Snapchat 1

TikTok 3

Pinterest 2

Amazon Advertising 2

Salesforce 0.5

Data Studio templates 1

Total 23.5

At a GBP 800 day rate for data engineering and development services you are looking at a total invest of GBP 18,800 for 23.5 days and at today’s exchange rate that is roughly USD 25,000.

You are saving USD 100k in pure licensing fees over 3 years!

You own everything; when we work with agencies we set up everything in your domain name with an email you give us so you own the data warehouse and you own the API connections: it’s your agency’s proprietary tech. You can even give it a cool name and advertise that you have these advanced capabilities on your website and in pitches.

You probably won't use the 500+ connectors that Funnel advertises, so why pay for them?

More about maintenance

Most agencies use out of the box data pipelines instead of building out their own API connections to pull data because they are worried that they don’t have the required resources in house to update and maintain them, they simply don’t have data engineers with this kind of expertise.

So let’s address that worry.

Scenario 1 - out of the box data pipeline and other platforms include support, but don’t think you won’t face costs of managing your data. The majority of agencies already have a person or two who primarily take care of reporting and analytics and someone in your agency will have to become a superstar so to speak. We have not even considered the cost of that in our calculation above.

Scenario 2 - custom API connections

The truth is that you won’t face much maintenance to update API connections at all. Due to the many product integrations that APIs cover the vendors have no incentive to make frequent changes and they generally keep their API updates to a minimum and breaking changes are very rare. And in any case you would be informed well in advance.

But for the sake of argument let’s be on the safe side of a budget calculation and overestimate maintenance costs massively and say it’s 10% of your set-up costs per year. In our above example that would add GBP 1,800 or USD 2,500 per year - max! In any case, we have included that number in the graph above.

It’s a huge overestimate, because realistically, as a service provider with a managed data warehouse service we are updating this for multiple clients at any given time and we spread that cost. But for the sake of argument we have included it in the graph above.

Key skills required

Here’s the key problem Acuto solves for agencies: nine out of ten agencies don’t have data engineers and developers who have the required experience and can set up API connections for you.

Our data engineers have PPC backgrounds from major agencies like Merkle, ForwardPMX and Brainlabs and we set-up and manage data warehouses and API connections for agencies on a daily basis.

The ROI calculation above shows that you can save you a lot of money, but the real key is in what else we can enable you to do once the data is all in one place.

After all, the whole point of a data warehouse is to open up options for your agency in the long run to go beyond standard reporting and to really gain insights that can make a difference in the performance of your clients’ accounts and your agency’s operational efficiency.

Find out more about what you can do with a data warehouse in this eBook on data warehousing for digital marketing agencies.

64 views0 comments