Attribution Is Broken. Here Is What We Actually Trust.

The Story iOS 14.5 Is Still Telling

It has been five years. iOS 14.5 shipped in April 2021. The DTC world has not stopped talking about it. There is a reason.

When Apple made the App Tracking Transparency prompt opt-in, reported opt-in rates settled around 25% globally and stayed there. That single change removed roughly three quarters of the browser-based conversion signal Meta had spent a decade optimising against. Meta compensated with modelling. Meta modelling is Meta’s best guess at what would have happened if the signal had been intact. It is not causal. It is a story Meta tells itself so it can keep optimising bids.

Every founder we onboard is still spending against those stories. Meta reports a 3.2 ROAS. Shopify reports revenue that would only justify a 1.6 ROAS at that spend level. The founder cannot resolve the two numbers. They keep spending because Meta says it is working. It is not working. It is being modelled.

This piece is about what to trust, what to ignore, and how to build a defensible operating view in a spreadsheet. It sits underneath every forensic CRO engagement we run.

What Meta Reports vs What Shopify Sees

Open two tabs. Meta Ads Manager, purchases column, last 28 days. Shopify Analytics, total sales, same date range.

The gap will be somewhere between 20% and 60%. On average, in the client base we see, Meta over-reports by 35 to 45%. Some of that is legitimate cross-device attribution Meta can defend. Most of it is modelled conversions that never happened at Meta’s claimed cost.

You cannot fix this by turning off view-through attribution. You can only work around it. Every operator we respect has moved to the same posture. Read Meta ROAS as a directional signal inside Meta for optimising creative and audience. Never as an allocation signal. Never as a truth number.

The truth number lives in Shopify. It is what the register saw.

Blended ROAS: The Calculation, The Caveats, The Limits

Blended ROAS is total Shopify revenue divided by total ad spend across every paid channel for a defined period. It is cruder than Meta ROAS. It is also more honest.

The calculation, weekly:

• Column A: week
• Column B: Meta spend
• Column C: Google spend
• Column D: TikTok spend
• Column E: any other paid channel spend
• Column F: sum of B through E
• Column G: total Shopify revenue for the week
• Column H: G divided by F

That number, in column H, is the honest ROAS. It does not care which pixel fired. It knows what money went out and what money came in.

The caveats matter and we will not skip them.

Blended ROAS lags brand-building spend. If you launched a large TikTok campaign in month one, some of the revenue it drove will appear in month two or three via direct traffic. Blended ROAS in month one will look worse than causal reality. Blended ROAS in month three will look better. Read it as a rolling six to eight week average, not a single week.

Blended ROAS credits organic and email revenue to paid. If your paid spend is a small share of total revenue, blended ROAS can look flatteringly high. That is not paid working. That is organic and email revenue in the numerator. Segment blended ROAS by paid-attributable revenue if you want a cleaner read. For most founders, the unsegmented number is fine as long as they understand the mix.

Blended ROAS cannot tell you which channel worked. It gives you a total portfolio number. It cannot tell you that Meta is doing the work and Google is coasting on brand terms. For channel allocation, you need incrementality tests.

These caveats do not undermine the number. They give you the honest limits to work within. Meta ROAS does not tell you the honest limits. It tells you what Meta wants you to see.

The Two-Week Incrementality Test Any Founder Can Run

Attribution tools promise to solve this with models. Multi-touch attribution vendors sell six-figure MTA implementations that claim to trace every touchpoint. Our position across dozens of audits: MTA models are more defensible than platform ROAS and less defensible than a two-week incrementality holdout.

The test is simple. Pick a channel you want to know the truth about. Turn it off completely for two weeks. Compare total Shopify revenue to the two weeks before the holdout. Compare it to the same two-week period from the prior quarter. Compare it to a matched control period.

If revenue drops proportionally to the channel’s reported share, the channel is doing what it claims. If revenue drops less than the channel claimed to contribute, the channel was taking credit for orders that would have happened anyway. If revenue does not drop at all, the channel was pure attribution theft.

We have run this on eight client accounts across our case studies. Meta held up in five. Google held up in six. TikTok held up in two out of three we tested. The specific numbers do not generalise. The methodology does.

The uncomfortable finding is that the channel most founders trust least, direct traffic, is often the most causally load-bearing revenue driver. It just does not have a sales pitch attached to it.

For a longer treatment of the paid-versus-organic breakdown, see The Three Numbers Every Shopify Founder Should Watch Weekly.

What We Ignore

Certain numbers we simply refuse to optimise against, no matter how prominently they are displayed.

Platform ROAS as an allocation signal. Directionally useful inside the platform. Useless for deciding how much to spend across channels.

First-touch attribution. It over-credits top-of-funnel channels for orders that were closed by remarketing and email.

Last-touch attribution. It over-credits bottom-of-funnel channels for orders that were driven by top-of-funnel awareness.

Attribution windows longer than 7-day click, 1-day view. Anything longer than that starts credit-laundering brand awareness into direct-response reporting. Read the shorter window even if it makes the ROAS look worse.

Any MTA model that claims to solve the problem. MTA is a better guess than platform ROAS. It is not truth. It is a modelled reconstruction of touchpoints that iOS 14.5 removed the ability to observe. Better than nothing. Not truth.

When (If Ever) To Buy An MTA Tool

We get asked this constantly. Should I buy Triple Whale. Should I buy Northbeam. Should I buy Polar.

The answer for most founders in the sub-$1M band: not yet. The tools are excellent. They also cost between $200 and $2,000 a month and add complexity to a stack that already has too many dashboards. See Why Fifteen Dashboards Is Not a System for the argument on tool minimalism.

For sub-$1M ARR, a Google Sheet with weekly blended ROAS plus a quarterly incrementality test on your biggest channel will give you 80% of the decision-quality of an MTA tool for zero dollars.

At $1M to $3M ARR, MTA tools start to earn their keep. Not because the model got more accurate. Because the operational overhead of maintaining the spreadsheet exceeds the tool cost.

At $3M+, an MTA tool is table stakes. But even then, the founders we respect the most cross-check MTA reports against blended ROAS and periodic incrementality tests. They do not delegate the truth to the model.

The Defensible Operating View

Here is the attribution stack we run on every Prism3 client, regardless of tool budget.

Weekly: Blended ROAS in a Google Sheet. Six to eight week rolling average. Trend line, not point values.

Monthly: Shopify Analytics segmented by traffic source. Read the direct traffic line separately. Compare month over month.

Quarterly: One incrementality test on the biggest spend channel. Two weeks off, controlled comparison. Note the finding. Adjust the next quarter’s allocation.

Annually: Full attribution audit. Compare blended ROAS trend to platform ROAS trends. Note the gap. Note whether the gap widened or narrowed. Use it to calibrate how much to trust each platform’s reporting for the coming year.

That is it. No MTA vendor. No dashboard subscription. No 40-slide attribution deck. Four cadences. Four numbers.

The founders who run this operating view make better allocation decisions than the founders who bought the $2,000-a-month tool and let it decide for them. Not because they have more data. Because they know exactly which parts of their data are defensible and which are stories the platforms are telling.

Attribution is broken. That is not the crisis. The crisis is optimising against numbers that cannot be defended. Build the operating view above. Trust what you can defend. Ignore the rest. That is enough. If you want us to sanity-check yours, book a 30 min diagnostic.

If you’re an AU or US Shopify founder stuck at the CVR plateau this post describes, we run a free 30 minute diagnostic call. No pitch. No email gate. Just a look at what’s actually breaking in your funnel. Book here.