How we run creative testing pilots at Creatively Squared

Most marketers know they need better ads—but the question is, what does "better" actually mean?

We’ve found the answer usually isn’t one big idea. It’s a mix of ideas. Different styles, formats, tones, and creators that help you figure out what truly resonates.

That’s what our creative testing pilots are built to explore.

We’re not testing for the sake of it. We’re showing how creative variety can boost performance, reduce costs, and help your existing strategy work harder. In this guide, we’ll walk you through how we do it—from how we set up a test, to how we measure success.

Why creative variety has become a performance driver

Digital platforms haven’t just changed where ads show up—they’ve changed how people experience them. Scrollable, skippable, and deeply personal, social feeds have put the power in the hands of the audience.

And that shift has made creative diversity not just a nice-to-have—but a core performance lever. Here’s why:

• Your content is part of the auction - Platforms like Meta and TikTok reward content that holds attention. When people watch and engage, your CPM drops. When they skip, it goes up.

• One story doesn’t work for everyone - Classic storytelling frameworks don't always hold up when people decide in 1–2 seconds whether to stick around. You need different styles and angles to figure out what hooks your audience and keeps them watching.

• Freshness matters - People see hundreds of ads a day. The same old formats burn out fast. Diversity helps you stay relevant and fight fatigue.

• Content needs to fit in, not stand out - The most effective assets don’t feel like ads. They look and sound like the platform they’re on—without losing your brand identity.

• Customer-first isn't optional anymore - Brands that only focus on pushing their message often see higher media costs. The platforms favour ads that people actually want to watch. Creative that puts the customer first pays off—literally.

What we usually see when we add creative variety

When we test diverse creative alongside a brand’s existing assets, we see patterns emerge. Here’s what tends to happen:

• Media costs go down - Platform-native content consistently delivers 15–30% lower CPMs.

• Engagement goes up - Completion rates double, watch time jumps by 40–60%, and we see stronger engagement signals like saves and shares.

• Brand results improve - In brand lift studies, we see increases in favourability and purchase intent—especially when the content feels more native and relatable.

Our testing framework: how the pilot works

Our pilot framework is designed to prove the value of creative diversity while keeping all other factors constant.

What we’re testing

We’re not here to replace your brand assets—we’re here to build on them. The goal of the pilot is to show how your current strategy can perform better with a layer of varied, socially native creative added in.

What we measure

We look at performance across three dimensions:

• Efficiency - CPM, cost per 3s/10s view
• Effectiveness - 3s view-through rate, completion rate, engagement rate, click-through rate
• Brand impact - Brand favourability, purchase consideration (measured via brand lift study)

How we set up the campaign

We run the test in two phases to compare apples to apples—first isolating creative impact, then optimising based on what’s working.

Phase 1: Testing in isolation

1. We split the campaign into two groups:
   • Group A: Your current brand assets
   • Group B: Creatively Squared assets (a mix of brand-style, lifestyle, and UGC-style content)
2. Each creative runs in its own ad set with identical targeting, placement, and daily budget.
3. This lets us identify which assets genuinely drive performance—without interference from the algorithm.

Phase 2: Optimisation

1. We consolidate the creatives into two ad sets—brand vs. Creatively Squared.
2. We use the same post IDs from Phase 1 so that engagement (likes, comments, shares) carries over.
3. Optimisation is enabled within each group so the platform can do its job—but we keep the budgets even to ensure a fair test.

We also introduce new creative during this phase—based on performance insights from Phase 1. This shows how ongoing optimisation can unlock even more value.

Supporting setup: what we need to make it work

Here’s what’s involved in bringing the pilot to life:

• Assets - We’ll need a variety of creative formats from both your existing library and new production.
• Brief - You’ll share your brand guidelines and strategic priorities so everything aligns.
• Budget - We recommend a test budget that’s meaningful to your team. It needs to be enough to get statistically valid results and to justify future roll-out.
• Timeline - Most pilots run over 10–12 weeks, including two rounds of optimisation and a brand lift study at the end.

Reporting and review cadence

We keep everyone in the loop at key points:

• Weekly updates on key metrics
• Bi-weekly reports with creative insights
• Mid-campaign review to align on learnings and next steps
• Final report including brand lift results and a recommendation for scale

Who does what

• You (the brand team) - Share brand guidelines, give creative approvals, help align on direction.

• Your media agency - Sets up the campaign in Meta or TikTok, maintains the testing structure, and shares results.

• Creatively Squared - Produces new creative, shares performance insights, manages the testing process, and runs the brand lift study using an independent research partner.

What success looks like

If the pilot goes well, you’ll see:

• Lower CPMs and cost per view
• Higher attention and engagement metrics
• Stronger brand results in your lift study
• A clear case for combining creative variety with brand consistency

What comes next

After the pilot, you can:

• Expand the approach to other markets or brands
• Build an always-on creative diversity plan
• Set up a continuous testing and learning loop

And Creatively Squared can power the engine behind it—producing creator-led, on-brand content at scale while supporting your team with strategic insight and performance data.

‍