Marketing Campaign Evaluation

A company runs an A/B test. Version B wins—12% lift, statistically significant. Champagne. 🎉 Six months later, revenue is flat. What happened? They averaged over their customers. Rookie move. (I've done it too.) Version B: +20% for new users. But -8% for returning customers. New users outnumbered returners in the test, so B "won." Then the customer mix shifted. More returners. The "winning" variant was slowly bleeding its best users. This is Simpson's Paradox—when aggregate trends reverse at the subgroup level. It's not exotic. It's everywhere. Data-driven teams walk into this constantly when the first rule of being data-driven is "run the test and trust the average." The fix isn't more data. It's asking: for whom did each version win? Averages describe populations. They don't describe people. The most dangerous phrase in analytics isn't "we don't have data." It's "the data is clear." For those wrestling with weird A/B results—I see you! Ask who's in your sample before you pop the champagne.

FIFA received 500 million ticket requests for the 2026 World Cup. Thousands of seats are currently unsold. Those two facts are not contradictory - they are the inevitable outcome of dynamic pricing that mistook demand signals for willingness to pay. The gap between expressing interest in a ticket and paying $33,000 for one is not a demand gap. It is a pricing gap. FIFA confused the volume of interest at any price with the volume of buyers at its chosen price points. The India and China media rights crisis is the same dynamic at a different scale. JioStar offered $20M against a $100M ask. CCTV offered the equivalent of $80M against a $250M demand. In both cases, FIFA priced against a demand signal that did not account for the commercial reality of the specific market. Aggressive dynamic pricing works when the product is genuinely scarce relative to the price-inelastic portion of demand. For a World Cup with 48 teams, 104 matches, and 3 host countries, the scarcity thesis was weaker than FIFA's pricing model assumed. The ticketing situation and the broadcast rights crisis are the same story. FIFA is the last major sports property that still believes its own demand narrative more than the market does. #sportsbusiness #sportsinvesting #sportscapital #sportsinfrastructure

Most SaaS companies focus on acquiring new customers. But the real revenue driver? Retention. - Acquiring a new customer costs 5X more than retaining an existing one. - Selling to a current customer has a 60-70% success rate. - Selling to a new prospect? Just 5-20%. The math is clear: If you’re not optimizing for renewals and reducing churn, you’re burning money. Every SaaS company eventually hits a wall where churn compounds. More customers leave → Higher acquisition needed → Growth stalls. David Skok puts it best: Churn is a leaky bucket. The bigger you grow, the more customers you need just to stay in place. So, how do you plug the leak? 1️⃣ Customer Success > Customer Support Proactive beats reactive. Retention starts before problems arise. - Onboarding, engagement, and expansion must be intentional. - Companies with dedicated CS teams cut churn by 41% in a year. 2️⃣ Measure the Right Metrics Not all churn is created equal. Track: - Customer churn rate → % of customers lost. - Revenue churn rate → % of revenue lost (more meaningful If the contract value varies a LOT!). - Net Revenue Retention (NRR) → Renewal + expansion revenue (gold standard for SaaS growth). 3️⃣ Design for Retention from Day 1 First-session success = Long-term retention. - Groove found that users who spent less than 2 minutes on their first login had a 60% churn rate. - The fix? Optimize Time-to-Value (TTV) to drive early engagement. 4️⃣ Incentivize Longer Commitments Monthly plans = Higher churn. - Annual/multi-year contracts lock in retention. - Offer compelling reasons to commit (discounts, premium features, exclusive support). 5️⃣ Use Behavioral Triggers - Buffer’s re-engagement emails for inactive users led to a 22% churn reduction. - HubSpot's CHI (Customer Happiness Index) predicted at-risk accounts before churn. What This Means for You Retention isn’t a support function—it’s a growth strategy. The best SaaS companies: - Build cross-functional retention teams. - Align Product, Sales, and CS to drive ongoing value. - Treat renewals as a natural progression, not a last-minute pitch. If you solve churn and renewals, you unlock sustainable, profitable growth. How is your SaaS business tackling retention? Let’s discuss. __ ♻️ Reshare this post if it can help others! __ ▶️ Want to see more content like this? You should join 2238+ members in the Tidbits WhatsApp Community! 💥 [link in the comments section]

Machine learning for dynamic pricing optimization offers businesses a competitive edge by enabling them to adjust prices in real-time, ensuring they remain responsive to market demands, customer behavior, and competition, ultimately maximizing revenue and profitability. Machine learning, a subset of AI, allows systems to learn from data and improve without explicit programming, identifying patterns and making predictions from historical data. In pricing optimization, it helps set prices strategically by considering demand, competition, costs, and customer perception. Fundamental data types used include sales history, market trends, competitor pricing, customer behavior, demographics, seasonality, and search trends. Standard algorithms, such as regression, decision trees, neural networks, clustering, and reinforcement learning, are applied to predict demand shifts. Dynamic pricing then adjusts prices in real-time, boosting revenue and competitiveness. For business implementation, ML models can be integrated with existing systems like sales, ERP, and CRM, allowing for real-time price adjustments. Challenges include maintaining high data quality, investing in technology and skills, and addressing ethical and regulatory concerns regarding dynamic pricing, customer perception, and compliance. #ai #MachineLearning #Pricing #CRO #COO

👀 Lessons from the Most Surprising A/B Test Wins of 2024 📈 Reflecting on 2024, here are three surprising A/B test case studies that show how experimentation can challenge conventional wisdom and drive conversions: 1️⃣ Social proof gone wrong: an eCommerce story 🔬 The test: An eCommerce retailer added a prominent "1,200+ Customers Love This Product!" banner to their product pages, thinking that highlighting the popularity of items would drive more purchases. ✅ The result: The variant with social proof banner underperformed by 7.5%! 💡 Why It Didn't Work: While social proof is often a conversion booster, the wording may have created skepticism or users may have seen the banner as hype rather than valuable information. 🧠 Takeaway: By removing the banner, the page felt more authentic and less salesy. ⚡ Test idea: Test removing social proof; overuse can backfire making users question the credibility of your claims. 2️⃣ "Ugly" design outperforms sleek 🔬 The test: An enterprise IT firm tested a sleek, modern landing page against a more "boring," text-heavy alternative. ✅ The Result: The boring design won by 9.8% because it was more user friendly. 💡 Why It Worked: The plain design aligned better with users needs and expectations. 🧠 Takeaway: Think function over flair. This test serves as a reminder that a "beautiful" design doesn’t always win—it’s about matching the design to your audience's needs. ⚡ Test idea: Test functional designs of your pages to see if clarity and focus drive better results. 3️⃣ Microcopy magic: a SaaS example 🔬 The test: A SaaS platform tested two versions of their primary call-to-action (CTA) button on their main product page. "Get Started" vs. "Watch a Demo". ✅ The result: "Watch a Demo" achieved a 74.73% lift in CTR. 💡 Why It Worked: The more concrete, instructive CTA clarified the action and benefit of taking action. 🧠 Takeaway: Align wording with user needs to clarify the process and make taking action feel less intimidating. ⚡ Test idea: Test your copy. Small changes can make a big difference by reducing friction or perceived risk. 🔑 Key takeaways ✅ Challenge assumptions: Just because a design is flashy doesn’t mean it will work for your audience. Always test alternatives, even if they seem boring. ✅ Understand your audience: Dig deeper into your users' needs, fears, and motivations. Insights about their behavior can guide more targeted tests. ✅ Optimize incrementally: Sometimes, small changes, like tweaking a CTA, can yield significant gains. Focus on areas with the least friction for quick wins. ✅ Choose data over ego: These tests show, the "prettiest" design or "best practice" isn't always the winner. Trust the data to guide your decision-making. 🤗 By embracing these lessons, 2025 could be your most successful #experimentation year yet. ❓ What surprising test wins have you experienced? Share your story and inspire others in the comments below ⬇️ #optimization #abtesting

So, how much did being genuinely nice to our customers earn us this quarter? Now imagine asking this question to your CFO. Today we are well aware and sometimes even obsessed with metrics: NPS, CSAT, churn rates…all perfectly calculated. But translating the warmth of customer happiness into cold, hard financial results? Well, that's not so simple. After all, it is not easy to connect a ‘smiling support rep’ to ‘higher EBIT’. However, the truth bomb here - Top CX performers consistently outperform their competitors. But the magic they create is not just in making customers smile. It is about connecting every delighted customer with revenue, retention, and even willingness to pay a little extra. The question for us to answer is - Are we connecting dots, or just coloring the margins? As business leaders, are we digging deep enough? What would happen if CX was tagged to every financial review, not just a customary part of the annual presentation? You could be walking into your next review, armed with not just satisfaction scores, but a clear graph of what those scores added to the bottom line. If you think ROI from customer experience is not just fairy dust, then here are 4 metrics to add gravitas to your next board meeting: ☘️ C - Customer Retention Track repeat purchase rate/ renewal rate. Know how many customers come back. Even a 5% increase in retention can boost profits considerably. ☘️ T - Ticket Size Happier customers spend more. We all do that. Measure if your CX improvements lead to higher average order value. ☘️ S - Share of Voice Delighted customers talk. Track organic referrals, online reviews and social media mentions. Don't forget - word of mouth reduces marketing costs. ☘️ S - Service Cost Zero-effort experiences reduce complaints and rework. When customers don't need to call back, your cost to serve drops. Measure cost per support ticket and first contact resolution rate. These may not happen in a day, but start somewhere. One step of transition a day leads to transformation over a quarter or a year. Let’s get past the vanity metrics and start making CX pay its own bills. About time no? #cx #customerexperience #serviceexcellence

Customer Success protects $67M with three people. Sales burns $4.2M chasing $8.3M with twenty-eight reps. CEO hired 15 more salespeople while customers canceled $12M. The Head of Customer Success could only watch. At this $80M ARR SaaS company,  her team of three managed $67M in existing revenue. Meanwhile, 28 sales reps chased new deals. The math was backwards. Customer Success cost $340K annually, generated $67M in renewals. Sales cost $4.2M, brought in $8.3M new ARR. Revenue per dollar invested: Customer Success: $197 for every $1 spent Sales: $2 for every $1 spent Board meetings always centered on pipeline. Customer Success was never mentioned until renewal season. The Head of Customer Success ran retention analysis: Current rate: 78% Churn reasons were documented and totally fixable: - Poor onboarding (32%) - Unmet sales expectations (28%) - Inadequate support (23%) - No value realization (17%) Her proposal: Instead of hiring 10 salespeople ($1.4M),  invest that amount in Customer Success. Projected impact: Retention increases from 78% to 88%. On $67M base, that's $6.7M additional recurring revenue. ROI: 479% vs maybe $3M from new salespeople. The CEO's response: "We need growth, not just maintaining what we have." Six months later, they missed annual targets  despite exceeding new customer acquisition. The Head of Customer Success left for a competitor. Her new company: 12 CS managers for $45M ARR. Retention rate: 94%. Growth rate: sustainable. Companies that prioritize retention foundation dominate markets. Companies that don't churn themselves out of business. You can't out-sell a retention problem.

founder learnings! part 8. A/B test math interpretation - I love stuff like this: Two members of our team (Fletcher Ehlers and Marie-Louise Brunet) - ran a test recently that decreased click-through rate (CTR) by over 10% - they added a warning telling users they’d need to log in if they clicked. However - instead of hurting conversions like you’d think, it actually increased them. As in - Fewer users clicked through, but overall, more users ended up finishing the flow. Why? Selection bias & signal vs. noise. By adding friction, we filtered out low-intent users—those who would have clicked but bounced at the next step. The ones who still clicked knew what they were getting into, making them far more likely to convert. Fewer clicks, but higher quality clicks. Here's a visual representation of the A/B test results. You can see how the click-through rate (CTR) dropped after adding friction (fewer clicks), but the total number of conversions increased. This highlights the power of understanding selection bias—removing low-intent users improved the quality of clicks, leading to better overall results.

Does putting pricing on your landing page help or kill your funnel? We ran a 60-day A/B test for a client. Real traffic. Real leads. Real consequences. Here’s how it went down: THE HYPOTHESIS: If buyers know the price upfront, they’ll self-qualify. Fewer tire-kickers = less time wasted by sales. That all sounds good but it also means less leads and more "Why are the leads down?!" emails. TEST SETUP: 🔹 Variant A (Control) • Standard demo request page • No pricing anywhere, classic “Talk to sales” vibes 🔹 Variant B (Test) • Identical page but added a full pricing table alongside the form • Clear pricing tiers, no surprises THE RESULTS: • Leads dropped 8.7% on the page • BUT lead to opportunity rates jumped 21.4% • Average deal size held steady • Net pipeline value? Statistically flat—within a ±3% margin Translation: You get fewer leads, but way better ones. The pricing filter scared off the browsers… but it attracted buyers who were ready to talk budget and timeline, not just features. Don’t hide the price. Lead with it. You’ll filter in serious buyers, and your sales team will thank you.

Static paywalls are leaving money on the table; intelligent pricing is how publishers reclaim it. Fixed paywalls block access and revenue potential. Relying on static pricing risks falling behind competitors like Schibsted, which saw a 19% increase in average revenue per user (ARPU) after adopting dynamic pricing (INMA, “Dynamic Paywalls Gain Momentum”, 2023). Traditional paywalls offer the same deal to every user, but not all readers are the same. Behaviour, loyalty, and content value vary, and a one-size-fits-all approach ignores these critical factors. This rigidity limits revenue yield and risks losing high-value audiences to more agile publishers. How AI-Driven Paywalls Maximise Revenue Yield Dynamic pricing, powered by AI, allows publishers to adjust subscription offers based on real-time user behaviour and perceived content value. Here’s how: ✅ Behavioural Targeting:  The Dallas Morning News increased conversions by 28% by offering discounts to frequent readers and trials to casual visitors (INMA, 2023). ✅ Content Valuation: The Financial Times uses dynamic pricing to align fees with content value, a strategy that contributed to a 14% YoY digital subscription growth (FT Group Annual Report, 2023). ✅Predictive Adjustments: Amedia reduced bounce rates by 18% using AI-driven exit-intent discounts (Reuters Institute, “Journalism, Media, and Technology Trends”, 2023). Instead of setting prices in stone, publishers use intelligent signals to flexibly match user willingness to pay, unlocking hidden revenue pockets. Three Practical Steps to Smarter Paywall Monetisation ✓ Audit Current Paywall Performance: Identify weak points like high drop-off rates or low conversion on high-value articles. ✓ Implement AI Segmentation: Use machine learning models to predict engagement and optimise when and how offers are shown. ✓ Define Dynamic Pricing Rules: Allow prices to shift based on real-time behaviour, content consumption trends, and traffic patterns. AI-driven dynamic paywalls aren’t about squeezing users—they're about aligning subscription offers with actual user value and intent. Early adopters have seen 20–35% higher conversion rates and up to 15% lift in average revenue per user (ARPU).Static pricing is no longer sustainable for publishers aiming to maximise revenue yield. Intelligent pricing strategies are the future. Here are key takeaways: 1. Static paywalls limit potential revenue growth. 2. AI-driven paywalls tailor offers based on user behaviour and content value. 3. Dynamic pricing improves both conversion rates and ARPU. 4. Publishers must audit, segment, and dynamically adjust pricing strategies to stay competitive. It’s time to audit your pricing model. If it can't adapt, your revenue won't, either. Is your paywall strategy optimised to maximise revenue yield in 2025? Share your thoughts with me in the comment section. #AIMonetization #DigitalPublishing #PaywallStrategy #SubscriptionRevenue #PublisherRevenue