How do you assess the success of a new thing?
I was battling with that question last week. I wanted to test a slightly different look and feel of my newsletter. I played around with the idea for a while and then decided to finally test it. I created two variations and ran A/B tests for a few weeks.
I ensured each test audience gets different variations in consecutive weeks to eliminate any opportunity for one group to be naturally more interested in clicking than the other. So, if you got the old format one week, you’d get the new one the next. And then I looked at the data.
It was all inconclusive.
There wasn’t any difference in performance metrics. Click rates were too close to call, no matter how I sliced and diced the data. And this was all I could go on.
Unless I asked.
Based on the data alone, it was clear that my readers were more interested in the content than the format. And while that felt like a great honor, it wasn’t the answer I sought.
So I asked around.
I posted both versions on social media. More importantly, I asked my subscribers themselves (basically, all of you lot!) And the answers I got were illuminating.
As expected, avid readers came because of the content, and the format wouldn’t have been a deal breaker. But people still had a preference:
They spoke about information retention, readability, scannability, focus, and so much more. People also commented on what looked fun, exciting, or interesting. And you might think this isn’t as relevant for business content, but it very much is. You want connection, and that’s based on emotion.
You can obviously see the end result here. I chose this version because it was the one most people preferred – and they picked it for some excellent reasons. It’s apparently easier to read and feels less like a magazine from a brand and more like a letter from a friend. Which is in line with my goal of fostering relationships through helpful content.
But the result isn’t the point here. The point is that you don’t get the full picture unless you ask. Fortunately, the A/B test was inconclusive – I’m not sure I would’ve asked more questions if it wasn’t. And the answers were what was important.
So ask away. Get the “why” behind the “what.” It may be the most important thing.
🧐 P.S. Have you ever been in a position where the why was more important than the what? Tell me about it!
Will this content help someone? Forward it to a friend with an invitation to subscribe right here: valchanova.me/newsletter.
📁 From My Archives
Topic research will help you get inspiration and make sure you’re prioritizing pieces that resonate with your audience. It’s the basis for a well-thought-out content backlog. In this post, you’ll see the best free topic research tools and the process I follow to uncover great ideas.
👌 Handpicked Stories for You
A Step-by-Step News Article Template with SEO Best Practices
This is a great checklist you can bookmark and keep close. While it’s designed to serve news articles, it’s pretty much applicable to any piece of content. The headline and title ideas are especially useful.
180+ Email Subject Lines from Real Brands (Sorted by Category)
This is not so much a blog post as a reference file. I’d bookmark it and open it up whenever I feel like I need extra inspiration for my next subject line. Still, I’d suggest you exercise good judgment – some of these are too clickbaity for my taste, and they definitely don’t fit any type of recipient.
Google Analytics 4 (GA4): Ultimate Guide and Tutorial
Backlinko is famous for these detailed guides that are both novice-friendly and especially thorough. In this case, I felt a bit let down by the level of depth – I expected a more in-depth walk-through and comments on some of the issues with GA4, especially sampling limitations and thresholding. But it’s a better place to start than many.
How the best companies measure content quality
Quality is a matter of perspective. At least that’s what some people will tell you – others know that you need to measure your work to manage and optimize it right. This post will give you very specific KPIs for each aspect of content: production efficiency, content performance, and the vitality of your content over time.
9 Ideas To Boost Your Internal Content Distribution Strategy
Getting your team on board with your content efforts can help push your pieces to more people, especially at launch. This post is a must for marketers working with bigger teams. It will help you get your team excited about your content and ready to share it with their audience – making your work easier along the way.
👀 Interesting stuff to click on
- This story about prehistoric children – most scientists don’t focus on children in their work. Archaeologists are first among them. But this may be changing.
- This exercise in creativity – aim for this level of artistry the next time you’re hanging clothes to dry.
- Weekly cat – kitty see, kitty do.
I hope these were fun for you!
🔥 That’s a wrap! Before you go…
If you enjoy my newsletter and want more, here are three options: