Correlation, Causation and Launch Data

A founder posts in a forum on Tuesday. On Wednesday, sign-ups double. The conclusion writes itself: the post worked. She plans three more.

It may well be true. It may also be that a newsletter mentioned the product the same morning, or that a competitor had an outage, or that Wednesday is simply a busy day for her audience. The graph cannot tell her. Launch data is full of such moments, where two things happen together and we feel certain about the reason.

This guide is about resisting that certainty without becoming paralysed. It explains correlation and causation in plain terms, shows the traps that launches set, and suggests habits for reaching conclusions you can trust.

Two words, two ideas

Correlation means two things vary together. When one goes up, the other tends to go up as well, or down. It is a statement about patterns in data.

Causation means one thing produces another. Changing the first changes the second. It is a statement about how the world works.

The old warning that correlation does not imply causation exists because patterns can arise for several reasons.

1. One really does cause the other.
2. The reverse is true. The second causes the first.
3. A third thing causes both.
4. Coincidence. With enough things measured, some will move together by chance.

Data alone usually cannot tell these apart. You need reasoning and, where possible, tests.

Why launches are especially tricky

A launch is a crowded event.

Many things happen at once. You post, email, list, tell friends, change the page and hope for mentions, all in the same few days.

Attention clusters. Interest arrives in bursts, so many measures rise together: visits, sign-ups, comments, links.

Timing overlaps. The effects of one action show up while another is under way.

Small numbers. Early products have few data points, so patterns are noisy.

You want it to work. Hope shapes interpretation.

All of this makes it easy to build a persuasive story from a coincidence.

The role of confounders

A confounder is a factor that influences both of the things you are comparing, producing an association that is not a direct cause. The textbook example is that ice cream sales and swimming accidents rise together in summer; heat drives both, and neither causes the other.

Launch examples:

• Weekday. Sign-ups and support questions both rise on weekdays. Neither causes the other. The weekday drives both.
• A mention. A newsletter feature raises visits, comments and links at once. The comments did not cause the sign-ups; the feature drove all three.
• Seasonality. Interest in your category rises in a certain month, lifting everything you do in it.
• Audience quality. A well-matched source produces both higher conversion and higher retention. The source drives both.

When two measures move together, ask: what else could be driving both?

Confirmation bias

People tend to favour information that confirms what they already believe, a pattern known as confirmation bias. After a launch, it shows up as selective memory.

• You notice the posts that preceded spikes and forget those that did not.
• You credit the activity you enjoy.
• You read ambiguous data as support.

Counter it with discipline: write down your expectation before looking at the result, and note the cases that do not fit.

Attribution is hard

In marketing, attribution is the practice of deciding which activities deserve credit for a result. It is imperfect even for large organisations with sophisticated tools. For a small launch, it is mostly judgement.

Be honest about this in your records. Instead of "the forum post brought forty sign-ups", write "forty sign-ups came on the day of the forum post; tagged visits from the post account for twenty of them; the rest cannot be traced."

Habits that improve your conclusions

Change one thing at a time

If you alter the headline, the price display and the screenshot on the same day, you cannot know which mattered. Make one change, give it time and record it.

Keep a baseline

Know what normal looks like. A rise only means something against what would have happened anyway. Record weekly numbers for the weeks before.

Keep a dated log

Write down every action and every external event you notice. When a pattern appears, the log gives you candidate explanations, including the dull ones.

Use tagged links

Tags make some cause-effect relationships visible: this visit came from that post. They do not cover everything, but they replace guesses with records where possible.

Look for the mechanism

A good causal story has a mechanism: a path by which the cause could produce the effect. "The post linked to the page, visitors clicked, some signed up" has one. "The post made the product seem popular, which somehow raised conversion on unrelated sources" needs more evidence.

Check the order

A cause comes before its effect. If sign-ups rose before the post went live, the post did not cause them.

Check the dose

If a cause is real, more of it often means more effect. If a post with a large reach produced the same lift as one with a tiny reach, be sceptical.

Look for other cases

Has it happened before? Did the last post also coincide with a rise? If the pattern repeats across several occasions, confidence grows. Treat a single occurrence as a hint.

Try to repeat it

The best test of a cause is to do the thing again under similar conditions and see whether the effect returns. A change that raises conversion twice is more believable than one that did once.

Run a simple experiment

If you have enough visitors, show half of them one version and half another, at the same time, and compare. This isolates the change from the calendar. With small numbers it may not be conclusive, but it is more reliable than comparing different weeks.

Beware of cherry-picked windows

A rise looks dramatic if the chart starts at the trough. Use standard windows and show the context.

Small numbers, big stories

The smaller the data, the more tempting it is to read meaning into noise. Three more sign-ups on a Thursday is not a trend. When numbers are small:

• Use counts, not percentages.
• Combine periods to see the shape.
• Hold conclusions loosely.
• Seek qualitative evidence: conversations, comments, messages.

A user who tells you "I found you through that post" is evidence of a different, and often more reliable, kind.

Ask people

Surveys and conversations complement numbers. A single optional question at sign-up, "How did you hear about us?", often gives clearer attribution than any report. People's memories are imperfect, but the answers add a second line of evidence. When the numbers and the answers agree, trust grows.

Reporting honestly

When you describe results to others, choose your verbs carefully.

• "Coincided with" for timing alone.
• "Was followed by" for sequence.
• "Is associated with" for correlation.
• "Probably contributed to" for plausible causes with some evidence.
• "Caused" only when you have tested it.

It may feel pedantic. It also protects you from repeating a story that is not true, and from teaching others to do the same.

Advertising and consumer rules in many places expect claims about results to be supportable, so a careful habit of language helps there too.

Do not give up on data

None of this means numbers are useless. It means they are clues. A pattern raises or lowers your confidence in an explanation. Several clues pointing the same way, a mechanism, a plausible order, a repeat and a user's own words, make a strong case. A single graph rarely does.

A short checklist before you say "it worked"

• Did the effect come after the action?
• What else happened at the same time?
• What was the baseline?
• Can I trace any of it with tags or answers?
• Is there a plausible mechanism?
• Has it happened more than once?
• Are the numbers large enough to mean something?
• Am I seeing what I hoped to see?

On this site

The launches and leaderboards pages show numbers with their dates, signs and context, and the developers page describes the free API and MCP server for pulling data into your own sheet. The categories page helps you find comparable products, and the blog has further guides in this series.

A story of a graph that fooled someone

A founder noticed that on days she sent a product update email, support questions rose. She concluded that the emails confused people and decided to send fewer. A colleague asked her to look again. The emails went out on Tuesdays and Thursdays, which were also the days when most new trial users signed up and needed help. The emails did not cause the questions. New users did, and both followed the weekly rhythm of her audience.

If she had cut the emails, she would have lost a useful channel and fixed nothing. Instead she kept a log, compared support questions per new user rather than per day and found they were steady. The pattern was real in the data and wrong in the story. A dull, ordinary explanation, the calendar, had been the confounder all along.

Questions that help in a meeting

When someone presents a causal story from a chart, a few friendly questions improve the discussion. What else changed that week? What was normal before? Has this happened before, and did the effect repeat? How many people is this based on? What would we expect to see if the explanation were wrong? Asking these in a spirit of curiosity, not challenge, turns a debate about opinions into a shared look at evidence.

The short version

Correlation means two things move together; causation means one produces the other. Launches are crowded with simultaneous events, confounders and hope, so patterns mislead easily. Change one thing at a time, keep a baseline and a dated log, tag links, look for mechanisms, order and repeats, ask people how they found you and choose your words honestly. Treat data as evidence that shifts confidence, not as proof.