This page in one lineWhen a betting market says something has a 70% chance, does it actually happen about 70 times in 100? The short answer: mostly yes on easy questions, only a little better than guessing on hard ones. Our own confidence score does not work.
Do Prediction Market Odds Come True?
Across 325 large prediction markets that have finished, when the crowd said a thing had a 97% chance a week before the end it happened 94% of the time. Overall the crowd’s week-before prices were 67% better than just saying 50% every time, but 71% of these markets were near-certain either way; on the genuinely uncertain ones the crowd was only 18% better than a 50-50 guess. Charlie’s own calls won 57% of the time. Split into four piles from least sure to most sure, the piles won 65%, 55%, 53%, 56%. That is not a climb. The pile Charlie was least sure about beat 3 of the 3 piles he was surer about, so the confidence score does not work and you should ignore it until it is rebuilt. Charlie’s separate forecasting tool for Bitcoin price levels has 231 forecasts still waiting to finish; the first can be graded from 2026-12-31.
When the crowd said 70%, did it happen 70% of the time?
Every dot is a group of finished prediction markets. Left to right is the chance the crowd’s price stated one week before the market closed. Up and down is how often the thing actually happened. If promises were perfect, every dot would sit on the dotted line. Dots above the line mean the crowd was too gloomy; dots below mean too confident. Bigger dots hold more markets.
Drag the chart sideways to see the rest of it
325 markets. The label on each dot is how many markets it holds.
What this does not mean. A dot near the line does not mean the crowd predicted each event. It means that across many events, its stated chances added up honestly.
How much of the crowd’s good score comes from easy calls?
Most finished markets end up near 0% or near 100% a week before they close, because by then everyone knows the answer. Getting those right is easy. So the crowd’s score is shown three ways: against a 50-50 guess on every market, against someone who simply knew how often “yes” wins in general (22% of the time here), and on the genuinely uncertain markets only, those priced between 10% and 90%.
Drag the chart sideways to see the rest of it
71% of the 325 markets were priced under 10% or over 90% a week out, so the answer was already obvious. The uncertain set holds 94 markets.
What this does not mean. A lower score on hard markets is not a failure. Hard markets are hard for everyone; the point is to know how big the crowd’s advantage really is.
Does the crowd get better as the deadline approaches?
Mostly the same markets (288 of the 325 had a price a month out), but now the stated chance is taken a month before the close instead of a week. Markets learn as news arrives, so the month-before dots should be a little further from the line.
Drag the chart sideways to see the rest of it
288 markets with a price available a month before the end.
What this does not mean. Better later is normal and not a flaw. The point is to know how much a price should be trusted at each distance from the deadline.
Did Charlie’s higher confidence mean more wins?
The Alpha Tracker is Charlie’s public list of short-term trading calls, each saying a small coin will rise or fall over the next hours or days. Every call gets a confidence score before it is published. The 493 finished calls are split into four equal groups by that score. If the score means anything, the bars should climb from left to right.
Drag the chart sideways to see the rest of it
All calls together: 57% won. The calls were published on only 35 separate days, so every group contains calls made in the same market conditions.
What this does not mean. With calls bunched on a few days, a group can look strong because it happened to be published on a good day.
Did higher confidence mean bigger wins?
Winning more often is one thing. This chart shows the typical size of the result in each confidence group, in percent.
Drag the chart sideways to see the rest of it
We also checked whether a higher score went with a bigger win. Scored from -1 (exactly backwards) through 0 (no link at all) to +1 (perfect): -0.05.
What this does not mean. Typical result here is the middle call, not the average, so one giant winner cannot carry a group.
Which part of Charlie’s score did the work?
The confidence score is built from three parts: the shape of the price chart, whether there was a news trigger, and where money was flowing. For each part, calls that scored higher on it are compared with calls that scored lower; the number in brackets is how many calls fell in each group, and when most calls share one value the groups are far from equal.
Drag the chart sideways to see the rest of it
A part that matters shows a clear gap between its two groups, with the higher-scoring group winning more. The number in brackets is how many calls are in the group.
What this does not mean. Which part works can change with the market. This is a report card for one period, not a permanent ranking.
When traders who borrow money felt sure about Bitcoin, were they right?
Some people borrow money so they can bet more on Bitcoin than they actually have. Each day we count how many of them were betting on a rise against how many were betting on a fall. Every day from 2020-06-03 to 2026-09-09 goes into one of five groups, from most sure of a fall to most sure of a rise. The bars show how often Bitcoin was actually higher 30 days later. If being sure meant anything, the bars should climb from left to right. Across the whole period Bitcoin was higher 30 days later 55 times in every 100, so that is the line each bar has to beat, and it is drawn on the chart. Where it says typical, that means the middle one: line every result up and take the one in the middle, so one wild month cannot drag it.
Drag the chart sideways to see the rest of it
2,290 days, but 30-day windows overlap, so this is closer to 76 separate tests. Typical 30-day move per group: most sure of a fall +4.2%, leaning to a fall +6.5%, middle +1.5%, leaning to a rise +1.2%, most sure of a rise -2.8%.
What this does not mean. A crowd being wrong on average does not mean betting against it is easy. The groups overlap in time, and the pattern can flip between years.
When people posting online felt sure about Bitcoin, were they right?
Each day we count the cheerful posts about Bitcoin online and subtract the gloomy ones, which gives one number for how cheerful people sounded that day. Every day from 2025-09-22 to 2026-08-23 goes into one of five groups, from most sure of a fall to most sure of a rise. The bars show how often Bitcoin was actually higher 30 days later. If being sure meant anything, the bars should climb from left to right. Across the whole period Bitcoin was higher 30 days later 48 times in every 100, so that is the line each bar has to beat, and it is drawn on the chart. Where it says typical, that means the middle one: line every result up and take the one in the middle, so one wild month cannot drag it.
Drag the chart sideways to see the rest of it
336 days, but 30-day windows overlap, so this is closer to 11 separate tests. Typical 30-day move per group: most sure of a fall +4.0%, leaning to a fall +3.6%, middle +1.7%, leaning to a rise -2.4%, most sure of a rise -15.9%.
What this does not mean. A crowd being wrong on average does not mean betting against it is easy. The groups overlap in time, and the pattern can flip between years.
What is still waiting to be graded?
Charlie also runs a separate forecasting tool that states a chance of Bitcoin reaching certain price levels by certain dates, beside the chance the betting price works out to. None of those forecasts has finished yet, so they cannot be graded here. The chart shows when they will be. All of them finish on the same day, so the first grade will be one batch judged on one day, not many separate tests.
Drag the chart sideways to see the rest of it
231 forecasts open. This page will grade them as they finish and show the result the same way as the charts above.
What this does not mean. An ungraded forecast is not a failed one. It is just not evidence yet.
What should you do with this?
Two kinds of reader use this page. One runs money for other people and has rules to follow. The other is deciding about their own savings. The same evidence leads to different actions.
If you run money for others
Funds, trading teams, the people who manage a company’s cash, research teams.
- You rely on prediction-market prices in your own workA week before the deadline, the price is a fair reading of the real chance, most of the time. Trust it most where the price is already near 0 or near 100, because those are the questions everyone has worked out. Where the price sits in the middle, take it as roughly right rather than exactly right.
- You are shown a confidence score by any providerAsk for this chart: win rate by confidence quarter. If the bars do not climb, the score is decoration.
- You want to grade a forecasterInsist on the full record including unfinished forecasts, and on the number of separate days behind it.
If it is your own money
Anyone deciding what to do with their own savings.
- A market says 90% and you want to bet on the likely outcomeNear-certain things did come in about as often as promised here, but you win very little when right and lose a lot on the rare miss.
- A market says a few percent and you like the unlikely outcomeLong shots priced around 1 in 100 actually happened about 3 times in 100 here, so roughly three times as often as the price said. That is a real gap, but they still lost about 97 times out of 100. Not free money.
- You follow Charlie’s callsIgnore the confidence score until it is rebuilt. The piles did not climb in order, and the pile Charlie was least sure about beat most of the surer ones. Judge each call on its own reasoning instead.
The conclusion
A check of whether stated chances came true as often as promised, for the prediction-market crowd and for Charlie’s own confidence scores.
A percentage is only useful if 70% actually means 70%. Most forecasters are never checked this way, so their numbers sound precise and mean little.
It tells you how far to trust a market price on easy and on hard questions, and whether Charlie’s confidence score is worth reading.
The crowd kept its promises well on average, but most of its 67% score came from easy calls; on uncertain markets it was 18% better than a 50-50 guess. Charlie’s confidence score does not work and should be ignored until it is rebuilt. Charlie’s Bitcoin price-level forecasting tool has 231 forecasts still waiting for their grade.
Words used on this page
Every technical word above is explained again here, in plain English.
- Prediction marketA place where people bet real money on whether something will happen. The price is the crowd’s stated chance.
- Stated chanceThe probability a price or a score is claiming. A price of 70 cents on a yes-or-no bet means a 70% stated chance.
- Promise lineThe dotted diagonal. A dot on it means the thing happened exactly as often as the stated chance said.
- Better than a 50-50 guessHow much lower the forecaster’s error was than someone who said 50% every time. 0% is no better; 100% would be perfect.
- Confidence scoreA number Charlie attaches to each call before publishing it, built from the chart, the news and the money flow.
- Alpha TrackerCharlie’s public list of short-term trading calls on small coins, each logged before its outcome is known.
- CallOne published prediction that a coin will rise or fall over the next hours or days.
- Long shotAn outcome the crowd thinks is very unlikely, priced at a few percent.
- Uncertain marketsMarkets priced between 10% and 90% a week before the close, where the answer was still genuinely in doubt.
- Forecasting toolCharlie’s separate model that states a chance of Bitcoin reaching a given price level by a given date.
- Money flowWhere money was moving into or out of a coin around the time of a call, one of the three parts of the confidence score.
- PileWe line the calls up from least sure to most sure, then cut the line into four equal piles. Pile 1 is the least sure, pile 4 the most.
- Wrongness scoreHow wrong a percentage turned out to be. Say 90% and it happens, you collect almost nothing. Say 90% and it does not, you collect a lot. Saying 50-50 to everything collects 100. Lower is better.
- GradedA forecast whose deadline has passed, so it can be marked right or wrong.
- Which way the borrowers are leaningBets that the price rises, divided by bets that it falls, among people betting with borrowed money. Above 1 means more of them expect a rise.
- Sentiment balancePositive social posts about Bitcoin minus negative ones, counted each day.
- GroupOne fifth of the days, sorted by how confident the crowd was that day.
- Leveraged crowdTraders using futures, contracts that let them bet on Bitcoin’s price with borrowed money, so their bets are larger than their cash.
- Contrary signalA reading that tended to be followed by the opposite of what it suggested.
Where this page could be wrong
- The prediction markets here are the largest finished ones from the last year; the smallest carried $14 million of bets. Small markets behave worse and are not included.
- The middle groups on the dot charts hold only a handful of markets each, so one market can move a dot by several points.
- A market’s stated chance is read from its price one week and one month before the end. Prices in between are not used.
- Charlie’s calls were published on a small number of days, so the four confidence groups share market conditions.
- The confidence score was never meant to be a percentage. It is judged on order, not on matching a number.
- All of this is about what has already finished. It says how honest past promises were, not whether the next one will be.
- The 30-day windows overlap heavily, so each crowd test rests on far fewer separate cases than its day count suggests.
- The social crowd series covers only about one year on the current data plan, and arrives with a delay of about a month.