The criterion · The decision
100% of the information never arrives.
You need to inform yourself, and you need to stop informing yourself. Between those two limits there is an interval, and we call it the decision window: the stretch in which you already know enough and have not yet overpaid for knowing it. It lies well before where caution would put it, and it is not an opinion: it is what decision theory has said since the sixties.
13 min read
The mechanics
Two curves that move in different directions.
Informing yourself has a value that grows, but less and less: the first data radically change what you know, the twentieth confirms what you already suspected. It is the law of diminishing returns applied to knowledge, and it makes the value curve flatten.
Waiting has a cost that grows, and more and more: the market window narrows, the team you had identified commits elsewhere, someone else solves the same problem, and the tax year closes. At the beginning waiting is almost free; at the end it is extremely expensive.
And there is a second cost almost never counted, because it does not appear in the classic models: the places are finite. Arriving late to a market is one harm; not arriving at a project because the position has already been taken by someone else is a different and blunter harm. In a round you do not compete against time in the abstract, you compete against someone who decided with less information than you are waiting for. Decision theory almost always models a single decider facing the world; here there are others deciding at the same time on the same place.
The difference between the two is what you really gain by going on informing yourself, and that difference has a maximum. It comes before the end. After that point, each new piece of data costs you more than it gives you: you go on learning and you start losing.
Where the point lies at which it is best to decide
Why the optimum is not at the end. In the model of the drawing the net reaches its maximum at around 58% of the information. But look how flat that maximum is: at 40% you already have 94% of the possible benefit, and at 70%, 98%. At 100% you have dropped to 79%. That is: deciding anywhere in the 40-70 band is almost equally good, and waiting until the end is measurably and clearly worse.
The grey band is not an ornament. It is exactly the interval recommended, by independent routes, by those who have had to decide a lot and fast. The figures are further down.
The theory
Information has a maximum price, and it can be calculated.
This is not a metaphor. In decision analysis there is a quantity called the expected value of perfect information (EVPI), formalised by Ronald Howard in 1966 and by Howard Raiffa in 1968: it is the difference between what you would gain deciding with absolute certainty and what you gain deciding with what you know now. It is, literally, the maximum a rational decider should pay to know everything.
And here is what almost nobody internalises: that number is finite, and it is often small. If the EVPI of a decision is one hundred thousand euros, spending one hundred and twenty thousand on studies to remove the uncertainty is irrational even if the studies work perfectly. Certainty is not a free good: it has a price, and above it buying it destroys value.
The same framework explains why paralysis is not prudence. Herbert Simon received the Nobel Prize in Economics in 1978 for, among other things, showing that real deciders do not optimise but satisfice: they look for the first sufficiently good option, because going on searching costs more than it improves the result. It is not a human defect to be corrected. It is the correct answer to a problem with search costs.
The paradox
100% is not slow: it is impossible, and not for lack of time.
So far we have spoken as if complete information existed and were merely expensive. It does not exist, and the reason is not that infinite time would be needed. It is more uncomfortable than that: gathering information consumes time, and while that time passes the conditions you were gathering information about change. The object you study moves while you study it.
The practical consequence is that data expires. The market study from eighteen months ago is not incomplete information: in part it is false information, because it describes a market that is no longer there. That is why the value curve does not only flatten: in a fast-moving environment, the oldest part of what you know deteriorates while you wait to complete it. Chasing 100% is chasing a target that moves at the same speed you advance.
Put another way: the choice is not between deciding with little information and deciding with a lot. It is between deciding with recent and incomplete information or with complete and old information. The second is not the prudent option it looks like.
The method
By levels: the macro first, and only then whatever detail is needed.
If 100% never arrives, the useful question is not «how much do I know» but «what do I know». And that is where the investors and business people who decide a lot do something that looks obvious and almost nobody does in a disciplined way: inform themselves by levels. First the macro parameters, the ones that rule out most cases with very little effort (is this market growing? how big is it? is there a structural reason why the problem is still unsolved?). Only when the macro level passes the filter do you go down to the detail, and only to some details: the ones that can change the decision.
This has a name in the literature and it is worth saying, because it is more solid than an intuition of ours. Kahneman and Tversky called it the outside view against the inside view: the systematic error of planners is to look only at the details of the specific case (the inside view) and neglect the distribution of outcomes of similar cases (the outside view). The procedure they propose, today known as reference class forecasting, is literally macro and then micro: choose the class of comparable cases, establish the distribution of outcomes of that class, and only then place your specific case within it. Kahneman received the Nobel Prize in Economics in 2002 for the line of work that includes these results.
What we do add to that framework are two things not in it, and we say them as our own reading and not as a discovery. The first is expiry: the outside view assumes a stable reference class, and in innovation the class moves while you study it. The second is that the places are finite, and that is what the next section is about.
The distinction
Data and information are not the same, and the difference explains why we never entirely agree.
A datum is an observation: a market figure, a patent date, a survival percentage. On its own it does not say what to do. Information appears when someone orders those data within a framework and interprets them. And there is the point almost never said out loud: the interpretation is part of the information, it is not an addition that can be separated off afterwards.
We gather data, interpret them and produce information. You arrive with other data, which we do not have; with another framework, formed by your sector and your experience; and you read our information in a way that need not match the one we meant to give it. The result is not a misunderstanding that can be avoided with more care: it is a structural divergence. It is always there, even between two honest people looking at the same table.
Hence a design consequence that runs through this whole site: we publish the data and the assumptions, not only the conclusions. The simulations have the controls on display so that you can move the assumptions and see what changes; the Tax Lease comes with its regulatory references; the sector curve comes with its source; and what we cannot yet demonstrate is in our figures, blank. It is not decorative transparency: it is the only reasonable answer to divergence. If your interpretation is going to differ from ours, let it at least start from the same data.
And there is something we ask of you, and it is an act of generosity: tell us your reading, above all when it does not match ours. If you have data we do not have, or if from the same data you draw a different conclusion, that is not an objection to be overcome: it is exactly what we are missing. We work on problems nobody has solved, and whoever lives with one of them sees things that cannot be seen from outside. We are not here only to explain: we are also here to learn, and it is the only way we know of improving steadily.
And there is a second consequence, the one that matters for deciding: an agreement is not closed when someone is right, it is closed when the two interpretations converge enough. That is why we do not try to convince you of our reading: we try to give you what you need to build your own and see whether they resemble each other.
The numbers
40%, 70%, 37%: the figures used by those who decide.
Three independent sources, three different routes, and all point to a threshold considerably lower than intuition suggests.
- Colin Powell · the 40/70 rule
Never decide with less than 40% of the information you are likely to get, and do not gather more than 70% of what is available. Below 40 you are guessing; above 70 you are losing time worth more than the missing datum.
- Jeff Bezos · the 70%
In his 2016 letter to shareholders: most decisions should be taken with around 70% of the information you wish you had, and waiting for 90% is, in most cases, being slow. He adds the part usually forgotten: if you know how to correct fast, being wrong may cost less than you think, whereas being slow is expensive for sure.
- Optimal stopping theory · the 37%
The classic optimal stopping problem (the «secretary problem») has an exact solution: observe the first 1/e fraction of the options, around 37%, and from then on take the first one that beats everything seen. The probability of picking the best is also 1/e, approximately 37%. It is the extreme and highly idealised case, but it gives the direction: exploring has an optimal end, and it comes before halfway.
It is worth noting where 80% falls in this company: above all of them. An 80% threshold is more conservative than Powell's, than Bezos's and than optimal stopping theory's. If you recognise yourself waiting for 80, the literature is not asking you for more prudence: it is asking you for less.
The limit
There is a part of the uncertainty that cannot be bought with analysis.
The value curve flattens for an underlying reason, and it is not the researcher's laziness: there are questions whose answer does not yet exist and cannot be obtained however much you study. Whether a customer will pay for something they have not seen is not a datum hidden in some report: it is a fact that happens when you put it in front of them. No amount of desk work anticipates it.
That is why the only way of buying that information is to produce it: prototype, MVP, first users. Which is exactly what the method's TRL pyramid does, and the reason we devote a quarter of the budget to early validation instead of the usual twentieth. It is not prudence: it is the cheapest way of buying the one datum that cannot be read.
The condition
Before applying the rule, look at whether the door turns both ways.
The 40-70 rule does not apply the same to everything, and Bezos himself points this out in the same letter: there are decisions that are two-way doors, reversible, and decisions that are one-way doors. In the first, deciding fast with partial information is the right thing, because the cost of being wrong is going back. In the second it is best to raise the threshold, because there is no way back.
The most expensive error organisations make is not deciding with little information: it is treating a two-way door as if it were one-way, and paying months of analysis for something that could have been undone in a week.
The fit
Our method is built to buy the cheap information first.
The phases of the path are not a bureaucratic sequence: they are a purchase order. First what costs little and rules out much (the problem is real, the market is growing, the state of the art leaves a gap), and only afterwards what costs a lot (building). Each phase exists so that the next is decided with more information and without having spent the large amount.
And for the investor the consequence is direct: when your capital comes in, the cheapest part of the information has already been bought by the method, and much of the expensive part has been paid for by the Tax Lease and public funds. You are not deciding at 20%. You are deciding on a project that has already passed the filter of market, idea and team and external validation.
That is why we have put on the site all the information we could, and not only what favours us: the rules that hold up the Tax Lease with their references, the calculation assumptions of the simulations, the sector's distribution of outcomes, and also what we cannot yet demonstrate. It is written so that you can review it with whoever you like: your adviser, your lawyer, or the artificial intelligence you use to read and check documents. The more information is open and ordered, the less your decision depends on trusting us.
What does not change, and it is worth saying here: the rest of the uncertainty is not eliminated. It is bounded. The distribution of what can happen, and what part of it depends on the method and what part does not, is drawn in the map of risk and reward.
«If you're good at course correcting, being wrong may be less costly than you think, whereas being slow is going to be expensive for sure.»
The asymmetry is that: the error gets corrected, the time does not come back. Tell us your problem and we will look at it with this criterion.
How long have you been gathering information?
John F. Kennedy, 1962