Market failure refers to a situation defined by an inefficient distribution of goods and services in the free market. In an ideally functioning market, the forces of supply and demand balance each other out, with a change on one side of the equation leading to a change in price that maintains the market's equilibrium.
And market failures are common. Moreover, scientists are producers of knowledge, thus they are market agents. They (and the knowledge they produce) are affected by forces of demand. Biased public knowledge is a leading cause of market failures (and of scientific failures), which may further amplify the forces of demand that affect the scientists.
While both the ideal market and the scientific process may seem stable at first sight, once we consider that the public knowledge is affected by supply and demand, we conclude that they are often unstable, since many small deviations from balance can be amplified by scientists and then by forces of demand, in an iterative process.
The way out is to decouple the economy from the scientific process, as much as possible. But since the scientific activity is part of the economy and vice-versa, what we need to do is to decouple some sectors of the economy from each other (and of public knowledge), which become then isolated from each other as much as possible.
This would lead to a more resillient economy, since a crisis in one sector would not imply a crisis in another sector. Also to more scientific progress, since statistically independent priors decrease the cost of a crisis (since it only affects one sector) allowing it to occur when it is needed and not when a catastrophe occurs. Note that there are no unbiased Bayesian priors, thus crisis (when a prior is replaced with another one) are often required for scientific progress to occur.
While there is a cost to organize the economy and public knowledge in such a way, the relevant question is whether this is a sustainable investment or not. Given a Bayesian prior which is the product of many statistically independent priors we need a lot of energy to make predictions about the real world (which is messy, thus the posterior is very different from the prior). Most functions are uncomputable, and for most computable functions we cannot say a priori if they are computable. In the real-world, prioritizing survival is the only viable strategy. But, when considering probabilities we can work with computable functions. It does not gurantee survival (because we do not know many things) but it increases our probability of survival.
Such probabilities allow us to compute predictions (with some intrinsic uncertainty) from many independent assumptions. The predictions can be well approximated by a few terms, which guarantees that at least the approximation is computable (and avoids the dependency hell[1], moreover).
We conclude that it is sustainable to split knowledge (a Bayesian prior) into independent components. Since the economy is increasingly based on knowledge, then it is also possible to split the economy into independent components, at least in principle. An explicit way of doing is already widely used (by the World Bzank, for instance): Catastrophe Bonds[2].
The “Catastrophe” in the name “Catastrophe Bonds” refers to the fact that these Bonds are often used as a means of insurance against extreme events (catastrophes), because they are mostly isollated from the rest of the economy, so isollated that even in a Catastrophe the Bonds still likely work. But they can be applied in other contexts. The fact that the risk in these Bonds is mostly independent (from most of the economy and from other Catastrophe Bonds) is their main advantage. Note that the Central Limit theorem implies that the overall risk in a Portfolio approximatelly decreases with
According to the World Bank[2]:
In a typical catastrophe bond structure, the entity exposed to the risk (known as the “sponsor”) enters into an insurance contract with a [special purpose vehicle, such as the the World Bank for instance] SPV that issues the bonds to investors. The SPV invests the proceeds of the bond issuance in highly rated securities that are held in a collateral trust, and it transfers the return on this collateral, together with the insurance premiums received from the sponsor, to the investors as periodic coupons on the bonds.
If a specified natural disaster occurs during the term of the bond, some or all of the assets held as collateral are liquidated and that money is paid to the sponsor as a pay-out under its insurance contract with the SPV. If no specified event occurs, the collateral assets are liquidated on the maturity date of the bonds and the money is paid to the investors.
We stress that a specified natural disaster can be replaced by many other specified events in this kind of Bons structure. What is crucial is that the trigger of the Bond is based on a credible measure or entity, mostly independent from the remaining economy.
For instance, many investments in information technology are made under the assumption that
In fact, one of the main problems in the funding of science is the problem of incentives. How to distribute the funds fairly between the scientists while at the same time making sure that most scientists have an incentive to really solve real problems. It is often the case that a scientist who is recognized as an expert in a specific problem (P vs. NP, for instance) has no incentive that someone else really solves it, since then his field would change in an unpredictable way which he does not want since he is already recognized as an expert on this problem (and not in other problems). When trying to solve a hard problem himself, it is much more likely that the expert will only make an indirect contribution to solve such problem , then the incentive for an expert to try to solve the hard problem (which would likely help someone else to solve the problem) can be null or even negative.
On the other hand, once there is a Catastrophe Bond assciated with P vs. NP, then every new relevant contribution is worth money (in a very direct way), since it changes how the public evaluates the odds of a solution of the problem and thus the value of the Bond itself. Someone who made such contribution can buy the Bond before announcing such contribution to the public and then sell it again (or sell before and buy after the announcment, depending on which contribution he made).