Introduction: How the First Great Finding in Empirical Science Was Lost for Decades

Believe it or not, humanity is completely capable of ignoring a $1,000,000,000,000 bill sitting in plain sight. It happened in 1747, when James Lind ran a controlled scientific study — maybe the first ever — and discovered that citrus fruits cured scurvy.

In Lind’s day, scurvy killed more British sailors than "shipwreck, storms, all other diseases, and enemy action combined”. Which is to say that Lind's cure was not just going to save countless lives; it was an incomparably powerful military technology. The stakes couldn't have been higher. And yet, despite having been proven with the most advanced scientific methods the world had ever seen, it was promptly lost for many decades after.

According to British Historian C.C. Lloyd, the British might have “lost the American War of Independence thirty years [after Lind’s study] because his cure was not adopted”. If Lloyd’s suggestion is correct, the discounted net assets of America is what gives you a roughly $1,000,000,000,000 bill that the British may have just left lying on the ground.

We will return to why and how this happened later. But for now let’s just observe that the incentives for the production and dissemination of scientific truths are often really, really poor. And so, as a result, humanity can lose even the most consequential truth.

Enter NFTs

The idea that NFTs could help rescue enormous scientific truths from obscurity might strike you as a little fanciful. You might even think... "aren't NFTs just some hyped-up blockchain art thing? Why should they help science funding at all?" So to start with let's take the simplest argument, and here we'll just quote it from our 2018 essay on NFTs for Science:

Historically, great knowledge is valued like great art. Bill Gates paid $40 million for Leonardo Da Vinci’s Codex Leicester. Einstein’s manuscripts sell for millions. And a single page of Darwin’s writing fetches hundreds of thousands of dollars[...] People buy Darwin’s original pages because they are connected to his idea — the fact that information from The Origin of Species is freely available is not a bug for its high value; it’s a feature. Today’s Darwin would type The Origin of the Species in a Google Doc or a versioned LaTeX file... But now with Planck, and the digital scarcity blockchain enables, she would create a unique Glyph [NFT] that serves as that first manuscript copy.

This "science as art" argument received some validation last month when we sold the first scientific result NFT (for $24,000). But we believe that this is just the beginning, and have been testing ways in which funds and proceeds from NFTs can be channeled to improve science funding more broadly. The aim, ultimately, is to enable progress and to try to help save some of the consequential ideas we might otherwise lose.

Within this essay we are auctioning off an NFT as a small test of the SplitStream, and incentive design for funding science through citations. To explain, let us first outline the project this NFT is a part of. Last month, after designing and crowdsourcing the first ever randomized control test of a long-untested theory, Planck sold the result as the aforementioned first science NFT. These funds will help us run a replication of the study we ran. Here is something nice that a very smart person said about the project:

Because we are interested in incentives for consequential ideas that it seemed possible we might lose, we looked for a promising candidate for the test. The most promising candidate we found was the late Berkeley Professor Seth Roberts' appetite theory.

Testing Seth Roberts' Appetite Theory

In 2004, Berkeley Professor Seth Roberts discovered that consuming flavorless calories reduced his appetite. Consuming flavorless or unfamiliar calories made his eating habits more consciously controllable and helped him lose weight. And beyond weight loss, such a tool for conscious eating seemed to have the potential to help people eat more nutritious food, more easily go vegan, etc. Though preliminary, it was an enormous discovery.

Seth was a serious empirical scientist and the evidence he published for his theory was intriguing, if unorthodox. But some very credible sources recognized the merit of his work and he was featured positively in the NYTimes. His book became a bestseller.

Glowing anecdotes appeared, including a notable one from Reddit co-founder Aaron Swartz. In that essay, Swartz confidently alludes to the "clinical trials" that would surely be carried out on Seth's Theory. His essential optimism, his confidence that the world would obviously figure out a way to run a clinical test on something so promising, is striking.

But the lab studies never came.

Within 5 years of its publication Seth’s idea had become little-discussed, never having undergone a single clinical trial. Some important outlets like Marginal Revolution LessWrong, and StatModeling kept the flame alive, but the discussion of Seth's theory all but vanishes:

By 2017, a few years after Seth had passed away, Neurobiologist Stephan Guyenet wrote that he expected to remain forever “puzzled” by Seth’s theory because “some sort of research” on it was “unlikely to ever happen.” Stephan’s answer reads more than a bit tragic; obesity kills millions each year, interest in using conscious eating to help improve the world, as in vegetarianism, is skyrocketing. Failing to follow up on a promising theory in such a world feels a bit like an echo of our tragic failure to follow up on James Lind's scurvy cure.

Why Do We Lose Ideas?

"Show me the incentive and I'll show you the result." - Charlie Munger

James Lind's scurvy cure didn't mesh well with the science of 1747. Science was then 150 years away from discovering vitamins and so, naturally, it would have been hard to understand how it was the vitamin C in citrus that was the cure. The top minds of the day, viewing Lind's results, quickly came to believe that the acid in the juice must be melting some big black scurvy blob that sailors got in their stomachs. "Great!" they said, "let's boil the juice to make it more acidic and work even better!"

Unfortunately, as we now understand, boiling the juice just destroys the Vitamin C content. Which means the boiled "cure" was no cure at all. Now there is nothing wrong about this; it's just good old normal science seeking understanding and trying out hypotheses. And 150 years later this did give us understanding, and that understanding was bigger than a simple cure for scurvy. The problem is that somehow for the intervening 150 years everyone didn't just go back to using the limes that were proven to work.

If Seth’s Theory proves out I think the story will be much the same as Lind's. Theories, like those of Seth Roberts and James Lind, which happen to integrate poorly with existing science of their day are bound to suffer. By failing to integrate into science they lose out on what is surely humanity's best protocol for rewarding, preserving, and developing openly shared knowledge. Patents might offer an alternative, but Seth's and Lind's theories were also hopelessly unpatentable. Citrus fruits are freely available and "flavorless calories" are easier to come by still; like a good scientist, Seth even set about demonstrating that a person could just hold their nose as they ate and get the desired effect. Good luck writing that patent.

Demonstrating How NFTs can Help Fund Unpatentable Science

When we first wrote about using NFTs as a unique way to fund science, Seth's was one of the major motivating cases. Here's how the simplest mechanism was intended to help fund such research:

1. NFTs of high-potential scientific work are sold
2. These NFTs, like a collectible, could grow in stature and value as the scientific work was validated
3. Prospective NFT sales of early scientific results could thereby profitably fund the validation of that research

As mentioned earlier, our current project is bearing this out: Planck crowdsourced the first ever randomized control trial of Seth Roberts' appetite theory, sold an NFT of the result, and are now using the proceeds to run an independent lab replication of the findings.

It's important to keep in mind the bigger picture here though. We hope Seth's theory is true (who wouldn't?), but our fundamental concern regards the incentives themselves. To become a full-fledged scientific truth a theory needs to be exposed to falsification. So it's worth remembering that, while our preliminary results on Seth's Theory are promising, every step along the journey is important, as are the incentives to take those steps.

In this essay, we are introducing a novel incentive structure for scientific funding that uses citations as a channel for funding. We were calling this a "CitationStream" but piggybacking on Jon-Kyle's terminology in his excellent essay, we are going to call it a “SplitStream”. The SplitStream is a system in which portions of an NFT sale may flow to some or all of the works the NFT cites, with those cited works then sending some of their funds further along to the works they cite, and so on.

The SplitStream

You might know the quote “If I have seen further, it is from standing on the shoulders of giants”. You might also know that it’s from Isaac Newton, a giant if there ever was one. So in science, giants tend to look down and see other giants. Citations, when they work, map those dependencies backwards through time.

Citations on citations.

Giants all the way down.

If you send funds through these citations, you get a cascade. A stream of splits. A SplitStream.

Here’s what it looks like:

We ran a test of this with an NFT of an independent data analysis of Seth’s Appetite Theory, selling for $4,200. We suspect this was the first ever funding of an independent data analysis! As a part of this independent analysis we were able to demo a powerful cryptographic technology from Aleo which we believe will be very useful to further open science. We used a Zero-knowledge Proof in Leo to enable independent statisticians to effectively "query" our dataset and see certain results without revealing the data itself.

Independent Analysis is a really valuable thing, and it is a cutting edge Open Science best practice. The analysis was conducted by two esteemed professors of statistics, Harry Crane and Ryan Martin, and thus the NFT was theirs with proceeds directed according to them. We requested that they keep 50% of the NFT sale but they agreed to test the SplitStream and, for the purposes of this incentive test, their analysis “cites” two NFTs:

The NFT of the Open Science Test of Seth’s Appetite Theory (currently owned by Molecule)
The NFT of the Splitstream Protocol (owned by Planck)

Planck then split our proceeds further to what we are citing:

1. Mirror because we are using their amazing platform and tech.
2. This Artwork is Always on Sale, suggested by Simon De La Rouvier as the proper target for the citation of his shared idea of Harberger taxes for NFT ownership (which is an important aspect of our protocol.)

We encourage the recipients of these proceeds, if they are at all substantial, to split back to those they would cite. Let's also note that this is a simple test of the mechanism and certainly does not represent the full range of actual citations we would use. Here is the NFT, an elegant violin plot of the results from the study we ran:

Design Challenges for the SplitStream

As a trained academic, I could hardly discuss a mechanism like the SplitStream without noting some potential pitfalls. Properly done, this has great potential to reward an effective division of labor in for open science and more. But it does not create obligations, or promises, or contracts, or anything like that. Participants are playing, roughly speaking, a live trust game.

And in fact, these incentives cannot become too enshrined and formal because citations are a fragile system. A person’s inspiration is purely their private knowledge and thus it must be shared voluntarily. A system which can't elicit honest attributions will fail.

For example, you could imagine that many people might appreciate being credited as the essential inspiration for a new Nobel-prize winning discovery in physics. Someone with sufficient resources, some prestige-hungry Jeff Bezos, might even be willing to pay for that honor. And then, of course, there’s nothing to stop our Nobel-winning physicist from claiming that her passing conversation with the dear leader Bezos was the fundamental inspiration for everything. Who could say otherwise? This sort of thing is absolutely going to happen sometimes, just as it sometimes happens in academia.

I’m going to call these types of behaviors “Mao effects”: the seeking of favor from those with status or power by over-attributing to them. Mao of course was named the “great helmsman”, the “great leader”, etc. and anything good that happened was often attributed to him. A network overtaken by these Mao effects breaks down, a Ponzi-scheme for those who claim complete originality.

In an evolutionary simulation it roughly looks like this:

These effects can get more subtle as well. I can’t do much more here than sort of gesture at this and signal that I take these ideas seriously (e.g. I actually coauthored a Behavioral Economics literature review called “When Do Monetary Incentives Backfire”). But a simple step toward preventing Mao Effects is observing the following two rules:

1. Favor citing things which are publicly shared.
2. Favor citing things which also cite.

The first condition helps provide a level of transparency and sense-making to the system. So, for instance, you can understand why I cite Mirror because you can clearly see that I'm using their tech. The second condition helps ensure that cited works offer further attribution to the work they've built on. Together these rules help ensure that a would-be Mao openly shares their knowledge while also citing that which contributed to it. More is needed if we are to one day get this right, but we hope it's a start.

Twitter @stephensonhmatt
p1anck.com