Thread
šŸ”„ If you read one thing about AI this week, let it be Googleā€™s leaked doc

ā€œWe have no moat, and neither does OpenAIā€
Full article here: www.semianalysis.com/p/google-we-have-no-moat-and-neither

Some quotes šŸ§µ
Open-source is innovating faster and cheaper than us, or OpenAI, ever can.

Examples:
Large expensive models are still slightly better, but the gap is quickly closing.
Metaā€™s LLaMA leaked, and the open-source community tan with it.
Some innovations are now coming from single people, tinkering on their laptop ā€œin their garageā€
After StableDiffusion it shouldnā€™t have come as a surprise. (Anyone remember DALL-E? many people sure donā€™t..)
Seems like having a lot of money to train large models might cause them to pay less attention to far cheaper methods (inversely, thatā€™s what the tinkerer at home works with)
And maybe those small, cheap, incremental updates can outrun progress on large, expensive models.
To save money, ā€œtinkerersā€ are training on smaller, but better curated data-sets. Outputs from ChatGPT that have been shared by users because they were cool are used.
Highly specialized individuals are making highly specialized products.
And Meta might be benefiting hereā€¦ because the leaked model was theirs, they can now use all the innovation thatā€™s happening on top of it šŸ„²
Is Google about to open-source a lot more stuff? ā€œIf you canā€™t beat ā€˜m, join ā€˜mā€?
And why be secretive? ā€œOpenAI already knows everything we do because they hire our people.ā€
Additional thoughts after 24h:

1. We donā€™t know if the doc is real, but I havenā€™t heard anyone argue with the analysis + we know that there are sometimes well written white-papers circulating within Google.

2. It does conveniently signal to the regulator ā€œwe are not a monopolyā€
3. Talking about regulations: open-source is definitely harder to regulate.

4. Apple and Amazon have been relatively absent in the recent AI race ā€” There have been rumors that Apple has vastly underestimated LLMs potential. Might they now benefit from open-source?
At Cradle.bio we are using AI to design better proteins to ultimately replace traditional farms šŸ„ and factories šŸ­

Read more in this threadšŸ§µ

(we are hiring)

Mentions
See All