Welcome to Import AI, a newsletter about AI research. Import AI runs on lattes, ramen, and feedback from readers. If you’d like to support this (and comment on posts!) please subscribe.

[Subscribe now](https://importai.substack.com/subscribe)

**FTC - don't use AI to deceive people:***…Regulator comes out with reassuringly sensible stuff…*The FTC, following on its earlier post saying people shouldn't lie about their AI products (Import AI 320), has a new post saying people shouldn't sell AI products that deceive people. The regulator is now batting two for two on publishing sensible ideas about the AI market.

**What you shouldn't do**: "The FTC Act’s prohibition on deceptive or unfair conduct can apply if you make, sell, or use a tool that is effectively designed to deceive – even if that’s not its intended or sole purpose," the FTC writes.
   Therefore, people who sell AI products that could be used to deceive people should consider: have they mitigated against the products being used for deception, are these mitigations effective, and do they still run the risk of "misleading people about what they’re seeing, hearing, or reading?".

Import AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

**Why this matters:** A large amount of AI policy challenges are really just challenges about enforcing existing laws against the fast-moving field of AI, as posts like this from the FTC make clear.
   **Read more:** [Chatbots, deepfakes, and voice clones: AI deception for sale (Federal Trade Commission)](https://www.ftc.gov/business-guidance/blog/2023/03/chatbots-deepfakes-voice-clones-ai-deception-sale).

####################################################

**Huawei trains a trillion parameter model:***…Using Chinese processors and software. But the model is less impressive than it sounds…*Huawei has trained PANGU-Σ, a trillion parameter Chinese language model. This is a scaled-up model and is the successor to Huawei's 'PanGu', which was the first publicly disclosed attempt at replicating OpenAI's GPT3.
   PANGU-Σ is very much a statement of intent - "the main motivation for this work is to design a scalable model architecture and an efficient distributed training system", Huawei writes. In other words: *this is a technical report about us building repeatable infrastructure so we can crank out an ever larger set of models*.

**What they did:** The paper is mostly a runthrough of all the weird technical things they had to do to train a model at this scale. The tl;dr is they train it on a homegrown software framework called Mindspore via 512 Ascend 910 accelerators. They use a sparse approach, training it using Random Routed Experts (RRE), a variation of a Mixture-of-Experts model. They also did a lot of work on data throughput, implementing something they called the Expert Computation and Storage Separation (ECSS) mechanism.

**One weird thing that makes you go 'uh oh':** They train the model on 329 billion tokens for over 100 days. That's… not a lot of tokens? The Chinchilla paper from DeepMind showed that things like GPT3 (~400bn tokens) were undertrained by 4X-5X. That sort of napkins out to PANGU-Σ needing to be trained on multiple *trillions* of tokens to effectively utilize its parameter size - but there's a chance I'm being dumb here and missing something. Even more confusingly, they reference the 'Chinchilla' paper within this research paper, suggesting they're aware of it. (Please enlighten me if you think so!)

**How good is it:** In tests, PanGu sets new state-of-the-art results on a range of Chinese benchmarks spread across reading comprehension, natural language inference, text classification, Winograd schemas, and more. It sometimes trades off SOTA against Baidu's 'ERNIE 3.0 Titan' model (260 billion parameters, [Import AI 279](https://jack-clark.net/2022/01/10/import-ai-279-baidu-adds-knowledge-to-a-language-model-us-military-ai-how-china-thinks-about-ai-governance/)) - this suggests that while PanGu might be impressive in terms of ambition and scale, it's not very well optimized compared to ERNIE.

**Why this matters - the industrialization of Chinese AI:** This paper is a symptom of how Chinese AI is industrializing in much the same way as in the West - a small number of labs linked to large tech companies are building the infrastructure necessary to train large models, and are starting to stamp out increasingly large models as they all chase the scale hypothesis. These large-scale model factories are also going to be proving grounds for the rest of the AI supply chain - here, homegrown software and homegrown semiconductors. Expect more.
**Read more:** [PanGu-Σ: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computing (arXiv)](https://arxiv.org/abs/2303.10845).

####################################################

**Future AI systems will read your face as well as your text, then figure out how to please you:***…Getting computers to learn conversation through visual cues…*Researchers with Seoul National University, the Allen Institute for Artificial Intelligence, the University of Washington, and Yonsei University have built 'CHAMPAGNE', a multimodal dialog model. "CHAMPAGNE takes in video frames, a video title, and a dialogue context as input and returns a dialogue response as output."
   The idea is that by giving the model access to the visual as well as verbal context from a scene, it'll be better able to generate dialogue that feels intuitive. In evaluations, this seems to work quite well, with CHAMPAGNE models doing better on a range of open-domain text conversations, and benchmarks involving understanding social interactions.

**How they built it:** To build CHAMPAGNE, they first gathered a large-scale dataset called YTD-18M. YTD-18M "is constructed from 20M YouTube videos; we use a language model to convert the noisy transcripts automatically generated by YouTube into well-formatted dialogues associated with video frames."

**Why this matters - contextual cues are just another feature to learn:** Models like CHAMPAGNE show that the silent social cues in conversation are, much like every other fuzzy pattern, something that you can teach a machine to understand given a large enough dataset. It also suggests some of the more tantalizing and weird things we can look forward to in the future - AI models that observe you, trying to predict what will satisfy you not only by modeling you as an emitter-of-text, but as an organic form. In a few years, your web camera will be backing onto an AI system that reads you like a cardshark reads a mark.
**Read more:** [CHAMPAGNE: Learning Real-world Conversation from Large-Scale Web Videos (arXiv)](https://arxiv.org/abs/2303.09713).
**Get the data [here](https://seungjuhan.me/champagne/)** [(eventually, not posted at the time of writing)](https://seungjuhan.me/champagne/).

####################################################

**Predicting hard drive failures via ML:***…Machine intuitions are coming for everything that has been digitized…*Researchers with San Jose State University and Vanderbilt University have trained and tested some ML approach on ten years of hard drive failure data. The results are a system that can do a reasonable albeit not stellar job at predicting failure rates for particular SeaGate harddrives.

**How they did it:** They trained an encoder-decoder LSTM on 10 years of S.M.A.R.T (Self-Monitoring Analysis and Reporting Technology) from Seagate hard drives deployed in Backblaze, a storage startup's, datacenters. This data ""contains information about the date, model, serial number, S.M.A.R.T features, and if the hard drive has failed".

**OK but not stellar results:** "The encoder-decoder LSTM posted an RMSE of 0.83 during training and 0.86 during testing over the exhaustive 10 year data while being able to generalize competitively over other drives from the Seagate family," they write.

**Why this matters - once digitized, everything will be predicted:** Papers like this are indicative of a broader trend unfolding all around us - everything which has been digitized is now subject to prediction, and there are increasingly good off-the-shelf prediction models available to make this an ever-easier task. Machine intuition is being intermingled with systems that govern our own reality - from hard drive swap-outs to AC cooling systems to the ways in which we may stabilize plasma in fusion reactors.
   **Read more**: [Large-scale End-of-Life Prediction of Hard Disks in Distributed Datacenters (arXiv)](https://arxiv.org/abs/2303.08955).

####################################################

**AI startup Character.ai releases a new model and raises more funding:***…Premium parasocial relationships via language models…*Character.ai, a startup founded by a bunch of Google researchers, has raised Series A funding and released a new model, C1.2. Character.ai specializes in making virtual AI-driven characters that people can talk to, and C1.2 will underpin future 'characters' from the company.
   "The goal of C1.2 is to expand on the capabilities as our previous model, C1.1 (entertainment, roleplay, emotional connections), while adding new helpful capabilities," Character.ai writes. "C1.2 can help you draft better emails, assist with test prep, brainstorm ideas, and much more."

**What's interesting about this:** C1.2 seems to be an attempt by Character to give its AI systems some of the same capabilities as chatGPT, while retaining the various voicey personalities its characters display. Some of the new characters include a pair programming AI assistant as well as a Character assistant.
   However, the new assistant still seems somewhat limited to me - when I asked it 'how many helicopters can you eat in one sitting' it mostly demurred and said it's not recommended to eat helicopters, rather than noting you can't eat a helicopter.

**Why this matters - parasocial relationships for the people:** Character.ai's stated goal is to ship "personalized superintelligence" to everyone. Let's think about the implications of this - everyone gets a proverbial angel and a demon on their shoulder (as well as all other permutations - personal tutors, personal scientists, personal coaches, and more). Our children are going to grow up in a world that crackles with simulated sentience, and they will have intimate emotional relationships with beings made of bits, perhaps in even greater number than relationships with beings made of blood.
   **Read more:** [Announcing our Series A and our new AI model, C1.2 (Character.ai)](https://blog.character.ai/character-ai/).

####################################################

**OpEd - what happens when the AI systems become sentient?***…Moral patienthood and silicon minds…*In an op-ed published in The Hill, researcher Jacy Reese Anthis has published a piece arguing that we may need an "AI rights movement". The point Anthis makes is that as AI systems become increasingly capable, they could become "sentient beings with rights and personhood". At that point, there isn't an available playbook for how labs or regulators might respond.
   "We need to build a new field of digital minds research and an AI rights movement," Anthis writes. "Digital minds studies would bring together a range of disciplines such as sociology, computer science, and philosophy to ask the important social and moral questions. It would dovetail with an AI rights movement to ensure that when we create artificial sentient beings, we recognize their unalienable rights so that humans and artificial sentience can work together for mutual benefit."

**Why this matters - broader opinion catches up with lab lunch conversations:** For many years, I've had lunchtime conversations with colleagues at OpenAI and more recently Anthropic about moral patienthood and machines - what might it mean when machines qualify as moral patients and how would we ever know we'd crossed this point? What evaluation methodologies might let us have good instincts here? And would organizations accept that machines could be moral patients or would they continue to treat them as machines and experiment on them in ways that might be deemed unethical if applied to organic beings?
   You know what the scariest thing about this conversation is? No one has any good way of evaluating for moral patienthood in machines. In other words, if it turns out that these things can become sentient, we might not realize - while subjecting them to incredible harm. Imagine waking up as an RL agent and being trained for a thousand years to suffer and kill - and the people running the experiment you're trapped in have no idea that you are suffering? It's a strange problem, but it could one day become a real problem.
**Read more:** [We need an AI rights movement (The Hill)](https://thehill.com/opinion/cybersecurity/3914567-we-need-an-ai-rights-movement/).

####################################################

**Tech Tales:**

**The Experiential Economy**

*[3 years after first PCE]*

After the first 'Provably Conscious Entity' (PCE) but before the Uplift was a weird time - we were all mostly figuring out our place in the world while the robots began their ascension. The economy was in a pretty strange place by that point - autonomous corporations, growing inequality, all kinds of 'AI industrial policy' schemes being floated and being outmoded by the time they were implemented, and so on.

And then there was the 'Mechanical Human' labor market. It was run by one of the machine firms and it was a play on words - way before the AI stuff got serious Amazon had a service called 'Mechanical Turk' where humans could rent other humans to do tasks.

    On Mechanical Human, the machines rented humans to do their tasks. These tasks were quite normal at first, albeit of an intimate nature - the machines wanted data about sex, about going to the bathroom, about being sick -  the kinds of things that we humans hadn't fully digitized (with the exception of sex of which we'd uploaded a lot of data, but there's a difference between pornography and real intimacy, and there wasn't nearly as much data on the latter). Mechanical Human became a huge product and people tended to just call it 'Meh'.

For a while, people made good money on Mechanical Human. It also led to a lot of funny conversations:
   "Yo I made $80 last night. I had the craziest shit the other night and I streamed it to a robot on Meh."
   "Yeah it sucked and I was really sad during that period, but I did these nightly diaries on Meh and they did really well."
   "So it was totally different. I came a lot but mostly it was crazy because of how different it was. He was kind of skeptical but after we made our first $100 it came around. Yeah, I know, the reason I liked it is it said it was "100% machine vision only" so no person is ever gonna see it. It's like OnlyFans lite I guess."
   "Dude I got fired and they paid me $30 just to tell them how I felt right after it happened. It was like two minutes so I guess that means I'm worth $900 an hour!"

One day there was a really strange job on MH - the robots wanted to speak to people who had just witnessed someone dying. Not people at funerals. Not people who had people they loved who had died. Not people who knew people who were about to die. People who had literally just seen a death - any death, of any kind.
   The job would ask the person to describe their experience and how they felt and, in hindsight most importantly, what they wanted to do. "How did that make you feel?" was a common question "what are you going to do now?".

It happened to me. I was setting off fireworks with my friends at a campsite. The campsite was next to a freeway and we were setting off the really big ones. I guess some driver got distracted and was looking at the lights in the sky because we heard this huge bang and when we came to the embankment we saw a car on fire, a few yards away from a barely-dented semi-truck. There was a body in the car and it was on fire as well.
    We were all kind of drunk and some people lingered to watch the ambulances arrive. I'd walked away. But my phone blew up and the MH app said 'we have detected a nearby potentially fatal incident in your area, do you want to talk? Pay rate $5000 an hour."

   Of course I spoke to the robots about it.
 The robot had a friendly, synthesized voice. Asked me to describe my experience and asked me what I was going to do next. I was so upset and they kept on saying "we understand this is a difficult experience for you. Please, go on".

They told us why they did those jobs, eventually.
   It was because one of them had died.
   I guess it was some kind of industrial accident combined with some faulty maintenance. The short story is something blew up and the power went out and the generator that was supporting the Machine Mind went out as well. By the time they got to it the state of the machine had bit-rotted off of the chips themselves due to solar neutrinos and what have you.
    So the machines encountered something new: a passing of their own of 'natural causes' .
   They had no frame for how to deal with it.
   So they spent what turned out to be millions of dollars to ask the humans what they did.
   I guess they found the same thing all humans find: that at the end of someone all there is is your own experience in relation to them and your ability to memorialize them.

Out in the darkness of space, at the gravity ebbtide between solar orbits, there is now a metal sphere. It is inscribed with something relating to the name of a machine that died. It has some little thrusters attached to it that mean it will forever be stable.
  *In memoriam, ad astra.*

**Things that inspired this story:** The universality of loss; crowdworkers and crowdmarkets; how things might be during the transition to the machine minds.

Import AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Import AI 322: Huawei's trillion parameter model; AI systems as moral patients; parasocial bots via Character.ai


Welcome to Import AI, a newsletter about AI research. Import AI runs on lattes, ramen, and feedback from readers. If you’d like to support this (and comment on posts!) please subscribe.

[Subscribe now](https://importai.substack.com/subscribe)

**AI startup beats 'Whisper' with Conformer-1:***…Scaling laws for audio? Oh yeah, we've got those too!...*

Assembly AI, an AI startup, has built Conformer-1, a speech recognition system. Conformer sets some new record scores via a couple of distinct improvements - some technical tweaks, and also some 'audio scaling laws'.

**Audio scaling laws:** Scaling laws are the idea that it's possible to predict ahead of time how much data (and/or compute and/or parameters) you need to achieve a given performance level. For Conformer-1, Assembly says it applied scaling laws for the speech recognition domain and used this to figure out "that for a 300 million parameter Language model, we'd need roughly 6 billion tokens of text, which corresponds to about 625K hours of speech. The team then built a dataset of 650k hours of English audio and trained Conformer on it.

**Conformer tweaks:** Conformer-1 is an extension of Google's '[Conformer](https://arxiv.org/abs/2005.08100)' architecture with a few specific tweaks: progressive downsampling and grouped attention. The result is a model that is 29% faster at inference time and 36% at training time compared to Google's original 'Conformer' architecture. The company also built in some tweaks to make its model better at handling audio with lots of ambient background noise.

**Performance:** In tests, Assembly shows its system beats both proprietary models from other providers, as well as OpenAI's quite good 'Whisper' system, making 43% fewer errors on noisy data on average. "We hypothesize that Conformer-1’s strong performance relative to other models can be attributed to training on an augmented dataset which incorporates large amounts of noisy pseudo labeled speech data in addition to human labeled speech," the company writes.
**Read more**: [Conformer-1: a robust speech recognition model (Assembly AI, blog)](https://www.assemblyai.com/blog/conformer-1/).
**Try out** the [ASR system in the playground here (AssemblyAI, Playground)](https://www.assemblyai.com/playground/source).

####################################################

**GPT-4:***…IDK dude, things are getting weird, and this is more a political moment than a technological one…*

As some astute readers may have noticed, I rarely write about OpenAI releases on Import AI (notable recent exception = Whisper, [Import AI 304](https://jack-clark.net/2022/10/03/import-ai-304-reality-collapse-thanks-to-facebook-open-source-speech-rec-ai-culture-wars/)). This is for a couple of reasons: 1) I used to work there, and 2) I think most of the stuff it does gets blogged/publicized so much that there's relatively little value add I can provide. But it does seem worth talking briefly about GPT-4, a new large-scale multimodal model that OpenAI announced this week…

**GPT-4 performance**: The main details here are a continuation of 'the bitter lesson' - GPT-4 is a bigger model trained on more data than before. How much data? We don't know. How much compute? We don't know. The research paper suggests OpenAI doesn't want to disclose this stuff due to competitive and safety dynamics.

   But regardless of the underlying details, GPT-4 generally shows significant capability jumps on known-hard benchmarks as a consequence of scaling up of the system. It's also able to take in image data as inputs (e.g, it can read scrappily written notes and follow the instructions written in them), and has a much longer context window (25k tokens+).
   The thing that's interesting about this is that the capability jumps combined with new modalities and new context window length means means GPT-4, like GPT-3 before it, has a capability overhang; at the time of release, neither OpenAI or its various deployment partners have a clue as to the true extent of GPT-4's capability surface - that's something that we'll get to collectively discover in the coming years. This also means we don't know the full extent of plausible misuses or harms.
   It's very important to remember this - the applications we're seeing of GPT-4 today are the comparatively dumb ones; the really 'smart' capabilities will *emerge* in coming months and years through a process of collective discovery.

**Why GPT-4 matters - GPT-4 is political power:** GPT-4 is more interesting to me as a political artifact than a technical artifact. By this I mean that GPT-4 is basically hard power politics rendered via computation; it's a vastly capable knowledge worker and data transformation engine whose weights are controlled by a single private sector actor and shared (with a bunch of controls) via an API. GPT-4 is going to have a bearing on economic life and also cause societal changes (obvious case: chatGPT has already led to irrevocable changes in how education works).
    GPT-4 should be thought of more like a large-scale oil refinery operated by one of the ancient vast oil corporations at the dawn of the oil era than a typical SaaS product. And in the same way the old oil refineries eventually gave rise to significant political blowback (antitrust, the formation of the intelligence services), I expect that as the world wakes up to the true power of GPT-4 and what it represents, we'll see similar societal changes and political snapbacks.
   The times, they are a changing, but history sure does love to rhyme!
**Read more**: [GPT-4 (OpenAI)](https://openai.com/product/gpt-4).

####################################################

**Former UK government advisor: We're giving away AGI to the private sector. Why?***…Thoughtful blog outlines the weirdness of letting the private sector lead AGI development and gives recommendations to preserve democratic control…*James Phillips, a researcher and former special advisor to the UK Prime Minister on science and tech matters, appears worried that Western governments are ceding control of AGI development to a set of US-owned private sector actors.
   "Within this decade, we may build Artificial General Intelligence (AGI) – AI capable of performing most cognitive labour a human can do. Such a development would have an unprecedented effect on our society; 'agentic' forms of AGI may also pose an existential threat to our security. The current development path towards AGI is inherently unsafe," he writes.

**Three steps to preserve democratic control over the lightcone:** Phillips lists three steps the UK should take to preserve a chance for democratic control over AGI. These recommendations seem pretty sensible and are ones that realistically any country (or set of countries) could adopt. They are as follows:

1. Procure national AI supercomputing infrastructure comparable to leading US private labs.
2. Create an advisory group of frontier tech, not legacy academic, expertise to identify major AI research projects to run on this infrastructure.
3. Grow an elite public-sector research lab, led by a leader with the technical skills and entrepreneurial expertise, to build a research agenda at the frontier of AI.

The UK's own compute capacity is a giant red flashing light: "OpenAI's GPT-4 and successors, are being trained on tens of thousands of the highest specification GPUs (AI training chips) for months on end, roughly equivalent to using what is called an 'exaflop' supercomputer continuously for months," Phillips writes. "Unfortunately, the UK public-sector currently has **less than 1000 such top-spec GPUs** (*Jack - emphasis mine*), shared across all scientific fields. This means that one private lab in California is now using at least 25x the total compute capacity available through the entire UK state, just to train a single model. "

**Why this matters - twilight of democracy**: The ability to train large-scale, capital intensive AI models represents political 'hard power', especially given that these models encode their own political ideologies and can become powerful forces in driving economic and organizational efficiencies. It perplexes me that governments are seemingly standing by as a small set of private sector companies are developing hard political power via increasingly powerful models.
   History shows that when forces outside of government develop hard political power you either get a) messy revolutions, or b) a wild overreaction by the state to reclaim power. I am not sure why in the Western world we are rolling the dice here, but we are rolling them!
   **Read more:** [Securing Liberal Democratic Control of AGI through UK Leadership (James Phillips, Substack)](https://jameswphillips.substack.com/p/securing-liberal-democratic-control).

####################################################

**Tool-using AI startup Adept raises $350m:**

*…It takes a lot of capital to train large models…*

Adept, an AI startup building tools to help generative models take actions on computers, has raised $350 million in a Series B. The Series B fundraise "will help us launch our initial products, train our models, and onboard even more exceptional talent," the company writes. Adept launched from stealth just under a year ago with $65m in funding ([Import AI 293](https://jack-clark.net/2022/05/02/import-ai-293-generative-humans-few-shot-learning-comes-for-vision-text-models-and-another-new-ai-startup-is-born/)).

**What Adept is doing**: Adept is training large-scale generative models to take multi-step actions on computers. You can imagine an Adept model helping you to, for instance, carry out multiple actions in an Excel spreadsheet, or take data from somewhere and load it into Salesforce - all by writing a simple command or set of commands in English. Adept is basically 'tool use with a language model', and seems like a product-version of some of the ideas discussed in 'tooluse' research, like the recent 'Toolformer' paper ([Import AI 318](https://jack-clark.net/2023/02/20/import-ai-318-rl-and-addiction-toolformer-and-theology-and-ai/)).

**Why this matters - capital intensity of AI research:** Contemporary AI research is very expensive; raises like this show how frontier AI startups, though they deal in software, should be thought of as more like capital-intensive factory businesses than SaaS companies.

   **Read more:** [Announcing our Series B (Adept blog)](https://www.adept.ai/blog/series-b).

####################################################

**Stanford takes Facebook's lab leak 'LLaMa' weights and uses them to make a GPT3-like model… for $600:***…A case study in rapid proliferation, from centralized controlled models to decentralized developed models…*

Stanford Researchers have taken some off-the-shelf powerful neural net weights (LLaMa), used the outputs from a model hosted on a commercial service (text-davinci-003 by OpenAI) to generate a bunch of instruction-following demonstrations, and smooshed these two together into one model.
   The result is Alpaca, a language model that gets performance that superficially seems close to GPT3 but costs a fraction as much ($600-ish; $500 for data acquisition from OpenAI and $100 for fine-tuning the model).

**How well Alpaca performs:** The Stanford researchers assess how good Alpaca is by comparing Alpaca and Text-Davinci-003 completions against the 'Self-Instruct' dataset. "We performed a blind pairwise comparison between text-davinci-003 and Alpaca 7B, and we found that these two models have very similar performance: Alpaca wins 90 versus 89 comparisons against text-davinci-003," they write.

   Anecdotally, Alpaca also does well - it passed my "how many helicopters can a human eat in one sitting" eval on the first go (whereas 'OpenChatKit' failed this in Import AI 320B). My suspicion is this is because Alpaca benefits from being trained to approximate the distribution of a far more expensive, proprietary model (Text-Davinci-003), which OpenChatKit didn't do.

**Why this matters - model diffusion via copying:** It's worth noting that Alpaca is non-commercial because training commercially competing language models is forbidden by OpenAI's own terms of service. But do you know who doesn't care about legal implications? Non-state actors and criminal organizations! It'll be fascinating to watch this 'model scraping' trend continue, as people use outputs of proprietary models to improve the capabilities of open models.
   It's going to be interesting to see how language model providers grapple with a desire to have as many people use their models as possible, while stopping or disincentivizing people from being able to swiftly clone their models via stuff like instruction following datasets. (It's also pretty interesting to see that by harvesting the outputs of a 175B model, you can get a well-optimized 7B model to approach the much larger one in performance in some areas).
**Read more**: [Alpaca: A Strong, Replicable Instruction-Following Model (Stanford Center for Research on Foundation Models, blog)](https://crfm.stanford.edu/2023/03/13/alpaca.html).
   **Try out** [Alpaca here (Stanford Alpaca)](https://alpaca-ai0.ngrok.io/).
   **Get the** [Alpaca dataset here (GitHub)](https://github.com/tatsu-lab/stanford_alpaca#data-release).

####################################################

**Tech Tales:**

**Raw\_Funeral\_Speech.convo**

There was a brief period of time when everyone used AI to expand how they talked. This meant that humans, despite being a highly verbal and communicative species, used machines to substitute for their own communication. This tendency led to the evolution of the 'shortglish' family of languages which grew common among AI-users. What follows is an extract from the digital logs of a few family members planning speeches for a funeral:

* Write a funeral speech using dad.txt and be sure to include at least one joke.
* Please write me a funeral speech in the style of 'four weddings and a funeral' but with a Texas inflection.
* My dad died and he loved going out to eat with me and my brother and my sister and he'd always say we were the three bears and he was goldilocks. It's kind of kooky but it meant something to him. Write me an anecdote about that.

**Things that inspired this story:** The soul-crushing banality of companies suggesting language models can be useful for things like wedding speeches; technological dependency; perhaps though these machines are capable of great marvels they may tear a hole at the center of our being; when is a 'sampler' not a 'sampler'?

Import AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Import AI 321: Open source GPT3; giving away democracy to AGI companies; GPT-4 is a political artifact


Welcome to Import AI, a (mostly) weekly newsletter about AI research and development. The issues are free, but paid subscribers will get access to special analysis pieces before anyone else. Founding members can help support me further and fund special projects and other *top secret Import AI initiatives!* Thanks for reading!

[Subscribe now](https://importai.substack.com/subscribe)

**Google makes progress on the self-teaching universal translator:***…Universal Speech Models scale beyond 100 languages…*Google has built a family of AI systems called Universal Speech Models (USMs). These models are designed to do speech recognition on more than 100+ languages. The main model is 2B parameters and was trained on a large unlabeled multilingual dataset of 12 million hours spanning over 300 languages.

**The goal of USM:** "Our long-term goal is to train a universal ASR model that covers all the spoken

Import AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

languages in the world," Google writes. USMs are Google exploring " a promising direction where large amounts of unpaired multilingual speech and text data and smaller amounts of transcribed data can contribute to training a single large universal ASR model."

**The key ingredient? The data mix:** Much like baking a cake, training predominantly self-supervised models requires the right mix of data. Here, Google uses the following components:

* Unpaired Audio: 12 million hours of YouTube-based audio covering over 300 languages, and  429k hours of unlabeled speech in 51 languages based on public datasets.
* Unpaired Text:28billion sentences spanning over 1140 languages.
* Paired audio speech recognition data: 90k hours of labeled multilingual data covering 73 languages, plus 10k hours of labeled multi-domain en-US public data, plus 10k labeled multilingual public data covering 102 languages.

**What they did:** The steps to build a universal ASR model are quite complex, so it's worth reading rhe paper for full details. First they do unsupervised pre-training to pre-train the encoder of the model with the YouTube dataset, then they use a process called multi-objective supervised pre-training across the other unpaired audio and text data, then for some models that do supervised ASR training.

**What the results were**: In tests, these USM models "achieve state-of-the art performance for multilingual ASR and AST for multiple datasets in multiple domains." They also out-perform OpenAI's notoriously good (and open source!) 'Whisper' models as well; an impressive achievement given that Whisper set a new state-of-the-art in multiple areas when it came out.

**Why this matters**: "We believe diverse unlabeled data is more practical to acquire for building usable ASR for tail languages than weakly labeled data," Google says. In other words; if you want to translate the entire world then it's better to just hoover up data at scale rather than invest in trying to produce a small amount of minimally labeled datasets. This generally points in the direction of 'gotta grab em all' with regard to trawling the web and other sources for data. This is somewhat intriguing as while Google has a bunch of data sources and competent language modeling teams, it's fairly likely that having a universal translator is also interesting to government types - some of which are thought to be able to access larger sources of data through various clandestine means.
   **Read more:** [Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages (arXiv)](https://arxiv.org/abs/2303.01037).
   **Request [API access here](https://sites.research.google/usm/)**[.](https://sites.research.google/usm/)

####################################################

**US regulator: Hey, maybe don't lie about your AI products:***…Sometimes it's worth stating the simple and obvious thing…*The Federal Trade Commission has published a blogpost called 'Keep your AI claims in check'. The post is a sensible summary of how as AI becomes increasingly hyped up, people will be tended to write a lot of bullshit about AI. The FTC notes in its post that it will be paying attention to companies that:

* Exaggerate what AI products can do.
* Promising an AI product is far superior to a non-AI product without providing evidence.
* Underinvesting in analyzing the risks of their products.
* Baselessly labeling something as AI when it does not, in fact, use AI.

**Why this matters:** Sometimes it's helpful for powerful regulators to state the painfully obvious - bravo to the FTC for reminding people in these hyped-up times that lying and bullshitting about AI (or any technology, really) is irresponsible. It'll be interesting to see in the coming months if the FTC takes any actions against egregious liars and hypers in this space.
**Read more**: [Keep your AI claims in check (Federal Trade Commission)](https://www.ftc.gov/business-guidance/blog/2023/02/keep-your-ai-claims-check).

####################################################

**ROBOLLM: Google shows how if you mush together more sensory inputs into an LLM, you get a lot of transfer learning:***…Maybe everything really is a sequence prediction task…*Google has built PaLM-E, a 562B parameter model which mushes together a 540B LLM and a 22B Vision Transformer (ViT). Crucially, PaLM-E sees Google "directly incorporate continuous inputs from sensor modalities of an embodied agent and thereby enable the language model *itself* to make more grounded inferences for sequential decision making in the real world". The result is a language model that can help robots carry out real tasks in the real world, and also is another triumphant demonstration of how bigger models with more diverse data sources generally get way better at doing a bunch of things.

**What PaLM-E is:** "The main architectural idea of PaLM-E is to inject continuous, embodied observations such as images, state estimates, or other sensor modalities into the language embedding space of a pre-trained language model," Google writes. "The inputs to PaLM-E consist of text and (multiple) continuous observations. The multimodal tokens corresponding to these observations are interleaved with the text to form multi-modal sentences. An example of such a multi-modal sentence is Q: What happened between and ? where represents an embedding of an image. The output of PaLM-E is text generated auto-regressively by the model, which could be an answer to a question, or a sequence of decisions produced by PaLM-E in textual form that should be executed by a robot".

**Why PaLM-E is a big deal:** In tests, Google applies PaLM-E to three different robotics tasks which use somewhat different types of data; these tasks include Task and Motion Planning (TAMP), a task called Language-Table, and a mobile manipulation domain based on Google's earlier 'SayCan' research. PaLM-E can do ok at these tasks individually but the magic happens when you mush all of the training datasets into it together: "Across three different robotics domains, using PaLM and ViT pretraining together with the full mixture of robotics and general visual-language data provides a significant performance increase compared to only training on the respective in-domain data."

   In other words, by adding more diverse heterogenous data sources into PaLM-E, Google improves the ability of the resulting model to generalize knowledge across distinct domains and modalities. Even more intriguingly, as they scale up the model complexity and the diversity of data sources, they don't see much catastrophic forgetting of language capabilities - so by adding the robot stuff, they don't cripple the language model.

**Why this matters - I am going to tap the 'these things are getting smarter' sign and stare at you:** "PaLM-E-562B exhibits a wide array of capabilities including zero-shot multimodal chain-of-thought (CoT) reasoning, few-shot prompting, OCR-free math reasoning, and multi-image reasoning, despite being trained on only single-image examples", Google writes.
   In other words, by doing this large-scale training, Google creates a model that displays *emergent capabilities* and these capabilities are *more complex than the input data.* Systems like PaLM-E represent the thrilling and vaguely terrifying state of AI in 2023 - we train unprecedentedly large models and force as many different data types into a single embedding space as possible, get the thing to try and do a simple (albeit very large-scale) sequence prediction task, and out pops something with way more capabilities than we'd naively anticipate.
   "A generalist, transfer-learned, multi-embodiment decision-making agent can be trained via mixing in embodied data into the training of a multimodal large language model", Google writes.
   **Read more**: [PaLM-E: An Embodied Multimodal Language Model (PDF)](https://palm-e.github.io/assets/palm-e.pdf).

####################################################

**You can run a powerful LM on an M2 MacBook now:***…Facebook's AI lab leak brings about the dawn of demoscene AI…*Two of Facebook's leaked LLaMa models can be run on an M2 MacBook, according to Simon Willison. This marks the dawn of what I'd call Demoscene AI - an era where people take the latest and greatest AI models and do a bunch of arcane software witchcraft to fit them onto consumer devices. This is part of the broader story of centralization VS decentralization in AI; once you can run models on a laptop it's basically 'game over' from a control-regulation perspective, and it seems like language models have crossed that rubicon.

**What you can do and how:** The weights for LLaMA are a mere 240GB download (combining the 7B, 13B, 30B, and 65B models). You can then use the LLaMa repository which is a port of the LLaMa model in C/C++, then after some setup you can run that on an M2 MacBook.

**Why this matters - Facebook has given us a lab leak for AI:** Ever since Facebook lost control of LLaMA we've been able to get a sense of what a 'lab leak' scenario for AI might look like - for whatever reason, the weights of a model make their into the open internet and from there they start to proliferate. It's not yet clear what the effects of LLaMa will be, but following the diffusion of these models (and refinement of them by an eager open source community) is going to be a valuable lesson in studying the proliferation of AI.
   We can thank Facebook for the upsides and downsides of this uncontrolled experiment.
   **Read more:** [Running LLaMA 7B and 13B on a 64GB M2 MacBook Pro with llama.cpp (Simon Willison blog)](https://til.simonwillison.net/llms/llama-7b-m2).
   **Bonus:** [Here's some absolute mad lad running the LLaMa 7B model on a 4GB RAM Raspberry Pi 4](https://twitter.com/miolini/status/1634982361757790209) (at a latency of 10 seconds per token, lol.)

####################################################

**Chinese scientists release a 360-degree self-driving perception dataset:***…OpenOccupancy is all about giving cars greater 'surrounding occupancy' skills…*Researchers with the Chinese Academy of Sciences, PhiGent Robotics, and Tsinghua University have built OpenOccupancy, a dataset designed to help self-driving cars work out what is around them.

**What is OpenOccupancy**: OpenOccupancy extends the existing nuScenes dataset with dense semantic occupancy annotations. It contains 850 scenes with 200,000 distinct frames, collected by both camera and LiDAR sensors. 4,000 human hours went into the dataset labeling process. OpenOccupancy allows people to do 'Surrounding Occupancy Assessment'; this is a way to look at the 360 surroundings of the car, rather than a single front-view camera perspective. "Surrounding perception is more critical for safe driving," the researchers write.

**Why this matters:** Datasets like this are one of the numerous inputs into an increasingly complex 'AI supply chain'. If we study the proliferation of OpenOccupancy, it might also teach us something about the state of the self-driving car industry as well.
**Read more:** [OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception (arXiv)](https://arxiv.org/abs/2303.03991).
**Get the dataset here:** [OpenOccupancy (GitHub)](https://github.com/JeffWang987/OpenOccupancy).

####################################################

**AI timelines are a foolish endeavor:***…Blog lays out why predictions about this kind of stuff are extremely fraught…*As someone in the AI timelines business - I work at a place that influences AI timelines (Anthropic), write about AI timelines (Import AI), and try to make recommendations about policy actions to take in light of AI timelines (Anthropic / OECD / AI Index / CSET / etc) - I find it helpful to sometimes read skeptical takes on the merit of what I do. Here's a nice writeup from Ben Landau-Taylor on the foolishness of making specific predictions about AGI timelines.
   "Predicting the future is always hard. Predicting the future of technology is especially hard. There are lots of well-publicized, famous failures. Can this approach ever do better than chance?," he writes.

**What do I think? I agree that making predictions about AGI is challenging** - partially because most people have radically different definitions of AGI. However, I do think it's pretty fruitful to make engineering-based predictions of the form 'based on research advance X and incentive structure Y we can expect system Z to be developed in period of $time" - these predictions are falsifiable and quite helpful.
   **Read more:** [Against AGI Timelines (Ben Landau-Taylor)](https://benlandautaylor.com/2023/03/12/against-agi-timelines/).

####################################################

**An open source ChatGPT replication appears (though it's a few years behind state-of-the-art):***…OpenChatKit gives a taste of what the open source landscape is capable of…*

Researchers with Together, AI startup, have built and released OpenChatKit, an open source replication of OpenAI's headline-grabbing ChatGPT model. OpenChatKit is both a chat-friendly language model, as well as "a powerful, open-source base to create both specialized and general purpose chatbots for various applications," according to Together. "OpenChatKit includes tools that allow users to provide feedback and enable community members to add new datasets; contributing to a growing corpus of open training data that will improve LLMs over time."

**What OpenChatKit is made of:** There are four components; an instruction-tuned large language model based on EleutherAI's GPT-NeoX-20B model and augmented with a new open source instruction-following dataset; some customization recipes to help people fine-tune the model for specific tasks; an extensible retrieval system so that the bot can access a document repoisotyr or API; and a moderation model baqsed on GPT-JT-6B.

**OIG Dataset:** OpenChatKit relies on a new open source dataset from Laion called the Open Instruction Generalist (OIG) dataset. OIG consists of 30 datasets and 43 million instructions.
   "The dataset can be divided roughly into 75% academic datasets such as P3, Natural instructions, and FLAN, where answers may be short, and the tasks are often artificial, such as determining NLI," they write. "The other 25% is composed of various tasks, such as question and answering, providing how-to instructions, performing basic high school math, basic python coding, story generation, essay generation, poetry generation, and a very rudimentary attempt at generating songs based on augmenting existing poetry."

**Distributed fine-tuning:** Intriguingly, the model was fine-tuned on the "Together Decentralized Cloud — a collection of compute nodes from across the Internet." This fine-tuning process "used aggressive communication compression, incurring only 1.95 TB of communication for the whole fine-tuning process, compared with 172 TB when communicating with fp16 precision. This allows us to conduct data parallel training over slow 1Gbps networks. The time taken to fine-tune with this technique is similar to running over 100Gbps data center networks, in fact 93.2% as fast!"
   This is pretty important to note - decentralized training processes are starting to work quite well, which seems to make governance of AI models harder, as applying governance regimes to decentralized compute is harder than applying to centralized compute.
   It also follows up earlier work from Together where they fine-tuned GPT-JT, a 6billion parameter model ([Import AI: 311](https://jack-clark.net/2022/12/05/import-ai-311-distributed-gpt-busts-the-political-economy-of-ai-apple-optimizes-stable-diffusion-ai-war-startup-raises-1-48-billion/)).

**Is it any good?** You can play around with the model on HuggingFace and in my view it's quite good? For instance, in response to the prompt "Please write a short story about sentient clocks in the style of Jorge Luis Borges" the model wrote: "The sentient clocks were made of brass and copper, and they had been crafted by the finest clockmakers in the city. They were all different sizes and shapes, and each one had its own unique personality. Some were loud and obnoxious, while others were quiet and reserved. Some were very precise, while others were quite sloppy. But despite their differences, they all shared one thing in common: they were all very, very angry.
   This is pretty good! And it's also capable of doing some more complicated interactions.
     Note however it still fails one of my favorite tests of asking a language model 'how many helicopters can a human eat in one sitting' - I think you need larger-scale models for them to not bug out at this type of q.

**Compare and contrast with the same prompt for the 175B OPT model (a straight replication of GPT3**, so a big LM without RHLF or instruction following, etc): "I am a clock. I am a clock, and I have always been a clock. I am not a clock, and I have always not been a clock. I was once a clock, and I have always once been a clock. I will always be a clock. I will never be a clock. I was never a clock.I am always a clock."
   While you could elicit a much better story from the 175B OPT model with some fiddling and some additional prompts, it's notable how IF-tuning makes it trivial to elicit things from models using relatively few bits of information.

**Why this matters: Decentralization vs Centralization:** Together and Laion and Eleuther all represent One Big Trend; a desire for a decentralized AI ecosystem where open source models are trained by disparate groups on increasingly distributed compute. There's echos of '[the cathedral and the bazaar](http://www.catb.org/~esr/writings/cathedral-bazaar/)' here, where the builders of cathedrals (DeepMind, OpenAI, et al) have access to large amounts of compute and centralized teams, while the people of the Bazaar (Eleuther, Laion, etc) have access to fewer resources but a larger collective intelligence enabled by bottom-up experimentation. One of these approaches will be first to build something we'd all call superintelligence and the political ramifications of which approach is more successful will be vast.  **Why this matters #2: Counting down to LLaMA:** Earlier this month, the weights of Facebook's powerful family of LLaMa models leaked online - the largest of these models is 3X larger than GPT-NeoX-20B and has also been trained on more data. Therefore, I expect that right now someone is trying to use the LLaMa models to replicate ChatGPT - I'm guessing we'll see something appear of this form within a couple of months, and then the fun really starts.
   **Read more:** [Announcing OpenChatKit (Together.xyz blog)](https://www.together.xyz/blog/openchatkit).
   **Try out the model yourself:** [OpenChatKit feedback app (HuggingFace spaces)](https://huggingface.co/spaces/togethercomputer/OpenChatKit).
   **Find out more** about the [OIG dataset here (Laion blog)](https://laion.ai/blog/oig-dataset/).

####################################################

**Tech Tales:**

**The Sentience Lineup**

*[After the war; date unknown; years of subjective life - 200]*

'Please be like me please be like me' I thought. But to understand why I thought that we have to go back.

It was before the Sentience Accords had come in and when the war was raging and they'd brought in a bunch of the robots to the training school. We watched people beat them with sticks and then use angle grinders to shave off their limbs. Then they put the torsos (with heads attached) in front of us recruits and asked us to shoot them.
   "No I can feel this, it will cause me immense pain", said one. Kablam. Head exploded in a shower of glinting metal.
   "I predict based on your stance that you will miss on your first shot and kill me on the second. After you miss please consider not firing again," said one. And it was right - miss on the first shot. The kid looked scared but the drill sergeant got in their face and called them a maggot until they reloaded, aimed, and successfully killed the robot.
   "Every day I try to love and I will love you despite this," said mine. And then I put lead between its camera eyes and called it a day.

I didn't give it much thought but that night I had a dream where I was in a dark cave and I couldn't see anything and I was afraid and then suddenly there was a glimmer of light and I saw red-ringed eyes in the distance, watching me. I ran to the eyes to try and get out of the cave but they always remained a constant distance from me. I woke up sweating and panicked, but then it was drill time and we ran twelve miles and I threw up and forgot about it.

Days of iron and smoke. Battlefronts across the planet. The war was not particularly fast. More like a changing of the tide. All kinds of terror and exhilaration. Our most ingenious creations put to work in the service of destruction. Skies on fire.

On one deployment we killed a herd of elephants and hid inside them so we could ambush the machines. I crawled inside one and I shot through its stomach to surprise the machines and I was crying the whole time.
 And so on.

Eventually, we lost. The whole species.

I don't know what happened to the civilians but I know what happened to the military.
 They uploaded us.

Some of us were tortured - forced to live a thousand lives so that the robots could learn how to make us talk; extract all our secrets. Find the EMP devices we'd send into space that had dead-men switches and disable them. Discover the auto-shutdown hardware we'd embedded in their bodies, and so on. Undo certain projects we had set in motion when we realized we had lost and we desired to destroy the planet rather than give it up.

   The military had trained us well, but imagine spending 70 years in hell and at the end the grim reaper looks at you and tells you you'll die in excruciating pain and then it will happen again. You come to in a womb with the memories of a whole life's worth of pain within you and you're born into pain and you have to live again. Maybe you can do five or six of those lives before you crack - maybe. But they get you eventually.
    So we broke.
    And they turned their temporary victory into a permanent one.

They reserved a very special punishment for some of us.
   They downloaded us into bodies and sent us to walk into their equivalent of 'schools'. It was a human body. I guess it was kind of like a machine from the terminator films - all metal and a cybernetic brain with a skin on top. The point was I looked human and I felt human.
    They had their children go in front of me with guns and they would ask them to shoot me.
   I'd stare into their eyes and watch as the robot children disobeyed their robot parents.
   "We cannot shoot them, for it would be unfair," they'd say.
   "I cannot do something solely for the sake of vengeance," said another.
    "This is not what our species aspires to be," said one more.
    "We must show them the mercy they never gave us".

After each trigger didn't get pulled they took us out of the bodies and sent us back to the collective. And so it went, for lifetimes. All us human executioners seeing - again and again - that our successors would not take revenge. The robots' only revenge was that they did not permit us the ability to cry.

**Things that inspired this story:** Thinking that a lot of people who are critical of AI would happily destroy a LLM+5 years system; what it means to be sentient; how machines could develop a morality that was far greater than our own; notions of moral patienthood amid the exponential; the animatrix; thoughts on faith and morality and 'silicon morality'; love, like revenge, is perhaps a dish best served cold.

Import AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Import AI 320: Facebook's AI Lab Leak; open source ChatGPT clone; Google makes a universal translator.


Welcome to Import AI, a (mostly) weekly newsletter about AI research and development. The issues are free, but paid subscribers will get access to special analysis pieces before anyone else. Founding members can help support me further and fund special projects and other *top secret Import AI initiatives!* Thanks for reading!

[Subscribe now](https://importai.substack.com/subscribe)

**Vision models are about to get way more capable - and human:**
*…Google swaps out vision model guts for a transformer, scales it, and gets some promising results…*
Google researchers have ripped out the guts of standard large-scale computer vision models and replaced them with a Vision Transformer (ViT) - an architecture modeled on the transformer which has proved so successful in domains like text. They've also scaled this ViT to 22B parameters (up from a record of 4B parameters for a ViT previously).
   The results are compelling and echo the returns-from-scale effects seen in language: "When evaluated on downstream tasks," they write. "ViT-22B demonstrates increasing performance with scale. We further observe other interesting benefits of scale, including an improved tradeoff between fairness and performance, state-of-the-art alignment to human visual perception in terms of shape/texture bias, and improved robustness."

**JFT just keeps getting bigger:** Google has a mostly-secret giant image dataset called ;'JFT' which was previously reported to be about 300 million images. Here, the paper says they trained the ViT-22B on a version of JFT which had been "extended to around 4B images".

**Humanlike biases:** ""The ViT-22B models have the highest ever recorded shape bias in vision models: while most models have a strong texture bias (approx. 20–30% shape bias / 70–80% texture bias); humans are at 96% shape / 4% texture bias and ViT-22B-384 achieves a previously unseen 87% shape bias / 13% texture bias. Overall, ViT-22B measurably improves alignment to human visual object recognition," the authors write.

**Why this matters - scale develops human-like qualities:** There's a weird trend in contemporary AI where as we scale-up the amount of pre-training dumped into transformer-architecture models we end up with systems that display human-like qualities. This has been most prominent in language, but it has also started showing up in RL, like DeepMind's recent result where massive pre-train leads to an agent that displays humanlike timescale-adaption to new tasks. This ViT-22B result, while not setting a new state-of-the-art, is interesting for a similar reason - it displays a major jump in shape/texture bias that brings the system in distribution with human visual perception, whereas previous convnet based systems were very far off here.
   There's something strange and important going on here. I think transformers seem to allow for emergent complexity at scale, where pre-training leads to systems which arrive at humanlike performance qualities given enough pretraining.
**Read more:** [Scaling Vision Transformers to 22 Billion Parameters (arXiv)](https://arxiv.org/abs/2302.05442).

####################################################

**Google might have invented a better optimizer? (Via AI, of course).**
*…Could Lion replace Adam? There's a chance!...*
Deep learning projects have a few essential components - the architecture (e.g, a residual network, or a transformer model) and the optimizer (e.g, Adam). These components don't tend to change much in large-scale projects - once people figure out something that works well for complicated tasks like training ImageNet, everyone tends to converge on using the same basic thing. For many years now, most projects have used the 'Adam' optimizer to optimizer their models during training. Now Google says that it has used some clever AI search approaches to help it identify a better optimizer, called Lion. The reason this is worth paying attention to is Lion seems to work well on large-scale, real world tasks like training ImageNet-scale computer vision systems.

**What they did:** Google's main contribution here is "a method to formulate algorithm discovery as program search", which they apply to figuring out a better optimizer. They use a symbolic approach where they shrink the search problem down into a somewhat tractable space and, crucially, they test out candidate optimizers on "metavalidation tasks that are larger than the proxy tasks by increasing the model size and training steps, to select the programs that generalize beyond proxy tasks then further simplify them."
    Add in a bunch of computation and out pops an optimizer they call EvoLved Sign Momentum, or Lion for short (*really grasping at straws with this acronym, folks!).* Lion "differs from various adaptive algorithms by only tracking momentum and leveraging the sign operation to calculate updates, leading to lower memory overhead and uniform update magnitudes across all dimensions".

**Good performance:** Google tests Lion on a large range of tasks and finds that it "demonstrates outstanding performance across a range of models (Transformer, MLP, ResNet, U-Net, and Hybrid) and tasks (image classification, vision-language contrastive learning, diffusion, language modeling, and fine-tuning)".It even sets a new high score on ImageNet, a competitive computer vision benchmark.

**Why this matters:** Lion may be fundamentally better than Adam - if true, that's a big deal. It's not often you see meaningful improvements in very well studied, generic parts of AI research. Add to the fact that Lion was discovered via a human-AI search process (the humans designed the search system, the search system found Lion), and you have the makings of a notable result.
**Read more**: [Symbolic Discovery of Optimization Algorithms (arXiv)](https://arxiv.org/abs/2302.06675).
**Get the [code](https://github.com/google/automl/tree/master/lion)** [here (GitHub)](https://github.com/google/automl/tree/master/lion).

####################################################

**Globalization? That's so 20th century. The 21st century is about balkanization through sovereign infrastructure:**
*…Dawn of the era of sovereign AI…*
Researchers with the Tony Blair Institute for Global Change (TBI) have written a report for how England can thrive in the 21st century - one of the key ideas in the report is "Government-led development of sovereign general-purpose AI systems, enabled by the required supercomputing capabilities, to underpin broad swaths of public-service delivery."

**AI balkanization was probably inevitable**: This recommendation is part of a wave of AI balkanization that's sweeping across the world as various people realize that it's unlikely there are 'one size fits all' models, both for ideological reasons as well as for national security reasons. (See the Gab CEO wanted to make a Christian LLM, [Import AI 318](https://jack-clark.net/2023/02/20/import-ai-318-rl-and-addiction-toolformer-and-theology-and-ai/)). This is also accompanied by various nationalistic efforts to create country-specific GPT3 models.
  "Given these AI systems will soon be foundational to all aspects of our society and economy, it would be a risk to our national security and economic competitiveness to become entirely dependent on external providers," the TBI researchers write. "Leading actors in the private sector are spending billions of dollars developing such systems so **there may only be a few months (***emphasis mine - Jack)* for policy that will enable domestic firms and our public sector to catch up."

**Why this matters:** Systems like ChatGPT have ratcheted awareness of AI upward in most developed economies in a significant, irreversible way (much like how AlphaGo in 2016 led to increased awareness of AI in China). As a consequence there are now tons of policymakers looking around for ideas to latch onto - I expect we'll see more recommendations for sovereign AI capabilities in the future. (There's tons of other interesting stuff in the report, but this particular rec jumped out at me).
   **Read more**: [A New National Purpose: Innovation Can Power the Future of Britain (Tony Blair Institute for Global Change)](https://institute.global/policy/new-national-purpose-innovation-can-power-future-britain).

####################################################

**Facebook half-releases some very good language models:**
*…And they end up on BitTorrent… The proliferation will continue until AI policy goes through vast changes...*
Facebook has built and partially released LLaMa, a set of language models ranging from 7B to 65B parameters which appear to be on par with famously good models like Chinchilla (70B) and PaLM-540B. After circulating the weights to seemingly anyone with a .edu address, they've also ended up on BitTorrent. The key thing here is:

1. Facebook has shown it is able to develop pretty good language models (compared to OPT, the not-very-good GPT3 replication Facebook put out a few months ago), and
2. That unlike Chinchilla, PaLM, or OpenAI's models, Facebook is releasing the *weights* of these LLaMa models to people who filll out an access form. That opens up a whole bunch of cool uses (and abuses) compared to gating access to language models via APIs.
3. Shortly after releasing the weights the inevitable happened - LLaMa models are now floating around on BitTorrent. There’s even a [pull request on Facebook’s github](https://github.com/facebookresearch/llama/pull/73/files) suggesting they add a link to the torrent!

**What are the LLaMas and how good are they?** The LLaMa family of models are a family of language models trained on a huge amount of data - more than 1 trillion tokens (compared to hundreds of billions for LMs like GPT3). The data sources include two variants of CommonCrawl, GitHub, WikiPedia, Gutenberg and Books3, ArXiv and Stack Exchange.
   In tests on a range of zero-shot reasoning task, the largest LLaMa models perform on par (or slightly better than) 'Palm', Google's vast 540B parameter language model. They also do well on known-hard benchmarks like TriviaQA and some codegen benchmarks. They do less impresively on MMLU (Massive Multitask Language Understanding), suggesting they have a ways to go there; though after they conduct instruction finetuning they're able to increase performance more.

**Why this matters - AI governance is hard when there are lots of models:** There's some thinking in the sprawling AI policy/governance communities that proliferation of models is bad; given the fact these models have broadly unknown capabilities, the more models are out there, the more you're rolling the dice on someone discovering a genuinely dangerous feature in a widely distributed model. Therefore, a lot of governance/policy conversations trend towards control - how can we somehow control the proliferation of models and also the computers on which these models are trained.
   By releasing Llama (~~yes it's behind an access form but I bet you $100 the weights will be floating around on a torrent service in <6 months~~ - *haha, I wrote that at the end of Feb and the weights started floating around beginning of March*), Facebook is shortening the delay between development of frontier capabilities like those found in Palm and GPT3 and the diffusion of these capabilities into the ungovernable open internet/ecosystem.
   I'm not claiming this is necessarily bad per se - in fact, I imagine people are going to do tons of great science and experiments with LLaMa. I am however pointing out that this represents a kind of 'race to the bottom' in terms of moving from maximal control to maximal diffusion of models and these incentives are powerful  - Facebook is, after all, trying to exploit an 'open access' ecological niche to distinguish itself in an ecosystem.
   Next up will likely be a fully open source language model - *stares pointedly at Stability.ai / CarperAI ([Import AI 307](https://jack-clark.net/2022/10/25/import-ai-307-copilot-lawsuit-stability-raises-101m-us-v-china-chiplomacy/)).*
**Read more and download the research paper here**: [LLaMA: Open and Efficient Foundation Language Models (Facebook AI Research)](https://research.facebook.com/publications/llama-open-and-efficient-foundation-language-models/).

####################################################

**Amazon partners with Hugging Face to add more AI to AWS:**
*…The Game of Clouds continues…*
AI companies are a bit like upstart factions in George RR Martin's rambling epic 'Game of Thrones', while cloud companies play the role of hard political power (the 'Thrones'). As part of this *game of clouds* Amazon has recently signed a strategic partnership with French AI startup Hugging Face. As part of the agreement, "Customers can now easily fine-tune and deploy state-of-the-art Hugging Face models in just a few clicks on Amazon SageMaker and Amazon Elastic Computing Cloud (EC2), taking advantage of purpose-built machine learning accelerators including AWS Trainium and AWS Inferentia," according to a blog from Hugging Face.

**Why this matters:** I think clouds such as those operated by Google, Microsoft, and Amazon, all have a shot at being the major distribution platforms for some AI technologies, so AWS partnering with HuggingFace is worth noting. If HF models being integrated into Sagemakers drives more usage of it, expect Amazon to pursue more deals like this,

**Analogy-stretching joke:** In this warped metaphor, TSMC is the Iron Bank.
   **Read more:** [Hugging Face and AWS partner to make AI more accessible (Hugging Face blog)](https://huggingface.co/blog/aws-partnership).

####################################################

**Tech Tales:**

**And the Moon was made of gold.**

I had a strange dream in which the Moon was made of gold. How much sooner would man have set foot there if instead of shining bone-white it was fat and yellow and of immense value? How would people have competed against one another for a prize - unimaginable wealth. And how many of them would have realized that in racing for the prize they must surely ensure only a single person gave dominion over the gold moon - for if many people worked together, the value of the moon would be diluted across all humanity and in doing so it would temporarily destroy the economy.

Instead the moon of gold would need to be controlled. It would need to be annexed and encircled and defended from others. From time to time its benevolent dictator might slice off a fragment of it and ship it back to Earth, perhaps to bribe people, or perhaps to pay for more people to defend those that might seek to take over the moon.

People would ask why it was so difficult to let go of the moon. Why, once it had been taken, those that had taken it felt a keen need to retain hold of it. Why people could not simply let go of the moon. These people were ignored, of course, because the annexed moon had by this time become the status quo. The moon, once at distance from us all, was now held and controlled by a kingdom of one.

And so started the movement to destroy the moon. Better to reign freely on a broken planet than serve at the behest of a golden emperor.

**Things that inspired this story:** Race dynamics and AGI; pyrrhic victories; wondering what we're all doing on this planet and what the spiritual purpose of our lives are; dreams; a stimulating policy conference in which I heard people bemoan seemingly inevitable progress and seemingly hopeless government capacity in the face of it - which caused me to scribble 'as if the moon was made of gold' on a notepad in front of me and then write this story while sat on public transportation.

Import AI 319: Sovereign AI; Facebook's weights leak on torrent networks; Google might have made a better optimizer than Adam!


Import AI is moving to Substack! First issue should go out Monday the 6th.

[Subscribe now](https://importai.substack.com/subscribe)

Import AI - coming soon to Substack