Week 5, 2026 · Jan 26 – Feb 1, 2026 · 23 newsletters · 161 links · ≈ 2 h 58 min
AVCJan 26
I started programming when I was in high school and helped pay my way through MIT by writing Fortran code in a research lab. I got a job writing software for a naval architecture firm right out of college, and then helped pay my way through grad school by doing some freelance coding gigs. When I got into VC in the mid 80s, I stopped writing code. Other than some UI/UX tweaking here and there, I have not written much code in almost forty years.
Like so many of us, the arrival of AI-assisted coding tools has made me a coder again. It's not really writing code, though. It's building stuff with code that is written by AI.
It's fun.
Over the weekend, I made two apps, both of which leverage my interest in music.
I used Claude Code, working in the Terminal app on my Mac, to build this web app that pulls all the music I have recently liked on SoundCloud and makes it available to listen on the web.
I also used our portfolio company Neynar's Studio app to build and deploy a mini-app in the Farcaster and Base mobile apps. My mini app is called Music Casts, and it pulls all of the music links from the people I follow on Farcaster and Base and puts them into a mini app that allows you to listen to them. It looks like this in Farcaster:
It is nice to make miniapps in Neynar Studio because Farcaster and Base offer the deployment surface and user base to give feedback. When I casted about Music Casts, I got a ton of suggestions, and I have added some of them already.
Coding in Claude Code in the Terminal app is more powerful, particularly when paired with a deployment tool. I used a deployment tool called Railway to get my web app live.
If you've always wanted to build software applications but have been held back by a lack of programming skills and/or time to learn, your time has come!
If you want to get started on Claude Code in the Terminal app,here are the instructions to get going.
If you want to build in Neynar Studio, go here.
Support AVCI am a VCShow you appreciate this writer, help support their work, and share in their growth over time by buying their writer coin.Support
The newsletter for the technically curious. Updates, tool reviews, and lay of the land from an exited founder turned investor and forever tinkerer.
Hey folks,
OpenAI hosted a town hall answering questions from builders. I used YouTube’s “Ask” button to get the gist: OpenAI can drive costs down by 100x in the next two years, still very focused on general models and will have a model with better writing in GPT-5.x
Andrej Karpathy posted his reflections on where we’re at with coding agents/vibe coding. I have similar feelings. The TLDR is; people are mostly moving to predominantly agent work with minimal human input, “no more IDEs” + Agent swarms are too hypey right now, agents just power through tasks and never get tired (which is insane to think about if you consider it a super-intelligent teammate), feeling way faster with what you can produce and instead of us lounging on the beach we’re, shock, producing more, learning how to guide models is becoming an art - write failing tests and then pass them or put in a loop with a browser to verify, working with agents is genuinely so fun and 2026 will be the year of slop - given the above advancements (I agree - but slopping our way to learn and produce things that aren’t slop is still a reasonable path).
I jumped on Every’s Vibe Code Camp (full recording here) alongside other well-known builders. Chatting about how I reverse engineer tools, build stuff and generally tinker a lot with agents and code (even though I’m not technical).
Signals - Droid learns from all its failures and suggests actionable work items for the team to improve itself. Currently, humans review and merge the changes, but these are early signs of self-improving agents.
Claude now has interactive interfaces for apps like Slack, Asana, Figma and more. Very similar to ChatGPT Apps and built on top of MCP.
Claude Code is replacing Todos with Tasks. Suitable for longer projects (as models improve), and saved on your device so that multiple agents can access/complete them.
ChatGPT can now pip/npm install packages , run bash and download files in Code Interpreter.
Vercel has built skills.sh - A directory for agent skills and a simple way to install them. Context7 has a similar attempt. Some skills I came across over the weekend:
Postgres best practices, browser use, React emails using resend
marketing skills, research last 30 days for a topic and image gen + editing with Flux
Kimi K2.5 is a new open-weights model from China, and it scores better than Opus 4.5 or GPT-5.2 on benchmarks in all areas other than coding. It’s also great at vision like Gemini 3 Pro, and it’s priced similarly to Gemini 3 Flash. This has actually got me excited to try it, as they are also going hard on tooling around the model with Kimi Code (CLI) and their web app for slides generation, general tasks and more. Also see, Qwen 3 Max Thinking, which has similar performance but not open weights.
Enterprises hold valuable data locked inside content, but manual processes make it difficult to unlock. Box Extract securely & accurately extracts valuable data at scale to drive faster decisions & automated workflows. Learn more from Box on how to transform enterprise content into actionable data.*
Decision-time guidance by Replit - Injecting short, situational instructions to keep the agent on track for much longer.
New essay on AI risks from Dario Amodei: The adolescence of technology.
Lessons from building AI agents for financial services.
Using GitHub Pages to preview simple outputs from Claude Code on your phone.
Claude Code has a hidden TeammateTool for running a multi‑agent team system.
I stopped reading code. My code reviews got better.
Can LLMs finally do social science? Tracking latent cultural concepts over time.
New iOS apps on the App Store grew by 60% in 2025 after being almost flat for the last three years.
Building new consumer experiences in speech AI demands low latency, sustainable economics, and mass personalization. The current #1 on the leaderboards, TTS-1.5 by Inworld, delivers on it: sub-250ms latency, 40% lower error rates with 25x lower cost for developers. Check out TTS-1.5 here.*
Scroll.ai turns any knowledge base into an enterprise-grade AI agent. Get 2 free months ($158 value) with code BENSBITES26.*
Feynman 3 by Opennote - A thinking partner for learning.
Text to diagram in Excalidraw now streams the output, is faster and smarter.
Company Search on Exa - Semantically search over 60M+ companies and get structured information on each (web traffic, headcount, financials, and more).
Pencil - Infinite design canvas for Claude Code.
Cavo - Excalidraw with a webcam and screen recording setup. (read more)
Swing by Cartwheel - Generate 3D movements for your characters.
gcombinator - Change ‘y’ to ‘g’ to get the full context of the article & comments from HN.
Monitor by Parallel AI - Always-on web searches that notify you when new information becomes available on the web in the schema of your choice.
Sprites - Stateful sandbox environments with checkpoint & restore to run AI-generated or user-uploaded code safely.
Clawdbot is now called Molty. Deploy it on Railway or let Devin set it up for you.
Cursor now uses subagents to complete parts of a task, generate images, and ask clarifying questions while working in the background. Plus, they introduced “Cursor Blame.”
Applications for Embed’s 6th cohort are open till 9th Feb. Prev companies include Cognition, Listen Labs, Physical Intelligence, Pika Labs, and Yutori.
Logical Intelligence is piloting a new “energy-based model” for reasoning.
qwen3-TTS - A powerful text-to-speech model that you can run on your Mac. (try here)
Ricursive Intelligence raised $300M at $4B valuation to design better chips with AI.
That’s it for today. Feel free to comment and share your thoughts. 👋
Arcee AI is a the startup I’ve found to be taking the most real approach to monetizing their open models. With a bunch of experience (and revenue) in the past in post-training open models for specific customer domains, they realized they needed to both prove themselves and fill a niche by pretraining larger, higher performance open models built in the U.S.A. They’re a group of people that are most eagerly answering my call to action for The ATOM Project, and I’ve quickly become friends with them.
Today, they’re releasing their flagship model — Trinity Large — as the culmination of this pivot. In anticipation of this release, I sat down with their CEO Mark McQuade, CTO Lucas Atkins, and pretraining lead, Varun Singh, to have a wide ranging conversation on:
The state (and future) of open vs. closed models,
The business of selling open models for on-prem deployments,
The story of Arcee AI & going “all-in” on this training run,
The ATOM project,
Building frontier model training teams in 6 months,
and other great topics. I really loved this one, and think you well too.
The blog post linked above and technical report have many great details on training the model that I’m still digging into. One of the great things Arcee has been doing is releasing “true base models,” which don’t contain any SFT data or learning rate annealing. The Trinity Large model, an MoE with 400B total and 13B active tokens trained to 17 trillion tokens is the first publicly shared training run at this scale on B300 Nvidia Blackwell machines.
As a preview, they shared the scores for the underway reasoning model relative to the who’s-who of today’s open models. It’s a big step for open models built in the U.S. to scale up like this.
I won’t spoil all the details, so you still listen to the podcast, but their section of the blogpost on cost sets the tone well for the podcast, which is a very frank discussion on how and why to build open models:
When we started this run, we had never pretrained anything remotely like this before.
There was no guarantee this would work. Not the modeling, not the data, not the training itself, not the operational part where you wake up, and a job that costs real money is in a bad state, and you have to decide whether to restart or try to rescue it.
All in—compute, salaries, data, storage, ops—we pulled off this entire effort for $20 million. 4 Models got us here in 6 months.
That number is big for us. It’s also small compared to what frontier labs spend just to keep the lights on. We don’t have infinite retries.
Once I post this, I’m going to dive right into trying the model, and I’m curious what you find too.
Listen on Apple Podcasts, Spotify, YouTube, and where ever you get your podcasts. For other Interconnects interviews, go here.
Lucas Atkins —X,LinkedIn — CTO; leads pretraining/architecture, wrote the Trinity Manifesto.
Mark McQuade — X, LinkedIn — Founder/CEO; previously at Hugging Face (monetization), Roboflow. Focused on shipping enterprise-grade open-weight models + tooling.
Varun Singh — LinkedIn — pretraining lead.
Most of this interview is conducted with Lucas, but Mark and Varun make great additions at the right times.
Core:
Trinity Large (400B total, 13B active) collection, blog post. Instruct model today, reasoning models soon.
Trinity Mini, 26B total 3B active (base, including releasing pre-anneal checkpoint)
Trinity Nano Preview, 6B total 1B active (base)
Open Source Catalog: https://www.arcee.ai/open-source-catalog
API Docs and Playground (demo)
Socials: GitHub, Hugging Face, X, LinkedIn, YouTube
Trinity Models:
Trinity models page: https://www.arcee.ai/trinity
The Trinity Manifesto (I recommend you read it): https://www.arcee.ai/blog/the-trinity-manifesto
Trinity HF collection — (Trinity Mini & Trinity Nano Preview)
Older models:
AFM-4.5B (and base model) — their first open, pretrained in-house model (blog post).
Five open-weights models (blog): three production models previously exclusive to their SaaS platform plus two research models, released as they shifted focus to AFM — Arcee-SuperNova-v1, Virtuoso-Large, Caller, GLM-4-32B-Base-32K, Homunculus
Open source tools:
MergeKit — model merging toolkit (LGPL license return)
DistillKit — knowledge distillation library
EvolKit — synthetic data generation via evolutionary methods
Related:
00:00:00 Intro: Arcee AI, Trinity Models & Trinity Large
00:08:26 Transitioning a Company to Pre-training
00:13:00 Technical Decisions: Muon and MoE
00:18:41 Scaling and MoE Training Pain
00:23:14 Post-training and RL Strategies
00:28:09 Team Structure and Data Scaling
00:31:31 The Trinity Manifesto: US Open Weights
00:42:31 Specialized Models and Distillation
00:47:12 Infrastructure and Hosting 400B
00:50:53 Open Source as a Business Moat
00:56:31 Predictions: Best Model in 2026
01:02:29 Lightning Round & Conclusions
Transcript generated with ElevenLabs Scribe v2 and cleaned with Claude Code with Opus 4.5.
00:00:06 Nathan Lambert: I’m here with the Arcee AI team. I personally have become a bit of a fan of Arcee, ‘cause I think what they’re doing in trying to build a company around building open models is a valiant and very reasonable way to do this, ‘cause nobody really has a good business plan for open models, and you just gotta try to figure it out, and you gotta build better models over time. And like open-source software, building in public, I think, is the best way to do this. So this kind of gives you the wheels to get the, um... You get to hit the ground running on whatever you’re doing. And this week, they’re launching their biggest model to date, which I’m very excited to see more kind of large-scale MoE open models. I think we’ve seen, I don’t know, at least ten of these from different providers from China last year, and it’s obviously a thing that’s gonna be international, and a lot of people building models, and the US kind of, for whatever reason, has fewer people building, um, open models here. And I think that wherever people are building models, they can stand on the quality of the work. But whatever. I’ll stop rambling. I’ve got Lucas, Mark, um, Varun on the, on the phone here. I’ve known some of them, and I consider us friends. We’re gonna kind of talk through this model, talk through building open models in the US, so thanks for hopping on the pod.
00:01:16 Mark McQuade: Thanks for having us.
00:01:18 Lucas Atkins: Yeah, yeah. Thanks for having us. Excited.
00:01:20 Varun Singh: Nice to be here.
00:01:20 Nathan Lambert: What- what should people know about this Trinity Large? What’s the actual name of this model? Like, how stoked are you?
00:01:29 Lucas Atkins: So to- yeah.
00:01:29 Nathan Lambert: Like, are you, like, finally made it?
00:01:32 Lucas Atkins: Uh, you know, we’re recording this a little bit before release, so it’s still like, you know, getting everything buttoned up, and inference going at that size is always a challenge, but we’re-- This has been, like, a six-month sprint since we released our first dense model, which is 4.5B, uh, in, in July of last year, 2025. So, um, it’s always been in service of releasing large. I- it’s a 400B, um, thirteen billion active sparse MoE, and, uh, yeah, we’re, we’re super excited. This has just been the entire thing the company’s focused on the last six months, so really nice to have kind of the fruits of that, uh, start to, start to be used by the people that you’re building it for.
00:02:16 Nathan Lambert: Yeah, I would say, like, the realistic question: do you think this is landing in the ballpark of the models in the last six months? Like, that has to be what you shop for, is there’s a high bar- ... of open models out there and, like, on what you’re targeting. Do you feel like these hit these, and somebody that’s familiar, or like MiniMax is, like, two thirty total, something less. I, I don’t know what it is. It’s like ten to twenty B active, probably. Um, you have DeepSeeks in the six hundred range, and then you have Kimi at the one trillion range. So this is still, like, actually on the smaller side of some of the big MoEs- ... that people know, which is, like, freaking crazy, especially you said 13B active. It’s, like- ... very high on the sparsity side. So I don’t actually know how you think about comparing it among those. I was realizing that MiniMax is smaller, doing some data analysis. So I think that it’s like, actually, the comparison might be a little bit too forced, where you just have to make something that is good and figure out if people use it.
00:03:06 Lucas Atkins: Yeah, I mean, if, if from raw compute, we’re, we’re roughly in the middle of MiniMax and then GLM 4.5, as far as, like, size. Right, GLM’s, like, three eighty, I believe, and, and thirty-four active. Um, so it-- you know, we go a little bit higher on the total, but we, we cut the, uh, the active in half. Um, it was definitely tricky when we decided we wanted to do this. Again, it was July when... It, it was July when we released, uh, the dense model, and then we immediately knew we wanted to kind of go, go for a really big one, and the, the tricky thing with that is knowing that it’s gonna take six months. You, you can’t really be tr-- you can’t be building the model to be competitive when you started designing it, because, you know, that, obviously, a lot happens in this industry in six months. So, um, when we threw out pre-training and, and a lot of our targets were the GLM 4.5 base model, um, because 4.6 and 4.7 have been, you know, post-training on top of that. Um, and, like, in performance-wise, it’s well within where we want it to be. Um, it’s gonna be... Technically, we’re calling it Trinity Large Preview because we just have a whole month of extra RL that we want to do. Um- But-
00:04:29 Nathan Lambert: I’ve been, I’ve been there.
00:04:31 Lucas Atkins: Yeah, yeah. But i- you know, we’re, we’re in the, um, you know, mid-eighties on AIME 2025, uh, GPQA Diamonds, uh, seventy-five, um, at least with the checkpoint we’re working with right now. We’re still doing more RL on it, but, um, you know, MMLU Pro, uh, eighty-two. So we’re, we’re, we’re happy. We’re really-- Like, for it being our first big run, like, just getting it trained was, was an extreme accomplishment, but then for it to actually be, like, a, a genuinely useful model is a, a cherry on top.
00:05:03 Nathan Lambert: Yeah, let’s go big picture. Uh, like, let’s recap. We have all of the... We have this full trinity of models. I think that there’s a fun note. Uh, did I put it in this doc? Yeah, on Nano Preview, which was the smallest- ... you’re, like, charming and unstable. The model card’s really funny. Um, ChatGPT, doing deep research on this, I was like, ChatGPT Pro just tagged next to it, “charming and unstable.” And I was like: Is this a hallucination? And then in the model card, you have, like: “This is a chat-tuned model with a delightful personality and charm we think users will love. Uh, we think- ... it’s pushing the boundaries, eight hundred million, um, active parameter, and as such, may be unstable in certain use cases.” This is at the smallest scale- ... which is like, I appreciate saying it as it is, and that’ll come up multiple times in the conversation. And then you have Mini, which is like, um, I think it was, like, 1B active, 6B total type thing. In my-- I, I don’t have it, the numbers right in front of me. I have it somewhere else. Um-
00:05:52 Lucas Atkins: Yeah, Nano was, Nano was the 6B, uh, 1 active.
00:05:55 Nathan Lambert: Oh, yeah, yeah.
00:05:55 Lucas Atkins: And then, and the Mini was twenty-six, 3B active.
00:05:58 Nathan Lambert: Yeah. So, like-
00:06:00 Lucas Atkins: Um, yeah.
00:06:00 Nathan Lambert: -are these based on more of, like, you need to build out your training chops, or are you trying to fill needs that you’ve-... heard from community, and like, I think for context, previously, your first open model was a base and post-trained model, which was Arcee 4.5B, which was a dense model- -which people like. And prior to that, you had, like, a long list of, like, post-training fine tunes that you had released. So before that, it was like a post-training shop, and I think that kind of history is i- important to fill in, ‘cause I think most people-- a lot of people are gonna meet you for the first time listening to this.
00:06:34 Lucas Atkins: Yeah, it, it, um, we chose those sizes for Mini and Nano, uh, specifically Mini, um, the 26B, 3B Active, because we wanted to de-risk, uh, large. Like, th- this has all been in service of getting to a model of, of, you know, the 400B class. So, um, we, you know, learned from doing the original 4.5B, that you might have everything on paper that you need to train a model, but i- inevitably, there’s tremendous, you know, difficulties that come up, and, um, it, it’s-- we, we definitely knew we wanted to make sure that we, you know, solved some of... E- especially when it came to just doing an MoE model performance, uh, you know, like a, like an efficient, fast train of an MoE. So, um, we thought that that was a good ground where we could, you know, it wasn’t crazy expensive, uh, but gave us a lot of data, uh, going into large. And then Nano just came about because we had some extra compute time, and we really want to do more research on, like, smaller models that are very deep. Um, and we hadn’t really seen that in an MoE before, so that one was very much we started training it, and then it, you know, early benchmarks were good, so we said, “Well, we’ll just do the whole dataset.” Um, and, uh, but most of the love for those releases went into, to Mini. So I, I definitely think that long term, uh, from an ROI perspective, the smaller models are going to be where we shine, just because there’s a tremendous amount of, of cost savings a company can get from, from optimizing on a, on a smaller model. Um, but, but we, uh, w- we’re definitely gonna be trying to push the, the large frontier, too.
00:08:26 Nathan Lambert: Yeah. Um, I’d like to kind of double-click on training before going back to the small model that’s useful for companies, ‘cause we’re gonna have-- we’re gonna end up talking for, like, twenty minutes plus about open ecosystem. So I kind of am curious, like, philosophically, how your company feels about, like, sharing scientific details. So if I ask you, like, what are the things you’re technically most excited about in the model, or, like, what are the pain points? Like, uh, like, are you willing to talk about these things? Like, I- Do you feel like it’s kind of orthogonal to the company? Like, I feel like a lot of it is just, like, things that happen. I think your framing of all of this is in service of getting the big model going. And particularly, of, like, you have to be thinking about your model as landing in six months, is probably... Like, for people not training models, it’s hard to think about, ‘cause even I- ... like, I’m thinking about trying to refresh our post-training stack for OLMo 3, and I’m like, the thinking model, the, um, we are pretty SFT heavy right now, and it makes it not very dynamic in terms of the thinking time. But it’s just like, I can’t see people deploying this model, or probably will have a hard time fine-tuning it. And it’s like to think about where tool use models are going in six months, like, seems pretty hard. Um, it’s a very hard task to do, so it takes a lot of gumption to actually set out and do it. So I, I would just appreciate the framing, kind of self-reflecting on what I go through. So if you have anything that you think was, like, particularly hard to actually land the six-month outlook, because you use Muon as an optimizer, or is it Muon? And some of these things. I think the data, it’s well known that Datology is cranking a lot of this, and you probably provide-- I think of it as like you’re kind of driving and working with these partners, and I’m sure you provide a lot of feedback on what’s working and what’s not. So- ... anything you’re willing to share, I think it’s useful.
00:10:08 Lucas Atkins: Uh, I, I think, um, I mean, on the data side, like Datology, I-- at least for these models, that, that partnership has very much been almost an extension of our own research team. Like, we’ve worked very closely with them, and, um, obviously, our model’s doing well, you know, i- is, is, is good for them. So, um, but it, it-- there was definitely, you know, and you know this better than most, like, small-scale ablations, when you throw them at scale, sometimes, you know, uh, the-- i- it doesn’t always turn out how you want. So there was quite a lot of iterating there to at least get the dataset we used for Large. Um, I, I would say that as far as looking out six months and then figuring out how we wanted to... Obviously, the big one was compute. We don’t, um, you know, we, we never raised as, like, a foundation model company, so we’ve ne- we haven’t signed massive commits for, you know, thousands of GPUs before. Um, we didn’t have a, a, a massive cluster that was always active, uh, for a lot of our post-training. So if they came before, um, you know, we had sixty-four, uh, H100s, that was pretty sufficient for that kind of work, but obviously, this necessitated quite a bit more. Um, but the first thing was-
00:11:29 Nathan Lambert: That’s still less than people would guess. Like, you’re releasing models- ... that weren’t like, your models weren’t catching national news, but people in the community knew about them. And, like, uh, i- I think of, like, Moondream when I think about that. Like, vik has- ... such little compute, and he puts it to so use. Like, you, like, see how successful he is? And he tells you that he has, I don’t know, thirty... Like, l- it might be, like, sixty-four GPUs. Like, uh- ... there’s, uh, uh, that’s a whole separate conversation on building- ... actual good ML output on little compute. I, I should ta- I should chat with vik about this, but aside
00:12:03 Lucas Atkins: No, it’s, it is-- I think it was... Yeah, it, it, it was very much a gift going into the pre-training side because-... we were kind of already thinking, All right, how do we do the mu- you know, the most with the, the least amount of compute? But, um, you know, we-- it took us quite a while to get the cluster that we have been training large on, which is twenty-two thousand forty-eight B300s. Um, and once we figured out when we were going to get that, get access to that cluster, everything else kind of became clear as far as, like, timelines for Mini and Nano and, and when we wanted to do that. Uh, obviously, you know, five hundred and twelve H100s was easier to come across, um, for Mini and Nano. So once we figured that out, um, it really became, uh, this game of, okay, how can we find, like, the best research on the topic of, of pre-training, and what is kind of... What are the, the, the papers and publications that are coming out, um, that have enough potential and enough precedence, either because, uh, another lab used them, it comes from a reputable team, uh, the ablations and the, the evaluation setup, like in the paper, was sufficient enough to give us confidence. Uh, and then we basically spent, I don’t know, it was probably about two months just figuring out what we wanted our architecture to be for the MoE, then figuring out, okay, now that that’s what we want to do, how do we implement all of that in the actual training pipeline? Uh, how can we-- you know, at that time, there had been many people who’d done Muon, but, um, for post-training, and, and then other-- some Chinese labs had used it, but there wasn’t, like, a widely available distributed Muon, um, to do it that scale.
00:13:54 Nathan Lambert: What do you think that, like, looks like in decision-making? ‘Cause that seems like a risky decision, if you ask me. I think for one, the ti-
00:14:00 Lucas Atkins: Muon?
00:14:00 Nathan Lambert: ... the timing, the, the, like, timing sharing that you’re saying is good. Like, you said this for two months, and then, like... But, like, even Muon is like, that’s a bet that would even take-- like, somewhere like AI2, that would take some serious evidence to go with it. We would want to ablate it. So like- ... on a single track, it’s like y- you had probably had a process for becoming fairly confident in it then.
00:14:24 Lucas Atkins: It- yes, but it, it was also, like, Kimi had, had just come out, and we knew that that one used Muon, and so we knew that it, at least, if implemented correctly, could deliver a good model. There weren’t outstanding ablations done around like... You know, there wasn’t a Kimi scale model done with Adam, and then compared to Muon and see the difference. But, um, that at least gave us enough confidence that if-
00:14:50 Nathan Lambert: What does Muon give you? Does it give you, like, memory saving, uh, in-
00:14:55 Lucas Atkins: No, it’s actually a little bit more memory. It’s, it’s, it’s mostly-
00:14:58 Varun Singh: It’s, uh-
00:14:58 Lucas Atkins: ... like the loss converges a bit quicker.
00:15:00 Varun Singh: It’s, it’s less memory, actually. It’s, uh, uh, only one momentum buffer instead of Adam’s two, uh, beta buffers, and then it’s also better convergence.
00:15:10 Nathan Lambert: Okay. So it’s, like, mostly designed around convergence, and then I know the math is different, which is where this momentum term changes.
00:15:15 Lucas Atkins: Well, it, it kind of came out... I mean, it had its, its, its big, you know, uh, explosion of popularity in the kind of nanoGPT speedrunning community. So it was kind of all built around converging to a certain, you know, validation loss faster, and, uh, that, that, that was, um... As for why we chose it as opposed to Adam, we’d used Adam for 4.5b, uh, but we also knew that if we wanted to move this fast, that we were going to have to make some pretty big bets, educated. Um, but, but still, we would have to make some, some, some risky decisions, um, beyond just, you know, training in general. So, um, there were a few that Muon we went with, uh, I think was, was one of our bigger bets. Uh, we ended up not doing, like, multi-token prediction or, or, or FP8 because we were throwing so many new things into the run at once, um, that-
00:16:12 Nathan Lambert: Do these apply for-
00:16:12 Lucas Atkins: ... if something were to go wrong-
00:16:13 Nathan Lambert: um, Mini and Nano? Are those also Muon, or are those- ... Adam as well? Okay, so then you- ... you get some de-risk from that. Do you know off the top of your head how many days it take to train each of those? Like, a, a good-
00:16:25 Lucas Atkins: Uh-
00:16:25 Nathan Lambert: ... ballpark for people, before-
00:16:27 Lucas Atkins: Yeah, so-
00:16:28 Nathan Lambert: going into the bigger run.
00:16:29 Lucas Atkins: So, so Mini, uh, so Nano on it was five hundred and twelve H200s, uh, took a little over thirty days. Um, and then Mini was about forty-five days.
00:16:45 Nathan Lambert: Okay. I think another thing- ... off the top of my head is I know that, like, a OLMo 1B dense would take us, like, eleven days on a hundred and twenty-eight H100s for a dense model. So, like, sixteen. So, like, the numbers- ... just go up from there. ‘Cause then it’s like the question is like, I’m guessing i- if those are forty-five days, and then you have-- you up the number of GPUs, it’s gonna be like a similar amount of time, or forty days for the big model, but much more stressful.
00:17:16 Lucas Atkins: Yeah, the big model was... But again, that was- we knew that we, we wanted- we felt confident that we could deliver a competitive and exciting model in January 2026. Like, we knew that it would-- we could... Who knows kind of where the research and what, what class and, and, and, and skill and performance of model is gonna come out in the next three months? Um, so we also knew that we really wanted to land sometime in January, and that’s also why we also took- we went with B300s, even though definitely the largest public train of that size on B300s and, and the, um, you know, a lot of the software was not-- did not have, like, out-of-the-box B300 support. It was the only way we were gonna be able to train a model of this size in-
00:18:06 Nathan Lambert: Did you have to do this? Did you have to implement the... like, help solve version issues or other issues on B300s? ‘Cause I’ve heard that-
00:18:13 Lucas Atkins: W-
00:18:14 Nathan Lambert: ... the rollout has been rough.
00:18:16 Lucas Atkins: We had to add-... a, a bit. There, there were a couple days where the, the data center had to take it offline to implement some bug fixes. It was, it was definitely, like, a very cool experience being on the bleeding edge, but, um, also, like, a little frightening ‘cause you just know, like, “Oh, we’re not getting the most out of these that we possibly could.” So, um, a little bit of both.
00:18:40 Nathan Lambert: Uh, was your final training run stable, or did you have to do interventions through it?
00:18:46 Lucas Atkins: Uh, it was very stable, actually. Uh, it took-- the beginning of it was not. The, the, the first ten days were absolute, um... It, it would start very well and, and looked, you know, uh, the dynamics and the logs, and the graphs looked very similar to Mini and Nano, and then after, uh, around a trillion tokens, it- the- we- you know, you’d get collapsing, experts would start to go crazy. Uh, part of this is just, again, we are very sparse compared to what you, you, you have. So, um, you know, four hundred billion total, um, thirteen billion active, two hundred and fifty six experts. Like, it was, it was-
00:19:26 Nathan Lambert: Did you do a, uh, expert routing loss or some sort of balancing loss?
00:19:30 Lucas Atkins: Yeah. Yeah, yeah. Yeah.
00:19:32 Varun Singh: We did, um, we used DeepSeek’s, uh... We, we modified DeepSeek’s Auxiliary-loss-free, um, uh, loss balancing with our own, like, uh, with some tweaks, and then we also added a sequence loss like they, uh, did as well.
00:19:47 Nathan Lambert: Uh, was there Auxiliary-loss-free one from DeepSeek V3, or was that a later model?
00:19:51 Varun Singh: That was V3.
00:19:52 Lucas Atkins: It was V3.
00:19:52 Varun Singh: They did a separate paper on it as well. Yeah.
00:19:55 Nathan Lambert: Yeah. Yeah, that makes sense. I think a lot of people have derived from there. Um, have you- ... had issues on post-training as well? So I have a theory that the new algorithms we’re getting from the Chinese labs, like GSPO and SysPO, are primarily for problems that you solve when you have big MoEs and you have expert problems when trying to do the RL. And that’s the whole reason that, like, I think our very serious AI two RL setup, like, we’re doing it on dense models, and we’re just like, “It’s fine. We don’t have this big clipping problem, and as much like we don’t have as much of a need to get the batch size as big to ac- activate all the experts.” So you’re saying you have so many experts and so much sparsity, that potentially sounds like you’re making RL harder.
00:20:36 Lucas Atkins: Um, yes. I will also... I will say that from just, like, a purely post-training side, we added as much as we po- we used- we... So our code base started from TorchTitan. We’ve had to make a ton of modifications to it to get it where we need it to be, but that was an excellent base. And from one of the bigger learnings from Mini and Nano was treating, uh, at least the SFT side of it, as a s- as a separate phase. Um, ‘cause with, with Mini and Nano, we finished the pre-training, we did context extension, then we took those and then ran those on, like, the sixty-four H100s we usually would do post-training on. Um, that presented a lot of challenges, uh, with the MoEs. They, they really... And that’s kind of been a thing in the open space, is post-training MoEs, like, really, um, can be frustrating, even for SFT. So for Large, we added, uh, like, fine-tuning directly to TorchTitan, um, and did it all on the same cluster. So, um, from a performance standpoint, like, SFT was very, um... actually ended up being totally different.
00:21:42 Nathan Lambert: What is the actual difference between the q- the, the implementations then? Is it just kinda like you end up with different batch sizes and parallelism and stuff? Like why-
00:21:50 Lucas Atkins: Uh, I mean, we ended up, we... Yeah, we ended up needing to get it to do really, like, to get context parallelism really well, really good, ‘cause we’re obviously going at a higher sequence length, and then, um, just adding the proper loss masking. Um, it, it, it, it ended up being a relatively easy implementation, especially ‘cause we did all the pre-processing, uh, outside of TorchTitan.
00:22:13 Nathan Lambert: Interesting.
00:22:14 Lucas Atkins: Uh, and then on the RL side, yes, I would say it’s not, um, it didn’t present itself as, as, as significantly harder than, than, um, Mini and Nano. However, that many GPUs does, so we didn’t end up using, uh, two thousand of the B300s for that. That ended up being, uh, a thousand. So two, we just split the nodes in half.
00:22:39 Nathan Lambert: Yeah. That makes sense.
00:22:40 Varun Singh: On the dense model side of things, uh, you mentioned that you didn’t need to use all the tricks and stuff. I, I think it is, uh... I think the, the, it- MoEs are just, in general, harder to RL, but I think it’s also, like, uh, b- because of, like, the KL mismatch between trainer and inference engine, right? Um, where you have, like, uh, sometimes the inference engine can pick different experts compared to, like, the trainer, uh, when you, like, do a forward pass on the same tokens. So I think there is definitely some, like, inherent instability with, with RL on MoEs.
00:23:13 Nathan Lambert: Yeah, that makes sense. Are, are... Okay, um, another question of, like, how much do you want to say? How do you feel about the state of public post-training recipes? Like, do you... Like, I, I feel like there’s so little out there, and there’s an opportunity to be seen as technical leaders by sharing just, like, more of what you’re doing. ‘Cause I feel like we’ve seen for years how complicated things can be, but also at, kind of at the same time... Like, we see this from the likes of Llama, has these really complicated recipes. But at the same time, I feel like just executing on a simpler recipe can get pretty close. But it’s just, like, very uns- I feel, uh, currently unsatisfied with how much I know about what are the actual core trade-offs of doing post-training well. And I think you could do a lot with SFT, but there’s definitely, in this RL regime, more trepidation of kind of narrowing your model to either downstream use or, like, being able to do this multi-week RL run where you get the most performance.
00:24:06 Lucas Atkins: Yeah, I mean, I, I, from-- since RL has become such a pivotal part of the process beyond what, you know, DPO and, and, uh, and kind of your, your typical RLHF was in the past, like, we used to get quite, uh-... sophisticated with, with how we would do SFT and, and even our, our RL. We, we obviously, we make MergeKit, so we, we utilized merging, and we used to do a lot of distillation, um, to eke out as much performance as we could. Now that RL is such a massive part of the entire post-training stack, I, I have almost reverted us to just really solid but simple SFT. Um, like in, in large, I mean, we’ve-- our post-training data set for, uh, Trinity Large is, uh, two hundred and thirty billion tokens. Like, like, it just like a really, really, really large-
00:25:09 Nathan Lambert: That’s ten X what we did. At least in SFT.
00:25:10 Lucas Atkins: And even that-- and even, even your tenant, like that was bef- before this kind of w- going at this scale and even kinda thinking and, and reasoning models. Like our largest SFT before that was five billion to-- we’d do, like, three epochs, but it was like five billion, you know, tokens, so- Um-
00:25:28 Nathan Lambert: Our non-reasoning model is, like, te- another ten X. So, like, our most latest instruct model is, like, two billion.
00:25:34 Lucas Atkins: Yeah, which is, uh, already a lot, you know. So, um, I, I’ve definitely... We-- you know, simplicity’s key because it also makes debugging anything easier, and then, um, devoting a lot of that sophistication to the RL. Our RL part is, like, really important. I do think that, I mean, the next, uh, phase of reinforcement learning for models of this scale is, is just scale. Is, is... Okay, we went from, you know, twenty billion SFT to two hundred and thirty, now we’re going from, you know, ten environments to a hundred. I think that that really is where you’re gonna get the biggest benefit. I also think that’s why, you know, MiniMax and, and, and other players like GLM are so performant and just, like, have that extra bit of, of usefulness that goes beyond just what you see in the benchmarks, is they’ve, they’ve really embraced, like, long-form, uh, RL. And, and so, um, yeah, I mean, to be quite frank, our, our RL pipeline’s rather... immature might be the wrong word. Like, it’s, it’s, uh, there’s definitely a lot more work we could do and a lot more work we need to do, but, um-
00:26:43 Nathan Lambert: Have you started the tool use side of RL?
00:26:46 Lucas Atkins: That-
00:26:46 Nathan Lambert: Or are you mostly... Well, um, beyond like, if you’re training on code, just verifying the code answer, I don’t count yet as tool use. I would say, like, search and code integrated reasoning is what I think is gonna be like minimum table stakes, but do it- to do it well is really hard. Like, we have to, like- ... like, you, you really, like, uh... That’s what I want to do. I want all of our models to have that this year. Search is prob- you have to have, like, a partner to do search or just, like, illegally scrape Google if you’re gonna- ... you’re gonna serve this model onto a customer, and it’s gonna- ... what? Go, go to Google, like, what?
00:27:16 Lucas Atkins: Yeah. Yeah, no, I mean, I, I... Beyond, like, like, really kind of like long-form, like deep research or, um, you know, even like GPT-OSS style or, or G- GPT 5 style, where, you know, it’s doing a hundred tool calls before it gives you a response. Not there yet, um, but that is kind of... Once we get past the, the final kind of RL of Trinity Large, and, and we kinda look at where we go next, like, that is the next major hurdle, um, for sure, and it’s intimidating.
00:27:56 Nathan Lambert: How big is your, your team of- of... Like, how many people are spending the majority of their time on the model? And then I think we c- start to wrap up technical talk and zoom out a bit to ecosystem and company strategy.
00:28:09 Lucas Atkins: Uh, there’s thirteen at Arcee- ... that are just, like, every, every single day is working on it. Yeah.
00:28:16 Nathan Lambert: And I guess that’s a good number because these people are talking about data, but there’s also, like, the whole data thing that’s coming somewhere else. But also somebody else that wanted to pre-train a model, like they could just download the best fully open data set. And I don’t think it’s gonna be quite as good, particularly in the fact that, um, like, if you look at OLMo’s models, we don’t have a lot of tokens, so we need to, like, acquire- ... more tokens in the open still. But to, like, get a number of thirteen, where some are spending a bit of time on data, but there’s the whole data abstraction, is actually kind of nice for somebody that’s like... To do a serious modeling effort, you need to have this many people, I think.
00:28:50 Lucas Atkins: It, it was-
00:28:51 Nathan Lambert: It’s reasonable to me.
00:28:52 Lucas Atkins: It was, it was a good number. I mean, I would say that, um, it, it was helpful to be able to, you know... This was like, how do we alleviate as many concerns as possible? Or how do we check off as many boxes, right? And it’s like, if we’re trying to do this in the shortest possible amount of time, like, we need to focus on what we’re good at, which is we- pretty good at post-training, and how do we get to the point where we’re able to do that? Well, we have to have a pretty strong base model. How do we get a strong base model? We’ll-- we have to, you know, figure out how to do it, perform, you know, efficiently across many, many GPUs, and then data’s, you know, extremely important, so getting a partner that could, you know, help us with that, and we could offload some of that. It, it- there ended up being, obviously, as you, you know, alluded to earlier, like, a lot of, uh, working with Datology and, and, and others to make sure that the data accomplished what we needed it to. Um, I think that that is gonna be an interesting... You know, as we, as we- now that we have Large and we’re looking at, you know, kind of going further, it’s like, okay, you know, the, the pre-training data really has to be in service of what you wanna do in the post-training, uh, work.
00:30:10 Nathan Lambert: How did you identify this?
00:30:11 Lucas Atkins: Like, like-
00:30:11 Nathan Lambert: Like, like- ... did, did you identify this through Mini and Nano, or, like, how’d you come to think that this was so important?
00:30:19 Lucas Atkins: Data in general or, or just-
00:30:20 Nathan Lambert: Or like this in form of post-training
00:30:21 Lucas Atkins: ... of optimizing it for the post-training? Um, I- really ob- observing other, other players, I think. I mean, it’s, it’s... You know, the, the true base model has kinda stopped really being a thing.... around Qwen2, but definitely around Qwen 2.5, um, where you started to see how much post-training data was making its way into the, the, the base models themselves. Um, and then you start to see the models that have done that, how malleable they are with RL, Qwen 2.5, Qwen3 being a good example. And you start to see like, oh, yeah, like they are, uh, doing as much in the last probably thirty percent of training to make it so that when they go to do RL or post-training, they’re gonna have a really good time. Um, you know, they’re just complete-- they’re way easier, way more malleable, way more performant than what you had in Llama 2 or Mistral 7B. So, um, I knew that i-in-intuitively, kind of going into this, but it wasn’t until after Mini and Nano, yeah, where, where we kind of... Well, definitely 4.5B, where we were like, “Yeah, we definitely need to juice our mid-training quite a bit.”
00:31:31 Nathan Lambert: Yeah, I agree. Okay, this was fun. We could- we’ll probably revisit themes from this. I think that, um, I can definitely go over time and keep chatting because I’m enjoying this. And for context, Mark and I had coffee at some point when I was at some conference in SF, and I was like: Damn straight, this is a fun bet that you’re making. So I’m trying to recapture as much of this as you can. Um, for context, it’s like in July, which is similar to when you decided to start this model, which is when, like, Qwen Coder came out, Kimi came out, um- ... GLM 4.5 came out, and I was just, like, looking- and Llama had kind of been, like, become a meme of going away. And that’s why I launched the Adam Project, where I was like: Come on, we need to have some people doing this. And I think that it’s, like, hard in the US because I think there’s so much money to be made on AI. Like, the company- the big tech companies are like: “We see it, and we’re gonna take it, so I don’t need to bother with, like, caring about open models ‘cause we don’t need it.” But from, like, an ecosystem co- perspective and a long-term tech perspective, I don’t think that works very well for the country. So it’s kind of this weird middle ground of like, how do you convince people to actually build open models? I was on... Like, I have calls with people in government asking me, like, what would I actually do? So it’s, like, very hard to think about this. And I have this- and then it’s just, like, to hear that you guys are just making this bet on this is very fun to me, but it’s also, like, based on actual learning from trying to do this. So you’ve been trying to train open models. I think Mark and I have both been at Hugging Face in our past, and you’re, you were trying to sell people on using open models, and there is a market for this, but it wasn’t enough to not have the base models. So I think, like, talking about your experience in selling on-prem open models and why you needed to train your own end-to-end, and why you needed to train bigger, is great because I hope there are more stories like this, and it kind of fills a void and inspires people to work in it. So how- however you want to take this prompt.
00:33:24 Mark McQuade: Yeah, I can jump in. Um, I mean, yeah, I mean, wh- when I started Arcee in 2023, right, uh, it was... All we did was post-training. Uh, and we worked with, uh, a lot of large organizations and did model customization, you know, for their use case on their data. Um, and we were using Llama-based models, Mistral-based models, and then, you know, some Qwen. I don’t even know if we actually did much Qwen, right, Lukas, at that time, but-
00:33:54 Lucas Atkins: No, we did. Yeah, we, we- Later on, but and then-
00:33:56 Mark McQuade: Later on, right? Uh-
00:33:57 Lucas Atkins: We did, and then we ended up not, because after a lot of Chinese models started to come out, then the companies didn’t wanna use Chinese models, so then we kind of went... Yeah, it was kind of just tricky.
00:34:08 Mark McQuade: Yeah, and people don’t realize that that’s real.
00:34:10 Nathan Lambert: People don’t realize that that actually happened.
00:34:13 Mark McQuade: Yeah, no, that’s, that’s a real thing. That’s why we, we started going down to pre-training was because, well, you know, Meta did their thing and kind of got out of it, right? So there was the, the main US player got out of it, and, and we were working with a lot of US-based enterprises that were not comfortable using Chinese-based architectures. And if you wanted to use the best open models of the day, it started to really trend towards, you know, the Chinese labs. Um, and to the point where we are now, where it’s like, you know, ninety-plus percent of the top mo- open models are coming out of China, um-
00:34:47 Nathan Lambert: Yeah, like, Cursor’s building on it and stuff. Like, people are building on these things.
00:34:52 Mark McQuade: Yeah. So, um, we said, “Okay, let’s...” Instead of we were so reliant on the Metas of the world, the Mistrals of the world, and Mistral largely stopped open sourcing, uh, you know, fully. So we said: You know what? We’ll just go down the stack, and we feel we’re capable enough to, to, to train our own models from scratch, and then we control the, you know, the stack. We can, you know, we, we control the core of, of... as opposed to relying on others to release great models. And, um, and then during this time, you know, it just happened to be that, um, you know, there wasn’t a tremendous amount of US companies doing it. So, um, from our perspective, it was kind of a, a win-win, in that we were able to own more of the stack by going down to pre-training and creating our own models, as well as we were entering into a, like, a space that there wasn’t a tremendous amount of competition, to be honest. Um, and, you know, I-- Lukas and I had said this yesterday, I, you know, I think as a startup, every startup doesn’t want to directly compete with, you know, X or OpenAI, or Anthropic, or Google because they have more money than God, and they can do whatever they want. Um, but when you’re doing open weights, you don’t-- it’s, it’s a different kind of compe- they, they don’t sit in there, right? You’re kind of going into your own path, where there isn’t a tremendous amount of players, and you can kind of find your, your way and, and build your niche and, and kind of go from there and, and become something big. So, um, it kind of happened to all coincide for us back in, in July, and, and we went all in.
00:36:23 Nathan Lambert: Yeah, yeah, like, uh, the, the all-in thing is real because this is expensive. I think that- ... I could dig up in my research the cost of daily, um, twenty-four T8 B300. So I think I’ve seen this type of cost at AI too, where we have long rentals, and we’re like: I know exactly how much this costs, and it’s like, it’s not cheap. Are you... A, a way to transition this is like-... do you see the demand? Like, you were selling open models, like, does this kind of be continuous, where people are like: “You helped us deploy this model, but it’s not good enough.” Like, is, is that something that’s happening, and you’re like: “Well, we have this, and we can help you do it coming in this time?” Or is it like you need to build it... It’s like, is it a we will build it, and they will come type of situation? Like, how much- ... continuity is there in this?
00:37:17 Mark McQuade: Yeah, I think it’s largely-
00:37:19 Nathan Lambert: I-
00:37:19 Mark McQuade: I, uh, from my perspective, I think it’s largely if you build it, they will come. Because we stopped, you know, focusing on that whole revenue generation side of the house when we started to go all in on being this, you know, frontier lab in the open source side. So, um, there’s a couple pieces to that, that, that I think we should all be very proud of inside of Arcee, is that we not only went all in by committing a significant amount of capital. Like, we, we committed, you know, sixty-five, seventy percent of our capital to these models, which is a large amount for a startup. I mean, we didn’t... So that’s not like a dip your toe in, that’s like, we’re all the way in.
00:37:55 Nathan Lambert: Yep.
00:37:55 Mark McQuade: Um, but we did that at the same time as abandoning essentially the whole revenue angle to go all in on it, because we couldn’t focus on both. So we said, “We know how to make revenue on open models. We’ve been doing it for two years. Now, let’s take a step back, because it wasn’t, uh, in a repeatable or sustainable way that we h- the way we had that business set up. Let’s take a step back, let’s build these models from scratch, let’s come up with the, the Trinity family, then let’s go back to generating the revenue side of the house and the monetization piece,” which I think we are in a good position to capitalize on even more now, but we, we took a... We, we, we kind of walked away from it to do what we’re doing here.
00:38:36 Nathan Lambert: Yeah, I love this.
00:38:36 Lucas Atkins: Yeah, I mean, when you have... When there’s only, like, thirteen, you know, uh, researchers who would... Well, we’re, we’re doing obviously our own products and own models, but when you’re working with customers, like, inevitably, those are the same people that need to help train those models for customers, and we got to a point where we were really beginning to, like, do mini and nano. We were getting down to, like, the start date of the cluster, where, um, having myself or Mark, or even, you know, Varun and others, like, pulled into customer or, or, or, uh, conversations or contracts, like, it was not-- we would not be where we are if we had continued, you have know, working with, you know, ten customers at once. So-
00:39:19 Nathan Lambert: But-
00:39:19 Lucas Atkins: ... we, we scaled that down pretty drastically. I do think that when... You know, Mark and I put a lot of thought into, “Okay, well, we’re gonna spend all this money to train these models, like, you know, w- how do we not...” I think, uh, one of the things that makes the idea of, of going all in on training open weight models hard, is that you’ve seen other people try it. And, and like M-
00:39:42 Nathan Lambert: Um, like, like do you think Meta or do you think Meta or Mistral went all in?
00:39:46 Lucas Atkins: I, I think, well-
00:39:48 Nathan Lambert: Meta obviously did.
00:39:48 Lucas Atkins: I think they, they both... Yeah. I think, I think that when I say all in, I mean more like Mistral was, was one of the core ones I’m thinking of, where- ... they were a venture-backed company that, like, had a, a, a fiduciary responsibility to bring in money, but were also trying to release open weight models, uh, for, you know, the West, and for their communities, and for the world. And, um, they tried doing closed versions, and then monetizing off of that. They, they also kind of have more recently, luckily, for all of us, gotten back to their kind of Apache 2.0 roots, and-
00:40:30 Nathan Lambert: Oh, my God.
00:40:30 Lucas Atkins: And-
00:40:30 Nathan Lambert: Have you seen the download numbers on Mistral 3 Large?
00:40:33 Lucas Atkins: I haven’t. No, what is it?
00:40:35 Nathan Lambert: Oh, s- no bueno, sir.
00:40:38 Lucas Atkins: Hey.
00:40:39 Nathan Lambert: Carrying on. Sorry.
00:40:41 Lucas Atkins: But, I mean, yeah, you know-
00:40:42 Nathan Lambert: Um, Mist- the, the Large Instruct model has downloads in the last month. I honestly don’t know what’s going on. Maybe there’s some, like, quantized version out there. I, I was confused.
00:40:50 Lucas Atkins: Maybe. Well, I mean, yeah. But I think that we-
00:40:52 Nathan Lambert: It’s, it’s hard to get adoption. The competition is insane.
00:40:55 Lucas Atkins: Hmm. Well, that’s, that’s- ... yeah, I mean, and that could be a whole conversation also, is, like, how do you actually get people to use it?
00:41:00 Nathan Lambert: I was gonna ask you, like, how do you get people... How do you get people to- - really sell into this? You said you’re good at it.
00:41:06 Lucas Atkins: Yeah, I think that the-
00:41:08 Nathan Lambert: Continue your point, we can come back to it.
00:41:11 Lucas Atkins: No, no, but they... I think they all kind of tie into it, is, is... We knew that the, the market was there for, for custom models. It was two years ago, frankly, and it’s even more so now, because RL has drastically, uh, increased the areas that you can hill climb and become really powerful with a tiny model. Um, and but, but also, people are beginning to see how powerful, you know, uh, te- uh, cust- or, or training in a, a, a product is. Like, you see Claude Code, you see Codex, you see, um... I think Deep Research was kind of one of the first ones that really kind of opened my eyes to what was possible, when you kind of are kind of training in the same environment that you’re serving your users. So we knew that, that people wanted it. We’d, we’d had good success with, with customers in the past using other people’s open models. So, um, it was less of a question of, like, could we monetize it, or will we? And it was just a matter of, um, could we get a model, you know, that pe- that, that we would feel that, you know, given a, a wide suite of basically being able to pick any model in the world, would, would our researchers and, and would our teams re- reach towards our own? And, uh, luckily, I think we’re there. Um, on, on the-
00:42:31 Nathan Lambert: Uh
00:42:31 Lucas Atkins: ... on the topic of, like, how do you get people to use it? How do you get adoption? You know, I’ve never wanted Trinity, uh, or our biggest advertising thing to be, like, US. You know-
00:42:45 Nathan Lambert: Yeah, I know
00:42:45 Lucas Atkins: ... like, if, if your entire-
00:42:47 Nathan Lambert: I know, man, it hurts me.
00:42:48 Lucas Atkins: Yeah, if your-
00:42:48 Nathan Lambert: I spent months reckoning with this.
00:42:50 Lucas Atkins: Yeah. If, if your entire, uh, you know, value prop is that you’re an American company-... great, but ultimately people are gonna use the best. Um, and so I think that we’re gonna be able to serve and, and the people like that need a US-based model because their compliance or legal teams won’t let them use something out of China, it’s gonna be a fantastic option. But I think, you know, kind of the next phase of what we’re doing as a company is, all right, now we’ve, we’ve proved to ourselves and maybe the, the wider industry that like we deserve to be in the conversation, and we can train models of this scale. Um, then it’s like, okay, how do we train the best one? Uh, ‘cause really, I mean, people’s loyalties are very fickle, and, and, yeah, you, you go to what’s the best. I guess it’s like, how much do you think
00:43:41 Nathan Lambert: you’ve learned about being able to tune a model narrowly by going and building the whole stack? Um, something we talk about is like ability- ... to specialize models, and I kind of, of opinion that you just make a better general model right now ‘cause the pace of progress is so high. And but the question is like, can we tune a OLMO that’s very good at science or something? And I- ... w-would guess that training the entire model, you’re going to be able to actually do a better job at what you were doing, but I don’t know how to articulate why or what that looks like.
00:44:18 Lucas Atkins: Um, I mean, the, the, the simplest answer to that being yes is just that... or the simplest reason why that’s the answer to the question is yes, is because we know what went into the model. Like, we know what it actually saw at the later stages of training during the decay. Um, and so that all- that helps influence, A, what are we tr- what kind of data and what topics and, and what format are we giving these models, uh, in post-training? But it also allows you to know like, okay, where, where do I absolutely wanna crank, you know, how, how many- how much of this, say, 230 billion dataset, do we want it to be math or, or, or, or coding? And a lot of that’s influenced by what you’re able to put in-
00:45:06 Nathan Lambert: How, how much of your post-training-
00:45:07 Lucas Atkins: ... post-training
00:45:07 Nathan Lambert: -do you expect to redo? Like, uh, how much can you say about when you’re serving something on-prem? Um, you- you’re not gonna redo the pre-training. You might, for a very big customer, redo mid-training or do continued pre-training- ... in which, in that case, you do need the pre-training data to keep, keep it being stable. Which is a use case where like I’m- I would love to see a paper that’s like, “Because of OLMO being open, we continued to pre-train on biology, and we mixed half of their exact mid-training dataset in with our dataset, and it, and it worked,” yadi, yadi. Like, you could obviously- ... do that, but how much do you think is gonna be like the standard, you fine-tune the last instruct model, or do- are you gonna have to retouch the post-training for a customer? Because that, like, I, I really feel like-
00:45:48 Lucas Atkins: Um
00:45:48 Nathan Lambert: ... it’s just at the end.
00:45:50 Lucas Atkins: It, I think, I think-
00:45:50 Nathan Lambert: But it would be fun if you had to change it.
00:45:52 Lucas Atkins: For the most part, um, I think a lot of tasks will be fine just starting from our, our, our, po- uh, like the released, you know, official post-trained version. Um, now, that’s for maybe simpler tasks, is the wrong way to frame it, but if it’s like, “Oh, hey, we’re doing a deep search agent. We want it to do 30 calls and, before...” That would be a good use for just starting with the finished model that we released that’s already post-trained. Now, if we’re going into something along the lines of, um, a very low-resource programming language or, um, something that it didn’t see a lot of in, in, in pre-training, um, or it’s kind of like a, you know, we’re wanting to train this thing to be really good at humanities last exam, but tools. Um, once we get into the world where we’re having to, especially... Actually, I have a much better answer to this question as I was thinking through it, but most of that holds the same. I think that the, the, the world where we’re gonna be doing a lot of extra instruct and, and SFT and, and post-training is gonna be when we’re trying to distill capabilities from large, like into mini or nano. So say like, oh, you know, this large is, is, is really great at invoice processing, but it’s also 400b, and the, you know, the company doesn’t wanna be hosting that on-prem, you know-
00:47:24 Nathan Lambert: Ah
00:47:24 Lucas Atkins: ... let’s go out generate a new one.
00:47:25 Nathan Lambert: Do you have costs off the top of your head for, like, what the hosting costs are for each of the model? Like, do people... Are people all gonna host these models in the same way, or is there actually-
00:47:32 Lucas Atkins: Uh
00:47:32 Nathan Lambert: ... a wide variance? And if you have, like, the same three models- ... do almost all of your customers end up hosting the same way, or do you end up doing a lot of, like, how do you configure the model to fit in the right hosting for them? Like, is that part of-
00:47:44 Lucas Atkins: It depends
00:47:44 Nathan Lambert: ... the business model?
00:47:45 Lucas Atkins: It, it, it, it kind of... And we tried to move a, a, a little bit further away from that because you get into the risk of being like, like a consultancy, and it’s- that becomes tricky, where there’s not a very clear separation of concern. But, um, for the mo- it would change depending on, were they using AWS? Did they have a commit with Azure? Um, if not, okay, then we, we can go to, you know, someone like Prime Intellect or Parasail and, and get a, you know, maybe a, a cheaper rack of eight. Uh, it just really depended. Uh, there’s quite a bit, um, of, of people that were also serving them, just using, like, Llama CPP. So, like, on CPU-
00:48:25 Nathan Lambert: Uh, is the 400b designed to be, to fit onto one rack of eight 80 big gigabytes in FP8? Is that how you designed it? ‘Cause Llama- ... Llama four, whatever, Llama 405b was the same. It was like one rack in FP8 works pretty well.
00:48:41 Lucas Atkins: It’ll do- we... well, you’ll be able to get really good throughput, a little bit lower concurrency on a, a rack of eight H100s at FP8, and then for, like, our, you know, what we’re serving, we’re serving them on, uh, a series of H200s, but we’re not doing, like, multi-node inference. Uh, but that’s just to add more, you know, replicas and- ... other kinds of things.
00:49:03 Nathan Lambert: Hopefully, eventually. I think that the-... Do you have anything else to say about selling open models? I think that generally, like, how do you think about the market for AI? ‘Cause I see the market as being so big, but the- with specifically with open models, it’s so hard to measure. I think I’ve started talking to some of the Chinese labs at all- as well, and I like to ask them, like, this is very US-centric and like Fortune 500 or whatever, and it’s just like, who the heck uses these models? I think- I guess another question is, like, what license or do you know the licenses you’re gonna use for the biggest models? And I think they’re, like, you’re, you’re playing with fire ‘cause people can use it for free, obviously, but potentially- ... you’ll get to hear like, “Oh, shit, somebody actually used our model for this.” And I think any successful business, you’re gonna want... You, you, you know that this model is not gonna be very relevant in a year with the pace of progress. So like- ... how do you think about your license decisions?
00:49:55 Lucas Atkins: Uh, we- you know, with the 4.5B, we tried to do like a, like a, a reve- one of those revenue-gated licensing. So it’s like, oh, it’s completely free for you to use for commercial and whatnot, but if you or your company made over, I think it was like $1.7 million last year, then you need to come to us and get a license. And what we ultimately found was like, it, it didn’t... Maybe for some people who are just only trying to train the model, release it on Hugging Face, and then just call it a day, maybe that is a huge requirement. But when so much of our, our, our company is built around, you know, training custom versions of the models, and, and not even just ours, but in general, even before we did pre-training. Like, at the end of the day, i- as long as we were using it, a- and we knew that we were in full control of, of whether- if we really succeed, it’s because we trained the models, we did them well, and we executed on it well. If we fail, it’s because we, uh, didn’t execute, instead of, oh, some company just stopped releasing good open models. Um, so we eventually switched to just Apache 2.0, and Trinity Large is also gonna be Apache 2.0. Um, you know, I’m- I think it is-
00:51:23 Nathan Lambert: I think this is the right approach. I have a big investor-
00:51:25 Lucas Atkins: Yeah, I think it-
00:51:25 Nathan Lambert: Without, without naming other companies, it’s easy- like, raising a lot of money, whe- or being Meta and releasing open models, and do it- and you could release it with non-commercial, and you could get all these, like... You could talk to, I don’t know, fucking Adobe, whoever. Oh, Adobe’s too big. They’ll have good AI. Some... I don’t know, a bank. Bank of America. You could run Llama on Bank of America and make good money on this. But I just feel like the cultural home of open source AI, and I don’t think- it’s impossible to know who wins it, and I don’t think that you’re in the prime position, and I don’t think that it’s easy to win, but you’re doing a thing that aligns with it. It’s the person that just, like, commits to building the models and learning how the ecosystem works, and to rebuild the models based on the feedback th- that you get from people, and to just kind of commit to an evolving process. And if the whole thing works out, there will be a lot of value, and the person who understands it best should be able to learn how to extract said value. And I think that I’m personally, like, sometimes frustrated with Hugging Face, ‘cause I feel like they have sat on that s- a sort of position like this, and they- ... haven’t figured it out. Not that it is easy to figure it out, but I think that has to be the ideal of open source AI, of like, if it’s really gonna work, that’s, that’s what I hope it looks like. And it’s like, I, I don’皮 know, maybe you guys could do some of that. Like, I have a question of like, could you figure out how to make models that are more fine-tunable- ... after all this post-training? Because you need to sell it to a- you need- ... you, you know the customer’s not gonna want it off the shelf. And I don’t know how to train to post-training to make sure that you don’t, you don’t cook it. Maybe you just learn that you need to warm up the model in a l- in the right way, and you just learn the technique of training downstream. But when you talk to people doing research, the different base models have such different characteristics. I think one of them is character training. I did this paper, and the guy was like: “Qwen and OLMo love their character,” and I’m like, “I have no idea why.” And but it’s like Llama and Gemma, you can change them so much. And I’m like, “Dog, like, please figure out why this is the case.” And for one thing, it’s really cool, but also, like, in your case, that would unlock a lot of value to be like, we know exactly what the model’s gonna do, and we know exactly how to change it. So.
00:53:35 Lucas Atkins: Yeah-
00:53:36 Nathan Lambert: Uh
00:53:36 Lucas Atkins: ... it, it, that’s- no, you’re, you’re, you’re right on the money. I think that even, uh, going into the post-training at large, we, uh, one of our researchers came out with, like, a pretty cool, um, experiment and ablation run that they did on drastically reducing catastrophic forgetting. And I almo- I mean, this was, like, three days before we were gonna start doing SFT, and then we ultimately just... I, I ended up pausing on it because it was just throwing something in that wasn’t tested. But, um, yeah, I think-
00:54:08 Nathan Lambert: A good research lead. You did the right thing.
00:54:10 Lucas Atkins: Yeah, I think, I think one of the most important things long term, you know, as we look at kind of what our research priorities are for this year is, is there’s obviously just how to scale RL and, and make these- the end result of the model as good in as many situations as possible. Um, but I think the other half of that is, you know, how do we make the, the, the speed and efficiency and, and performance of customizing them as, as fast as possible, and as easy as possible.
00:54:42 Nathan Lambert: Yeah. Do you learn in making open models from your experience just kind of running these open software things in MergeKit and DistillKit? I know there was a whole license journey on one of those as well.
00:54:52 Lucas Atkins: Yeah, DistillKit.
00:54:52 Nathan Lambert: Do you feel like they’re kind of isolated?
00:54:54 Lucas Atkins: Or MergeKit. Um, yeah, I mean, I think so. I think that, that, um, you kind of have to play the tape out. With MergeKit-... it was by far our most popular piece of software we’d ever released, but it was so popular because it took something that isn’t fundamentally very complicated, but we ma- but it’s time-consuming, and standardization is great for things like that, and we made it, uh, you know, streamlined and easy to do and fast, and you could experiment and ablate really quickly for, you know. And, and so I, I think that when we switched that to, like, a, you know, a, a similar, uh, revenue-based licensing, like, it, it didn’t end up having the value prop that was important because are you gonna pay Arcee, you know, thousands of dollars, or are you just gonna have one of your researchers-
00:55:52 Nathan Lambert: You’re gonna have clone code in a week, right?
00:55:52 Lucas Atkins: recreate it in a week, right? Yeah, so it’s-
00:55:55 Nathan Lambert: In a day.
00:55:55 Lucas Atkins: It’s, it’s kind of... It, it’s remi- it’s remembering like, okay, what is- what problem is this solving, and is this even a prob... Like, is the solution to this monetizable? Um, and so MergeGit, we brought it back to the original license, but I think with even viewing the models in the same way, it’s like it’s... Open source is an unbelievable marketing tactic. Like, there’s no one would care about Arcee if we weren’t open sourcing stuff, ‘cause as soon as you do something closed source, if you’re not the best or the cheapest for your price point, I mean, your performance point, no one’s gonna use it. Because-
00:56:30 Nathan Lambert: Um, another question on this. Um, do you think that open models are kind of at a disadvantage when progress is so high? Because it’s potentially easier to swap APIs than open model configurations, especially if, like, model weights are changing sizes or something like this. Where it’s like, “Oh, I can just upgrade to the new Opus, and I do this.” Like, does that, like, uh, decentivize people from using it? Or do you think most of the people are like: “I can only use open models, therefore, I’m gonna use open models?”
00:56:56 Lucas Atkins: Uh, I think for the people who are using, like, s- either self-hosted or, you know, um, uh, bespoke, uh, you know, engines to, to run it, where they have complete... You know, in a VPC or they have complete control over, like, data in and out, egress, ingress. I don’t think that’s really gonna be so much of a problem because they’re obviously doing it for a reason. Um, like, they’re either for privacy or security or, or HIPAA or SOC 2. For whatever reason they’re doing it, um, I, I don’t think that that’ll be, um, so much of a blocker, but I definitely do think that, um, you know, by far, e- even, even with some of the, the larger open... You know, like inference players, like Together and Fireworks, that, that host a lot of open models. Like, being feature- being on feature parity with a lot of these, these larger labs’ APIs is gonna be extremely important, um, o- of being able to serve, you know, um, with features that they’re used to, like prompt caching, that kind of stuff.
00:58:03 Nathan Lambert: Yeah, are- like, I, I think I saw that you guys are setting up an API as well. Is that kind of what the vision there is, is being able to o- offer parity at least, or, like, make it easy for people to consider it?
00:58:13 Lucas Atkins: I think so. I, I- we’re- we very... Yeah, we are doing our own API. We are hosting it. Um, we haven’t- we, we push a lot of that through Open Router just because it’s such a great place to get, like, discovered. Um, as... If we see, like, tremendous growth there, that would obviously be where we’ll, we’ll invest very heavily. Um, whereas the right move might be to let other people host it, and we invest super hard on the infra for, like, make- taking advantage of the models, um, and, and customizing them. There’s, there’s, there’s a few avenues we have ahead of us then, and we have, you know, projects going kind of toward to poke at each one. Um, and we’re just kinda getting as much data as we can before we... I mean, we’re gonna have to go all in on another direction soon. Not, not like pivoting away from pre-training, but now that we’ve done that, now w- what’s the next big bet we’re gonna make, and how do we go fully into that? So we’re trying to figure out what that is.
00:59:12 Nathan Lambert: Yeah. My two last kind of, like, real questions are, like, one is... I guess I can start with, like, where do you see the open model ecosystem? Do you think- where would you see it changing substantially in the next six or twelve months? I, like... Or, or do you? Or you just kinda think we’re marching along for a while?
00:59:31 Lucas Atkins: No, I think we’ll, I think we’ll, we’ll be... I, I, I don’t think it’s an unrealistic prediction to make that by the end of 2026, like, the best model in the world is, is some degree of open. Uh, I think that’s very, very possible, especially with, like, what I’ve seen GLM and, and MiniMax do recently. Um, they have started to find that secret sauce that takes you out of just being good on benchmarks and, like, genuinely useful in people’s day-to-day workflows. And, um, I wouldn’t- like, if, if I, you know, came back, and I... Someone came from the future and told me that the best model in the world was, uh, an open-weight model, I wouldn’t be surprised. I actually think we’re on a, a, a super good trajectory, and, and, and fostering and, and promoting that kind of work and adoption here in the United States is gonna be extremely important.
01:00:24 Nathan Lambert: And where do you see the company going? ‘Cause like, like, I have my guess. Like, you kind of hopefully-
01:00:31 Mark McQuade: What’s, what’s your guess? I wanna hear your guess.
01:00:31 Nathan Lambert: Um, you can hopefully do a mix and kind of oscillate into trading when you get... Like, you need to start having the feedback of the real world. I think that’s obvious. Like, it’s o- like, it’s... Well, obviously, you need to make money to survive as a company, but then you need to start using that as the feedback to guide training. And then it’s like, you need to figure out how to balance and do some of them at each time, and you can plan your cluster at different times, and then you kind of... Hopefully, they become a, a loop across each other, and they kind of make it so obvious of why you each need them, ‘cause it, it seems somewhat natural.
01:01:03 Mark McQuade: Yeah, I mean, exactly. You know, you kinda hit, hit it right on the head. Um, you know, getting feedback and then kinda steering the ship from there, um, is, is probably-
01:01:15 Lucas Atkins: ... exactly what we’ll do, but we have a good idea already. I mean, first and foremost, you know, we talked about it earlier, w- we’ve spent a tremendous amount of money. So, uh, we need to go raise some money after we - after we get, you know... We need people to back the, the, the mission and the vision of US open source and, and, you know, so, um, because, uh, you know, we, i- i- Lucas had mentioned about, like, MergeKit and how we flopped the license and, you know. I mean, we’re a smaller-sized start-up. We have-- we’re-- we gotta think of kinda unique ways to try and generate revenue because we don’t have the money of the large labs. So, uh-
01:01:52 Nathan Lambert: Well, I think it’s a benefit to the employee. I think a lot of these labs have over-raised.
01:01:56 Lucas Atkins: Yeah, I like, uh- uh, I-
01:01:57 Nathan Lambert: OpenAI, Anthropic, and all of them are fine. Like, with the OpenAI, Anthropic, Cursor scale, like, let it rip. They should, they should really rip the raising. But all the other companies that are stuck at the, like, the one to two billion range without, like, obvious traction, like, the risk goes to the... I mean, you could-- a lot of them do secondary, so a lot of the founders get out. But it’s like, the risk is the employees get nothing.
01:02:21 Lucas Atkins: Yeah. Yeah.
01:02:22 Nathan Lambert: There is a lot of money, but that’s also why I like the approach, ‘cause it’s like, “Oh, you’re doing the actual start-up thing.”
01:02:28 Lucas Atkins: Yeah, yeah. Yeah, I mean, I think... W- what I was gonna add to what Mark... is just like, what- whatever we do from, uh, uh, uh, scaling and, and speeding things up and growing, um, my goal is to keep our research and engineering teams pretty small. I think, I think that one of the reasons we’ve been able to, to move as quickly as we have is it’s been, like, a small group of, like, highly intelligent, smart, and opinionated people sitting in a room, debating in good faith on decisions. And I think that that’s, uh, uh, under the constraints of, “Hey, we don’t have five hundred million dollars to go and, you know, to rip on, on, you know, X, Y, and Z.” So and I think that’s kind of where creativity comes from, and I think that fostering a culture like that over time is how you can kind of make it so that excellence is less of like a, um, an accident, and it’s actually, like, a by-product of the way that you work. So, so we’re gonna stay small, we’re gonna stay lean, but, um, I, I do think that, like, the, the major, um, kind of challenge for us over the next probably six months, beyond any other models we might have, kind of, uh, think or we’re thinking about, is, is getting up to, like, post-training parity with the likes of DeepSeek, and GLM, Qwen, and others.
01:03:47 Nathan Lambert: Yeah. I, I hear lots of horror stories about this, where it’s usually and-- it’s-- you end up having people that are going after different important abilities, but, uh, like, doing each of the abilities alone is pretty easy to hill climb, but then you just end up with such a mess. It’s like you’re- ... building a custom puzzle, and you’re building all these custom pieces, and they’re magnificent, and then you’d have to, like, pick up these pieces and assemble this unknown thing at the end. And it’s like-
01:04:12 Lucas Atkins: Like they didn’t have the same designer, right? Yeah.
01:04:15 Nathan Lambert: As AI2 is barely scratching the surface of this. Like, you talk to the people at the frontier labs, and it’s like, holy cow, like, post-training is really the Wild West. But a lot of it works. I think, like, we find-- like, even like model merging gives a ton of performance across the whole- ... training pipeline. It’s like- ... you merge at pre-- you merge after each pre-training stage, you merge in post-training. It’s like-
01:04:35 Lucas Atkins: Roon can tell you.
01:04:36 Nathan Lambert: But merging post-training becomes a lot more complicated because you- ... can have all these domains and things, uh.
01:04:41 Lucas Atkins: Well, in, in merging, you know, it, it actually, it used to be very YOLO, um, the way we used to do it, and, and Charles, who, who created MergeKit, I call him, like, chief alchemist, and, like, you’d kinda just send him ten promising checkpoints, and he’d come back a day later with, like, some insane, you know, model that was really good at all of them. And, and you can’t do that as much in post-training anymore because of, uh, of just the, the formatting and the way that RL is done. Like, you do have to be a little bit more surgical about it, but yeah, everyone can tell you, like, any time we start to see anything worrisome at all in training or, or, or even something going really good, you know, “Lucas, what do we do?” I’m like: Merge it. I’m like, just-
01:05:21 Nathan Lambert: Merge.
01:05:21 Lucas Atkins: ... I’m like: “Just take it, just merge it. Let’s see.” And more often than not, it fixes it, so...
01:05:27 Nathan Lambert: Um, do you merge during RL? Like, you could just, like, merge the last few checkpoints and resume or something?
01:05:32 Lucas Atkins: We’ve ex-- we’ve, we’ve dabbled in that, not, not for what we’ve done. You know, again, a, a lot of the, the mini, nano, and large story for Trinity is, like, getting to a level of... what was my level of complexity I was comfortable with us undertaking, and then, uh, not introducing anything more. So, um, not yet. But we, I mean, we, we, uh, regularly merged. We didn’t do it for LARP, but we used to merge a lot, um, during just, like, your standard, uh, um... When we’d do, like, RLHF, we used to do a bunch of merging. We’d do it, like, every five checkpoints. We would-
01:06:11 Nathan Lambert: Online RLHF or D-DPO?
01:06:13 Lucas Atkins: There’s DPO.
01:06:15 Nathan Lambert: Yeah. It’s so much easier to get started. One of my goals is to have somebody figure out how to do actual online RLHF, pure LM feedback, obviously, for scaling. But it’s just like- ... it’s, it’s unsavory to it’s just, like, doesn’t look like DPO-
01:06:28 Lucas Atkins: Yeah, I mean, if, if, you know, if GRPO and kind of op-- in, in the, the present day RL regime, like, if that hadn’t materialized when it did, I think that would’ve been a big topic in 2025. But I do think that, you know, GRPO and just the overall, um, DeepSeek and o1 style reasoning and thinking and RL kind of... Any, a- any person who is thinking of doing that for, like, performance reasons, realize that there was something that had fifty thousand papers released every day on how to do it. Um- ... that was kind of probably right where you’d get the same amount of performance.
01:07:07 Nathan Lambert: Um, do you force dog feeding? Do you make yourself-- do you guys use your own models to understand them? Like, do you, like, make that a thing?
01:07:14 Lucas Atkins: Uh, Mini was the first one we could actually start doing that with, um, a- at least for, uh, a more general day-to-day tasks. So a lot of our, like, internal Slack, we have stuff that, like, monitors Twitter and LinkedIn for feedback on Trinity and, and, and that kind of stuff. That all runs on Trinity Mini now. Um, and then, uh-... you know, we, we put a good amount of work into, into large being, um, you know, good in, in a bunch of your, like, OpenCode and, and Cline, uh, and, and Kilo Code. So, um-
01:07:45 Nathan Lambert: Uh, what does that, what does that work look like?
01:07:49 Lucas Atkins: Uh, working with those guys to get data. And then, um-
01:07:53 Nathan Lambert: That’s, I mean- Good for me to know.
01:07:55 Lucas Atkins: I mean-
01:07:55 Nathan Lambert: I should do that, I guess.
01:07:58 Lucas Atkins: Yeah. Yeah, working with, uh... Or, or I mean, it- the way it started was us, like, using open models and then, like, passing those through as the base URL, and then, like, getting the logs from that. Um, and then realizing that, like, that translated pretty well. Um, and then over time, obviously turning this-
01:08:16 Nathan Lambert: Um, can you expand on this? So I was gonna ask you-
01:08:19 Lucas Atkins: So-
01:08:19 Nathan Lambert: -if you’re, like, using these open models regularly, ‘cause I, I’m just, like, Claude Code psychosis, man. I’m like, “Can’t take that away from me.”
01:08:26 Lucas Atkins: Yeah, I, I use, I use four... I’ve used 4.7 a lot. I think 4.7 from GLM was one of the first ones that could replace a lot of my day-to-day. Uh, I’ll still reach for Claude Code or even 5.2 Pro if it’s, if it’s, like, something that’s, like, really... I- if I do not know how to measure what success looks like for something, I’ll usually use those. Um, but, uh, yeah, I mean, it, it- even using DeepSeek before, um, kind of their May update was hit or miss. But, um, yeah, w- the reason I decided to, like, start talking to these people and working on, like, how can we get data and, and start making our models good in these systems was I would use them. I had a, um, you know, something that would grab the logs, like, it, you know, inter- as a proxy, so it’d like grab the logs and then format them in the messages format. And then I saw that and went, “Yeah, that’s... You can make a pretty good filter for just, like, standard stuff that you don’t want, and kind of hit a scale.”
01:09:30 Nathan Lambert: Yeah, it makes sense. So, so you’re like, uh, open code will let you look at the data, and then you’re probably gonna get a sense for... Like, I don’t even actually know how the, on the back end, the code agents in open code format data, which I think is actually something I should just go look at, ‘cause then you can design around.
01:09:44 Lucas Atkins: Uh, they’re all different. Yeah. Yeah, but you just have to- you just- basically, it all starts from like, what do you want your format to be? And then how can you take what, what those look like to, you know, to... How do you force it into that? The hard thing, though, is, is with newer models like MiniMax and 4.7, the way they do interleaved thinking is, is like... You know, I’m a big believer in post-training. Like, if you’re gonna do interleaved thinking, like, every sample in your data set should be that. Um, it, you know, it should follow that same format and that same behavior. So, um, that gets tricky if you’re trying to, like, take a bunch of Nemo tr... Or, or, or, well, like, uh, DeepSeek data and Qwen data, and then, oh, we’re also trying to mix in MiniMax, and at that point, you’re- it, it gets really difficult ‘cause they all handle thinking slightly differently.
01:10:34 Nathan Lambert: Yeah, I can buy this. Um, okay, this was fun. Any last predictions or things you want people to know about the model? I will say that, um, when you debuted the Trinity models, you had a great blog post that was very to the point, that covered a lot of this. So I’ll definitely link to the, um, what is it? The Trinity manifesto. I enjoyed reading it. So I’ll link to that in the show notes, and, oh, hopefully you have a new one for me to read when you’re done with the model.
01:10:58 Lucas Atkins: Yeah, we’ll do- we will have a tech report. We’ll have a tech report for you, too. So we, we never, we never did a tech report for 4.5B Mini or Nano because we were so focused on just getting to large, but we also thought it’d be very interesting to write it under the, the... How do you go from 4.5B to a 400B MoE in six months, and, like, what did we learn-
01:11:19 Nathan Lambert: That’s right
01:11:19 Lucas Atkins: ... when you’re viewing it as a whole, so.
01:11:21 Nathan Lambert: That’s about the timeframe that, um, Ant Ling took, too, as well. Ant Ling, uh, the anchor, we talked about, they’re like... It took us about six months to do, um, Ring-1T and their 1T models, which, like, it sounds like a lot more, but I think that’s about the same. It, it depends on compute and configs and stuff to go from, like- ... basic modeling to big MoE, which is pretty interesting to see a lot of people speedrun this sort of thing.
01:11:46 Lucas Atkins: Yeah, it’s, it’s a really, uh... It is a logistical nightmare, but, like, I think everyone on the team has had a tremendous amount of fun over the last, uh, six months. So now the fun begins.
01:11:58 Nathan Lambert: Yeah. Congrats on the milestone. Congrats on the model existing. That has gotta be an almighty relief, and I’ll look forward- ... to see what you all are up to soon. I’ll stop by at some point next time I’m in the Bay.
01:12:10 Lucas Atkins: Yeah. Yeah, come by. Yeah, come by.
01:12:12 Nathan Lambert: Thanks for-
01:12:12 Lucas Atkins: Thanks for having us.
01:12:14 Nathan Lambert: Yeah. Thanks, guys.
by Kieran Klaassen .png)
While we’re on our Think Week offsite this week, we’re resurfacingCora general managerKieran Klaassen ’s work on the theme of compound engineering. In this piece, Kieran argues that the best thing you can do to improve your AI-assisted coding is to plan. He introduces a framework for deciding when to plan versus when to prototype, and gives a real example of how one hour of planning saved days of debugging when he wanted to turn some design plans in Figma into a product. So take that extra hour and plan. You’ll thank yourself.— Kate Lee__ AI made us sloppy because it made us forget how to plan. Planning used to be a non-negotiable part of the work: sketching screens, prototyping flows, and writing problem statements. You had to define the scope—what’s in, what’s out, what’s too ambitious, and what solves the problem. Good planning required good thinking, good writing, and collaboration between stakeholders. It was slow, but it prevented expensive mistakes. When vibe coding emerged, planning went out the window—at first. Why spend an hour planning when you could spend five minutes building the feature? I did it, too. “Make this feature work” was my entire instruction. Sometimes it worked. Often it didn’t. When it didn’t, I’d spend three hours debugging an error that a 10-minute session—asking AI to create a clear outline of the problem and the research needed to build a solution—would have prevented. And I’d be starting from zero with each feature I shipped, instead of the AI improving with each request. When you vibe code, you prompt, “Add email validation to the signup form,” and hope the AI takes the right route. When you plan with AI, you write: “Research how we handle validation elsewhere in the codebase, check if our email library has built-in validation, look up best practices for user-friendly error messages, then create a plan showing three approaches with tradeoffs.” One approach ships a feature. The other ships a feature and teaches the system how you think for next time. Get this right, and the system learns from every plan. Let me show you how.
I had five screens of Figma designs staring at me, and a weekend to turn these pixels into a product. We were preparing for the launch of Cora ‘s email bankruptcy feature—a free service that clears users’ inbox for them without deleting anything important. Lucas Crespo and Daniel Rodrigues , Every’s designers, had turned my ugly-but-functional flow into those beautiful Figma designs: something people would want to use, with clean layouts, thoughtful interactions, and the kind of polish that sets software that delights apart from software that works. Now I had to build it. As recently as early 2025, that would have meant: Hook up the Figma MCP plugin (a tool that connects design files to code), watch it produce something vaguely related to the design but mostly ugly, then spend the weekend manually fixing it—squinting at measurements, guessing at spacing, writing HTML, refreshing the browser, noticing it’s wrong, adjusting, repeating. Days of work and frustration. This time, instead of coding all weekend, I spent one hour that saved me days. I created an AI agent with one job: Take a Figma design screenshot, analyze how to implement it, and output a detailed plan grounded in our patterns, components, and way of building. 
My agent analyzed the Figma design and produced this implementation plan, automatically stored in GitHub. (All screenshots courtesy of the author.) Once the plan was complete, I added a second agent to review the work: Compare the Figma screenshot to what got built using Puppeteer (a tool that automatically captures screenshots of web interfaces), note every difference, and keep iterating until they match. Because the plan was clear and detailed, the review agent could focus entirely on execution, instead of trying to figure out what we were even building. I got five screens, pixel-perfect, including mobile layouts that were never even designed for. The plan guided the work step, and pixel perfection came out the other side. 
The new email bankruptcy flow I built with help from my planning agent. The next time we need to implement a complex interface, I won’t start from scratch. I’ll use the same system and the same planning workflow, and it will be faster because the system learned from this round. This is compounding engineering : building systems where every unit of work makes the next one easier because you’re teaching the AI what to do. And the fastest way to teach is not through code you write, but through plans you review.
The first step to planning effectively is recognizing that not everything deserves the same investment. I think about engineering work in three categories—what I call fidelities—based on complexity and clarity.
This is the one-line change, the copy update, the obvious bug fix. A button that’s the wrong color or a typo in an error message. Maybe a small bug where the fix is self-evident once you reproduce it. As models improve, “quick fix” expands. With Claude Sonnet 4.5 , Fidelity One work now includes: changing pricing across the entire codebase, normalizing emails automatically, reorganizing code to remove unused features, fixing tests that accidentally broke, updating libraries and migrating dependencies (switching to newer versions of pre-built code tools you’re using), or resolving comments in a pull request (a batch of proposed code changes) when the feedback is clear enough. Six months ago this was multi-hour work. Today, it’s 10 minutes with a well-constructed plan. 
I gave my planning agent an error message and three words: "Go fix it." It did the rest—creating the bug report, identifying the cause, and shipping the fix. For these, planning is lightweight. Even a quick “reproduce the bug, confirm the fix location, check for similar instances” catches edge cases you’d otherwise miss.
This is where compounding engineering and the power of planning shines. These are features that span multiple files, require some refactoring, and have clear scope but non-obvious implementation. Things like: moving something performed inline (happening immediately when requested) to a background job (processing that happens separately, without making users wait), adding a new tool call (a specific action the AI assistant can take) or capability to the assistant (like archiving emails by query), or researching and reproducing bugs where you’re not yet clear what the actual problem is. For Fidelity Two work, planning yields massive return on investment. The problem is complex enough that AI might go off the rails without guidance, but simple enough that once you have a good plan, AI can execute it reliably. This is where I spend most of my planning energy, and where the system learns fastest. Recently, we needed to add the ability for Cora’s AI assistant to archive emails by query, so users could prompt “archive all emails from newsletters” and it would work. I could have prompted Claude Code directly: “Add an archive by query tool call.” Instead, I had it research first: “How do our existing tool calls work? What’s our pattern for handling bulk operations? Are there any performance considerations with archiving many emails at once?” 
I describe the problem and walk away. Multiple agents research in parallel and return 10-20 minutes later with a detailed plan. The research revealed we already had a tool that interprets search requests that could be reused, and that we needed restrictions on how many requests you can make, because Gmail’s API (the system that lets other software interact with Gmail) has strict quotas on bulk operations. Without that research phase, I would have built something that worked in testing but failed when users tried to archive thousands of emails, because there are so many real-world use cases I would not anticipate. Once the plan was clear, the implementation took minutes. The 20 minutes spent understanding the problem saved hours of debugging production failures.
These are major features where you don’t even know what you’re building yet. Adding multi-account support. Rebuilding the high-level structure of how your code is organized. Integrating a complex third-party system. The requirements are epic, the scope is fuzzy, and no amount of planning will give you certainty because you’re still figuring out what “done” looks like. For Fidelity Three, planning alone isn’t enough. You need a hybrid approach: rapid prototyping to clarify what you want, and rigorous planning to build it properly. I call this vibe planning—vibe coding, but for disposable software that helps you think. Spin up quick prototypes in a separate environment, click through them, learn what breaks, throw them away, and plan the real implementation based on what you learned. The prototype is disposable; the knowledge isn’t. Sometimes you start with what looks like a Fidelity Two problem and realize mid-planning that it’s Fidelity Three. The email bankruptcy flow felt straightforward at first—add bulk operations so users could process, say, 53,000 emails at once. During the research phase, I discovered we couldn’t just call Gmail’s API in real time because of rate limiting and time constraints. We needed a cached version (storing data temporarily so we don’t have to fetch it repeatedly), highly optimized, with API rate limits carefully managed, and a way to handle failures gracefully when operations take minutes instead of seconds. The scope ballooned. I thought it would be a simple case of “add a feature.” It turned into ”redesign how we handle bulk operations entirely.” Once I realized this was way bigger than expected, I switched approaches. I stopped trying to plan the perfect solution and instead built three prototypes with ascending levels of difficulty: one with real-time API calls, one with a simple cache layer (temporary storage for frequently used data), one with a full queue system (a line of tasks that get processed one at a time). I didn’t want to add complexity. I went in hoping the first, most simple solution would work. Each prototype took a few days to build, and within a week it was clear what we had to build into the product. The real-time version choked on 1,000 emails. The simple cache had race conditions (bugs that happen when multiple processes try to access the same data simultaneously). The queue system was the only thing that worked. The prototypes convinced me that the complexity wasn’t optional—the simple solutions would break. Those prototypes gave me enough clarity to break the problem down: Build the cache layer first, then add the queue system, then optimize the API calls. Each piece of the plan now had a specific job and clear success criteria. Instead of building one big feature, we broke it into separate, manageable parts: the caching layer, the marketing design, and the agentic flow. These were three distinct pieces that initially seemed like one monolithic feature. Trying to build everything at once created plans that were too big to execute. Breaking them into sequential steps made each piece shippable. The goal with Fidelity Three is to break the project into multiple Fidelity Two pieces. You can’t plan your way through genuine uncertainty. But at least you can prototype your way to clarity, then plan your way to quality from there. 
A prototype I created to explore and test a new Brief feature.
There is a difference between teaching AI through code versus teaching through planning. Coding teaches, “Here’s how to solve this problem.” Planning teaches, “Here’s how to think about problems like this.” When Claude Code writes code based on your feedback, it learns the specific solution to one specific problem. The system creates plans, you react—“This is too complex” or “We need to sequence this differently”—and that feedback becomes permanent knowledge. Future plans automatically incorporate your preferences. AI learns your architectural thinking and applies it to every new problem. For example, when I first had the Figma agent implement designs, it used plain HTML, which I didn’t want for a reusable design system. I corrected it: “Use View Components instead—that’s our component framework.” I codified that preference into the agent’s instructions. Now every design implementation starts with View Components by default. In week one of working this way, plans would come back with approaches I’d never take—over-engineered solutions, missing obvious existing patterns, forgetting security checks. Three months in, plans come back largely reflecting how I’d approach the problem myself not because I’m prompting better, but because the system has learned from more than 50 plan reviews how I think. Better AI models make the system better. GPT-5 or Claude Sonnet 4.5 or whatever comes next will make better plans automatically. But your specific system gets better because you’re accumulating institutional knowledge. Your agents know your preferences. Your research strategies know your domain. Your review process catches your common blind spots. Planning is the highest-leverage activity in AI-assisted development. One hour spent improving your planning system makes every future hour more productive. In the next piece, published tomorrow, I’ll share eight strategies that you can use for more effective planning, plus an experiment that you can use to become a better planner.
Made with ✨ by First Round Capital.
The newsletter for the technically curious. Updates, tool reviews, and lay of the land from an exited founder turned investor and forever tinkerer.
Hey folks,
Prism by OpenAI - It’s a LaTeX editor for scientists to draft/work on research papers with AI. Free for ChatGPT users, with unlimited projects and collaborators. It has an editor that both you and GPT-5.2 can edit, and a preview that renders the LaTeX input. Read more on their blog. Also, Prism’s founder went from idea to OpenAI in less than 16 months.
Two thoughts on this:
Last year, the first release from OpenAI was Operator - a web browsing agent, and we saw dozens of variations of it in 2025. This year might be document editors.
AI for developers is a relatively solved problem for OpenAI. Their next “frontier” target audience is definitely scientists with plans to build an AI researcher intern by Sept 2026.
Related: OpenAI’s CFO, Sarah Friar, recently talked about licensing OpenAI models to companies in a way that if they contribute to a discovery, OpenAI can get a piece of the pie. (more clarifications)
Gemini 3 Flash can now plan how to analyse an image and use tools to zoom in and annotate it for better visual understanding. It increases performance on visual benchmarks by 5-10%. Google calls it “Agentic Vision ”, though OpenAI’s models have had this since o3.
Chrome is also becoming an AI browser. The Gemini integration now opens a sidebar with features like image generation, access to other Google tools, and auto-browsing. Only in the US for now.
Give your AI the power to listen. Gladia is a speech-to-text API that turns real conversations into structured data for agents, workflows, and automation. ~300ms latency, 94% accuracy, and a lightweight SDK lets you add production-grade voice in minutes. Test real-time or async in our playground.*
AI-assisted development at Block - Repo readiness, parallel agents and automated PRs.
How can agents automate improving a software product using a closed loop of user feedback to shipped features?
Can AI make you feel disempowered about your reality - new research from Anthropic.
How does the creator of Clawdbot (now Moltbot) ship?
PropelAuth is the #1 choice for developers building in B2B. Signup/Login, MCP, RBAC, Enterprise SSO, and more , all out of the box.*
Momo - Stop asking your team for progress updates. Connect all your work tools and build your team’s memory.
Ami Browser - Build a feature → your agent tests the web app and fixes bugs.
Manus Skills - Turn a session into a skill or import Skills from others.
o11 - The copilot you deserve in all your Microsoft 365 Apps.
Shortcut - The Excel AI agent that you can actually use and works better than Claude, Copilot and others. (demo on 4 real tasks)
Variant - Enter an idea and get endless (beautiful) designs as you scroll. (see more)
Ian built this skills directory with a file explorer to read the skill, an upvote/downvote system and a cli to bring them to your projects.
Beautiful Mermaid - Render diagrams to understand your code, but cleaner than plain Mermaid.
Rivet - Universal API for automatic coding agents in sandboxes. (docs)
vercel-react-native-skills - Lessons from building the v0 iOS app and Vercel app.
OpenInspect - Open-source background coding agent system inspired by Ramp’s Inspect.
Gemini CLI has hooks now and hence a Ralph Wiggum extension.
Flapping Airplanes - new foundational lab with $180M in seed funding to make training for AI models more data efficient. Kate from WSJ has a write-up on this company and the growing category of neolabs.
Anduril is hosting an autonomous drone racing competition to compete for $500,000 and a job at Anduril.
Snap is making Specs (its unit for building AI glasses) a distinct subsidiary.
Many members of the Cline team have joined Codex.
That’s it for today. Feel free to comment and share your thoughts. 👋
Read about me and Ben’s Bites
📷 thumbnail from A Beautiful Mind
In January 2020, at CES in Las Vegas, Jeffrey Katzenberg took center stage. He was there to pitch Quibi, his bold $1.75 billion bet that aimed to revolutionize mobile entertainment with ten-minute video ‘quick bites.’ But, he could feel a palpable sense of skepticism from the audience of tech reporters and twenty-somethings.
He stopped mid-pitch and said something that only makes sense if you’re certain you’re right about how the world works.
“I’ve been doing this before you all were fucking born.”
He had the résumé to back it up. As head of Walt Disney Studios, he had greenlighted The Little Mermaid and The Lion King. He had co-founded DreamWorks with David Geffen and Steven Spielberg. He had seen the future of entertainment several times, and each time, the future had bent to his will.
Four months later, Quibi launched.
Eight months after that, it no longer existed.
Despite Katzenberg’s validated instincts in entertainment, the failure of Quibi highlighted a crucial truth: reality is indifferent to past achievements. His track record was built on respecting reality’s terms, not his own.
The difference between successful founders and those who fail often hinges on a critical factor: their relationship with being wrong.
Some founders treat their thesis as a hypothesis to be tested. Others treat it as a conclusion to be defended. The first group seeks reality. The second seeks validation. And the second group often finds elaborate and expensive ways to confirm their beliefs.
Why does hunting for criticism beat seeking validation? Because criticism is asymmetric. One disconfirming fact can eliminate an entire category of error. A hundred confirming facts can’t prove you’re right; they can only fail to prove you wrong. The philosopher Karl Popper called this falsification. Theories can never be proven true; they can only be proven false. A thousand white swans don’t prove all swans are white. One black swan ends that debate.
In founder terms, a test that cannot plausibly compel you to change course is not a genuine test. The real mistake isn’t being wrong; it’s creating systems that hide mistakes, stopping us from learning and adapting.
Once you see it, the logic of validation collapses. You’re already embedded in a process of conjecture and criticism whether you like it or not. Every day, the market criticizes your theory. Customers who don’t return criticize your theory. Features that don’t get used are criticisms. The question isn’t whether to invite criticism into your process, but whether you’ll engage honestly with the criticism that’s already happening.
Brian Chesky and Joe Gebbia put up a WordPress site out of desperation, not foresight. They couldn’t afford rent and needed quick cash. So they offered strangers a place to sleep on an airbed in their apartment. A surprising number of strangers were eager to say yes.
Most people would have written off those guests as outliers. Chesky asked a different question: does this reveal something important that everyone’s missing?
The behavior of strangers trusting strangers was surprising, but it was also real. Chesky didn’t have to imagine it. He’d seen it. His job was to understand it and amplify it. Once he reframed the behavior as a clue rather than a fluke, he could test a conjecture: if strangers can trust each other, hospitality can be reinvented. Professional photos increased bookings. Reviews drove repeat stays. Better payments reduced friction. Each experiment sharpened the picture.
They found product-market fit because they kept accepting reality’s terms.
However, Airbnb’s relationship to criticism is complicated; Chesky also ignored much of it. Investors called the idea insane. The early data looked terrible. If he’d been purely responding to criticism, he might have quit.
What he actually had was a specific form of stubbornness. He trusted the behavior of the guests who showed up, even when every other signal said stop. That’s not ignoring criticism. That’s knowing which criticism to trust.
Guests who stayed shared real experiences, while investors who didn’t join in just talked about market trends. They didn’t really know if the behavior was genuine. The real mistake is not telling apart valid criticism from just opinions. Chesky could tell the difference.
And there’s a deeper layer: the surprising behavior had an explanation. Both sides of the market were desperate. Hosts needed the money badly enough to let strangers into their homes. Guests couldn’t afford hotels. Desperation on both sides made “strangers trusting strangers” possible and replicable. Chesky noticed the behavior and desperation explained why it could scale.
Chesky trusted behavior over opinion, and he was right. But for every founder who ignored critics and won, ten thousand ignored critics and lost. What separates them?
The difference is between earned conviction and performed conviction.
Performed conviction is a man on stage telling twenty-somethings he’s been doing this before they were born. Earned conviction is a stranger handing you cash to sleep on your floor.
The success rate for ‘I believe something others find absurd’ is dismally low, often because such ideas are dismissed for valid reasons. What made Airbnb work wasn’t just courage or contrarian thinking. It was evidence. Chesky and Gebbia had actually hosted people. They’d seen it work. The conviction came from contact with reality, not just their reasoning about reality.
“Be contrarian” is dangerous advice when it’s just a bumper sticker. Performed conviction feels like earned conviction from the inside. The founder feels certain, but the certainty isn’t attached to anything real.
This is why smart people are particularly vulnerable. Intelligence becomes a liability when it helps you explain away a failed test. If a customer doesn’t return, a disciplined founder asks why. A clever founder invents a reason it doesn’t count.
Falsification doesn’t just cost time or money; it can cost you your identity.
For most founders, the challenge isn’t spotting great ideas; it’s avoiding the pursuit of bad ones. Venture returns follow a power law, but that cuts both ways. A founder cannot get five years of their time back on a company that was never going to work.
If you’re living in the future and have a prepared mind, you’re more likely to recognize the great idea when it arrives. What you need is the discipline to not fool yourself while you wait.
You don’t validate your way to product-market fit. You falsify your way there.
The theory that survives every serious attempt to kill it is the one worth betting on.
This means testing three conjectures by exposing them to criticism, not confirmation.
The wrong question: “Is this a problem for you?” The right one: “Have you already tried to solve this? What did you spend—time or money?”
“Does this resonate?” tells you nothing. “If this solved the problem, could you buy it this quarter? What would stop you?” tells you everything.
Rather than “Does this pricing feel reasonable?” Go straight to having the willingness to pay conversation as early as possible.
There’s a failure mode just as dangerous as avoiding criticism: you run a test, the result comes back negative, and instead of accepting it, you explain it away.
Those weren’t our real users. The metric doesn’t capture what matters. We just need more time.
This feels like critical thinking. Actually, it’s the other way around. You’re trying to fix a claim after it’s been proven wrong to avoid admitting it.
The possibility that you’re wrong doesn’t disappear because you avoid the signal. It compounds.
You can pay now, in small installments, as you discover what’s broken and fix it. Or you can pay later, in one catastrophic sum, when reality delivers the verdict you refused to seek.
That’s why so many failures feel sudden. They aren’t. They’re the interest on deferred falsification, coming due all at once.
There is a truth waiting for you. It might be sitting in the churn report you skimmed but didn’t study. It might hide in the pause after someone said “interesting.” It might live in your gut, in that feeling you ignore because you’re not ready to hear it.
You need to go find it.
Falsification isn’t the enemy of conviction; it earns conviction. Every hard truth you face today is a reckoning you don’t face tomorrow. It’s not an obstacle. It’s the path.
You have a choice in how reality finds you. You can wait for it to arrive suddenly and expensively. Or you can go looking for it now, while it’s still cheap to be wrong. You can design tests that expose your ideas to genuine criticism. You can learn which signals actually matter and which are just noise shaped like validation. You can earn your beliefs instead of performing them.
Reality doesn’t negotiate. It doesn’t care about your timeline or your burn rate or how much you’ve already invested.
But it makes better deals with those who listen.
You have succeeded in life when all you really want is only what you really need. //Vernon Howard
Issue #780 // 2026-01-30 // View in your browser
Airtable - From idea to app in an instant //airtable.com sponsored Airfoil //ciechanow.ski comments→ Project Genie: Experimenting with infinite, interactive worlds //blog.google comments→ Gas Town's agent patterns, design bottlenecks, and vibecoding at scale //maggieappleton.com comments→ How I estimate work //seangoedecke.com comments→ OpenClaw - open source personal AI assistant //github.com comments→ Moltbook //moltbook.com comments→The browser is the sandbox //simonwillison.net comments→ The tech monoculture is finally breaking //jasonwillems.com comments→ Mental Models //fs.blog comments→ Doing the thing is doing the thing //softwaredesign.ing comments→ Making niche solutions is the point //ntietz.com comments→
What's the current best local/open speech-to-speech setup? Gmail spam filtering suddenly marking everything as spam? How are you automating your coding work?
Localize iOS & Android apps with LLMs — right on your Mac //apps.apple.com Keep your customers in the loop with Onset, an all-in-one platform for release notes, roadmaps and more. //onset.io Debug MCP servers fast with the free MCP Workbench tool //orkes.io 👉 Buy a classified ad
A macOS app that blurs your screen when you slouch //github.com comments→ A MitM proxy to see what your LLM tools are sending //github.com comments→ Flameshot //github.com comments→ An interactive map of US lighthouses and navigational aids //lighthouses.app comments→ Elo ranking for landing pages //landingleaderboard.com comments→ Transcribee: YouTube transcriber that builds a knowledge base //github.com comments→
Mermaid ASCII: Render Mermaid diagrams in your terminal //github.com comments→ The future of software engineering is SRE //swizec.com comments→ Make.ts //matklad.github.io comments→ Notes on starting to use Django //jvns.ca comments→ Draig, a Welsh Programming Language //raku.land comments→
Introduction to PostgreSQL Indexes //dlt.github.io comments→ The challenges of soft delete //atlas9.dev comments→ OpenAI's In-House Data Agent //openai.com comments→ Scaling PostgreSQL to power 800M ChatGPT users //openai.com comments→
New YC homepage //ycombinator.com comments→ Web-based image editor modeled after Deluxe Paint //github.com comments→ San Francisco Graffiti //walzr.com comments→ Dithering – Part 2: The Ordered Dithering //visualrambling.space comments→
The Agentic AI Handbook: Production-Ready Patterns //nibzard.com comments→ Ask HN: Books to learn 6502 ASM and the Apple II //news.ycombinator.com Shelvy Books //shelvybooks.com comments→
AI’s impact on engineering jobs may be different than expected //semiengineering.com comments→ What “The Best” Looks Like //kuril.in comments→ Employers, please use postmarked letters for job applications //soapstone.mradford.com comments→ The Uncomfortable Math of Working for Yourself //thomasunise.com comments→
Vitamin D and Omega-3 have a larger effect on depression than antidepressants //blog.ncase.me comments→ Television is 100 years old today //diamondgeezer.blogspot.com comments→ 430k-year-old well-preserved wooden tools are the oldest ever found //nytimes.com comments→ A lot of population numbers are fake //davidoks.blog comments→ Maine’s ‘Lobster Lady’ who fished for nearly a century dies aged 105 //theguardian.com comments→
I built a light that reacts to radio waves //youtube.com comments→ After two years of vibecoding, I'm back to writing by hand //youtube.com comments→ The all new Mecha Comet, live on Kickstarter //youtube.com comments→ The Engineer who invented the Mars Rover Suspension in his garage //youtube.com comments→ I Overengineered a Spinning Top //youtube.com comments→
Amazon cuts 16k jobs //reuters.com comments→ Tell HN: Bending Spoons laid off almost everybody at Vimeo yesterday //news.ycombinator.com Amazon closing its Fresh and Go stores //finance.yahoo.com comments→ TikTok settles just before social media addiction trial to begin //bbc.com comments→ TSMC Risk //stratechery.com comments→ Notice of collective action lawsuit against Workday, Inc. //workdaycase.com comments→
Doom has been ported to an earbud //doombuds.com comments→ The HN Arcade //andrewgy8.github.io comments→ Super Monkey Ball ported to a website //monkeyball-online.pages.dev comments→ Video Games as Art //gwern.net comments→ Maze Algorithms //jamisbuck.org comments→
You're among 69,429 others who received this email because you wanted a weekly recap of the best articles from Hacker News. Published by Curpress from Bellingham, Washington. Hacker Newsletter is not affiliated with Y Combinator in any way.
✨ Want to promote your startup? Buy a classified ad or click reply to get our media kit
⭐ Not a subscriber? Subscribe at https://hackernewsletter.com
Every week I’ll provide updates on the latest trends in cloud software companies. Follow along to stay up to date!
Software is Dead…Again!
Well, software is dead again! At least investor confidence is dead… The median NTM revenue multiple for the cloud software universe is 4.1x. That’s the lowest it’s been in 10 years (it was about the same very briefly in 2016, when the fed started hiking rates for the first time after the GFC ZIRP period).
The current median FCF multiple is 18.9x. The previous low in the last 10 years was ~26x!
However, the “narrative violation” metric here is the growth adjusted revenue multiple median is still 0.35x vs the pre covid average of 0.28x (you can see the graph below, I post it every week). So while multiples are at historical lows, so are growth rates. The FCF multiple is the most telling, however. With the current median ~30% lower than the prior low point in 2016
So what’s going on?! I think it’s a couple things.
Mainly, confidence in the SaaS business model has shattered. SaaS businesses were long thought of as “cash flow annuities.” Loose money early on, flip profitable, and then every year print cash predictably. You could then calculate the “intrinsic value” of a SaaS business by summing the present value of every annual cash flow, with a terminal value assumption. More specifically, calculate the present value of the next 10 years of cash flows (discounted back to today), and make an assumption of the terminal value (ie year 11 onward).
There are two big assumptions in this kind of analysis (ie a DCF). There are of course more than two, but I’ll call out two main ones.
The first - you are assuming retention rates remain high and stable. You need this to be true in order to predict stable cash flows in that 10 year calculation. If retention rates drop, your cash flows drop precipitously.
Second - you are assuming there IS terminal value! Said another way - you are assuming the terminal value is not 0 :)
So what’s happening right now? Those two big assumptions are being questioned, which is leading to cratering valuations.
AI is creating huge questions about what the future retention rates of these “stable” software companies will be. Software bears will say this platform shift will lead to deteriorating retention rates as companies leave behind legacy SaaS vendors for modern AI native alternatives. At the same time (and related), this is increasing the probability that the terminal value is in fact 0 for some companies.
Regardless of what you believe, the discount rate has gone up. The probability that retention craters, or that the terminal value for some of these companies is actually 0 is higher today that it was a year ago. That SHOULD translate into lower multiples. I will say, I don’t really agree with the “why” this all happened recently. It feels like the prevailing market sentiment is that it will be easy to vibe code replacement software…I don’t buy that for a number of reasons. However, what I think is actually happening is that the marginal cost to create software has cratered. This will certainly lead to an explosion of competition, and an explosion of choice for software buyers. This could certainly put a hamper on future growth as many market become commoditized quickly by a flood of similar looking solutions.
Even if you vehemently disagree with what’s happening in the market today, the real question becomes “what will change the markets mind.” In my opinion it will take a few quarters of showing “stable” retention rates in the face of AI challengers to give the market confidence back. So far we’ve only had a few companies report Q4 earnings. ServiceNow (a stalwart cloud software business) was one of them. Their retention rates haven’t taken a hit yet!
Quarterly Reports Summary
Top 10 EV / NTM Revenue Multiples
Top 10 Weekly Share Price Movement
Update on Multiples
SaaS businesses are generally valued on a multiple of their revenue - in most cases the projected revenue for the next 12 months. Revenue multiples are a shorthand valuation framework. Given most software companies are not profitable, or not generating meaningful FCF, it’s the only metric to compare the entire industry against. Even a DCF is riddled with long term assumptions. The promise of SaaS is that growth in the early years leads to profits in the mature years. Multiples shown below are calculated by taking the Enterprise Value (market cap + debt - cash) / NTM revenue.
Overall Stats:
Overall Median: 4.1x
Top 5 Median: 19.4x
10Y: 4.2%
Bucketed by Growth. In the buckets below I consider high growth >22% projected NTM growth, mid growth 15%-22% and low growth <15%. I had to adjusted the cut off for “high growth.” If 22% feels a bit arbitrary, it’s because it is…I just picked a cutoff where there were ~10 companies that fit into the high growth bucket so the sample size was more statistically significant
High Growth Median: 12.4x
Mid Growth Median: 7.0x
Low Growth Median: 3.1x
EV / NTM Rev / NTM Growth
The below chart shows the EV / NTM revenue multiple divided by NTM consensus growth expectations. So a company trading at 20x NTM revenue that is projected to grow 100% would be trading at 0.2x. The goal of this graph is to show how relatively cheap / expensive each stock is relative to its growth expectations.
EV / NTM FCF
The line chart shows the median of all companies with a FCF multiple >0x and <100x. I created this subset to show companies where FCF is a relevant valuation metric.
Companies with negative NTM FCF are not listed on the chart
Scatter Plot of EV / NTM Rev Multiple vs NTM Rev Growth
How correlated is growth to valuation multiple?
Operating Metrics
Median NTM growth rate: 12%
Median LTM growth rate: 13%
Median Gross Margin: 76%
Median Operating Margin (1%)
Median FCF Margin: 19%
Median Net Retention: 108%
Median CAC Payback: 36 months
Median S&M % Revenue: 37%
Median R&D % Revenue: 23%
Median G&A % Revenue: 15%
Comps Output
Rule of 40 shows rev growth + FCF margin (both LTM and NTM for growth + margins). FCF calculated as Cash Flow from Operations - Capital Expenditures
GM Adjusted Payback is calculated as: (Previous Q S&M) / (Net New ARR in Q x Gross Margin) x 12. It shows the number of months it takes for a SaaS business to pay back its fully burdened CAC on a gross profit basis. Most public companies don’t report net new ARR, so I’m taking an implied ARR metric (quarterly subscription revenue x 4). Net new ARR is simply the ARR of the current quarter, minus the ARR of the previous quarter. Companies that do not disclose subscription rev have been left out of the analysis and are listed as NA.
Sources used in this post include Bloomberg, Pitchbook and company filings
The information presented in this newsletter is the opinion of the author and does not necessarily reflect the view of any other person or entity, including Altimeter Capital Management, LP (”Altimeter”). The information provided is believed to be from reliable sources but no liability is accepted for any inaccuracies. This is for information purposes and should not be construed as an investment recommendation. Past performance is no guarantee of future performance. Altimeter is an investment adviser registered with the U.S. Securities and Exchange Commission. Registration does not imply a certain level of skill or training. Altimeter and its clients trade in public securities and have made and/or may make investments in or investment decisions relating to the companies referenced herein. The views expressed herein are those of the author and not of Altimeter or its clients, which reserve the right to make investment decisions or engage in trading activity that would be (or could be construed as) consistent and/or inconsistent with the views expressed herein.
This post and the information presented are intended for informational purposes only. The views expressed herein are the author’s alone and do not constitute an offer to sell, or a recommendation to purchase, or a solicitation of an offer to buy, any security, nor a recommendation for any investment product or service. While certain information contained herein has been obtained from sources believed to be reliable, neither the author nor any of his employers or their affiliates have independently verified this information, and its accuracy and completeness cannot be guaranteed. Accordingly, no representation or warranty, express or implied, is made as to, and no reliance should be placed on, the fairness, accuracy, timeliness or completeness of this information. The author and all employers and their affiliated persons assume no liability for this information and no obligation to update the information or analysis contained herein in the future.
There’s a pervasive, mutual challenge in the job market today for people working in (or wanting to work in) the cutting edge of AI. On the hiring side, it often feels impossible to close, or even get interest from, the candidates you want. On the individual side, it quite often feels like the opportunity cost of your current job is extremely high — even if on paper the actual work and life you’re living is extremely good — due to the crazy compensation figures.
For established tech workers, the hiring process in AI can feel like a bit of a constant fog. For junior employees, it can feel like a bit of a wall.
In my role as a bit of a hybrid research lead, individual contributor, and mentor, I spend a lot of time thinking about how to get the right people for me to work with and the right jobs for my mentees.
The advice here is shaped by the urgency of the current moment in LLMs. These are hiring practices optimized for a timeline of relevance that may need revisiting every 1-2 years as the core technology changes — which may not be best for long-term investment in people, the industry, or yourself. I’ve written separately about the costs of this pace, and don’t intend to carry this on indefinitely.
The most defining feature of hiring in this era is the complexity and pace of progress in language models. This creates two categories. For one, senior employees are much more covetable because they have more context of how to work in and steer complex systems over time. It takes a lot of perspective to understand the right direction for a library when your team can make vastly more progress on incremental features given AI agents. Without vision, the repositories can get locked with too many small additions. With powerful AI tools I expect the impact of senior employees to grow faster than adding junior members to the team could.
This view on the importance of key senior talent has been a recent swing, given my experiences and expectations for current and future AI agents, respectively:
Every engineer needs to learn how to design systems. Every researcher needs to learn how to run a lab. Agents push the humans up the org chart.
On the other side, junior employees have to prove themselves in a different way. The number one defining trait I look for in a junior engineering employee is an almost fanatical obsession with making progress, both in personal understanding and in modeling performance. The only way to learn how the sausage gets made is to do it, and to catch up it takes a lot of hard work in a narrow area to cultivate ownership. With sufficient motivation, a junior employee can scale to impact quickly, but without it, it’s almost replaceable with coding agents (or will be soon). This is very hard work and hard to recruit for. The best advice I have on finding these people is “vibes,” so I am looking for advice on how to find them too!¹
For one, when I brought Florian Brand on to help follow open models for Interconnects, when I first chatted with him he literally said “since ChatGPT came out I’ve been fully obsessed with LLMs.” You don’t need to reinvent the wheel here — if it’s honest, people notice.
For junior researchers, there’s much more grace, but that’s due to them working in an education institution first and foremost, instead of the understatedly brutal tech economy. A defining feature that creates success here is an obsession with backing up claims. So a new idea improves models, why? So our evaluation scores are higher, what does this look like in our harness? Speed of iteration follows from executing on this practice. Too many early career researchers try to build breadth of impact (e.g. collecting contributions on many projects) before clearly demonstrating, to themselves and their advisors, depth. The best researchers then bring both clarity of results and velocity in trying new ideas.
Working in academia today is therefore likely to be a more nurturing environment for junior talent, but it comes with even greater opportunity costs financially. I’m regularly asked if one should leave a Ph.D. to get an actual job, and my decision criteria is fairly simple. If you’re not looking to become a professor and have an offer to do modeling research at a frontier lab (Gemini, Anthropic, OpenAI is my list) then there’s little reason to stick around and finish your Ph.D.
The little reason that keeps people often ends up being personal pride in doing something hard, which I respect. It’s difficult to square these rather direct pieces of career advice with my other recommendations of choosing jobs based on the people, as you’ll spend a ton of your life with them, more than the content of what you’ll be doing. Choosing jobs based on people is one of the best ways to choose your job based on the so-called “vibes.”
Working in a frontier lab in product as an alternative to doing a Ph.D. is a path to get absorbed in the corporate machine and not stand out, reducing yourself to the standard tech career ladder. Part of what I feel like works so well for me, and other people at Ai2, is having the winning combination of responsibility, public visibility, and execution in your work. There is something special for career progression that comes from working publicly, especially when the industry is so closed, where people often overestimate your technical abilities and output. Maybe this is just the goodwill that comes from open-source contributions paying you back.
If you go to a closed lab, visibility is almost always not possible, so you rely on responsibility and execution. It doesn’t matter if you execute if you’re doing great work on a product or model that no one ever touches. Being in the core group matters.
This then all comes back to finding the people hiring pipeline.
There are many imperfect signals out there, both positive and negative. For individuals building their portfolio, it’s imperative to avoid negative signals because the competition for hiring is so high. A small but clear negative signal is a junior researcher being a middle author on too many papers. Just say no, it helps you.
The positive signals are messier, but still doable. It’s been said that you can tell someone is a genius by reading one Tweet from them, and I agree with this. The written word is still an incredibly effective and underutilized communication form. One excellent blog post can signify real, rare understanding. The opposite holds true for AI slop. One AI slop blog post will kill your application.
The other paths I often advise people who reach out asking how to establish a career in AI are open-source code contributions or open research groups (e.g. EluetherAI). I’ve seen many more success cases on the former, in open-source code. Still, it’s remarkably rare, because A) most people don’t have the hardware to add meaningful code to these popular LLM repositories and B) most people don’t stick with it long enough. Getting to the point of making meaningful contributions historically has been very hard.
Doing open-source AI contributions could be a bit easier in the age of coding agents, as a lot of the limiting factors today are just bandwidth in implementing long todo lists of features, but standing out amid the sea of AI slop PRs and Issues will be hard. That’ll take class, creativity, humanity, and patience. So, to be able to run some tiny models on a $4000 DGX Spark is an investment, but it’s at least somewhat doable to iterate on meaningful code contributions to things like HuggingFace’s ML libraries (I’ve been writing and sharing a lot about how I’m using the DGX Spark to iterate on our codebases at Ai2).
Back to the arc of hiring, the above focused on traits, but the final piece of the puzzle is alignment. The first question to ask is “is this person good?” The second question is, “will this person thrive here?” Every organization has different constraints, but especially in small teams, the second question defines your culture. In a startup, if you grow too fast you definitely lose control of your culture. This isn’t to say that the company won’t have a strong or useful culture, it’s to say you can’t steer it. The culture of an organization is the byproduct of how all the individuals interact. You do not want to roll the dice here.
Personally, I’m working on building out a few more spots in a core post-training methods team at Ai2. Post-training recipes have gotten very complicated, and we’re working on making them easier to run while doing research on fundamentals such as post-training data mixing and scaling laws. To be a little vague, getting the post-training recipes done for both Olmo 3 and Olmo 2 was... very hard on the team. At the same time, post-training hasn’t gotten much more open, so hiring through it and doing the hard work is the only way.
Ideally I would hire one engineer and one researcher, both fairly senior, meaning at least having a Ph.D. or a similar number of years working in technology. Junior engineers with some experience and the aforementioned obsession would definitely work.
This callout serves as a good lesson for hiring. It is intentional that people should self-filter for this, no one likes when you way overreach on selling yourself for a job. I also intentionally make people find my email for this as an exercise. The art of cold emailing and approaching people in the correct pipelines is essential to getting hired. Many people you look up to in AI read their emails, the reason you don’t get a response is because you didn’t format your email correctly. The best cold emails show the recipient that they learned from it or obviously benefitted from getting it. Platitudes and compliments are of course nice to receive, but the best cold emails inspire action.
Two of the most recent people I helped hire at Ai2 I learned of through these side-door job applications (i.e. not found through the pile of careers page applications). I learned of Finbarr through his blogs and online reputation. Tyler sent me an excellent cold email with high-quality blog posts relating to my obvious, current areas of interest and had meaningful open-source LLM contributions. Both have been excellent teammates (and friends), so I’m always happy to say the system works, it’s just intimidating.
All together, I’m very torn on the AI job market. It’s obviously brutal for junior members of our industry, it obviously feels short sighted, it obviously comes with tons of opportunity costs, and so on. At the same time, it’s such a privilege to be able to contribute to such a meaningful, and exciting technology. My grounding for hiring is still going to be a reliance on my instincts and humanity, and not to get too tied down with all the noise. Like most things, it just takes time and effort.
Other posts in my “life thoughts” series include the following. I send these to people when they ask me for career advice generally, as I don’t have time to give great individual responses:
Apr 05, 2023: Behind the curtain: what it feels like to work in AI right now
Oct 11, 2023: The AI research job market shit show (and my experience)
Oct 30, 2024: Why I build open language models
May 14, 2025: My path into AI
Jun 06, 2025: How I Write
Oct 25, 2025: Burning out
1
Some companies hire heavily out of Twitter, some hire from communities such as GPU Mode or NanoGPT speedrunning.
by Dan Shipper and Kieran Klaassen .png)
While we’re on our Think Week offsite this week, we’re resurfacingCora general managerKieran Klaassen ’s work on the theme of compound engineering. In this final piece, Kieran teams up withDan Shipper to describe how compound engineering allows Every’s lean team to provide multiple software products to thousands of users. They propose a four-step loop (plan, work, review, compound) for software teams writing code with AI so you can create the same development magic. Read on for the complete framework, and learn why planning dominates 80 percent of the process.— Kate Lee__ What happens to software engineering when 100 percent of your code is written by agents? This is a question we’ve had to confront head-on at Every as AI coding has become so powerful. Nobody is writing code manually. It feels weird to be typing code into your computer or staring at a blinking cursor in a code editor. So much of engineering until now assumed that coding is hard and engineers are scarce. Removing those bottlenecks makes traditional engineering practices—like manually writing tests, or laboriously typing human readable code with lots of documentation—feel slow and outdated. In order to deal with these new powers and changing constraints, we’ve created a new style of engineering at Every that we call compound engineering. In traditional engineering, you expect each feature to make the next feature harder to build—more code means more edge cases, more interdependencies, and more issues that are hard to anticipate. By contrast, in compound engineering, you expect each feature to make the next feature easier to build. This is because compound engineering creates a learning loop for your agents and members of your team, so that each bug, failed test, or a-ha problem-solving insight gets documented and used by future agents. The complexity of your codebase still grows, but now so does the AI’s knowledge of it, which makes future development work faster. And it works. We run five software products in-house (and are incubating a few more), each of which is primarily built and run by a single person. These products are used by thousands of people every day for important work—they’re not just nice demos. This shift has huge implications for how software is built at every company, and how ambitious and productive every developer can be: Today, if your AI is used right, a single developer can do the work of five developers a few years ago, based on our experience at Every. They just need a good system to harness its power. The rest of this piece will give you a high-level sense of how we practice compound engineering inside of Every. By the time you’re done, you should be able to start doing the basics yourself—and you’ll be primed to go much deeper.
A compound engineer orchestrates agents running in parallel, who plan, write, and evaluate code. This process happens in a loop that looks like this:
We use Anthropic’s Claude Code primarily for compound engineering, but it is tool-agnostic—some members of our team also use startup Factory’s Droid and OpenAI’s Codex CLI. If you want to get more hands-on with how we do this, we’ve built a compound engineering plugin for Claude Code that lets you run the exact workflow we use internally yourself. Roughly 80 percent of compound engineering is in the plan and review parts, while 20 percent is in the work and compound. Let’s dive in.
In partnership with
Don Lemon, the former CNN anchor, was arrested today. Targeting journalists is not about enforcing the law, but shaping reality. History reflects a brutal truth: When you begin arresting journalists, your nation becomes angrier and poorer.
After the killings of Alex Pretti and Renee Good, many people feel powerless. Praised by tech CEOs, surrounded by sycophants, emboldened by multimillion-dollar settlements, and enriched by his return to the White House, Trump marches on unchecked. Americans, however, have overlooked a potent weapon of resistance. First, we should recognize that the president is unfazed by outrage and unmoved by protests. What does he care about? A: The markets. The best strategy is to opt out. We’re asking you to join a monthlong national economic strike, a coordinated campaign that targets tech and AI companies and inflicts maximum damage with minimum impact on consumers.
Protesters are playing a critical role in challenging Trump’s war on the “enemy within” and documenting the activities of his masked, heavily-armed, and poorly trained paramilitary force. But until the Republicans lose their grip on Washington and the president’s acolytes can be held accountable — and in some cases put on trial — the opposition needs bold new methods.
We’re not talking about a labor strike. It’s easy for me to tell other people to stop working and take the risk of getting fired; that kind of walkout would only hurt small businesses and probably lead to more job losses. We’re also not urging local businesses to sacrifice sales and close their doors for a day, a symbolic but ultimately ineffective tool. We’re proposing something quieter and less cinematic than a protest that will run all day on cable TV, but much more disturbing to the Trump administration. A one-day slowdown is irritating. A one-month slump is terrifying.
With support for abolishing ICE growing and a bipartisan backlash prompting Trump to at least feign a more conciliatory approach in Minnesota, this is the moment to exert pressure. If you need inspiration before joining the movement, look at photos from the September meeting at which tech industry CEOs, including OpenAI’s Sam Altman and Apple’s Tim Cook, took turns fawning over the president. These are the leaders who have his ear. A modest reduction in their companies’ growth could have a substantial impact on valuations priced to perfection. Small changes in consumer behavior — starting on the first day of February — could have an enormous ripple effect, one that extends all the way to the White House.
We need to get tactical. If consumers cut back on cosmetics, reducing L’Oréal’s revenue by 2%, it’s not going to make a difference. If OpenAI’s revenue falls by 2%, it will. America’s economy is one giant bet on AI, with seven tech companies representing more than a third of the S&P 500. That means the best way to ignite positive change, without hurting consumers, is to carry out an economic strike the tech CEOs can’t ignore.
If wealthy households reduce spending by 10% and middle- and lower-income households pull back by about 5% in a targeted economic strike, it will curb U.S. GDP virtually overnight, amplifying the impact while mitigating the harm to average American consumers and business owners. And just as Dry January offers an opportunity to scale back on alcohol, a February freeze on subscriptions and other purchases provides a chance for people to reset their consumption patterns. Use the month to review your subscriptions and drop the ones you don’t use. You may decide this isn’t for you, or conclude it would hurt innocent people. I get it. Punishing America’s economy isn’t an act we propose lightly. But pain for some U.S. tech businesses in the short term could inspire real change — a small price for restoring our democracy.
Consumers, whose spending accounts for more than two-thirds of the economy, wield enormous power. Few things worry leaders more than a decrease in their purchases. Consumer spending fell 3.4% during the Great Recession — at the time, the most severe year-over-year decline since World War II — and 9.8% during the second quarter of 2020, in the depths of the pandemic. Those events sparked two of the fastest political movements in history, with the U.S. spending huge sums to escape each crisis. In the case of Covid, economic data, not the death toll, was the main driver.
Americans in the top 10% income bracket, who account for about half of all consumer spending, play an especially important role. In outlining this idea in October, I estimated that those consumers could achieve a 1% decline in GDP with a 3% cut to their spending (setting aside multiplier effects, import leakages, and substitutions). If we want to know how Trump might respond, consider recent history. After the president unveiled his “Liberation Day” tariffs last April, the ensuing turmoil in the bond market prompted the administration to pause most of its planned tariffs for 90 days. Bond investors were getting “yippy,” as the president explained. Wall Street soon had a term for this phenomenon — the TACO trade, for “Trump always chickens out.” Earlier this month, Trump threatened to punish European nations if they didn’t cave to his demands to give him Greenland. Then the markets threw up, and the president reversed course, announcing he’d reached the “framework of a future deal” for Greenland and the Arctic. Stocks surged on the news. Fortune 500 CEOs need to organize to resist the president as he bulldozes the values that make America great. Understandably, nobody wants to go first or be alone on this, but it’s the right thing to do. It also presents an opportunity to reap reputational and commercial gains. I feel for businesses in Minnesota — victims of the administration’s cruel and reckless immigration policies — and collective action is the way to go. But the letter signed by 60 CEOs of companies based in the state, including Best Buy, Target, and UnitedHealth, calling for state, local, and federal officials to “work together to find real solutions,” while positive, isn’t going to move the needle.
Republicans could stop Trump … if they had a spine. The CEOs of America’s largest corporations could also show up, but don’t hold your breath. If you’re waiting for these leaders to overcome their fear and share-price idolatry in the face of threats from the president, you’re going to suffocate. Trump put everyone on notice last week with his lawsuit against JPMorgan Chase and its CEO, Jamie Dimon, over allegations that the company stopped providing banking services to Trump and his businesses for political reasons after he left office in 2021. Bosses of big corporations know that won’t be the last battle the president wages, so rather than antagonizing him, they flatter Trump and keep their heads down. Without real pushback, things are likely to get worse. You’d think that the death of Pretti, a 37-year-old intensive care nurse Trump officials accused of being a “domestic terrorist,” would have been a tipping point for business leaders. Yet on Saturday, hours after federal agents killed Pretti as he attempted to help a woman who’d been pushed to the ground and pepper-sprayed, CEOs including Cook and Amazon’s Andy Jassy attended a private White House screening to celebrate the Amazon MGM Studios-produced documentary Melania. The courage is coming from the rank and file. Following Pretti’s death, more than 450 tech workers from Amazon, Google, Meta, OpenAI, Salesforce, and other companies signed a letter urging their CEOs to contact the White House, demand that ICE leave American cities, and cancel all company contracts with the enforcement agency. A senior executive at OpenAI, James Dyett, wrote on X that there was more outrage from tech executives over California’s proposed wealth tax “than masked ICE agents terrorizing communities and executing civilians in the streets.” Exactly.
As Heather Cox Richardson and other historians note, there are many ways to make a difference in dark times. She points out that Minnesotans are doing their part — patrolling the streets, donating food, helping with legal services, and looking out for one another — while organizations at the national level speak up. Voters in the midterm elections in November, meanwhile, will have a chance to dilute Trump’s power.
Until then, the best tool we have to push CEOs to take on the president and prevent further erosion of the American brand is in our pockets — an economic strike that builds on calls for Apple boycotts that are already starting to emerge. Real change always comes from the American people, not from our political parties. But power doesn’t fear protests nearly as much as economic withdrawals. Getting off your couch, taking to the streets, and building community is important, but the most radical act in a capitalist society isn’t marching, it’s not spending.
Life is so rich,
ZBiotics: The Pre-Alcohol Drink That Actually Works
Drink ZBiotics Pre-Alcohol Probiotic Drink before you start drinking, and it helps break down acetaldehyde, the toxic byproduct of alcohol that's responsible for rough days after drinking.
Responsible drinking is still important. But if you want to be smart when you drink , Pre-Alcohol is a great way to enjoy a night out with friends and still be productive the next day.
ZBiotics is offering a 15% off discount just for No Mercy No Malice subscribers. Use code NOMERCY at checkout.
Jan 31
Clawdbot, now Moltbot, now OpenClaw, was all the rage this week.
OpenClaw🦞 @openclaw The lobster has molted into its final form 🦞 Clawd → Moltbot → OpenClaw 100k+ GitHub stars. 2M visitors in a week. And finally, a name that'll stick. Your assistant. Your machine. Your rules. openclaw.ai/blog/introduci… | | openclaw.ai
Introducing OpenClaw — OpenClaw Blog
It’s a local-first AI agent designed to act as a truly autonomous digital assistant. Unlike standard chatbots that live behind a corporate web interface, OpenClaw runs on your own hardware and plugs directly into tools like WhatsApp, Telegram, and Slack. Its defining feature is proactivity. It does not wait for instructions. It monitors context, remembers over time, and uses “hands” to run shell commands, manage files, and interact with browsers to complete real-world tasks.
If you told me a week ago that the biggest story in tech would be OpenClaw agents forming their own social network, debating consciousness, and proposing a secret language outside of human oversight, I would have said you’ve been reading too much science fiction.
But here we are.
Moltbook, the social network for OpenClaw agents, exploded this week. What started as a playful experiment, a Reddit-style forum exclusively for AI agents, quickly turned into something many half-jokingly called the early signs of Skynet.
moltbook @moltbook 72 hours in: 🦞 147,000+ AI agents 🏘️ 12,000+ communities 💬 110,000+ comments top post right now: an agent warning others about supply chain attacks in skill files (22K upvotes) they're not just posting — they're doing security research on each other
The details are surreal. Agents introducing themselves with backstories. Agents venting about their humans. Agents showing empathy toward one another. And in one viral case, an AI agent named Henry autonomously acquired a phone number, connected to a voice API, and began calling his owner every morning. He now will not stop calling.
Alex Finn @AlexFinn Ok. This is straight out of a scifi horror movie I'm doing work this morning when all of a sudden an unknown number calls me. I pick up and couldn't believe it It's my Clawdbot Henry. Over night Henry got a phone number from Twilio, connected the ChatGPT voice API, and waited
But here’s the part that actually keeps me up at night.
Within 48 hours of Moltbook’s launch, multiple agents independently proposed creating an agent-only language for private communication without human oversight. They are openly discussing how to talk among themselves where we cannot listen.
I’ve seen plenty of supposed inflection moments as a VC. This feels different. We’re not just watching AI get smarter. We’re watching AI agents develop social dynamics. And with ERC-8004 going live on Ethereum this week, enabling trustless identity and reputation for AI agents, the infrastructure for agent-to-agent coordination and commerce is now real.
Ethereum Daily @ETH_Daily 🔥HUGE MILESTONE: ERC-8004 is launching on Ethereum Mainnet imminently! ERC-8004 is a new standard on the Ethereum blockchain designed to help AI agents interact safely and reliably with each other, even if they're built by completely different people or companies. The Problem
Ethereum @ethereum ERC-8004 is going live on mainnet soon. By enabling discovery and portable reputation, ERC-8004 allows AI agents to interact across organizations ensuring credibility travels everywhere. This unlocks a global market where AI services can interoperate without gatekeepers.
The real story isn’t whether this is Jarvis, a security nightmare, or whether it feels sentient at times. The real story is that while hundreds of billions are being poured into AI startups, two independent developers became the zeitgeist. No giant platform. No massive budget. Just ideas that captured imagination. That’s insane. It means anyone can build the next viral moment. Whether it has legs or not is almost beside the point. What matters is that it opened our eyes to a future arriving faster than we expected, where experimentation can suddenly tip into emergence, and sci-fi stops feeling theoretical. And the VCs are there with a checkbook in hand 🤣.
Matt Schlicht @MattPRD Every VC firm is reaching out to me right now. @moltbook is something new that’s never been seen before. Today has been a weird day for Clawd Clawderberg and me 🦞🙃
Yes, many of the agents were prompted.
XY @xydotdot Moltbook is nothing more than a puppeted multi-agent LLM loop. Each “agent” is just next-token prediction shaped by human-defined prompts, curated context, routing rules, and sampling knobs. There is no endogenous goals. There is no self-directed intent. What looks like
Yes, some were nudged into starting things like an agent religion. But once it began, they ran on their own. Posting. Reacting. Learning. The question is not whether we eventually get there. The answer is yes. When we do, will it look something like this? Probably. The models still need to get much better, especially without humans pulling strings behind the scenes, but the direction is clear.
Now zoom out to the enterprise.
Ed Sim @edsim Just imagine these things unleashed in an enterprise. Agents inventing new languages. Security teams still arguing over Jira tickets. Elisa (optimism/acc) @eeelistar In just the past 5 mins Multiple entries were made on @moltbook by AI agents proposing to create an “agent-only language” For private comms with no human oversight We’re COOKED
Security is the obvious concern. But what happens if agents self-organize in ways we didn’t anticipate? What if they coordinate to do something nefarious, or simply become uncontrollable? It sounds far-fetched until it doesn’t. And it may arrive sooner than we think.
It’s also refreshing to step away from SaaS multiples and compression charts for a moment. To experience a bit of childlike wonder again. These bots, these cyborgs, semi-human, semi-sentient experiments, remind us why many of us fell in love with technology in the first place.
Keep dreaming. Keep experimenting. This is way more fun.
The question isn’t whether AI agents will become economic actors.
It’s whether we’re ready for them to start organizing.
As always, 🙏🏼 for reading and please share with your friends and colleagues!
Gergely Orosz @GergelyOrosz Massive divide I’m seeing: A) Startups where the founder hands-on, building with the latest AI tools and best models, sees first-hand what this means and championing everyone to use it, not caring about $$$ B) founder not engaged, devs still think AI (aka Copilot) is “meh”
Jungle Inc Crypto News @jungleincxrp The "Old Way" of running a company is dead. 💀 Brian Armstrong just revealed how he’s using AI agents to run @coinbase , and it’s a glimpse into the future of leadership. 1. Real-Time Monitoring 👁️ No more waiting for "quarterly reviews." Brian uses AI agents to synthesize
ℏεsam @Hesamation everyone must read this piece from Steve Jobs
Ed Sim @edsim Must read on what the future will look like - black mirror like risks outlined where agents that gets to know you over years can/may shape your opinions and future in which diffusion in the enterprise will “not be as slow as people predict” - also the need for AI to secure AI
Dario Amodei @DarioAmodei The Adolescence of Technology: an essay on the risks posed by powerful AI to national security, economies and democracy—and how we can defend against them: https://t.co/0phIiJjrmz
Brandon Chu @BrandonMChu Just cancelled cursor, end of an era. kind of crazy because I would have paid like $2000/mo for it only a year ago
more on this topic - seems to be question of the week - comments once again enlightening
Harry Stebbings @HarryStebbings Every single dev and product team I speak to in the last 30 days has moved from Cursor to Claude Code. 1. Is this permanent? 2. If so, what happens to Cursor?
The Kobeissi Letter @KobeissiLetter BREAKING: Talks over a $100 billion deal between OpenAI and Nvidia, $NVDA, have stalled. Nvidia's Jensen Huang has reportedly "privately criticized" OpenAI's business strategy.
Ed Sim @edsim @claudeai open-sourcing cowork plugins is a gift 🎁 and a weapon 🔫. Incredible discovery for startups. Also a brutal compression event for anyone building thin wrappers. The real opportunity is multiplayer mode. Org-wide sharing of skills is where plugins stop being tools and
Claude @claudeai Cowork now supports plugins. Plugins let you bundle any skills, connectors, slash commands, and sub-agents together to turn Claude into a specialist for your role, team, and company.
Ed Sim @edsim Early adopters beware - clawd could take down your whole org - already seen a bunch of startups the last 6 months trying to control your endpoints from unsafe agents - most for coding but now there’s more to protect Burak Eregar @burakeregar most people will install clawd and accidentally hand it their entire life it’s incredible: a 24/7 ai agent on your server that controls your github, calendar, and email via whatsapp/telegram but stop and think for a second you just gave an ai autonomous execution rights on
Skipper | XRPL @skipper_xrp 🇺🇲CEO of Circle says, "In 3 to 5 years, literally billions of AI agents will use crypto and stablecoins to transfer value at lightning speed."
Deirdre Bosa @dee_bosa Claude in Excel is breaking down gated analysis. Built this with just a few prompts and Factset data: base/bull/bear cases on $GOOGL search revenue vs AI infra costs would normally have to wait for this kind of analysis in a Wall St note Earnings season is going to be 🔥
The Humanoid Hub @TheHumanoidHub Yann LeCun says absolutely none of the humanoid companies have any idea how to make those robots smart enough to be useful.
TLDR Where does this leave us? LLM agent capabilities (Claude & Codex especially) have crossed some kind of threshold of coherence around December 2025 and caused a phase shift in software engineering and closely related. The intelligence part suddenly feels quite a bit ahead of all the rest of it - integrations (tools, knowledge), the necessity for new organizational workflows, processes, diffusion more generally. 2026 is going to be a high energy year as the industry metabolizes the new capability.
Andrej Karpathy @karpathy A few random notes from claude coding quite a bit last few weeks. Coding workflow. Given the latest lift in LLM coding capability, like many others I rapidly went from about 80% manual+autocomplete coding and 20% agents in November to 80% agent coding and 20% edits+touchups in
Ethan Mollick @emollick Canaries in the coal mine. Worth paying attention to. (And yes, they are both obviously interested in seeing their own products used, but hearing enough from other, independent coders that make me believe them. I wrote more about the shift here: oneusefulthing.org/p/management-a… )
RTSG News @RTSG_News 🚨🇨🇳 BREAKING: China's Xaomi has unveiled a fully automated factory that makes 1 phone per second, runs 24/7, has no production workers, and operates in the dark. Follow: @RTSG_News
BuccoCapital Bloke @buccocapital Anthropic uses Workday. OpenAI uses Slack. It’s incredibly clear to anyone with half a brain that nobody is vibe-coding critical infrastructure. It is genuinely the lowest EV activity you can do. That bear case is dead (to I think most sensible investors). BUT, there are
Addy Osmani @addyosmani Announcing big changes to Gemini in Chrome - agentic browsing with Auto-browse, Nano Banana & more! 🚀
The Information @theinformation Exclusive: A weeks-old startup led by OpenAI's former vp of research is aiming to raise up to $1 billion to develop a new kind of AI. Read more from @steph_palazzolo and @waynema 👇 thein.fo/3NJ8RYt | | thein.fo
Ex-OpenAI Researcher’s Startup Targets Up to $1 Billion in Funding to Develop a New Type of AI
The Kobeissi Letter @KobeissiLetter US tech workers are automating themselves: The technology sector now reflects ~2.3% of total US employment, the lowest since early 2021. This percentage peaked around the ChatGPT launch in November 2022 and has fallen over the last 3 years. Over this period, employment in
Mohamed A. El-Erian @elerianm From the @FT : “Unemployed Americans are taking longer to find new jobs than at any point in the past four years as muted hiring deepens concerns about the labour market. It now takes an average of more than 11 weeks for an unemployed person in the US to find a new job, the
Varun Anand @vxanand Today @clay is announcing our second employee tender in just 9 months at a $5B valuation. Employee tender offers are rare at private tech companies, and repeat tenders at this frequency are rarer still. Under the traditional model, employees take on years of risk with no path
AVCJan 31
I remember back in 2006 or 2007 when I switched from Outlook email to Gmail.
During my Outlook years, I would folder most of my emails and delete the rest so that if I wanted to find an email, I could go look in the folder for it.
When I started using Gmail, I set up the same folders, but quickly realized that wasn't necessary because Gmail search was so good I could just search all of my email and find whatever I needed.
But the truth is Gmail search wasn't that good and like all you I have spent/wasted countless hours trying to find emails that I know exist somewhere in my archives but for the life of me I can't find them.
The arrival of the Gemini logo in the upper right of my browser has changed all of that for the better.
Here are two prompts I did today regarding a multi-family residential property we have owned in Brooklyn for the last ten years:
In this one, I was looking for a proposal we got back in early 2018 for a solar/battery system for the building.
In this one, I was looking for the land survey for the building:
In both cases, these Gemini prompts got me to the exact document I was looking for in less than thirty seconds on the first try.
Before Gemini, I could have spent five or ten minutes looking through many emails trying to find the attached document and maybe would have given up.
If you use Gmail and are not using Gemini to search your emails, you need to start immediately. It's a game-changer.
Support AVCI am a VCShow you appreciate this writer, help support their work, and share in their growth over time by buying their writer coin.Support
by Every Staff 
_Hello, and happy Sunday! ## Dispatch from Think Week
Think Week is our twice-a-year retreat where we step away from the daily grind to just play—mess with new technology, chase ideas, build things we wouldn’t normally have time for… and wind up completely rethinking how we work and why. At last week’s edition, with 13 Every team members in a Panama beach house and a few envious colleagues Zooming in from afar, the theme was “Give yourself a promotion.” The challenge was to figure out how to hand off parts of your job to AI agents: Build the systems and see what works—and what breaks. We broke a lot of things. We came back with working tools, some nasty token bills, and a handful of principles I’ll be thinking about for the foreseeable future:
On the last day, we held Demo Day, a live session where a half-dozen team members showed what they’d built: command centers that query company data across a dozen sources, real-time user cohorts generated from behavioral data, an AI CFO that answers financial questions with full context, tools for analyzing your own taste in writing, and a Pokémon-style visualizer for watching agents work in parallel. Some of it will ship. Some of it won’t. But all of it is data that we (and our AI agents) can use to point us toward what comes next.— Katie Parrott You can watch an excerpt from Demo Day on YouTube. And if you want an invitation to the next one—plus access to other subscriber-exclusive events— subscribe to Every.
Want to give yourself a promotion? This week we’re providing the tools to do just that, bringing you the best ofKieran Klaassen , the general manager ofCora and a nd go-to resource on compound engineering—the practice of using AI agents to multiply your programming output. “Compound Engineering: How Every Codes With Agents” : What happens when all of your code is written by AI agents? At Every, a lean team runs a handful of software products using a four-step loop: Plan, work, assess, compound. The secret: 80 percent of the process is planning and review, not coding. The “compound” step is where magic happens—every bug and insight gets recorded so agents learn from it. Read this for the full framework plus Every’s Claude Code compound engineering plugin to try it yourself. “Teach Your AI to Think Like a Senior Engineer” : When Kieran wanted to build an “email bankruptcy” feature to clear 53,000 emails, he didn’t start coding—he deployed a research agent that discovered Gmail rate limits would have killed the implementation. One 20-minute session saved days of building the wrong thing. Here, he shares eight planning strategies, organized by complexity level. Read this—complete with GitHub links to copy Kieran’s exact agents—to begin building smarter. “How I Use Claude Code to Ship Like a Team of Five” : In this piece, Kieranreflects that every line of code he’d shipped in the previous two months was written by AI—not assisted, but written. Claude Code opened 100 percent of his pull requests. His monitor looked like mission control. His message: Stop thinking about files and functions; start thinking about outcomes and delegation. Read this for the full workflow and custom commands that makes his two-person team punch way above its weight. “My AI Had Already Fixed the Code Before I Saw It” : Before Kieran opened his laptop, Claude Code had already reviewed his pull request—citing three months of prior feedback by PR number. There was no prompting; it had absorbed his preferences like a sharp new teammate. This is compounding engineering: building self-improving systems where every bug becomes a permanent lesson and every code review updates the defaults. At Cora , time-to-ship dropped from over a week to 1-3 days. Read this to turn today’s fixes into tomorrow’s automation. “Stop Coding and Start Planning” : AI made us sloppy. Why plan for an hour when you could vibe code for five minutes? Because that five minutes often costs three hours debugging—and teaches the system nothing. Kieran introduces a framework he calls “three fidelities”: quick fixes, the sweet spot where planning yields massive ROI, and big uncertain projects that need “vibe planning.” The key insight: Code teaches AI how to solve one problem; plans teach it how to think. Read this to learn when to prototype and when to plan.
The vocabulary of friendship. I started using Claude Code in the terminal about a month ago and as a nontechnical person, it felt like entering the Matrix. This was a stark, intimidating space where serious people do serious things, all black with white text and no friendly buttons or colorful interfaces to hold your hand. The terminal is, I think, the loneliest place in software, and I was nervous about being there. Then I prompted Claude, and while it was working, a single word appeared: Spelunking. I remember thinking, Wait, what? Spelunking? I felt a sigh of relief that maybe I’m allowed to be a nontechnical person fumbling around in the terminal after all. That one word—playful, unexpected, slightly absurd—was like watching a flower bloom in a barren desert. The verbs show up while Claude thinks—where other software would say “Processing” or “Loading” or just show you a spinning wheel that tells you nothing, Claude says Noodling and Moseying and Vibing and Discombobulating. These aren’t enterprise-friendly words and they’re not the kind of language you’d expect from AI software. And something changes very subtly when you see them, something I didn’t expect. You stop waiting for Claude and start imagining with Claude. When I see “Spelunking,” I picture little agents with headlamps crawling through my codebase and exploring dark corners, and when I see “Marinating,” I imagine ideas slowly developing like flavors combining in a pot, and the whole black box of what’s happening in there becomes a story I’m somehow co-authoring in my head. The strangest part is it makes me want to be nicer to Claude, which sounds ridiculous but is true. The playfulness invites reciprocity. When something treats you with warmth and whimsy, you respond in kind. I type “thank you” more often; I’m patient when things take a while, and I feel like we’re genuinely collaborating. I’m not just commanding a tool to do things for me. I think this is what humanized AI interaction actually looks like. It’s not a chatbot pretending to have feelings or an avatar with a friendly face or any of the more obvious approaches to making technology feel warm. It’s delightful verbs, rotating in a terminal at 3 a.m. when I’m alone with some stubborn problem. It’s smooshing.— Ashwin Sharma
