Open and closed models are on different exponentials
Interconnects by Nathan Lambert · Monday, June 1 2026 · 7 min read · ↑ top
Where marginally higher intelligence drives value, and where it doesn't.
Listen to post · 7:21
The largest debate that’ll define the future balance of power between the open and closed AI model ecosystems is primarily economic — it’s if users of AI will continue to pay dramatically more, i.e. large margins, for the top closed models. Early 2026 is a seminal time for the AI industry, as the coding agents¹ have shown the first area where a huge AI market will continue to pay a substantial premium for better intelligence.
The other side of this dichotomy is the inevitable decay of API businesses at these same labs. These labs will realize they need to protect their best models, rolling them out later in APIs to both protect token supply, avoid distillation, and stick to use-cases with higher margins. All of these effects will be clearly visible in 5-10 year timelines, as in the near term markets, prices, margins, and demand will be dictated by a rapid buildout of compute (supply-limited in the near term) and mass subsidization of tokens (through continued investment in new AI companies).
The core of this argument rests in the obvious habit changes that are setting in with coding agents past the Opus 4.5 and Codex 5.2 thresholds. People are not making this switch because they are lazy, but because their net output is obviously higher when using an agent as an implementation aid for complex knowledge work. For people who rely on coding agents to work, they will always pay more for the best rather than settle for good enough. There are so many ways to make the product better, speed, intelligence, specialized models, etc.
I would pay $2000/month for the tools today, especially knowing they’ll get much better. At the same time, it is likely that many companies are forcing agents and usage onto people that actually will get very little out of them in their current form, which helps the AI buildout (or bubble) continue.
The best closed labs — right now this list is just Anthropic and OpenAI, but it’s reasonable to expect Google to catch up — will always make the most efficient models for intelligence at a given cost. Building models is a mass capital investment of talent, data, and compute. These systems, a combination of model weights, harnesses, tools, and serving infrastructure have massive returns on integration (where open models are designed to work across many, diverse serving situations). These integration benefits — the integration of hardware and new forms of software — can be expressed in any possible way of making models better.
The models in the near future may saturate on benchmark scores, but if that intelligence ceiling really is a cap on utility then the labs will optimize utility per second or per watt, serving users in another way. Improving the models is possible in every direction — there have been no walls in progress. We’re early in the mass buildout of intelligence, which involves harnessing the physical world to build numerous datacenters, organizing many AI researchers so that a large team can contribute to one model, and of course solving many small, low-level puzzles that unlock performance. Every indication is that there is still meaningful performance to be unlocked and the closed labs are the best set up to extract it.
The collective wisdom of the labs is that making the models smarter, in terms of the frontier of absolute intelligence, has the most value. This is the right call to me because it unlocks large new markets. Optimizing models at a fixed intelligence level locks in markets, expands accessibility over time, and increases return on investment for users (while potentially lowering margins for selling intelligence).
Many people are making this bet that models will keep getting better and are learning to work well in these harnesses, even though some workflows are still a bit clunky. This is the right bet. These people all will continue to use the absolutely best models available. It’s like buying an iPhone as a consumer. You could get an Android and suffer from a bunch of paper cuts to save money, but why would you? The returns to performance are even higher in the workplace, which drives pricing power.
In this mental model, the frontier labs as businesses, will look like new, reimagined forms of a mix of Apple and Microsoft. The Apple side is that they’re selling an integrated, extremely hard to replicate technology. The Microsoft side is selling high-leverage subscriptions across the economy. In 5-10 years I expect both OpenAI and Anthropic to be valued in the $2-10T range. The true frontier labs will be an oligopoly that looks like the cloud market today.
On the other side of this equation is the open model economy. This isn’t to say that the frontier labs will dominate all aspects of AI use. Yes, I expect OpenAI and Anthropic to be the most representative companies of the AI boom (new companies, alongside Nvidia of course), but the collective value capture around open models will be far bigger overall, it’s just that the revenue and margins will be shared across a wide stack of companies.
Many businesses want to switch to open models but the models today are not good enough in out-of-distribution tasks. Eventually open model builders will stop chasing Claude and GPT on the Artificial Analysis index and fill this niche. This fork could be driven by economic factors, where they no longer have the revenue to support the growing R&D costs for continuing to scale models. It can also be driven by pure demand, where certain AI solutions only can exist at low price points present in open models. Where closed labs are an oligopoly, open model builders and users will be far more diverse and numerous. The total market value will dramatically exceed the cumulative value of OpenAI and Anthropic.
Open models are by their nature not integrated, so they will rely on multiple companies coordinating to serve them. Each of these layers will have alternatives, driving prices down to commodity pricing. These low, predictable prices will be where many enterprises enter to build in-house agents and tools for niche tasks. The predominant mode of deployment here is that enterprises find a model that hits a sufficient performance threshold on a task of interest and does not replace the model later (setup costs are high). As customizing models becomes easier, again in the open model finetuning stack we are seeing emerge (Tinker, Fireworks, Prime Intellect, etc.), this market becomes even bigger.
What this will look like in the coming years is a steady rise in open model inference proportion across the entrenched hyper-scale clouds of Google, Amazon, Microsoft and new AI infrastructure companies of Together, Fireworks, OpenRouter, etc when compared to OpenAI and Anthropic.
The closed models hit incredible product-market fit with the current agents, starting their integrated exponential by monetizing the top end of the knowledge work. The open model economy will take far longer, but it will also be far more satisfying to follow, as it tracks the broader diffusion of AI into the entire economy and world.
1
The term coding agent is funny because we barely write code in them. They’re general agents that are so capable because they write a lot of code.
Liking, sharing, commenting, or recommending Interconnects from your Substack is the only way Interconnects is possible. Thank you for your support.
If you liked this, consider upgrading to a paid subscription to cover my growing subscription and API fees. We offer group Interconnects subscriptions at tiered discounts for 5+ heads.
If you want all of the best AI writing on Substack through one subscription for your team of 20+, check out https://readsail.com/.
Americans pay more for healthcare than any country on earth — and they're getting less for it every year.
In this week's Prof G+ Deep Dive, Scott explains why hospital consolidation is the real driver of America's healthcare cost crisis, what solutions actually exist, and what's standing in the way.
If you’re not a Prof G+ subscriber yet and want access …
No ads on pods, because ads tax your most valuable asset: time
Prof G+ exclusives, including breaking livestreams, deep dives, keynotes, and more
Community privileges to leave comments, make friends, and engage in a series of bad decisions that might pay off
Drake Dukes · Monday, June 1 2026 · 7 min read · ↑ top
Former Apple Siri & Roblox ML scientist enters stealth, Ex-Ogilvy PE operator builds AI for the full PE deal lifecycle, & Ex-Lyft hardware director exits stealth with $12M to test hardware in seconds
We’re tracking company launches as they happen and surfacing the most interesting new founders and startups (yes, all outside of this stealth activity too). If you want to stay ahead of the curve, this is where you’ll find them first.
Right now we’ve got 5 subscribers: my wife, my mom, and that high school buddy who won’t stop pitching me his app. Be smart like them and get in early 👇
We run a live feed inside Gravity that tracks founders entering stealth and companies quietly exiting it.
What you’re reading here is about 1% of the stealth activity we pick up. The full tracker updates in real time as things change and new activity emerges.
Co-Founcer & Co-CEO:Maximiliano Casal(Co-Founder at Nowports - YC W19 unicorn)
Arkimedes is an AI infrastructure platform for private equity firms, offering five integrated modules covering the full deal lifecycle built for large-cap PE, mid-market PE, and corporate development teams.
HQ: New York, United States
Industry: FinTech, Private Equity, Enterprise AI | Team Size: 8
FounderDNA: Serial Founder, Top 10 University, Masters Degree, Doctorate Degree
Prior Experience: Data Scientist at McKinsey & Company, Computational Biologist at Harvard Medical School, Co-Founder & CEO at Expand Consulting, Co-Founder, ML at Maka Media
Prior Experience: Security Team Lead at Axis Security, Cyber Security Researcher at Rafael Advanced Defense Systems, Security Research & Development at Israel Defense Forces
Ocean is an agentic email security platform that uses purpose-built AI agents to detect malicious intent in targeted and novel email attacks, replacing legacy pattern-based detection with context-aware analysis at enterprise scale.
HQ: New York, United States
Industry: Cybersecurity, Email Security, AI | Team Size: 2
Latest Funding: $28M Series A Round on 5/19/2026
Key Investors: Lightspeed Venture Partners, Picture Capital and Cerca Partners
FounderDNA: Serial Founder, Technical Founder, Masters Degree, Doctorate Degree
Prior Experience: Director of Hardware Engineering at Lyft, Head of Electrical Engineering (Jump) at Uber, Director of Engineering at Flex, Co-Founder/Director of Hardware Engineering at Olio Devices
Panacea pairs experienced ex-FDA regulatory consultants with an AI platform to deliver the fastest and lowest-cost pathway to FDA approval, offering fixed outcome-based pricing across all major regulatory pathways.
HQ: San Francisco, California, United States
Industry: HealthTech, Regulatory Technology, Biotechnology | Team Size: 2
Time Spent in Stealth Mode: 5 Months
🕵️♂️Key Talent Going Under Stealth
Illuminating clues left behind by world class talent and influential innovators who just went into stealth mode
Mahesh Nandwana - Founder at Stealth Startup
FounderDNA: Technical Founder, Masters Degree, Former FAANG
Prior Experience: Technical Lead/ML Scientist (Siri Perception) at Apple, Senior Principal ML Scientist (Foundation AI) at Roblox, Advanced Computer Scientist (Speech Technology and Research Lab )at SRI International
Co-Founder and CEO:Mark Miller (General Manager and Head of Product - Business Prime at Amazon)
HQ: Seattle, Washington, United States
Time Spent in Stealth Mode: 1 Month
Kyle Morgan - Founder at Stealth AI Startup
FounderDNA: Former FAANG, Top 10 University
Prior Experience: Lead UXR (Bard & Gemini) at Google DeepMind, Lead UXR, Applied AI / Enterprise AI Agents at Google, Head of Digital Experience at Vocus Group Limited, UX Design Research Manager at AGL Energy
Vera Chichagova - Co-Founder & CEO at Stealth Startup
FounderDNA: Masters Degree
Prior Experience: Director Regulatory Compliance at OKX, Group Head of Regulatory Compliance at Revolut, Vice President Compliance at Barclays Investment Bank
🚨Here’s the deal 🚨This email has gotten too big. Exciting, but with more people following it, the edge diminishes. I’ve thought long and hard about what to do to preserve the value in the signals. I’m not sure about the final direction yet, but in the meantime I’ve been sending an email 48 hours earlier to a select group of paid subscribers. The feedback has been pretty positive so I’m going to open up the list for another 100 spots. To get signals early, Apply here!
Stay Stealthy,
Drake
Thank you for reading. If you liked it, share it with your friends, colleagues and everyone interested in staying ahead of the hidden developments in tech. Subscribe below and follow us onX / Twitter to never miss a company operating under stealth again.
Stealth Startup Spy is a data-driven newsletter for investors, journalists and tech enthusiasts interested in uncovering the next big move for key talent, real-time stealth company launches and technology advancements not in plain sight. We leverage the technology built at Gravity to shine a light on the hidden world of stealth startups.
Scott Galloway · Tuesday, June 2 2026 · 2 min read · ↑ top
Plus, the Markets Tour is live from NYC… tonight
I foster a decent amount of loyalty among the people I work with. It’s not a function of character or empathy, only the recognition that nothing wonderful happens when you’re on an island. Greatness is in the agency of others. Behind my bold predictions, sharp analysis, and smart takes there’s a talented research team, led by Mia Silverio. Extra Credit is a newsletter where that team steps into the spotlight.
Each week, Extra Credit brings you one new story that digs deep into a business or business-adjacent topic that isn’t aggregating attention — but should be. Every post also includes a page from the Prof G Media storytelling playbook, with the writer sharing the narrative techniques we frequently deploy. It’s a twofer: Get smarter, become a better storyteller.
Since you’re already a Prof G subscriber, you’ll automatically receive the free Extra Credit posts, but if you want the all-access pass you’ll need to upgrade here. Or, you can make a bad decision, i.e. opt-out by clicking here.
First post drops tomorrow. Enjoy.
Students Eat Free
Well, not quite free, but we’re celebrating the launch of Extra Credit by offering 50% off annual and monthly Prof G+ subscription plans to students and educators with an academic email address. Matriculate below (or share with your favorite student).
Live from New York
… it’s Tuesday night!
That’s right, we’re livestreaming the sold-out final stop on the Prof G Markets tour from NYC tonight at 7:30 p.m. ET , exclusively for Prof G+ subscribers.
Love listening to me and Ed Elson on Markets? Tune in, we’re even better live. Love the Mooch? Tune in, Anthony Scaramucci joins us onstage as a special guest.
Still not enough for you? Maybe our surprise guest will move the needle. I’d say who, but I’ve been briefed on why keeping things under wraps is a matter of national security (seriously). So, yeah… they’re kind of a big deal. Curious? Join us tonight at the link below.
Raging Perspective
Last week, we debuted Raging Perspective , our Substack-exclusive live show brought to you by my Raging Moderates co-host, Jessica Tarlov, and Gen Z political analyst, Aaron Parnas. Watch a replay of the first episode in all its unscripted, unfiltered, unedited glory here.
Raging Perspective is back again this Wednesday at 2 p.m. ET (note the time change, this week only). Register below to be notified when the stream goes live, and don’t forget – this show is interactive, baby. Bring your rage, and your questions.
See you tonight on the Prof G Markets tour livestream.
ben's bites · Tuesday, June 2 2026 · 6 min read · ↑ top
NVIDIA and Microsoft birthed a new computer
Hey folks,
I’m spending as much time as possible off Twitter, and any other general doomscroll distractions. I’ve got work to do (the course/manual/whatever it’ll be called).
Aiming to get at least the preview ‘lessons’ out this month.
I’m not going to force any wisdom out in today’s intro because it’s not my style.
Pulse is world’s fastest speech-to-text model (#1 on Sierra’s μ-Bench for P95 latency) with top performing accuracy (under 5% WER on Artificial Analysis leaderboard), works across 39 languages and 100+ accents.
Get $25 free credits, use code BYTE25-N3CX3UKV (valid till 6/6)
Headlines
Claude Opus 4.8 is out, with dynamic workflows in Claude Code. Claude now writes an orchestration script, then spins up subagents in parallel to work through complex tasks.
Dex’s take: this doesn’t prove loose multi-agent systems work. Deterministic workflows around small agent loops are more reliable.
Claude Opus 4.8 - Simon Willison calls it a modest but useful upgrade, mostly because it’s more honest about uncertainty and less likely to miss flaws in its own code. Every’s vibe check is more bullish: they found it a big jump from 4.7, strong at coding/writing/knowledge work, and competitive with GPT-5.5 on their internal senior-engineer benchmark. The catch is the harness: the model is back, but Claude’s app still feels messier than Codex.
Stacker's AI Accelerator is offering $500k in inference credits to businesses ready to go AI-first. Selected companies get credits and hands-on mentoring to deploy AI agents across their operations. Applications close June 9th. Apply now.*
My feed
Stacker is an AI coworker that joins the dots in your business. Stacker lives in Slack, connects your tools, and does the work.*
Emil Kowalski
@emilkowalski
To get good animations from an AI you need to get good at telling it what you want: - "stagger this list of items" - "make this animation direction-aware" - "spacial consistency", "crossfade", "layout animation", I made a motion vocabulary for this: animations.dev/vocabulary
Wes Winder
@weswinder
this codex prompt is literally the best thing ever if you are building react apps and want the cleanest possible code simply run "/goal run npx react-doctor@latest and fix issues until you get a score of 100. do it properly without taking any shortcuts" in codex thank me later
Lenny Rachitsky
@lennysan
My biggest takeaways from @benedictevans : 1. We’re in 1997 for AI—it’s as big a deal as the internet or mobile, and only as big a deal as the internet or mobile. We’re at the stage where most stuff kind of doesn’t work yet, most of what people will build hasn’t been built, and
Lenny Rachitsky @lennysan
A rational conversation on where AI is actually going with @benedictevans For 20+ years, Benedict has been one of the clearest, most reliable thinkers on where technology is heading, and how it'll impact our lives. He was @a16z's resident "thinker" for 5+ years, and has spent
Farza 🇵🇰🇺🇸
@FarzaTV
Watch me control my computer with just my voice. This is the future of operating systems. No hands. GPT-Realtime 2.0 is very, very underrated. Demo:
Interconnects by Nathan Lambert · Tuesday, June 2 2026 · 15 min read · ↑ top
Listen to post · 15:50
I’m departing the Allen Institute for AI (Ai2), where I got the great privilege to work on the Olmo models, to grow, to learn, and to have broad lasting impacts. This post is an attempt to reflect on why what we did was influential, despite obviously being far from the frontier in performance (even when within size buckets), and how this reflects on various paths to impact in AI today.
To start, I shared the following note with the company yesterday:
Dear Ai2.
As many of you know, today is my last day working at Ai2.
I joined Ai2 largely as an accident. I met Luca at ICML 2023 in Hawaii and realized I could level up my open post-training work dramatically if I got the chance to join. When I got an offer it was an absolute no-brainer, it was such a welcoming and exciting environment.
It has been a wonderful ride that has transformed my life, and I couldn’t be prouder of the work we did together. Ai2 has a wonderful scientific culture at its core and I’m excited to see this continue. I feel very lucky to have been here and that I personally have benefited massively from everyone who has worked so hard to cultivate that culture and environment. It is and has been a team effort. This includes all the people whose longest interactions with me were brief chats at the coffee machine. I drew so much energy and excitement from all the different ways people at Ai2 showed up for the mission.
I’ve already thanked much of the OE team directly, but I wanted to thank everyone else that went into this. Legal, IT, Comms, and the Office team all do a great job enabling and leveling up our research work. It’s often work that is forgotten, outside of the lime light, or remembered at the last minute, but it all has been crucial to achieving our goals. I’m excited to keep visiting the wonderful Northlake space in the coming years.
Even though I’m leaving, I’m more excited than ever about Ai2’s mission. Ai2 operates in such a rare niche between academia and industry, where we can explore and influence the most important technology of our lifetime. Doing this openly is the best way to ensure the technology diffuses safely to everyone who may benefit. Ai2 needs to stay as ambitious as possible, trying to influence the cutting edge of AI and the biggest issues of the field. Do not shy away from these challenges – AI needs independent voices as it only becomes more geopolitical, socially disruptive, and central to the economy.
I will still be working in this space, working to make the open ecosystem better coordinated and more useful.
So as I go off to try something new, don’t be strangers. I’ll always be reachable at nathan@natolambert.com and will still live in Seattle for most of the year.
Nathan
I have loved and will still love Ai2. Ai2 has a deep culture of caring about the research process, the outputs that get shared, and most importantly the people who do the work. This is why the institution creates countless wonderful people that go and spread the gospel throughout the research community. This core culture will remain through the rebuild, and there are plenty of resources to do impactful research across the spectrum of AI.
In the last two years of my time at Ai2 I’ve done so much meaningful work. Of course Olmo is at the top and has been my priority, but making time for consistent practice here on Interconnects, weekend cram sessions for ATOM, and also the fun RLHF book make for a list that makes me wonder how I did it all. I was obviously obsessed with work, but not in a way that made me lose sleep or lose my overall wellness. It was the right long-term approach.
This impressive list is one where I was ruthless in saying no to things that didn’t matter and got all my work out to see the light of day. I had no medium-sized projects that didn’t succeed in the last few years. It makes me wonder if I wasn’t taking enough risk. It shows you can truly do so much with your time, and it’s actually harder to find the right problems and environment to do it. Many people are in environments where their work never becomes public or they’re forced to change topics consistently.
From zero to hero
To start, I’d like to do a short recap on my path to Ai2 to show what Ai2 was just as much a growth story for me as an execution story.
I studied electrical engineering in undergrad, focusing on linear systems math and microelectronics.
I was admitted to the UC Berkeley EECS Ph.D. program to study microelectromechanical systems (MEMS).
I showed up at Berkeley in August of 2017 and realized AI was obviously the thing I should be doing. I asked the likes of Sergey Levine or Pieter Abbeel if they could advise me – they said no.
I threw all my energy into learning what I could about AI. I got a break to get advised by one of Sergey’s post-docs in 2018 or 2019. I went all in on that, I fought for funding, I fought to have an AI paper.
This process worked out by the end of my Ph.D. in 2022: I had access to the Berkeley AI Research (BAIR) building and collaborations in the department. It was a bumpy road.
I wanted to go to industry research, to get a nice paying job with intellectual freedom, something like FAIR or Google Brain at the time. HuggingFace was the only job that fit that bill, it was easy to say yes to.
I joined HuggingFace in May of 2022 and wasted my time at the company until ChatGPT was released. I used my RL background to write a blog post on RLHF which went viral. HuggingFace decided it would be good for me to form a team around this success.
In 2023 I learned NLP and about language models. I had a lot of fun and built an initial community. I got burned out by working remote with a huge time difference. I met Luca Soldaini at ICML in Hawaii, where I was giving a tutorial on RLHF, and they told me Ai2 was hiring.
I got the job at Ai2 largely because of my excitement and how I was saying I wanted to do a lot of stuff that sounded cool to them but no one was likely to do (RL related things). My interviews were far from a sure thing – this is a great job to land!
I started at Ai2 in October of 2023. I worked remotely for a while. I was doing normal research, I made the first reward model evaluation, RewardBench. It was a solid success, but nothing like how the pretraining team was getting ready to release the first Olmo.
I helped coach Ai2 on how to release models well, helping the Tülu 2 project land (the first model to do DPO well, publicly at the 70B scale).
The first Olmo was released in early 2024, I squeaked onto the papers just by trying to be helpful and doing some basic post-training. I was already good at paying attention to which projects are actually important.
That summer I started rounding everyone up to do a “big frontier post-training project.” This became Tülu 3, one of my favorite projects ever released, in fall of 2024. The goal was to beat Llama 3’s post-training with their own base model. The team morale was incredibly high and the execution was so timely, allowing us to coin the term Reinforcement Learning with Verifiable Rewards (RLVR) in the paper.
The crazy lengths I went to get the Tülu 3 and Olmo 2 post-training done had me sending 40% more slack messages than anyone at the company and got me the award “The Cat Herder.”
2025 was a much simpler year. We were too slow to react to reasoning models, given we had been doing similar stuff with Tülu 3, but sometimes that happens.
Originally we wanted to release Olmo 3 by June or July of 2025. That obviously didn’t happen, but we got the slim chance to train a bigger model, and it really landed. We threaded the needle.
Since Olmo 3 was released, it was clear that some changes were coming and I personally never got a big post-training project off the ground after that. Many other people managed great work in the spring of 2026.
This all leaves me here today showing you that only about half of my story at Ai2 is what I was known widely for, and the rest was building momentum. It often takes a year of building relationships and direction before really big successes can happen in a career.
I was just about a nobody when I joined Ai2 and I got to join a team that was willing to learn from the skills I had brought from HuggingFace. With how media works, I often think I get more recognition than I deserve for Ai2’s success.
The likes of Tülu 3, Olmo 2, and Olmo 3 felt like generational team efforts. The amount of personal successes and breakthroughs that happened for those projects is immense – and to sustain them over such a long time period is incredibly hard to replicate. The sum far exceeded the individual parts.
I’ve heard many times in the last few months how people wouldn’t know about Ai2 if it wasn’t for my writing. Statements like this are overblown, but they are partially true and reiterate how crucial building relationships and getting the word out is today.
When you write a plan that is feasible, the world bends towards that plan. When you convince people it’s going to happen it only becomes more likely. Vision and compelling explanations are one of the items in shortest supply in the tech industry. Often building the thing is easy and explaining it is hard. If no one knows about your work, the value is often close to 0. So much of building reputation is about building relationships with people who will receive your work.
Reflecting on all of this, I’ve had a shockingly linear path through my career to incremental success. I would expect the first 10 years of most careers to be in search of finding one opportunity as good as Ai2, and you will not always be able to seize it. There are some ways to create more opportunities.
I’ve discussed before how a large part of my rise is down to many more senior and more established scientists being drawn into the closed ecosystems at the same time as an immense swell in interest for AI. This created a power vacuum that I, and a few other prominent scientists that I think form my “generation”, got to grow rapidly into.
The role of public scientists
With my work at Ai2 and Interconnects, I summarize my role and mission as trying to accomplish three things:
Provide clarity in the evolution of frontier models. This is easiest when the science has caught up, but even applying a scientific lens to how the models are changing is very useful to building trust in the broader AI ecosystem.
Create a vibrant and diverse open (model) ecosystem. This is crucial to mitigating some risks of AI, particularly with concentration of power and myopia in studying frontier safety, that has motivated me now for 3-4 years. The risks haven’t abated.
To build institutions that create people and ideas that further the above missions, and generally mission-driven individuals that are willing to advocate and build a future they believe in. AI is a grand problem, and not one that I can do alone, so I need to build brands to rise through the noise and attract likeminded people.
At my best, I have many avenues for impact. I help open researchers work on impactful problems – not wasting the precious compute and time they have during the AI boom. I help policymakers know what is true. I build models that people use. I tell stories that make people smile. I keep the list wide so that I can stay motivated.
I see all of this continuing, and have been thinking about the broader impacts of this repeatedly over the last few months. Hearing that Andrej Karpathy was joining Anthropic prompted me to finally share more of my opinions:
For a long time, academic researchers being at the cutting edge of new technologies has been a great social equilibrium. Neutral, unbiased technologists have been the people to spread new ideas to the world.
As AI research takes off in velocity, it is also going behind closed doors. The tech industry has sowed distrust, and now they are the ones trying to tell the world about incredible changes coming. It’s a big loss to a form of social contract in America.
There’s been a history of scientists helping society understand new technologies. There is a public service in the culture of science that I want to see continue.
It’s being exacerbated by feelings of FOMO, especially financially driven, where I’m seeing many people who previously wanted to be professors -- and likely still do deep down -- feel a need to conform and chase money, in a pocket of industry. I get it, I grapple with this.
For those with a safety net, there will be great returns to some who choose to zag, and try to build something good, for people who need something different. For me, this is building interesting, fully-open models, to show what you can do with a variety of open weight sizes.
Yes, AI’s immediate future is dictated by the frontier, but it’s long-term trajectory still deeply includes academic institutions and open science. Knowledge will always diffuse, but to whom?
As of today, I think China is positioned to be the global home of AI research in a few years. The home of research is where ideas are accessible, spread rapidly, and are nurtured. The U.S. seems to be unwinding many institutions and relationships.
The largest returns go to people who build something differentiated, at least in reputation, and a lot of people are not being shown that this path exists.
To elaborate on this, I don’t fault any of the individuals who are going to industry today. I’ve been very close to doing this myself in the past weeks of job searching, or rather job exploring. It’s a systematic problem where scientists cannot easily get the support to take bold stances, especially stances that are designed around the public good.
To go a step further and say that only the research within closed, frontier labs matters is very myopic. Yes, there’s a sort of research you can only do with vast compute resources, and they will directly impact the most revolutionary tools of the day. But, I see the relative opportunity to do good elsewhere as higher for plenty of people.
Open research will always be the standard that sets the language people use to understand AI. It’ll always be how the next generation is trained – even if it’s behind what industry has built. It’ll be the ecosystem where new long-shot ideas are built. Without investing in this open ecosystem, all of these cycles will be kneecapped.
At the end of the day, so much of my role now is just showing the path to impact in this domain. To show how clever, mid-sized open models can impact real problems in the world. To show how policy-makers and educators need open research to structure the rest of society around AI. This is a fun role too! It would be very sad for me to see this light diminish ever further, into the lightest embers of a fire that looks almost entirely out.
Even if the pace of research were to slow further, if the folks remaining like myself got financial offers they can’t refuse for their families’ sake, the torch of open research will never fully go out. It’s core to how science is taught and done. There is a next generation coming, they just look for guidance and role-models.
What’s next
I see the best Ai2 work as research infrastructure. Building recipes in public gives countless researchers the ability to ask very specific questions of training processes. We need these researchers in the broader community, as Ai2 could never answer all the interesting questions themselves. One of my great joys in recent months has been visiting a top ML university and hearing so many graduate students say they’re building on Olmo. This is how the world should work!
Going forward, I still plan to operate in similar spaces, fighting for open-science, imagining what the future of the open model ecosystem can be, and doing my best to make the social transition to an AI-native era smooth. I’m most excited by how you can train medium sized open models on specific tasks that become useful tools in complement to the frontier models – massively winning on price. I want to invest in the ecological diversity of open models and coordination across builders.
For something that isn’t surprising given my past focus areas, I’m watching the pace of releases from all labs open & closed, and how they’re hillclimbing on super ripe new post-training veins (on-policy distillation, agentic workflows, etc.), it’s clear that fully-open post training recipes are about as far behind as they ever have been & falling further behind. I’d like to fix this. It’s not 100% clear yet if I will this year, but I’ll try.
To do this best and to execute, mostly personally, I needed a new start and fresh perspectives. I’ll be carefully building what I’m doing next over the next few months and am eager to share more about it when I can. One of my close teammates at Ai2 shared this quote with me in a farewell card, and I found it very apt in where I’m going next.
The object of life is not to be on the side of the majority, but to escape finding oneself in the ranks of the insane. — Marcus Aurelius
Liking, sharing, commenting, or recommending Interconnects from your Substack is the only way Interconnects is possible. Thank you for your support.
If you liked this, consider upgrading to a paid subscription to cover my growing subscription and API fees. We offer group Interconnects subscriptions at tiered discounts for 5+ heads.
If you want all of the best AI writing on Substack through one subscription for your team of 20+, check out https://readsail.com/.
A guide for identifying which state of AI adoption matches your needs, complete with sample prompts and advice for when it’s time to move to the next level
by Mike Taylor All it takes is one viral post to make you feel like you’re using AI all wrong. Someone’s running 12 Claude Code sessions in parallel. Someone else’s agent answers emails while they sleep. Meanwhile, you’re still arguing with ChatGPT. Here’s the thing: Keeping up with the power users isn’t the point. The best way to get value from AI is to use it in a way that fits your work—and to check in now and then to see whether you could be getting more from it. With that in mind, today we published a guide that maps all eight levels of AI adoption, from chatbot basics to full agent orchestration. We explain how each level works in practice, with sample prompts, so you can figure out which ones match your current needs and workflows, what’s possible at each stage, and when it’s time to move to the next one.
Level 1—Chatbot: You ask, it answers.
Level 2—Copilot: The AI works alongside you, inside your files.
Level 3—Agent: It executes a task step by step, checking in for approval.
Level 4—Autopilot: It runs on its own; you review the result.
Level 5—Workflows: You build a system that makes its output more reliable.
Level 6—Assistant: It works in the background, without being prompted.
Level 7—Multi-agent: You manage several long-running agents at once.
Level 8—Orchestrator: A manager agent runs a team of sub-agents for you.
A higher level isn’t necessarily better. The right level for a task is generally determined by how much you trust the AI to do a good job without intervention, and how big a deal it’ll be if it does mess up. If you want to know where you fall on the AI adoption spectrum—and whether it’s time to experiment with higher levels—this guide is for you. Read the 8 levels guide
Prof G Research Team · Wednesday, June 3 2026 · 7 min read · ↑ top
The autonomous vehicle rollout is about to get a lot more complicated
Every year, more than 36,000 Americans die in car accidents. Compared with human drivers, Waymo’s autonomous vehicle (AV) technology results in 92% fewer serious accidents, suggesting it could prevent many of those deaths. But after a year of the company operating eight cars with zero accidents in New York City, Mayor Zohran Mamdani let Waymo’s testing permit expire anyway. When asked why at a press conference, Mamdani made his position clear:
“Look, if a company like Waymo finds itself in New York City, what they will also find is a City government that is committed to delivering for the workers who keep the city running, and those workers also include our taxi drivers who, for far too long, have been sold a dream of being able to work their way to the middle class, only to have the rug pulled out from under them.”
To translate: The mayor of New York City will not let technology companies automate the work of taxi drivers and rideshare drivers. He won’t be the last politician to take that stance.
Hit the Brakes
For as long as autonomous driving has existed, there has been organized resistance to it. Mamdani is not an outlier — he’s the latest and most prominent face of a movement that has been working for nearly a decade to slow, stall, or stop autonomous vehicles from replacing human workers. That movement has a name, a headquarters in Washington, D.C., and 1.3 million members: the International Brotherhood of Teamsters, America’s largest private-sector union and the most powerful force standing between autonomous vehicles and the open road.
Nearly a decade ago, the Teamsters successfully lobbied Congress to exclude autonomous semitrucks from legislation that paved the way for autonomous vehicle testing. Four years later, in 2021, they shut down a bill that would have relaxed federal autonomous vehicle rules. In 2023, the Teamsters backed a California bill that would’ve required human drivers in all autonomous trucks. The bill made it past the state Legislature, but Gov. Newsom vetoed it.
As autonomous driving develops into more serious technology, so do the Teamsters’ efforts to halt it. Last year, the conflict intensified when Teamsters Local 25 called on Waymo to pause their planned Boston rollout altogether.
“Waymo is steamrolling into cities throughout our country without concern for workers or residents,” said Local 25 President Tom Mari at a rally outside of Boston’s City Hall. “They’re doing this because they want to make trillions of dollars by eliminating jobs.”
As of now, Waymo is continuing their testing in Boston despite the Teamsters’ opposition. Under the current Massachusetts permitting process, all autonomous vehicles need ahuman operator behind the wheel. There’s currently a bill in the Massachusetts State House that would change this — but it’s been met with a competing Teamsters-backed bill that would officially codify the requirement for human operators. It’s unclear how or when this conflict will be resolved, especially as Waymo continues to ramp up its lobbying spend.
What is clear, however, is that these legislative battles are not one-off events. They’re the beginning of what will be a long path to seeing autonomous vehicles on the roads of all 50 states. And for good reason — the Teamsters have a point. Job destruction is coming.
Driver Destruction
Since ride-sharing services began popping up in the early 2010s, they’ve become the backbone of the gig economy. According to Deloitte, about a third of the American workforce participates in the gig economy. From there, it’s estimated that at least a quarter of American gig workers drive in some capacity, whether that be delivering food, groceries, or humans. Autonomous vehicles put all of these jobs at risk.
If we zoom out to include other driving occupations, it gets even worse. According to the Bureau of Labor Statistics, there are roughly 440,000 taxi and limousine drivers, 460,000 food delivery drivers, 1.5 million small-package delivery drivers, and 2.2 million long-haul truckers. That’s a total of 4.6 million jobs.
Driving is also the most common occupation among young men without a college degree, by far. All of this means that, as you read this, nearly 3% of the American workforce is in the crosshairs of Big Tech.
Nearly all of those occupations are, in some capacity, represented by the Teamsters union. The transportation sector as a whole has a union penetration rate of about 14% — a minority, obviously, but still more than double the rate of the private sector in total.
The Teamsters’ fight against autonomous driving is important not only to its members but also to its very existence. If the Teamsters begin bleeding members due to automation, they’ll start losing dues, which would place pressure on the union’s finances and dampen its political influence.
Outside of unions, it can be assumed that there will be plenty of political capital to be gained in future elections for politicians who choose to take a stance against Big Tech and autonomous vehicles. The tide is turning on all forms of AI, and public polling already reflects this reality.
The share of Americans who think AVs improve road safety has actually fallen 12 percentage points since 2018 — and that’s despite the technology becoming exponentially safer over the same period.
Compared with human drivers, Waymo vehicles have 82% fewer injury-causing crashes, 92% fewercrashes involving injured pedestrians, and 85% fewer crashes involving injured cyclists. Last year, car accidents were the leading cause of death for Americans ages 5 to 29. In an entirely Waymo’d world, more than 33,000 lives would be saved annually. But without public buy-in, none of this matters.
The raucous boos at graduation ceremonies across the country illustrate how unpopular artificial intelligence and autonomous technologies more broadly are in America. Autonomous vehicles are possibly the most visceral andphysical embodiment of AI that the average American might interact with on a regular basis. In cities where Waymo or Tesla or Zoox operate, autonomous vehicles are everywhere. They serve as a constant reminder of the once-farfetched, unrecognizably transformed future that awaits us — a future that most Americans interpret as dystopian.
Let’s Talk
The Teamsters are not the first union to stand in front of automation. In the 1810s, English textile workers — the original Luddites — smashed power looms with hammers in the dark of night. In the end, it didn’t save their jobs.
A century later, elevator operators’ unions fought to keep humans at the helm of automatic elevators. Those jobs are long gone. In the 1960s, longshoremen’s unions battled the introduction of shipping containers. The longshoremen lost that fight, but not before negotiating severance funds and job guarantees that softened the blow for existing workers. The pattern is undefeated: Technology always wins, the only question is how much protection workers are able to take with them on the way out.
This is exactly why Mamdani and the Teamsters are wrong. Letting the permit expire is not good for labor — in fact, it has the opposite effect.
The more honest and useful fight is the one the longshoremen eventually settled for: not blocking the technology altogether but demanding that the companies deploying it bear some of thecost of the disruption they’re causing. Retraining funds, transition payments, even a Waymo-funded safety net for displaced drivers are all policies worth proposing. Instead, the Teamsters’ current Luddite-inspired strategy will leave workers empty-handed. The question they should ask themselves is: Do you want to be right or do you want to be effective?
Each week, this section will give you an inside look at how this article came about, as well as the analytical and storytelling techniques the author used to report it.
Working with Ed on Markets has made me acutely aware of the public’ssouring perception of all things artificial intelligence. Months before commencement speakers were getting heckled for mentioning AI, Ed pointed out the mounting opposition to data center construction across the country and its relation to politics.
I’ve been following the autonomous race closely, and Ed’s findings made me wonder: Is the same thing happening with robotaxis? I realized that the answer was a resounding yes. But when you think about thetrade-off between safety and jobs, things get complicated … and a lot more interesting.
One more thing: Part of being a good analyst is recognizing patterns across different domains. When a new story reminds you of something you’ve seen before, follow that instinct. History might not always repeat, but it’s always instructive.
Dan Chiolan is a research analyst on the Prof G Markets team. He started as an intern in 2024 before joining Prof G Media full time after graduating from Temple University.
Yesterday Microsoft added a new metric to a model release card, one that will likely become a standard.1 Average token usage. In the first row, the Microsoft model hits 71.6 on SWE-Bench Verified using about a third of the tokens Claude Haiku 4.5 burns. Benchmarks are now measured on two different dimensions, the overall performance & the cost to achieve that intelligence. This is yet another sign that the era of subsidies2, tokenmaxxing3, & all-out performance for many use cases is over. Even the most valuable companies in the world cannot afford state-of-the-art intelligence for every conceivable use case.4 Uber capped employee AI spending after blowing through its budget in four months.5 Salesforce is spending $300M on Anthropic tokens & has frozen engineering hires.6 This new dual benchmark answers the buyer’s only question : what is my intelligence per dollar? Artificial Analysis already benchmarks this.7 GPT 5.5 & Claude Opus 4.8 land within a point of each other on the Intelligence Index, around 60. Running the index costs $3,357 on GPT 5.5 & $4,685 on Opus 4.8. Same answer, 40% more expensive. Model companies must now compete on both dimensions. The application layer will compete one level up, on dollars per outcome, what a closed ticket, a shipped PR, or a resolved support case actually costs. Every layer in the stack now has to price the same way the customer thinks : per result, not per token.
1. Introducing MAI-Code-1-Flash — Microsoft announces a new coding model with average token usage on the release card. ↩︎
2. The Unsustainable Subsidy — The era of AI subsidies is ending. ↩︎
3. Tokenmaxxing — Models that game benchmarks with extra tokens are losing their edge. ↩︎
4. Microsoft cancels Claude Code licenses, shifting developers to GitHub Copilot CLI — Microsoft cancelled Claude Code licenses across its Experiences and Devices division (Windows, Microsoft 365, Outlook, Teams, Surface) after engineering usage outran budgets. ↩︎
5. Uber caps employee AI spending after blowing through budget in 4 months — Uber caps employee AI spending after blowing through budget in four months. ↩︎
6. Salesforce Spends $300M on AI, Freezes Engineering Hires — Salesforce Spends $300M on AI, Freezes Engineering Hires. ↩︎
7. AI Model & API Providers Analysis — Independent analysis of AI model costs. ↩︎
A year ago, USV decided to explore what an AI native VC firm would look like. We hired Spencer who built a platform and a bunch of agents and wrote this post about all of that before departing to do a startup. And we paused our longstanding analyst program last year and saw how far we could get with agent analysts instead of humans analysts.
ben's bites · Thursday, June 4 2026 · 7 min read · ↑ top
Codex Sites and open models
Hey folks,
I’m making progress on my agents manual! I think I finally figured out how I want the thing to look and feel.
I’ve built and rebuilt this damn thing so many times in this process, which is actually part of the process. I am a lazy workaholic (h/t Rick Rubin) - I have to spend time in the work, even if it feels like it’s not going anywhere, until ‘suddenly’ things click.
Whilst in the process, you find yourself wanting tools to exist to make things easier for yourself—that’s a huge part of why learning agents and how to steer them is so good.
You can build tools to enable you to build things.
I spun up this tool before bed last night where I can comment/delete on copy whilst I’m building, which I copy as one big block as agent feedback.
Ben Tossell
@bentossell
spun up a little text-editing tool with codex before bed like agentation but just for copy (supports keyboard shortcuts too ofc)
Attio is the CRM for the new way of GTM. Get agents working on every account, surfacing opportunities, and handle the work that used to take your team days. Open your inbox, the follow-ups are drafted. Walk into a meeting, you're already briefed. Got a question, just Ask Attio. Start for free today.
Headlines
Codex has two new additions: Plugins and Sites. Plugins are pre-built collections of skills, connectors to relevant apps (like Figma for designers) and instructions tuned for specific roles like data analysis and product design. Sites lets users create a shareable website/app with a database, file storage, env vars, access controls, and more. Initially only available to business and enterprise users.
A bunch of new open models released recently -
Gemma 4 12B - Multimodal (i.e. accepts images and audio as input) and performs nearly as well as the two-month-old 26B variant.
Ideogram 4.0 - 9.3B model for image generation. Trained on JSON prompts for control over the layout, colours and text for each element on the image. Also check Reve 2.0 for the focus on layout of elements in an image (but it’s closed-source).
Miso One - 8B text-to-speech model claiming expressive speech with 110ms latency.
Also, just like Cursor’s Composer, more companies are trying out fine-tuning big open-weights models for their domain-specific work. Latest entry → Harvey got a Kimi 2.6 agent to beat Opus 4.7 on its legal benchmark at ~11x lower cost.
Microsoft Scout is an always-on Microsoft 365 agent built on OpenClaw (reminder: openclaw is open-source). Different approach from what Google is doing with Gemini Spark.
Ramp Stack - An accounting assistant that helps with month-end close work: reconciling accounts, preparing schedules/accruals and more with reviewable sources. They also published a nice blog post explaining their efforts to benchmark Stack against other frontier models.
Financial fraud is evolving fast. It’s time to fight back—with AI. Read MIT Technology Review and Plaid’s report to see how technology is reshaping financial defenses. Learn more and see how smarter tools and industry collaboration can help fight against the rise of fraud. Read the report.*
My feed
Smallest AI lets you deploy voice agents at scale, powered by realtime STT & TTS and production-ready telephony infrastructure.*
Bloom turns your brand assets, site, decks, Figma and socials into a callable system that agents can use via API/MCP to generate on-brand assets.
Windsurf is now Devin Desktop. It manages fleets of local and cloud agents from the editor. Nous also released a desktop app for its CLI agent Hermes.
Hallmark v1.1 - open-source design skill for coding agents.
ViBench - benchmark from Replit with tasks focused on end-to-end app creation; Opus 4.8 beats GPT-5.5 on price/performance for vibe coding.
Skills for macOS - app for browsing and editing local skills, MCP configs and plugins.
Ollie - AI assistant for parents to manage the chores to free up time for family.
Television - visual workspace for personal agents. Notion-like kanban board vibes but with each tile attached to an agent.
Building software is learning - it’s an iterative process that will run into questions and obstacles. You should want that to happen as fast as possible.
Modern Engineering Values - a workflow and engineering values built after shipping several mostly or fully AI-written projects.
SDKs I’ve come across:
Email SDK - unified API for sending emails. works across multiple providers.
storagesdk - object storage with snapshots and forks.
Afters
Ethan Mollick
@emollick
Had Claude Code build a snake game where the snake becomes aware it is in the game and then... stuff happens. Some impressive creative decisions by the AI (& also some very AI ones), I just gave a first prompt and some feedback on the game as it went. snake-awakening.netlify.app
jules
@julesrosenberg
5 things @rauchg does differently > counts every keystroke he types per day > built a tool that lets him retroactively screen-record bugs > gives feedback in v0 instead of writing it out > gets a full company brain dump from an agent every Monday > doesn't keep a to do list
Paul Graham
@paulg
The most important component of writing clearly is simply to have high standards for clarity. Then if you write something unclear, you notice, and ask: what did I mean to say? You can just keep doing this over and over. And if you have high standards for clarity, you will.
Guillermo Rauch
@rauchg
YES-CODE An entire category of software, "no-code", was built under the presumption that code is expensive, difficult, and scarce. Coding agents have forever changed the equation. Code is now cheap, easy, and abundant. I remember @cramforce being asked by an analyst long ago:
Warp @warpdotdev
Our warp[dot]dev site gets 10M visitors/year. We migrated the whole thing from a no-code editor back to code in just 3 weeks. Very few hiccups, and SEO actually improved. Plus, the marketing team is free to use Warp to ship future changes
AA
@measure_plan
i made fruit ninja but you've got a guitar instead of a sword
AA @measure_plan
i made snake but the only way to move is by playing guitar chords
0xSero
@0xSero
I had a conversation Mario about electrical engineering, Pi, and parenting. Very grateful to get another chance to chat with one of my favorite builders and people. Enjoy (:
Peter Steinberger 🦞
@steipete
Here’s the video of my talk at MS Build: Build the thing that builds the thing.
| | build.microsoft.com
by Marcus Moretti Figma/ TL;DR:Spiralv4 just shipped with four major updates: a style engine that generates writing indistinguishable from your own 87 percent of the time, agent-native access via MCP, CLI, and API, team workspaces for writing in a shared voice, and a $10 price drop, bringing personal plans to start at $15 a month. Spiral will continue to be free for paid Every subscribers along with access to all our tools and content.Try Spiral 4.0
Today we’re announcing a number of updates to Spiral, the writing partner for you and your agent. Spiral is built by writers for writers, to help you from idea to line edit, matching your writing style throughout.
The highlights:
With stylometry (or the study of writing styles), Spiral now sounds more like you. We’ve built a new Style Engine from the ground up, so Spiral computes your writing fingerprint and picks relevant samples for new drafts.
Use Spiral wherever you do work. With a new MCP, plus our existing CLI and API, Spiral can step in if you’re underwhelmed by your agent’s writing output, or need good writing in any workflow.
For teams, use Spiral to speak with one voice. Team workspaces let you share styles, prompts, knowledge, and now chats and drafts.
And finally, we’ve given Spiral a new coat of paint and logo, designed by Daniel Rodrigues. The primary brand font is now Edgar, from Frere-Jones Type.
Since re-launching at the end of last year, Spiral has:
Created 5,524 from 168,464 writing samples
Generated 113,165 drafts
Made 350,078 revisions
It also now averages a 4.9/5 conversation score on our internal LLM-as-judge eval. We built Spiral to help people who write for work write better. Just as Cursor is a coding harness, Spiral is a writing harness, supporting you at every stage of the writing process. Here’s how:
When it’s time to draft , Spiral uses stylometry to reproduce your voice, working in Every’s know-how where appropriate. For example, if you ask Spiral for tweets, it will incorporate best practices from X’s latest algorithm update.
When you need help polishing a draft, Spiral is your editor. Along with a built-in guardrails against AI-speak, you can set custom writing rules that Spiral applies in a “top edit,” the final expert-level edit on a piece—a term I learned working at Every.
We’ve written about the challenges of getting LLMs to write like you. It’s difficult to prompt an LLM to write like you, let alone get it to stop using common AI phrasing and punctuation. Spiral’s Style Engine is the best solution to this problem we’re aware of. An eval runs on every draft Spiral produces, challenging an LLM-as-judge to spot the generated draft among real samples in a blind lineup. Today we’re at 87 percent on this eval, meaning Spiral’s generated draft blends in with users’ samples almost nine times out of 10. When a draft is spotted, the judge explains why, creating a feedback loop to refine the Style Engine further. Try Spiral 4.0
Spiral goes agent-native
As Dan Shipper has pointed out , Claude and Codex are increasingly becoming the central interface for all computer work. So we’ve made Spiral available to agents via MCP, CLI, and API. To try it out, copy and paste this command in your agent:
Help me set up Spiral, my AI writing tool, so you can write in my voice. Read https://writewithspiral.com/agents.md and follow the steps. In short: add Spiral’s remote MCP server at https://api.writewithspiral.com/mcp/ (Streamable HTTP). The first connection opens a browser to sign in to Spiral and authorize access (OAuth, no API key to paste). Then help me write something.
The CLI, or command-line interface, is personally how I use Spiral the most. After I merge a pull request, a cleanup command runs in Claude Code, which calls Spiral to generate tweets about the new feature for the Spiral X account. Spiral markets itself. This technique is now bundled into the compound engineering plugin in the form of the ce-promote command. In addition to the main spiral write command, the CLI and MCP, or model context protocol, expose “personalize” and “humanize” functions. “Personalize” takes a given piece of text and rewrites it in your voice. “Humanize” does a pass to remove common AI tells, including the dreaded em-dash (which Every’s house style uses, hence its appearance in this piece). Over 500 agents have been connected to Spiral since we launched the integration last month. Those agents are revising blog posts, generating marketing copy, drafting email replies, and more—automatically, and in the user’s voice. On some days, API sessions outnumber web sessions. And as agent-native usage of Spiral picked up, we realized we needed to adjust our pricing model. As a result, we’re adopting a new token-based pricing model, which is more in line with AI apps like Claude, Codex, and Cursor.
From session limits to token limits
In May alone, Spiral generated billions of LLM tokens, or units of text. While drafts typically range from 500 to 1,000 words, a lot of tokens are processed under the hood to make those drafts great. I’m reminded of the line attributed to French mathematician Blaise Pascal : “If I had more time, I would have written a shorter letter.” It takes a lot of tokens to generate a few good ones. Before this release, Spiral limited the number of sessions, or unique chats, users could start per month. This approach had two problems. First, some users sent hundreds of messages within a single chat, consuming tens of millions of tokens, while using only 2 percent of their session allotment. Second, API users hit their session limit quickly, because the shape of API usage tends to be many single-turn sessions. We’re moving to a token-based model, which is in line with how billing works in AI products like Claude and Codex. The personal and team plans come with millions of tokens each month. Once those tokens are consumed, it’s pay-as-you-go for extra token usage. Customers can disable extra usage and set their spend cap. The good news is that the base prices of the personal and team plans are both dropping by $10. Personal plans now start at $15 per month (down from $25), and team plans start at $25 per user per month (down from $35).
Tell your stories, express your ideas
Technology is at its best when it augments our skill sets—amplifying what we’re good at, assisting with what we’re not. Figma and Canva help designers do better work, and allow people without a design background to manifest what they imagine. Claude Code and Codex help engineers ship more software, and allow people without engineering backgrounds to create the software they always wanted to exist. Our hope is that Spiral helps writers sharpen their work, and allows people without a strong writing background to put their stories and ideas into words. One Spiral user is a retired musician in Australia. He’s accumulated a lifetime of stories in the studio and on tour. He’d never written them down, because he didn’t quite know how to tell them. Since signing up for Spiral, he’s recorded many chapters of his life stories with the tool’s help. He told me that Spiral has taught him how to be a better storyteller. That’s what we’re building toward: a writing partner that helps people say what they mean and get better at saying it. Spiral produces good writing fast, but it also explains its writing and editing decisions along the way: the rationale behind rhythm, structure, rhetoric, and more. As my colleague Natalia Quintero observed, the best AI tools teach you things as you use them. If any of this sounds useful, try Spiral. Share your feedback on X (@tryspiral) or get in touch: hi@writewithspiral.com. Try Spiral 4.0Marcus Morettiis the general manager of Spiral (@tryspiral).To read more essays like this, subscribe to Every , and follow us on X at @every and on LinkedIn.Subscribe
We’re launching a new section highlighting founders actively raising capital.
Every week I get flooded with messages from founders we’ve featured (and plenty we haven’t yet) wanting to share what they’re building, how fast they’re growing, what their revenue looks like, and who they’re raising from.
Some of those conversations are too interesting to keep behind closed doors.
With the founders’ permission, we’ll occasionally feature startups that are actively fundraising and give you a peek behind the curtain on what they’re building, where they’re at, and what their raise looks like.
Think of it as a curated founder-to-investor matchmaking section built right into the newsletter.
Back to the good stuff…
In this issue of the Stealth Startup Spy, here is what we will uncover:
MIT and Stanford-trained founder behind Topolabs (acquired by Autodesk) is building semi-autonomous robotics for wildfire prevention and precision forestry
Serial founder with exits to Intel, Check Point, and Proofpoint enters stealth
Former SVP at Cambr (acq. by NYSE: NBHC) is building an AI-powered treasury platform for local governments
Former Chief Strategy Officer at Boston Dynamics launches stealth robotics startup
Harvard MBA and former Uber Freight GM is building an AI-native logistics OS for healthcare and life sciences
And more…
Now let’s shine the spotlight… 💡💡💡
💰 Featured Founders Fundraising
Discover startups currently fundraising before they hit everyone’s radar.
FounderDNA: Serial Founder, Technical Founder, Doctorate Degree, Masters Degree, Top 10 University
Prior Experience: Ex-Senior Program Manager for Power and Propulsion, Lunar Fission Surface Power and Directed Energy at Lockheed Martin; ongoing Industry Advisor to NASA, US Space Force, and the Pentagon
Vaxon Space is developing satellite constellations in VLEO focused on missile defense and space-based interceptors, as well as AI / data center connectivity and enhanced ISR. They are a dual-use company supporting the DoW and commercial customers.
HQ: San Jose, California, United States
Traction and Highlights: Vaxon is currently fundraising through June 15 and filed provisional patents on inlet and pumping system with testing beginning in June. Anticipating approximately $3M in government funding wins across DARPA, NASA, AFRL and USSF.
🕵️♂️ Founders Coming Out of Stealth
Real-time updates from founders who debut what they’ve been working on under stealth mode
Prior Experience: Multiple GM Roles at Uber Freight, MBA at Harvard Business School, MS at UC Berkeley, Investment Manager at Verdane, Investment Manager at Equinor Ventures, Advisor at Porterbuddy
ZoomLogi is an AI-native logistics operating system for healthcare and life sciences, unifying carrier data, sensor feeds, and workflows to predict and automate exception resolution for pharmaceutical manufacturers, specialty pharmacies, biotechs, and logistics service providers.
FounderDNA: Serial Founder, Masters Degree, Former FAANG, Top 10 University
Prior Experience: Master of Public Policy at University of Chicago, Management Intern at Amazon, Associate Consultant at Altair Advisers, Commercial Strategy Intern at Gilead Sciences, Congressional Intern at U.S. House of Representatives
Nolro is an AI control plane that enables enterprises to approve releases, enforce guardrails, monitor drift and hallucinations, and generate audit-ready compliance evidence across AI systems, vendors, and workflows.
HQ: Chicago, Illinois, United States
Industry: AI Governance, Enterprise SaaS, Compliance Tech
FounderDNA: Serial Founder, Technical Founder, Masters Degree, Former FAANG, Top 10 University
Prior Experience: Engineering Manager at Amazon, Tech Fellow at Microsoft, Stanford University GSB (Corporate Innovation & Leadership), Founder & CEO at Voicy.AI, Founder at Jobsmunk
FounderDNA: Serial Founder, Technical Founder, Masters Degree, Prior Exit, Top 10 University
Prior Experience: Founder & CEO at Topolabs (Acquired by Autodesk), VP Software & AI at Stratasys, Founder & CTO at Riven, Senior Engineer at Intuitive Surgical, MIT and Stanford alum
Prior Experience: SVP, Head of Product and Operations at Cambr (acquired by National Bank Holdings Corporation, NYSE: NBHC), Strategic Initiatives and Operations at Keystone Industries
Edwin is the AI-powered treasury and financial platform built for local governments.
HQ: Santa Monica, California, United States
Industry: GovTech, FinTech, B2B SaaS | Team Size: 2
Time Spent in Stealth Mode: 6 months
🕵️♂️ Key Talent Going Under Stealth
Illuminating clues left behind by world class talent and influential innovators who just went into stealth mode
Neeraj Pradhan - Founder at Stealth Startup
FounderDNA: Technical Founder, Masters Degree, Former FAANG, Top 10 University
Prior Experience: Founding AI Engineer at LlamaIndex, Machine Learning Engineer at Meta, Research Engineer at Uber, Tech Lead AI Applications at Tribe AI
Etay Bogner - Co-Founder & CEO at Stealth AI Startup
FounderDNA: Serial Founder, Technical Founder, Former FAANG, Prior Exit
Prior Experience: VP and GM Zero Trust Products at Proofpoint, Founder & CEO at Meta Networks (Acquired by Proofpoint), Founder & CTO at Neocleus (Acquired by Intel), Founder & CTO at Stratoscale, Founder & Managing Director at SofaWare (Acquired by Check Point)
Prior Experience: Chief Product Officer at Self Financial, SVP/GM Commerce at HubSpot, Vice President of Product at Coinbase, Head of Product at Venmo, Staff Software Engineer at PayPal
Marc Theermann - Founder & CEO at Stealth Robotics Company
FounderDNA: Masters Degree, Former FAANG
Prior Experience: Chief Strategy Officer at Boston Dynamics, Director Global Partnerships at Google, EVP Strategy at Millennial Media (Sold to AOL/Verizon), Head of Mobile Platform Sales at Google
Prior Experience: Head of Engineering - CDP at Coinbase, Head of Engineering at ZORA, Staff ML Platform Engineer at Clubhouse, Member of Technical Steering Committee at x402 Foundation
🚨Here’s the deal 🚨This email has gotten too big. Exciting, but with more people following it, the edge diminishes. I’ve thought long and hard about what to do to preserve the value in the signals. I’m not sure about the final direction yet, but in the meantime I’ve been sending an email 48 hours earlier to a select group of paid subscribers. The feedback has been pretty positive so I’m going to open up the list for another 100 spots. To get signals early, Apply here!
Stay Stealthy,
Drake
Thank you for reading. If you liked it, share it with your friends, colleagues and everyone interested in staying ahead of the hidden developments in tech. Subscribe below and follow us onX / Twitter to never miss a company operating under stealth again.
Stealth Startup Spy is a data-driven newsletter for investors, journalists and tech enthusiasts interested in uncovering the next big move for key talent, real-time stealth company launches and technology advancements not in plain sight. We leverage the technology built at Gravity to shine a light on the hidden world of stealth startups.
Spiral 4.0 introduces a new style engine, why enterprise roadmaps are hard, and a workflow for making your coding agent more efficient
by Laura Entis ### Launch
Spiral 4.0
Today we’re launchingSpiral4.0, which writes drafts in your voice from idea to line edit. Spiral has a new MCP alongside the existing CLI and API, so any agent or workflow can write in your voice too. For teams, we’ve expanded workspaces, which let you share styles, prompts, knowledge—and now chats and drafts. Finally, Spiral has a new pricing model: We’ve switched from session limits to token limits, so costs match your actual usage rather than how many times you opened a new chat. A vast majority of users will end up paying less: Personal plans now start at $15 a month—down from $25—and team plans are $25 per user, down from $35. Try Spiral 4.0
Signal
Enterprise AI product roadmaps are hard
Microsoft is moving fast. Three months after OpenClaw came out in November 2025, Microsoft CEO Satya Nadella described it as a “virus”-like security risk. By May, the company’s “Project Lobster” was internally testing “ClawPilot,” an OpenClaw-based desktop environment. This week at the Microsoft Build conference, the company released Scout , a personal agent for work built on OpenClaw. For a company employing 100,000 engineers, this is blindingly fast. Unfortunately, it may already be too late. The Google Trends graph for the term “openclaw” shows search interest spiked in January and began its descent soon after. (Screenshot courtesy of Mike Taylor.) OpenClaw search traffic spiked in early January, after everyone had a chance to experiment with Opus 4.5 over the holidays. The sharp rise in interest died down almost as quickly as it took off, helped along in early April by Anthropic ending support for subsidized Max plan usage —thereby forcing everyone to scramble to get OpenClaw working on cheaper models. This doesn’t mean OpenClaw is dead; the open-source project saw a recent uptick in download and is still under active development, with millions of dollars of patronage from OpenAI, which hired its creator Peter Steinberger. AI agents as a category aren’t dead, either, as traffic has moved to other agents like Hermes, Google has just rolled out Gemini Spark (first announced last month at its I/O developer conference), and Claude and Codex have both adopted more agentic features inspired by OpenClaw. That said, it must be tough to manage enterprise AI product roadmaps these days. You do everything right, watch the latest trends, pivot your focus to supporting new tools and making them secure in enterprise environments. You move mountains to explain to stakeholders why this is a good idea. You plan the keynote of your big conference, which has to be scheduled months in advance. Then a month after the internal beta (just three months since the tool went viral), you’re already behind the news cycle. Everyone has moved onto the next shiny thing. You go back to the drawing board and think “maybe next time, we’ll just announce it on X.”— Mike Taylor
Log on
Get hands-on with how Every uses AI. These are the live camps, workshops, and meetups where team members teach the workflows behind our work.
These days, Monologue ’s general manager Naveen Naidu spends most of his time in the Codex app with Fin—formerly Intercom, a customer support platform—open in the coding agent’s in-app browser. Working from a repository-local project, he has Codex investigate the customer issue displayed in the browser, create a bug report in Linear, link the Intercom ticket to the Linear issue, and draft a reply to the customer with information about the bug report—all without having to leave the app. Fin has an MCP with 13 common actions, like searching conversations or reading and writing messages. Naveen’s workflow required a more specific one: Turn the active Fin conversation into a markdown file the coding agent could read. Here’s Naveen’s workflow for creating a more focused setup ...
You're among 70,562 others who received this email because you wanted a weekly recap of the best articles from Hacker News. Published by Curpress from Bellingham, Washington. Hacker Newsletter is not affiliated with Y Combinator in any way.
✨ Want to promote your startup? Buy a classified ad or click reply to get our media kit
Every week I’ll provide updates on the latest trends in cloud software companies. Follow along to stay up to date!
Where Are the American Open Source Models?
I’ve been investing in open source companies for nearly my entire venture career. I love open source businesses, think they’re generally great for the ecosystem, and can also create a lot of commercial value (but this can be tricky!). We’ve seen all kinds of open source businesses become successful. Databases (Mongo, Clickhouse, etc), Data Infrastructure (Databricks, Confluent, etc) Developer Tools (Hashicorp, GitLab, etc). And many other categories.
The nuance lies in how you define “open source.” A lot of this comes down to what open source license the open source project uses. At a high level there are three main buckets. First is fully open and permissive (MIT, Apache 2.0, etc) - anyone can use, modify, and distribute the code however they want, including building commercial products on top of it. Second is copyleft (GPL, AGPL, etc) - the code is open, but if you distribute or build on it you have to open source your own code too. Third is source available - the code is publicly readable but the license restricts commercial use in specific ways. This is where you see licenses like BSL (Business Source License) and SSPL, which HashiCorp and MongoDB have both moved to at various points. The main point with source available licenses is that you can basically do everything you can in the permissive license, EXCEPT monetize the project (ie you can’t take the project, create a hosted version, and charge for it). The most “pure” opensource proponents would say only the fully open and permissive licenses are truly open source. Everything else isn’t.
The playbook for many of these open source companies was / is to first become the default in their field. This is the most important part. Spark became the default in large scale data processing, Kafka became the default for real time data streaming and event ingestion. Mongo for NoSQL databases. Etc. Each project became widespread and ubiquitous in their respective fields - they were / are the dominant platform for their use case. Then the commercial companies behind each project figured out a way to monetize. Sometimes this was through hosted offerings - where the company runs and manages the infrastructure for you, so you don't have to worry about deploying, scaling, or maintaining the project yourself. In other instances, proprietary features were gated behind a commercial offering (this could be better performance, enterprise features, etc). Regardless - there were playbooks to monetize, and successful examples of large commercial businesses being built on top of successful open source projects.
But this wasn’t always the case. For a long time, the conventional wisdom was that open source couldn't be a real business. If the code is free, who pays? And the skepticism was genuine - why would a company pay for something their engineers could just download and run themselves? Early open source companies had to fight hard to prove out the commercial model, and a lot of investors passed on these deals early on because the business logic wasn't obvious. "You're just going to give it away?" was a common reaction. And honestly, it wasn't an unreasonable question. The answer turned out to be: yes, you give the software away, but you charge for everything around it - the hosted version, the enterprise features, the support, the integrations, the proprietary engines, etc. The open source project served as distribution, and the commercial product was something different. And after a wave of successful open source commercial companies emerged, it became more obvious what was possible (commercially). And this was very important - investors (like myself) deeply believed successful open source projects could translate into successful commercial companies. This created an important incentive set - an incentive to fund open source companies (by investors) and an incentive to start commercial open source companies (by founders)
Let me tie this back to what’s happening in open source AI. Today, one of the biggest disappointments is a lack of a strong open source model ecosystem in the US. All of the frontier models are closed source from OpenAI / Anthropic. The leading opensource models are largely all from Chinese labs. DeepSeek being the most prominent example, but also Qwen from Alibaba, Kimi from Moonshot AI, and others. Jensen Huang has talked about the importance of having American leaders at every layer of the AI stack - chips, models, infrastructure, applications. The US is clearly winning at chips (Nvidia) and has strong incumbents at the application layer, but the open source model layer is a real gap. And that gap matters more than people realize. Open source models are how developers learn, experiment, and build. They're the distribution layer for the next generation of AI applications. If the dominant open source models all come from Chinese labs, that has long term implications for where developer communities form, where tooling gets built around, and ultimately where commercial ecosystems emerge.
So why hasn’t there been more of a blossoming US open source model ecosystem? I think it all comes down to incentives. In the same way investors had questions about the business model of open source companies many years ago, they have similar questions about open source labs today. How do you monetize an open source model? You could “host” it - which in the case of models probably means just means hosting an API endpoint people can hit for inference. BUT if you’re doing this you’re also going to directly compete with anyone else who can pick up your model and host an inference API endpoint. Baseten, Fireworks, all the hyperscalers, etc. And the question then becomes do you as the model provider have an advantage that lets you serve the model either more performantly or more cost efficiently (or both). The answer to this question is probably “maybe” but not by enough to win with high margins over the long term.
The next natural place is to move into fine tuning / eval feedback loops. Most folks want to take an open source model, fine tune it for their use case / data, observe / monitor / improve that fine tuned model (and rinse repeat). So the open source labs could move into this line of business and charge for it. But then you’re still going to compete with the Fireworks / Baseten / Hyperscalers who will also offer this.
And then there’s the question of what does it mean to be an open source model? There's actually a spectrum here too - and it maps pretty cleanly to the licensing buckets we talked about earlier. The MIT / Apache equivalent in the model world would be truly open weights with no commercial restrictions - download it, fine tune it, host it, build a business on top of it, do whatever you want. The source available equivalent would be something like Llama - the weights are publicly available, you can experiment and build, but there are commercial restrictions that kick in at scale. And then true open source, the equivalent of releasing the full source code, would mean releasing not just the weights but the training data, the training pipeline, the evals, everything needed to fully replicate the model from scratch. Almost nobody does this. So when we talk about the "open source model ecosystem," we're really mostly talking about open weights models with varying degrees of commercial restrictions. Which matters, because just like source available licenses changed the commercial dynamics for traditional open source companies, the restrictions baked into open weights models will shape where developer communities form, what gets built on top, and ultimately who captures the commercial value.
I think the main issue facing the open source model ecosystem is incentives. People (ie investors) just question the business feasibility of open source models. Today, most of the world is using the frontier models for everything. However, what’s really come into focus very recently is costs. It just doesn’t make sense to drive the Ferrari down the street to the grocery store…The Honda will do just fine. You don’t need the “expensive frontier” tokens for everything. I don’t know where the line is, but let’s just use the 80/20 rule. Maybe 80% of the use cases are generic enough where a smaller, less powerful, but cost efficient model will do fine. And for the 20% of use cases you’ll really need the frontier expensive tokens (i’m completely making up that ratio, just using an illustrative example).
So then - why shouldn’t open source dominate in this next part of the cycle?? When people shift usage from most expensive tokens to cheaper tokens, wouldn’t open source be the natural alternative? Yes - but where are the models?? The other baked in assumption is that the open source models are “good enough.” Said another way, the gap between the frontier closed source models and open source isn’t “that wide” (and more importantly is the trajectory widening or closing).
Right now all the best open source models Chinese models… And these models have largely gotten to where they are by distilling frontier models. But I think that was much easier when the models were more “basic” language models. Question / answer type models where you could create a data set by asking millions of questions to the frontier model. I think distillation is getting harder. The most frontier models are now being released with increasingly complex harnesses. Or the frontier models aren’t released via an API but are wrapped into a product themselves. Or look at Claude - playing around with Claude Code, I have workstreams that kick off 50+ agents in parallel. How do you distill a model with 50+ agents running in parallel?? It’s much harder than just getting millions of question / answer pairs… All of this makes it much harder to “distill.” And the cost to pre-train (without the head start of distilling) is massive. All of this to say - I find myself today probably on the side of the gap widening (not shrinking) between the frontier and open source.
So who will be able to raise massive amounts of money, to pre-train a model, that will maybe work, but might end up being too far behind the frontier to capture real market share, and even if you get all of the previous points right there are still questions about how you monetize over the long haul (the first half of this post). Everyone WANTS this to happen. But who’s going to fund it?
I go back to the early days of Neoclouds where Nvidia really propped up the ecosystem. They funded (massively) a class of companies who might not have gotten the funding elsewhere. Do we need King Jensen again?? Or do we need governments to fund it out of a sovereign interest?? I’m not sure. But I really hope the US open source ecosystem finds a way to thrive.
What I think it will take - some sort of research breakthrough that open source gets first. Let’s hope that happens! I also think (no inside info) at some point the closed source frontier labs will get back into open sourcing models. Maybe not the frontier. Maybe a slimmed down version of the frontier, or maybe open sourcing one model release prior to the frontier. This is the most hopeful scenario!
Quarterly Reports Summary
Top 10 EV / NTM Revenue Multiples
Top 10 Weekly Share Price Movement
Update on Multiples
SaaS businesses are generally valued on a multiple of their revenue - in most cases the projected revenue for the next 12 months. Revenue multiples are a shorthand valuation framework. Given most software companies are not profitable, or not generating meaningful FCF, it’s the only metric to compare the entire industry against. Even a DCF is riddled with long term assumptions. The promise of SaaS is that growth in the early years leads to profits in the mature years. Multiples shown below are calculated by taking the Enterprise Value (market cap + debt - cash) / NTM revenue.
Overall Stats:
Overall Median: 3.4x
Top 5 Median: 28.9x
10Y: 4.5%
Bucketed by Growth. In the buckets below I consider high growth >22% projected NTM growth, mid growth 15%-22% and low growth <15%. I had to adjusted the cut off for “high growth.” If 22% feels a bit arbitrary, it’s because it is…I just picked a cutoff where there were ~10 companies that fit into the high growth bucket so the sample size was more statistically significant
High Growth Median: 16.3x
Mid Growth Median: 5.0x
Low Growth Median: 3.0x
EV / NTM Rev / NTM Growth
The below chart shows the EV / NTM revenue multiple divided by NTM consensus growth expectations. So a company trading at 20x NTM revenue that is projected to grow 100% would be trading at 0.2x. The goal of this graph is to show how relatively cheap / expensive each stock is relative to its growth expectations.
EV / NTM FCF
The line chart shows the median of all companies with a FCF multiple >0x and <100x. I created this subset to show companies where FCF is a relevant valuation metric.
Companies with negative NTM FCF are not listed on the chart
Scatter Plot of EV / NTM Rev Multiple vs NTM Rev Growth
How correlated is growth to valuation multiple?
Operating Metrics
Median NTM growth rate: 12%
Median LTM growth rate: 16%
Median Gross Margin: 76%
Median Operating Margin 2%
Median FCF Margin: 21%
Median Net Retention: 110%
Median CAC Payback: 42 months
Median S&M % Revenue: 34%
Median R&D % Revenue: 23%
Median G&A % Revenue: 13%
Comps Output
Rule of 40 shows rev growth + FCF margin (both LTM and NTM for growth + margins). FCF calculated as Cash Flow from Operations - Capital Expenditures
GM Adjusted Payback is calculated as: (Previous Q S&M) / (Net New ARR in Q x Gross Margin) x 12. It shows the number of months it takes for a SaaS business to pay back its fully burdened CAC on a gross profit basis. Most public companies don’t report net new ARR, so I’m taking an implied ARR metric (quarterly subscription revenue x 4). Net new ARR is simply the ARR of the current quarter, minus the ARR of the previous quarter. Companies that do not disclose subscription rev have been left out of the analysis and are listed as NA.
Sources used in this post include Bloomberg, Pitchbook and company filings
The information presented in this newsletter is the opinion of the author and does not necessarily reflect the view of any other person or entity, including Altimeter Capital Management, LP (”Altimeter”). The information provided is believed to be from reliable sources but no liability is accepted for any inaccuracies. This is for information purposes and should not be construed as an investment recommendation. Past performance is no guarantee of future performance. Altimeter is an investment adviser registered with the U.S. Securities and Exchange Commission. Registration does not imply a certain level of skill or training. Altimeter and its clients trade in public securities and have made and/or may make investments in or investment decisions relating to the companies referenced herein. The views expressed herein are those of the author and not of Altimeter or its clients, which reserve the right to make investment decisions or engage in trading activity that would be (or could be construed as) consistent and/or inconsistent with the views expressed herein.
This post and the information presented are intended for informational purposes only. The views expressed herein are the author’s alone and do not constitute an offer to sell, or a recommendation to purchase, or a solicitation of an offer to buy, any security, nor a recommendation for any investment product or service. While certain information contained herein has been obtained from sources believed to be reliable, neither the author nor any of his employers or their affiliates have independently verified this information, and its accuracy and completeness cannot be guaranteed. Accordingly, no representation or warranty, express or implied, is made as to, and no reliance should be placed on, the fairness, accuracy, timeliness or completeness of this information. The author and all employers and their affiliated persons assume no liability for this information and no obligation to update the information or analysis contained herein in the future.
by Mike Taylor As I rode in my Uber to Microsoft’s annual Build conference on Monday, I fondly recalled a time when you could get anywhere in San Francisco for $5. Those days are long gone. Venture capitalists lost their appetite to supply unlimited funding in a viciously competitive market, and Uber needed to show a path to profitability ahead of its 2019 IPO. There are signs that the “$5 Uber era” of LLMs is over now, too. AI labs are subsidizing subscriptions to the tune of thousands of dollars , which can’t continue forever. This year Anthropic, OpenAI, and SpaceXAI are all going public—and like Uber seven years ago, they’ll need to take a hard look at their books. On June 1, the eve of the event, Microsoft sparked outrage by switching to token-based billing on GitHub Copilot. Some users said their bills jumped from $39 to over $3,000per month. Rather than backtracking on billing, Microsoft used the conference stage in California to make the case for using AI more pragmatically in the face of rising costs. I came away from the event thinking that Microsoft is the first company to get real about a world where intelligence is available on tap, but constrained by how many coins you can put in the meter. Here is what the company’s vision looks like in practice, and what it might tell us about how we’ll be paying for and pricing AI in the future.
Intelligence on and off the meter: A product approach
In his opening speech, CEO Satya Nadella addressed pricing concerns head-on. He promised “unmetered intelligence to every desk and every home,” an AI-era update to Bill Gates’s vision of “a computer on every desk.” Microsoft CEO Satya Nadella promised “unmetered intelligence to every desk and every home.” (All images courtesy of Mike Taylor.) The most tangible way to experience that vision is with the RTX Spark, a new laptop Microsoft designed for AI workloads with Nvidia. The device is able to run a medium-sized 128-billion-parameter model locally (frontier models are in the trillions of parameters) so developers can get a lot of work done without paying a penny for tokens. Microsoft is taking advantage of the fact that the leading open-source models like Kimi-K2.6 , which have a trillion parameters, are too big to fit on most laptops, and is betting that budget-conscious coders might not mind being a year or two behind the frontier and use a smaller model. The device will be released in the fall. The RTX Spark laptop follows earlier feature announcements that show that Microsoft wants to decrease switching costs for customers by being the place where you can use any model, agent, or harness. The laptop has a rebuilt smart terminal app that allows you to run any coding agent harness and has adopted popular terminal commands from the Mac ecosystem to make the shift easier for developers. Even the GitHub Copilot Desktop app, also released at the conference, makes it easy to switch providers between OpenAI-built, Anthropic-built, and local open-source models running on your device. When questioned about the affordability of agentic coding, Mario Rodriguez , GitHub’s chief product officer, cited the automatic model routing feature in GitHub Copilot, which can delegate less complicated tasks to cheaper models. In my interview with Kyle Daigle , GitHub’s chief operating officer, he lamented that developers tend to choose “the model of the day, or week, or hour,” even when the task doesn’t merit that kind of power. A person probably will not manually switch to a cheaper model for that final step, “but the tools could.” I’ve also long argued that not every task needs to be done by a frontier model. I get the feeling the team built this model router feature for themselves after facing the same problem everyone else is right now—Microsoft itself has been cancelling Claude Code licenses to reduce costs. Features like automatic model routing show that Microsoft understands how runaway costs hurt enterprises that need tighter control over spending. The AI labs won’t let large companies buy highly subsidized individual “Max” plans, so big companies end up paying full freight on every token they burn. One that wasn’t properly monitoring usage is rumored to have spent an eye-watering half a billion dollars on Claude tokens in a single month. That wasn’t the only news that day: Microsoft’s research lab, led by Mustafa Suleyman , released a set of new (cheaper) smaller models spanning image, voice, transcription, coding, and reasoning.
Tackling costs through model optimization
But when you don’t use the latest models to save cost, there’s a higher risk of making a costly mistake. One answer was a phrase I heard over 100 times at the one-day event:...
Wednesday night was absolutely insane. The city is fully abuzz and if the Knicks can pull it off, it will be the end of NYC as we know it. Midtown will be a bombed out DMZ. The sports bars of the city will be torn down to the studs.
This is already hilariously inaccurate/out of date
Anthropic filed its S-1 and SpaceX goes public next week. There’s ≈$3T of market cap hitting in the next few months with at least another trilly on the way (OAI). Google is selling ≈$80B of stock and it’s not much of a story.
We are witnessing the total takeover of already-concentrated financial markets by a single story; debt and equities, publics and privates, newcos and established names. This will be an entirely unprecedented wealth creation and liquidity event. The exuberance about AI the technology will have become fully overtaken by exuberance for AI the financial asset.
These are (or at least have been) obviously great stocks. Whether or not they are great businesses is the most important question in financial markets right now. I don’t know ¯\(ツ)/¯
The Bull Case
World-changing technology. They wind up with infinite surface area, and everyone pays them for everything. Intelligence is the most valuable market in the world (all white-collar work), and they become both an accelerant for and a tax on global GDP. LLMs either scale in other domains like they’ve done in coding or it turns out that every task is actually a coding problem after all.
And this is not even accounting for true AGI and a post-scarcity / post-economic world.
The Bear Case
Foundation models are obviously incredible products but turn out to be terrible businesses.
Training is a cost of revenue, not an R&D cost. Stop training, everyone churns and moves on to your competitor or a distilled Chinese version. The labs are perpetually investing unbelievable sums in building these fast depreciating assets (capex and training).
Every model is bigger and more expensive to run and also gets distilled faster, so these companies have to make huge up-front investments that depreciate extremely quickly. They are competing against one another and with the in-house labs of huge companies that throw off huge amounts of cash. They’ll never be able to charge enough because of competitive pressures. They will just run out of money and places to get it.
So what?
On balance, it’s almost certainly better to bet on the bull case, despite how much upside has already been sucked out / pulled forward at these prices.
If there’s any chance the bull case is right, and you have either a low enough cost of capital or are otherwise short the proliferation of intelligence, you have to get exposure here.
And everyone who works a white-collar job is structurally short. We’re all waking up every day betting that our intellect and creativity will matter. Nobody has any incentive not to be in it and every incentive to be in it, even beyond all the high-falutin’ stuff. (You just look like a fucking moron if you don’t have exposure).
So we’re definitely in a reflexive bubble. If you don’t believe the incentives/structural reflexivity just look around and use your eyes.
Is the bubble worth it?
Will it be like railroads, which ultimately produced a ton of valuable infrastructure (I think yes), or like crypto, which was basically a wash (though of course we wouldn’t have AI without all the chips and stuff from crypto)?
Here, the intelligence and capabilities are a positive externality of the bubble/exuberant build outs. And the more distillation happens, the more the frontier gets eaten by open source, the better for everyone. It does seem clear by now that as long as we don’t completely bungle the second order impacts (on society, politics, and business) that we are all going to be the winners. I mean this literally. Productivity is the main thing. It doesn’t require AGI or LLM maximalism to believe that we’ll look back on today’s standard of living (our material conditions) and standard of toil (the drudgery we accept for ourselves) the same way we look back at a pre-electrified world: impoverished, immiserated, and base.
It’s possible the labs ultimately get wiped out. I don’t know, and I’m not betting on it. But the bubble being worth it for the world and worth it for the equity are not the same question. If information wants to be free, then intelligence wants to be unbounded.
None of this answers the actual question of the business vs the stock and the bonanza won’t settle it one way or the other. A business is not a stock, and a stock is not a product; the price won’t be a verdict.
My name is Yoni Rechtman. I’m a partner at Slow Ventures, where I lead pre/seed rounds from a ≈$325M fund. I’m a generalist investor looking for weird takes on important stories: N-of-1 companies taking non-obvious approaches to markets that matter. I’m interested in real world businesses, hybrid software companies, AI’s second-order effects, healthcare, network effects, and fintech. If you’re building something ambitious or think I’m wrong, I’d love to hear about it.
SWL Week in Review - The Whole Point is You Can't Measure the ROI on AI
sam lessin · Friday, June 5 2026 · 7 min read · ↑ top
MORE OR LESS POD
The SpaceX IPO cultural moment — Storing value in ‘stories’ vs. the broad-based ‘financial’ story, the real capabilities of agents, why startups should be making movie trailers not pitch decks - lack of ROI is the defensibility… "The narrative gap in AI is crazy.”, “Is Taylor Swift jumping the shark?"
HOT TAKES
Narrative Asset Swap (BTC vs. SpaceX) Tsunami — It is funny to watch my narrative asset portfolio get smoked ahead of SpaceX IPO (the mega-narrative asset) — Warren Buffet didn’t anticipate that the ‘tide would go out’ not because of the moon, but because of a big Tsunami coming. I should have thought of this, after all — the thing I have said forever is that the only thing that replaces a story, is a BETTER story (this already happened with crypto to AI… for power, etc) — but now it comes for retail.
The Numerology Consumer Use Case for AI — speaking of …. A good guy / friend texted me some numbers a dream told him I should pay attention to — I didn’t know what to do with the numbers, but my claude with access to a huge amount of personal data sorted out what the numbers meant… finally an AI case for consumers.
Let’s hope for Anthropic and OpenAI’s sake no one ever figures out the ROI on AI - The Illegibility is The Moat — This feels important to tease out… Anthropic costs 10x the open source models to use, and for all the stuff no one knows how to think about ROI … which is exactly why they can change what they want. If the market was efficient (and if it ever really is) then the margin on multiplying big numbers will snap to zero… so actually the fact that people can’t really benchmark a given task / know where to spend based on curve / probability of ‘right’ outcome AND value of incremental rightness is precisely what keeps AI possibly profitable. Let’s hope for Anthropic and OpenAI’s sake no one ever figures out the ROI on AI — Fll’er up
The SpaceX IPO is a Cultural Moment More Than Financial — Do we store value in stories of cults, or do we store value in the ultimate broad global consensus narrative of DCFs… SpaceX is going to be the test — the broad rotation to cults, etc. regionalization and fictionalization / de-globalization says SpaceX valuation will be just fine (and that the global DCF narrative is nothing more than lowest common denominator reserve price on an asset) — but we will see.
GDP really is a broken metric … Fun looking back on my 2014 column about why it is, and why we should be focusing on increasing power-use-per-person — everyone deserves their own blackhole’s worth of energy… energymaxxing.
Slow Instagram got Hacked… Hilariously … I know this was stressful for many… but I really get a lot of joy out of Slow’s Instagram account being taken over by a random person in a turban. I am only annoyed he made our account private AND hasn’t posted anything interesting in days! Hacking is a great way to get a job as a social media manager… but once you have the job to keep it you gotta perform!
The Multiplying Big Numbers Market … talked about this market for matrix multiplication on CNBC which was fun… the full CNBC segment — you really have to think about the question of if calculating is a defensible market, or is multiplication the ultimate globally liquid commodity (depends on if you trust the calculating AND/OR the cost of validation!)
Putting AI In Orbit& Planetary Defense — no one seems to be registering that for a sci-fi driven narrative… putting your brain on the OUTSIDE of your scull doesn’t seem to be too smart… that was clearly an evolutionary dead end… better to make the Boring Company a defense co and burry compute underground — It does seem like in this moment the key is to just do things that move a lot of atoms some way, and the use case will emerge!
5000 Years Ago, Only 5% of Men Were Reproducing — we do have a problem with the birth rate…. And it does seem like women will mostly keep reproducing but a lot of men will not — interesting that this happened before pretty clearly… Monogamy is great for society / ultimate way to keep lots of roving angry men from doing angry disconnected men things… BUT maybe not our truly base state… we do need new religion if we like the 1990s version of the world.
Who To Blame For The Enhanced Games — clearly this totally sucked / didn’t deliver. Is it an FDA fail (not allowed to use really good drugs) OR a games fail (didn’t have the balls to really full send it) — anyway, real credit to the Olympics.
AI’s threat to the US consumer economy — I keep coming back to this — you don’t need to erase all the jobs — you just need to kick out / fire the upper-middle-manager class (maybe like 5-10% of Americans) who have disposable income BUT aren’t so rich that they just save / spend very low percent of income/ wealth — that or figure out ways to get very rich people to buy more stuff / have MORE demands authentically (oh, trying that… with foundation farms, also my $100K docs — any category where you would pay more than you CAN pay for more quality).
Founder’s Fund Understood The Marketing Assignment — a bunch of pretty famous people in startup world doing highly produced game of Mafia… yes, this is exactly the type of post-modern VC advertising that is pitch perfect. Again, high quality video is winning.
Red Pearled — thanks Stanford seniors… this was a fun gift to get, and I really get a kick out of having people that appreciate special moments around.
Why Aren’t Non-Profits Racking H100s — the Vatican is talking about AI, but what they should be doing is building tax-free data centers… They have the real estate footprint, the tax structure advantage, etc… Monks used to sit around copying books — now we can have monk machines multiplying numbers… same thing new era. Plus who doesn’t want their calculations blessed?
Best,
Sam
P.S. Alex Karp can dead hang for 5:30 —… really? I want to see it because that is wildly impressive if actually not a psyop … although someone has pointed out to me that apparently he only weighs 90 pounds (which makes it slightly more plausible)
P.P.S. Dan Sullivan vs. Dan Sullivan — also awesomely entertaining… between this, karp, founders fund, etc. I think we are finally entering a post trump media era of power. Between stuff like this and Spencer Pratt MAGA better up their marketing strategy because they are for the first time looking a bit outclassed / loosing the iron grip on best social media strategy.
P.P.P.S Here is a crazy one — the guy who created Heated Rivalry was a director on Shorsey and Letterkenny (in order the two best TV shows ever mades, and the greatest expression of modern male psyche)
Scott Galloway · Friday, June 5 2026 · 9 min read · ↑ top
Metrics Maxxing
“What gets measured gets managed” is often misattributed to Peter Drucker, the father of modern management theory. The full quote, from business journalist Simon Caulkin, is a warning not a promise. “What gets measured gets managed — even when it’s pointless to measure and manage it, and even if it harms the purpose of the organization to do so.” In other words, our mania for measurement can obscure what matters most. Consider the popularity of health and fitness apps and the personal-optimization trend those technologies enable. To a point, the more data we collect on ourselves, the better able we may be to improve our lives. But metrics aren’t the arbiters of living well, nor is optimization life’s end-goal. I believe this trend isn’t about optimizing life; it reflects a growing obsession that’s consuming life’s purpose and meaning.
Perfection Maxxing
The digital economy has created a winner-take-most ecosystem. Life/America is exceptional for the top tier, and increasingly anxious for everyone else. This K-shaped life offering awesome or anxiety fuels a “maxxing” culture: How do we look? How much protein do we consume? How well do we sleep? How many books do we read, etc.? The optimization and gamification of life has created a Hunger Games we’re all playing, all the time. As journalist Nitsuh Abebe wrote in the New York Times in October, the concept of maxxing comes from 1940s academic game theory, but it’s been repurposed by online communities to describe a strategy for “relentless optimization” where balance goes to die. “The language that comes from this layer of the internet has a mechanistic, gamelike aura, as if life were mostly just a web of tactics and hacks and mutual manipulation.”
According to clinical psychologist Catherine Houlihan, the “optimization mindset has many of the hallmarks of perfectionism.” Some commonalities: constantly pursuing high standards such that falling short of a goal is seen as failure; being preoccupied with results to the point of worry or rumination; constantly measuring performance to an obsessive degree; avoiding tasks if we fear we won’t be perfect; slipping into binary thinking, e.g., your diet is either “healthy” (perfect/optimal), or “unhealthy” (imperfect/suboptimal). “We don’t yet have much research about how adopting an optimization mindset might affect mental health and well-being,” Houlihan wrote. “But the negative effects of perfectionism are well established.” A 2023 meta-analysis of 121 studies found that when perfectionism takes the form of obsessive fear of failure — replaying mistakes, tying self-worth to performance, etc. — it correlates meaningfully with anxiety, OCD, and depression in young people.
80:20
In economic terms, optimization means getting the greatest return on your investment. Investors, however, aren’t perfectionists. They’re pragmatists who operate with an understanding of the Pareto Principle, which states that, for many outcomes, roughly 80% of the results come from 20% of the effort. When applied to the personal investments we make in our own fitness, health, and longevity, the lesson is that we make the biggest gains going from zero to one, but there’s a point, likely around 80%, where the efficiency frontier begins to collapse. If you don’t exercise at all, getting moving 4x/week will confer significant benefits. If you’re a gym rat, however, working out every day vs. 4x/week yields diminishing returns.
Value Capture
Bryan Johnson, an entrepreneur whose philosophy is “don’t die,” spends $2 million a year optimizing for longevity. Each day, he tracks hundreds of biomarkers, adheres to a strict vegan diet where every calorie that enters his body “must fight for its life,” uses shockwave and red-light therapies, and hangs out in his home sauna and hyperbaric chamber. He ingests a stack of prescription drugs and dozens of supplements, and exercises up to 90 minutes a day — without rest days. Bedtime is 8:30 p.m., sleep temperature is strictly regulated at 65° to 68°F, and he wakes up between 4:30 a.m. and 5:00 a.m. without an alarm. My Pivot co-host Kara Swisher, who interviewed Johnson for her CNN series, Kara Swisher Wants to Live Forever , observed that Johnson has an “obsession with measurement.” I’d add he has an aversion to “l-i-v-i-n,” as Matthew McConaughey famously put it in Dazed and Confused.
Our obsession with metrics, says journalist Derek Thompson, is akin to a modern religion that’s making us miserable. “Modern life is awash in statistics,” Thompson wrote in March. “Often, the quantification of modern life makes us play the games we can easily measure rather than the games we deeply value.” When we do this, we’re succumbing to “value capture,” according to University of Utah philosophy professor C. Thi Nguyen. “Value capture occurs when an agent’s values are rich and subtle,” Nguyen wrote in 2024. “They enter a social environment that presents simplified — typically quantified — versions of those values; and those simplified articulations come to dominate their practical reasoning.”
Some examples: We adhere to dietary guidelines to improve our health, but fixate on BMI such that the metric replaces the original goal; we pursue education to learn, but chase GPA at the expense of knowledge; we use social media to connect, but we come to value likes and other parasocial metrics over meaningful relationships. “Metrics are useful because they compress information, [and] they are dangerous because they compress information,” Nguyen told Thompson. “[It’s] not that these metrics aren’t measuring something real and that they aren’t objectively tracking something that we want to know about; it’s that they speak so loudly that they threaten to drown out other nearby qualities that are also incredibly valuable but are harder to measure.”
Enjoy Every Sandwich
In October 2002, with only months to live, frequent guest Warren Zevon appeared on David Letterman’s show for the final time. The musician retained his dark wit, joking that not visiting a doctor for more than two decades was “one of those phobias that really didn’t pay off.” In a more serious moment, Letterman asked Zevon if he had any insights about life to share. “I really always enjoyed myself,” Zevon said. “But it’s more valuable now. You’re reminded to enjoy every sandwich and every minute of it, playing with the guys and being with the kids and everything.” Zevon’s answer is memorable — Enjoy Every Sandwich became the title of a posthumous tribute album — because he articulated his life’s purpose, rather than the metrics he’d registered along the way.
I frequently encounter people who ask about my diet and fitness routine. It’s simple: I eat a diet that, mostly, hits my targets for calories and macros, try to get a good night’s sleep, and exercise regularly. As someone who’s obsessed with data, I code as an optimizer. I am not. I work out harder so I can drink … more. The first thing I do when I arrive in Los Angeles — if you know you know — is go to In-N-Out Burger. I often order (gasp) dessert, especially if I’m with my boys. I regularly stay up too late talking to friends back in the states. Two nights ago, after interviewing Secretary Clinton for a live pod in NYC, I came home, ingested edibles, binge-watched Season 3 of Euphoria and washed down chocolate-covered almonds, lifted from the mini-bar at the Faena hotel earlier this week, with two Peronis.
A. Great. Night.
Pattern Recognition
If there’s a pattern, it’s this: I’m health conscious 80% of the time, so I can devour the other 20%, and create a whole that is greater than the sum of its parts. The question isn’t will I live longer, but will I live better? A: Yes. Research supports this. Dieters who adhere to rigid meal plans are more likely to experience mood disturbances than those who don’t; flexible dieters are less moody and more likely to reduce their BMI. Harvard happiness researcher Shawn Achor tested multiple variables — background, income, activities, and sleep — and found that social connection was the strongest predictor of happiness, suggesting that a late night with friends is better for your health than a perfect sleep score. Consuming alcohol in moderation is associated with higher death rates, but a large-scale study of 1.5 million people found that moderate drinkers report higher life satisfaction than abstainers. Then there’s the work of Bronnie Ware, a palliative-care nurse who collected the regrets of her dying patients. They shared about not living their truth, wishing they’d worked less, expressed their feelings, kept in touch with friends, and been happier. Nobody said they wish they’d done a better job “optimizing.”
The Little Prince
I gave my oldest son a ring that he wears as a necklace. The inscription comes from Antoine de Saint-Exupéry’s The Little Prince , a 1943 novella about friendship, loneliness, loss, and love. “What is essential is invisible to the eye.”
My work and life are narrowing, distilling to a few goals: one of them is to prepare my sons for others. Many things I do don’t advance that goal, and some things undermine it. I’m a work in progress, i.e., suboptimal. When I’m gone, if I’ve accomplished this goal, my sons will have, among other things, receipts in the form of grief — proof that they loved deeply, as Nicole Avant, former U.S. ambassador and film producer, wrote in her memoir, Think You’ll Be Happy. The boys won’t remember my VO2 max. They won’t know my sleep score or my macro splits. What they’ll carry with them is … me, the man who showed up, imperfectly, at the dinner table, at their games, and in countless fleeting moments that didn’t register on any dashboard. The metrics were never the point. The sandwiches we shared were.
Life is so rich,
The Markets tour ended this week in New York, where I got to share the stage with one of my heroes, former Secretary of State Hillary Rodham Clinton. Total badass. The full, unedited show is available exclusively for Prof G+ paid subscribers by clicking here.
Ed Sim from What's Hot 🔥 in Enterprise IT/VC · Saturday, June 6 2026 · 12 min read · ↑ top
From Model Wars to Workflow Wars?
Jun 6
Your employees, your business processes, your workflows, your data…so why not your AI?
Until now, building AI around your enterprise has been difficult, time consuming, and expensive. The tradeoff was simple: use frontier intelligence and pay for it, or use cheaper models and sacrifice capability.
That tradeoff is starting to break.
The messaging this week was coming from all directions, but it pointed to the same idea: enterprises no longer have to choose between intelligence and economics.
Some are building AI around their own data, workflows, and institutional knowledge. Others are going further, combining proprietary data with open models, post-training, and routing. Different approaches, same destination: more control, better economics, and AI that increasingly reflects the business itself.
A week ago this felt like a pitch. This week it started looking like a roadmap, although we’re still early.
As discussed in last week’s issue #500, the budget pressure is real and it’s forcing the conversation. Listen to Dara at Uber:
Ed Sim
@edsim
reiterating: "We're using the more expensive models to explore. Once we scale some of these experiences, we'll look to bring in more efficient models that are more efficient on a token basis or are open source." wrote more about this here last week: whatshotit.vc/p/whats-in-ent…
Patrick OShaughnessy @patrick_oshag
Dara (CEO of Uber) on their AI spend: "We blew through our AI budget in a quarter, for the whole year. It is forcing us to adjust. We are going to meter headcount increases because to the extent that my engineers are getting much more efficient, their throughput is https://t.co/V1tuIxaFoa
Let's look at three vendors to dig in further. First, Palantir. It costs you money, but it’s smarter because it understands your data, workflows, and institutional knowledge while keeping them under your control. Alex Karp lays it out:
Palantir
@PalantirTech
At AIPCon 10, Palantir CEO Alex Karp shares our secret to sales: “We’re hoping that you’ll go to a large language model company and learn that they don’t care about you at all.” “What you will find is there are a myriad of problems that these very important models solve, and
Kirkland & Ellis doubles down on that message.
The frontier labs heard it too. Microsoft's answer is to make the model yours. They launched 7 new models, yes they're in the race with OpenAI, Anthropic, and Google, but more importantly read this from Mustafa's post:
With Frontier Tuning, you’re building your own model, trained on your your own data, within your environment, controlled by you. Ok, this is not open source and neither is Palantir but you do get my point - why use a frontier lab model at the highest cost when perhaps i can rearchitect a better way. Microsoft claims that Land-O-Lakes has a 10x more cost efficient model than Open AI’s GPT 5.5.
Here’s the fork worth naming out loud.
“Own your AI” is splitting into two camps.
Context is yours: Palantir and others are helping enterprises build AI around their institutional knowledge, workflows, permissions, and data. The model underneath can change, but the enterprise context becomes the asset.
Model is yours: Nemotron, GLM, and the post-training crowd give you the model, the data, and the weights, with routing on top.
Both beat renting raw frontier intelligence for everything.
Frontier models aren’t going away. But long term I’m betting the biggest winners will combine frontier intelligence with proprietary data, enterprise context, workflow orchestration, and increasingly their own models.
Control compounds.
The company that owns the AI behind its critical enterprise workflows has more control over its economics, its roadmap, and ultimately its competitive advantage.
That’s not a hedge. That’s the call.
Which is why I love Nvidia putting significant dollars behind Nemotron, which is catching up fast and was released under a new MDW open license: open model, open data, and open weights.
NVIDIA AI
@NVIDIAAI
Today we're shipping Nemotron 3 Ultra. A 550B MoE frontier-intelligence open model built for long-running agents. It delivers 5x faster inference and lowers the cost of complex agentic tasks by up to 30% versus other open frontier models.
Clem from Hugging Face tells it like it is - routing and post-training open-source models gives you smarter, faster, and CHEAPER systems.
clem 🤗
@ClementDelangue
Routing and post-training open-source models won't only give you more accurate systems but also meaningfully faster and cheaper systems as most companies are currently learning (in addition to giving you more control and privacy). The idea that a "frontier" model (by frontier we
Harvey @harvey
We partnered with @FireworksAI_HQ to train open-source models for legal. Here's what we found: 1) Hybrid legal agents can beat frontier models on quality and cost by routing selectively to a frontier advisor. We tested a hybrid setup where GLM 5.1 served as the primary worker,
We will move to a world like Clem says where post-training with more control of your data and evals will become an important way to build agents. This is also a world we invested behind at boldstart a couple of years ago with Uare.ai. Here’s an excerpt from founder and CEO Rob Locascio talking more about what and why (stay tuned for release soon!)
Everything above is the same argument told in bits. Generalist AI is that argument told in atoms.
Your data, your model, your AI. The catch in the physical world is that there is no Reddit for robotics data. You can’t scrape your way to a foundation model when the foundation doesn’t exist. So the moat isn’t the model, it’s how you manufacture the data no one else has.
Generalist went and built it. 3d printed data hands on real human hands, video of every kind of work getting done, over 500,000 hours of proprietary training data feeding their own model. With GEN-1, deployments are live and the model learns new skills in hours to days. Same playbook as the software camp, just in robots: own the data, own the model, own the outcome.
Generalist
@GeneralistAI
Everyday for the past 2 weeks, we've been sharing something new from GEN-1, our latest milestone in scaling robot learning. This has never been done before. Going from ideas to skills in days (or faster) is what physical AI models should deliver. More coming. Stay tuned. Read
Which is why I'm 🔥 up that the company announced its most recent round, $400M at a $2B valuation. New lead Radical alongside 8VC, Union Square, Hanabi, Norwest and more, with existing investors Nvidia, boldstart, Spark, Jeff Bezos and more. Angels include Eric Yuan, Bin Lin, Fei-Fei Li, and Naval Ravikant.
Ed Sim
@edsim
huge congrats to the @GeneralistAI team on its $400M raise to build the frontier lab for robot intelligence the autonomous enterprise is coming for the physical world faster than you can imagine Amazing what @peteflorence , @andyzengineer , Andrew Barry and team have accomplished
Generalist @GeneralistAI
We've raised $400M in new funding. This capital goes toward one mission: building general intelligence for the physical world and making it useful to everyone.
We led the inception round in March 2024 with Nvidia because we believed the autonomous enterprise doesn’t stop at software. It extends into the physical world, where the real value of labor lives. GEN-1 is the proof. If you want the founding story, read my partner Ellen Chisa’s post on Pete Florence, Andy Zeng, and Andy Barry, the PhDs, DeepMind, and the lifelong passion behind it.
And as Masa says, the opportunity is simply massive
Eren Chen
@ErenChenAI
“The next trillion-dollar opportunity is Physical AI and Robotics.” — Masayoshi Son, CEO of SoftBank, also just became the richest in Asia
So back to where we started. What if?
The tradeoff is starting to disappear. Frontier or affordable, owned or rented, smart or cheap. Those were yesterday’s either/ors. This week made them ands.
The question is no longer whether you can own your AI. It’s whether you can afford not to.
The future isn’t frontier versus open. It’s who owns the context, who owns the model, and ultimately who owns the outcome.
As always, 🙏🏼 for reading and please share with your friends and colleagues
Scaling Startups
follow this thread - so many reposts of horror stories raising from ummm…venture capitalists…
choose your partner wisely, not just the firm, the partner
GREG ISENBERG
@gregisenberg
I was once pitching in a board room at a top 3 VC firm for a $15M Series A. 12 people in the meeting. One of the GPs fully fell asleep. Out cold for 30+ minutes. Nobody acknowledged it. Everyone just kept going. I kept presenting my Series A slides to an unconscious man in a
Enterprise Tech
let’s all celebrate a major milestone - agentic traffic has finally surpassed human traffic!
Matthew Prince 🌥
@eastdakota
Welp, that happened faster than I predicted. Thought it would be end of 2027, then early 2027, but agentic traffic growing so fast that bots have now passed human traffic online for the first time in the Internet's history. radar.cloudflare.com/traffic#bot-vs…
🤯 will AI build itself? Anthropic sees a path to recursive self-improvement - read the data - engineers ship 8× more code/quarter, 80% of codebase is AI-written, coding success at 76%, 52× training speedups, and beats humans 64% on research decisions. This closes the loop for AI to autonomously design better successors. Massive upside…kind of insane!
Anthropic
@AnthropicAI
Our internal data shows Claude is accelerating AI development—a possible path to recursive self-improvement, or AI autonomously building a more capable successor. It’s happening faster than we thought, and the implications deserve greater attention.
| | anthropic.com
When AI builds itself
don’t count Microsoft out - back and just launched 7 AI models
Mustafa Suleyman
@mustafasuleyman
Super excited to announce seven new world-class MAI models today. They represent what we consider a new era in AI designed to keep you in control and on the frontier. First is our text foundation model, MAI-Thinking-1, exceptionally strong on reasoning and SWE tasks. - It’s a
i like Jensen’s thinking on this - best time ever to be a software company but make sure it’s built for agents!
Wall St Engine
@wallstengine
$NVDA CEO Jensen Huang: This is actually an incredible time to be a software company. But the software has to be presented to the agent in a way that the agent can use it.
more on that - deep systems of records on most of these or incredible context with the Palantir ontology
The Transcript
@TheTranscript_
$NVDA CEO: Software isn't dead "And we're working with so many companies: Cadence and CrowdStrike, and also Palantir, SAP, and ServiceNow. People always said, 'Jensen, the agents are going to disrupt these markets.' I said completely the opposite. And you can now see it. Agents
💯 which is why companies need intelligent process mining and mapping to understand workflows in order to figure out how the org works and how to improve
Tom Blomfield
@t_blom
Imagine replacing 90% of your employees with a team of geniuses who have no idea how your company operates. Total chaos. Nothing works. That’s what AI feels like today. The missing piece is extracting all the domain knowledge from people’s heads and providing that as
and also why many cos are using Forward Deployed Engineers (FDEs) to kick off the process as well and more importantly measure what success is - read the full post here
model optionality increasing
Ara Kharazian
@arakharazian
NEW: DeepSeek, the Chinese AI company, is one of the fastest growing vendors on Ramp. In probably the biggest sign that companies are looking for cheaper alternatives to OpenAI and Anthropic, some are willing to use cheaper, Chinese models, sending U.S. data back and forth from
super cool video and post from Guy Podjarny of Tessl on the Emerging Agentic Stack and why “context and skills are the new code.” Software development is transforming from writing code and implementation to defining intent and instructions. Code is becoming “disposable” and easily regenerated, while context (guiding the AI agent) is becoming the primary unit of work.
along with a deeper discussion with friend James Kaplan, CTO McKinsey, on the future of software development with Guy from Tessl (watch here)
more of the same but for security harness engineering
Ed Sim
@edsim
Simply unsustainable to do continuous scanning as more code and code bases than ever before Huge opportunities for security harness engineering to help enterprises get SOTA but cost way less and do more continuous scanning
aaron holmes @aaronpholmes
NEW from me: Anthropic’s Mythos isn’t cheap—one tester burned through $1M of tokens in a couple weeks—but companies are still budgeting for it to prevent hacks: https://t.co/Q6nck5ThSm
going to be an insane year for robotics - owning the brain, the intelligence layer going to be big business
Sam Altman
@sama
OpenAI Robotics is hiring, looking for exceptional full-stack hardware, ops, systems, and ML engineers to help us program and manufacture robots that are useful for society. AI should be able to help people in the physical world. In the short term, we are focused on robots to
and Nvidia
NVIDIA AI
@NVIDIAAI
Introducing Cosmos 3: Our latest frontier model for Physical AI Cosmos 3 is the world’s first fully open omnimodel with native vision reasoning, world and action generation. Today we’re releasing Super (32B) and Nano (8B) variants.
drones are working…
Atoms Not Bits
@AtomsNotBits
BREAKING: Walmart, the world’s largest retailer, has crossed 1 million drone deliveries across Texas, Arkansas, Florida, and North Carolina. The company says its drone delivery network now averages 23-minute deliveries
Markets
the SpaceX IPO roadshow presentation is here!
Sawyer Merritt
@SawyerMerritt
Here is @SpaceX 's full IPO roadshow presentation from CFO Bret Johnson, who has been CFO at the company for the last 15 years. It's worth watching.
speaking of SpaceX - 100x increase in revenue forecasted by 2030!
*Walter Bloomberg
@DeItaone
GOLDMAN SEES SPACEX AI REVENUE EXPLODING TO $322B BY 2030 Goldman Sachs projects SpaceX AI revenue rising from $3.2B in 2025 to $322B by 2030, a ~100x increase, forming the core justification for its $1.78T IPO valuation. Total revenue is forecast to reach $474B, with Starlink
SaaS back? just select names which is the correct view - more nuance needed
that’s some AI concentration
The Kobeissi Letter
@KobeissiLetter
This is incredible: AI-related companies have issued ~$140 billion in investment-grade bonds year-to-date, accounting for 49% of the total IG issuance. AI-related companies have also attracted ~$220 billion in venture capital funding year-to-date, making up 87% of the total.
it’s never as bad as one thinks…or as good - amazing comeback
Jon Erlichman
@JonErlichman
A story published on this day in 1999. Amazon’s stock is up 9,000% since.
State of Data May slightly delayed by my forays at CVPR. State of Data June to follow around June 25th, after a large announcement I have about a new company I’ve been involved with. I’m excited to unveil a lot of the work, formalized, that I’ve been doing with many data companies in the space.
You may see me and my work on the model scoreboards soon :).
What data spend actually is
I recently made a post on twitter made viral by MTS which echoed the line that data spend was a 10-15B category by lab. People seemed to be confused and befuddled by, in their view, how much labs were spending on “labelled images and traces.” They would right to be so if that were actually what I referred to as the data industry. But it is not (here is a reminder piece as well).
An RL dataset sold comprises many things. A top human data buyer at a top lab, in search of research partners who they think will be able to scale quality with quantity, will often look for genuine research partners. And a genuine research partner does not churn out data at a purely industrial pace. Often, to get data in the right formats for RL (especially as useful regimes make RL datasets grow smaller in problem amounts, but longer in difficulty), substantial costs for iteration and compute are baked in. The cost of creating the environment, tools, compute for testing, etc. are all baked into the per unit pricing of tasks, which is often the most common unit of data pricing that one looks at here.
One should pay attention to the notion of “project-based pricing” as well. Lab teams are often given budgets for data acquisition which they must divide amongst vendors. A sample project may look like: “we want x tasks for a general capability, and this vendor is promising us x tasks at y pricing in a 3 month delivery interval.” These budgets, post-pilot often being capped in the range of 1-10M per project (although some can certainly go higher), are often arbitrarily given based on lab spend directions at the app layer, though there is certainly always “kitchen-sink like spend” in various research teams for improving general model capabilities.
Projects don’t always play by the same metrics in terms of model improvement (or even the right ones). While some base the quality of their tasks entirely by their potential and rigor to hillclimb extremely specific benchmarks, others trust that whoever supplies their tasks can be judicious in helping them decide which benchmarks matter. Of course, usually not only 1-4 benchmarks are considered (reward in post trained analysis) but a plethora, including many internal to labs that vendors will never see. You may ask - how would a vendor propose to provide post training data that improves model capabilities on a certain task when they don’t know the KPIs? Well, perhaps nobody actually knows the KPIs. In this way, we can realize why being a “researcher” like data vendor here is of paramount importance, as that services level expertise becomes a real product.
What I’ve described is a mainstay dynamic with RL data. An RL dataset, then, should also be noted as not the primary data modality bought and sold by your largest data players. Human staffing, SFT data, and more banal labeling/pre-training data aggregation jobs are the primary mandate of some of your larger data players. Often, the margins are juiced here by research services reputation or timeliness of project delivery.
Data Labeling, eschewing the layman’s sense
One striking thing about data markets is that people still conflate the data market model of Scale from 2018-2021 as how to think about data markets. Labeling, insofar as hiring bodies from India and the Philippines to label image data. While a semblance of this still is a large part of the 10-15B lab-specific data markets, this is entirely untrue for data markets as it relates to RL data.
The axis that changes (as human labor is, though, still needed for frontier data that pushes outward model capabilities) is the educated-ness or “judgement” of the human labor. For RL data targeting white collar knowledge work - we need more educated people. The way in which we wield these educated people who have proper judgement to create good benchmarks is also an axis of skill issues in of itself. Consider the following, commonly repeated argument from Surge:
Surge is LMArena’s number one hater
LMArena is a perfect example of improper application of judgement labor to grade LLM responses in an area which clearly require higher order expertise - and the results are hillclimbing on the wrong things. That many data companies then take this as golden ground truth for “realism” and employ dubious synthetic data pipelines to scale this up mean also that these flaws are amplified all throughout training data. This data doesn’t expand the frontier - but places brambles all around efforts to do so.
When you look at the cost economics, the mechanisms to scale up data for proportional model advancements are very different than Scale in 2019. On the pendulum of “operational” work versus “engineering” work, good datasets to scale RL data are incredibly engineering heavy. They are so engineering heavy, that they demand creation of adjacent infrastructure to support them - a notion I crystallize as “Antikythera Mechanisms” in my previous work - that actually give them the premise of moat and unit economics investments that make them seem like venture bets. But altogether, the investment into such infrastructure paints the launch stage for properly executed RL environment companies to look like Facebook teams pre-the proliferation of the internet, representing agnostically good talent aggregator bets.
But this is not easily discernable from the surface (the notion of whether a team is a talent aggregator or engaging in the right contracts). An RL environment company will always say, per their website, that they are pushing “frontier capabilities” by matching the best “human experts” to expand “AI capabilities.” They will not tell you whether most of those contracts with AI labs are lower margin staffing solutions, whether they are primarily synth SFT data generators, whether they are mostly providing selective RL-data only engagements. They will not be clear about whether their labor and QA comes from weekend MBA students and Nigerian annotators, or whether they prefer to spend as much time teaching domain experts how RL works to produce proper labor alignment. They will also mix contracts, as they are often paired together for larger projects when the aim is to produce general model advancement.
Altogether, what this means is that data is a nuanced industry with many subdomains that I consider venture-investable, and many domains who aren’t. Investors who dismiss “data” companies without clearly understanding the difference between a Mechanize/Fleet/Preference Model and Scale-type player need to update their priors.
A Pre-training Miasma
We generally find that robotics data markets continues to attract antagonism for the following reasons:
There are genuinely few buyers, all with bespoke specifications, such that over-optimizing for one modality of scaling leaves one to undue risk
There are genuinely buyers whose scaling directions will be proven wrong in the interim - meaning that serving their customers won’t bring vendors anywhere towards building a more sustainable business
Simulation infrastructure startups genuinely threaten the data narrative - if we are successful in bridging the Sim to Real gap, or making genuine strides in sample-efficient innovative directions - it is genuinely unclear whether diversity in today’s requested datasets will continue to be requested at their current scale
Data sales, then, must lean more political in nature. Buyers cannot assess data vendors without a costly inventorization process; a process whereas they pass on to their researchers the mandate to experiment whether:
The data is useful - incorporated in some training run and form fit to some downstream policy improvement metric
The data partner is iterative in both speed and research taste
The data partner genuinely cares about the quality of data - whereas they are immediately available for a call/email to rectify mistakes in data quality that inevitably occur at scale
But the cost to assess these things are too great in lieu of a warm referral, or simply an overactive internal champion who relentlessly puts a data company into researchers’ group chats, or an overwhelmingly, shockingly, good display of quality data (rare).
So, indeed, it is by the above that most of these data sales are done today, and most of the ego vendors that do pick up contracts do it for contracts where researchers have no forward looking permanent spend direction bolstered by an agreed upon scaling architecture (not a transformer yet here) and are instead experimenting (see Figure’s egocentric data purchases). The only winning move, as seen from RL environment companies, is to extend into the above head research layer or below deployed application layer, if one wants to accrue real venture scalable value.
Today, Mecka, Build AI, and xDof are amongst those that are successful at somewhat completing the above at scale. There are increasingly a few exits from large robotics labs professing to do some sort of “outsourced post-training data for these labs” in light of all the direction constraints. But data, just as in its best iterations in post-training land for text based models, must be a service, with the medium to deliver that service being data, where our work must keep in mind the best opportunities to capitalize on when “models get better.”
Benchmark Psychosis
What is a good benchmark? How do we trust a good benchmark? Every month, it seems that a data company releases a new benchmarks that ascribes groundbreaking flaws to benchmarks of yore.
From Datacurve’s DeepSWE
A consequence of RL environment companies, AI roll ups, and RLaaS companies extending their infrastructure into real world businesses is the proliferation of the ability to translate raw business context into model actionable formats (Antikythera Mechanisms). A 2nd order consequence of this proliferation is faster, more autonomous, and more realistic benchmark task creation. A 3rd order consequence of more benchmarks is our ability to create “benchmark bundles” that adequately measure a bundle of skills/tasks that we care about, in order to create our blended assessment of whether to use a model or not.
This “Benchmark Bundle” approach somewhat assuages our Benchmark Psychosis concerns by leaving tailoring up to the individual and dissuading arguments against “realism” - one figures that they can incorporate both the long horizon “create a version of x software completely from scratch benchmarks” alongside the actual realistic benchmarks which may be consigned to shorter horizon work in order to test both theoretical limits and practical utility. Of course, a benchmark must also comply with agnostic measures for good benchmarks, like n gram contamination testing, cross-harness differences, etc. (not too dissimilar from the grade school’s first teachings of the scientific method, where one has to have only one independent variable!).
I’ll give an example from recent evaluations I’m doing with Opus 4.8:
Opus 4.8 sits at the bottom of arr_waterfall (0.603 mean, sixth of seven) but leads pe_valuation (0.725) and pair_trade (0.444). 56 of 102 arr_waterfall criteria regress by ≥15pp from Opus 4.7 to Opus 4.8, against only 5 improvements. The regression concentrates on contradiction penalizing judgement behavior - putting a customer in two buckets simultaneously, computing beginning ARR as (BoP+EoP)/2 instead of point-in-time, conflating logo retention with revenue retention. Opus 4.7 makes none of these errors while Opus 4.8 makes them 35-40% of the time. The same self-reflection that introduces contradictions on the scalar-precision rubric lifts Opus 4.8 to first place on the tool-orchestration rubric, where iterative re-screening of comp sets and precedent multiples is exactly what the rubric rewards.
GLM-5.1 (open-weight, Z-AI 32B-active MoE) takes the #1 arr_waterfall spot at 0.843 mean, ~1pp ahead of Gemini 3.1 Pro and ~24pp ahead of Opus 4.8. Once the EmptyModelResponseError terminator pattern is corrected via a Verifiers shim, GLM-5.1 pe_valuation lifts from 0.337 (raw) to 0.674 (engaged subset), tying Opus 4.8 within 5pp. I generally see this as direct evidence that post-training-engineering choices at frontier labs are not yet dominating over base-model capability on real-world rubrics.
All of my tasks are sourced from real world pipelines where I directly work with real world businesses, examine their most important workflows, and autonomously convert them into post-training actionable formats, with reward rubric creation and battle-testing being the primary human input. In these, I see over engineering of “self-reflection” in latest models that produce contradictions in earlier work (that my reward rubric penalizes) that probably play well to their harnesses in respective app layer products. The result is that cross-harness differences cause models you wouldn’t even imagine to be frontier to leapfrog “frontier” models in actual tasks - leading one to wonder if frontier models have been exceedingly post-trained on frontier “toy” datasets produced by RL environment companies that make tasks artificially hard at the expense of realism.
This theme of “overcorrection” in a somewhat over-post-training-engineered solution is not uniquely observed. We see it as a precursor for a litany of issues around harnessing and benchmark performance:
Sclar et al. document up to 76-accuracy-point spreads from format variations alone on Llama-2-13B. The same prompt under different system-prompt templates produces different answers from the same model.
HAL ran 21,730 rollouts across multiple scaffolds on the same dataset and found higher-reasoning-effort scaffolds reduced accuracy in 21 of 36 runs which is the opposite of the intuition that ‘more chain-of-thought = better answer’.
‘Efficient Benchmarking of AI Agents’ finds 7 of 10 widely-used agent benchmarks have scaffold confounding affecting the expectation of the score, not just its variance.
Single-harness numbers are one observation drawn from a distribution whose width the benchmark hasn’t measured, but they disproportionately make up the majority of our benchmarks.
The harness-sensitivity is amplified by a recent training-pipeline change at frontier labs. Anthropic post-trains Claude against the Claude Code harness and bash + str_replace + apply_patch are first-class affordances the model expects. Similarly, OpenAI post-trains GPT-5.5 against the Codex CLI and Google post-trains Gemini against the agent loop used inside Gemini CLI. The model knows what shape of tool surface to expect, and produces output calibrated to it. When you run the same model under a different scaffold - Verifiers, Inspect AI, tau-bench, OLMES - you measure the gap between the model’s training scaffold and yours.
Three changes in the post-training pipeline have shifted the harness from ‘one of several evaluation knobs’ to ‘the primary measurement instrument’:
Reasoning-budget tuning at the model level. Reasoning models (GLM-5.1, Opus 4.8, Gemini 3.1 Pro) allocate variable amounts of reasoning per turn. The same query under a low-reasoning-budget scaffold and a high-reasoning-budget scaffold produces qualitatively different outputs from the same model. HAL’s finding that higher reasoning effort hurt accuracy in 58% of cases (21/36) is direct evidence that the relationship between scaffolding choice and reward is non-monotonic.
Tool-shape-specific post-training. Models now expect specific tool signatures (Anthropic’s str_replace, OpenAI’s apply_patch, Google’s tool-calling JSON schema). A scaffolded evaluation that exposes a different tool signature measures the gap between the model’s training distribution and yours, not the model’s underlying capability.
Content-policy filtering layered on top of the model. Vertex content-filter aborts on our Gemini pe_valuation rollouts ~33% past step 14 which is a policy decision highly attuned outside of model-capability. The same Gemini under a different routing (AI Studio direct vs Vertex) would produce different abort rates. This is only direct evidence that we should suspect Gemini much more capable than the product scaffolding around it allows.
Twelve months ago we could ship benchmark numbers under a single scaffold and the noise floor was small enough that the cross-model deltas dominated. Today, with reasoning-budget tuning, tool-shape post-training, content-policy filtering, and provider-route fragmentation all stacking, the noise floor is approaching the signal.
On the data vendor side - much of the slop data hitting labs professes to have discovered genuine model differences - but actual fails on all counts of failing to identify cross-harness differences. Today, I work with datasets from many human data vendors that would do much better having shipped:
Surface-stratification table: at least 2 scaffolding configurations per model, one of which is a production-deployed harness (Claude Code for Anthropic, Codex CLI for OpenAI, Gemini CLI for Google). Spread reported per cell. Spread >5pp flags the dataset as scaffolding-sensitive and the vendor must declare which configuration the headline number is calibrated against.
Sandbox manifest: docker_image_digest, network_egress_policy, tool_approval_policy, isolation_granularity, max_turns_per_rollout, observation_truncation_policy. Without these the benchmark cannot be re-run by a buyer against the same scaffolding contract.
Provider-route disclosure: which API endpoint served each model. Anthropic-direct vs Prime Inference vs OpenRouter vs Bedrock vs Vertex are different surfaces. The standardizing practice of including a harness-declaration check matches the harness against a list of accepted production harnesses; the routing layer is part of that surface.
Per-model failure-class breakdown: whereas we start to see 18-label taxonomies for investigating model failures - here is the breakdown I use in my benchmarking: (capability / 6 prompt sub-classes / 7 scaffolding sub-classes / rubric-defect / training-data-defect / orchestration-defect / triangulation-failure / ambiguous-needs-human-review).
A Product and a Service
ML Practitioners outside of the data industry will remark that data is useless without understanding the full end use case; if you do not understand the KPIs for which data is meant to train on, then to be a data aggregator is to be a book collector with a pupil that only wants to learn physics. Then the question belies - how difficult is it to aggregate books? And moreover, even if I do accumulate a decent amount of physics books, how will I be sure that certain passages will exist in each that allow my pupil to learn truly generalizable information?
The plight described above is the plight of the OTS, or “off-the-shelf,” business model in data companies. In a way, the OTS model as compared to the on demand, contrived model, allows one to controls ones costs by optimizing ones’ operations. To optimize one’s operations is the only hope that a data company builds scalable systems. On the other hand, one cannot enter a new relationship professing to sell any “OTS” data even under the guise of improving general model capabilities. After all, in learning (both human and machine), we profess to learn best when something is foreign, but in a way where we grasp how exactly it bounds the limits of our knowledge.
Practically, this will mean that all companies that want to sell OTS data must prove their mettle first by serving “on-demand” contracts, or contracts that have an acute sense of what labs currently want. In further examination, one will find that most data companies today will have obtained their first data contracts by support from an internal “champion” who relentlessly puts them in group chats with human data teams and lead researchers in a political way. Ironically, once a researcher has “trained” a data company to understand their taste and register them as an approved vendor (particularly the case at Google), they will generally regard the hassle of onboarding a new vendor (even in case of subpar data) to supersede marginal gains. Nevertheless, there a few, but small, group of vendors that researchers have sworn to not touch forever anymore owing to poor data quality.
I mentioned before that an RLaaS company is a data company, and a data company is an RLaaS company, just serving different customers with different infrastructure needs (one lacks data, another lacks talent/post training infrastructure). At this point, everyone in the valley is aware of how certain RLaaS companies have sold data - somewhat synthetically altered, based on their real world engagements. The recent overtures of Harvey, Ramp, Cognition, Sierra, Decagon, etc. into public open source post-training experiments are actually part of a larger cooperative plan with frontier labs whereas frontier labs want to work with sophisticated app layer companies for co-developing evals/benchmarks. This serves a three-fold purpose:
Frontier labs understand the harnessing being used for their models - and adapt accordingly
Frontier labs, in ever-hungry search for better evals, can obtain more evals from orgs who amass elite engineering talent and who are also domain experts
We still see a power law effect where, even if post-training occurs in elite app layer companies, the resultant models are treated more as lower level tooling to be orchestrated by higher level models in their complex harnessing. To adapt GPT/Opus to power law token spenders’ evolving harnessing, then, is to directly adapt products for the enterprise harness benchmarks that matter.
The last point goes under-appreciated by the average API connoisseur and likely the root cause of over-engineered model behaviors in recent individual testing. The benchmarks that we’re trying to benchmark max are, sure still the SWE-agent pros and the litany of benchmarks our vendors sell to us, but now increasingly geared towards the internal benchmarks of breakout app layer companies.
This realization alone should make RL environment companies want to move into the app layer/applied layer as soon as possible themselves.
No room for sycophants
Increasingly - the knowledge to sell good RL data compounds as a series of internal learned senses that one could define as “taste” for good data. Many post-training datasets and evals sold today to decry “poor model performance” lack post-review judgement by their creators that ask, simply, whether rollout failures are emblematic of actual model failures.
Moreover, when one becomes overly fixated on selling data and the 1st order attributes of what makes a dataset valuable from a buyers perspective, they forgo thinking about a macro view of what makes a dataset truly valuable. This presents a lack of foresight in ability to predict the need for N-gram contamination testing, when a dataset is truly at risk of already being represented in training corpuses, or a lack of ability to examine harness level differences and reason that a certain model does better at a certain agentic task because it was trained in “bash” rather than “apply_patch.”
That labs are increasingly split among those who “benchmark-max” and those who do not (a vanishingly singular segment) mean that such behavior is encouraged. In a way, one expects a process where black boxes sell to black boxes to be inefficient. I view it as a training exercise in revealing strong lab partner companies who use it as an accelerator to build companies to eventually capture accrued value in the app or model layer, or supercharge learned expertise in app layer domains.
At this point, that top labs are going to top application layer companies with good benchmarks and asking them to produce data (thereby revolutionizing their whole economics) and app layer companies doing this themselves in certain domains is so commonplace, that wonders how a generalized RL environment company competes if not without carried customer accounts from a pre-existing vendor. Many wonder how, if GDM mandates vendor diversity on the scale of 15-50 companies, which companies are large data sellers if not the flashy ones on the post training data vendor list. The truth is, that many of the RL environment companies powering extreme domain specific advancements in biology and cybersecurity, are life sciences/cyber companies themselves.
Indeed, some of the breakout companies recently in the cybersecurity and biological sciences space who’ve made seriously blockbuster revenue expansions in the past few months are exactly those app layer companies who’ve been approached by labs to produce data. Most will be good judges of how to produce good rubrics, and generally be helped by the advent of models to being better at their domains. Many of them are alerted to the space by being contacted by the likes of Mercor for subcontracting expertise for reward rubrics, then seek to cut out the middlemen and go direct to lab. Many are subsequently, and hopefully, able to return to their own application use cases equipped with a semblance of post-training expertise to regain their app layer market share.
A sycophant of a mercor-esque model without domain-specific expertise, and lukewarm on the counts of data/research taste, will arrive to the party with only white claws to get to the fun. They will be consigned to working with N-2 labs, be the victim of unexpected model advancements rendering their data useless, and without perpetual personal connections to researchers, generally falter and be bought for SPL/market share fodder by the likes of larger data companies. Every week, I am approached by doomed data vendors who ask me about the market opportunity for selling data, without any thought as to the downstream uses of it.
We are all neolabs
But an RL environments company that hits escape velocity will buy themselves enough breathing room to think outwardly about their place in the value chain. I’m often criticized for being a data markets bull; the truth is that I think that data markets are an excellent place to start building a company, but not an excellent place to stay as a company. The vast majority of the end value of commoditized intelligence will always accrue in the services and app layer of actual work. The model will be trained by the frontier lab, benchmark leaders gaining the right to be sold at a markup, and app layer harness wrappers increasingly rolled out to capture more of the margin. Anthropic and OAI’s ventures into services are only evidence of more ability to capture the spread.
Handshake, Mercor, and others ought to enter the enterprise services and application layer themselves. It is the hardest work, but it is the work to which all token value accrues, and the work that one should be good at, from an infrastructure level, by an RL environments company done right. This is because:
A well run RL environments company will have developed relationships with real world services businesses such as to negotiate access to their token streams, either by ownership, profit share, or another app layer mechanism
A well run RL environments company will have impeccable understanding of harnesses to optimize model performance for economically valuable activities, because this it the work they do with labs hand in hand
A well run RL environments company will have strongly technical FDE talent and researchers, as they are the forward deployed research arm of labs investigating model capability in truly economically valuable capacities
A well run RL environments company will have strong financing, as they will generally be profitable from lab engagements and not engaged on overly CapEx timesinks, having the same model learning fruits as the labs without any of the compute costs
A well run RL environments company will have Antikythera mechanisms to automatically ingest and infer context from real world services businesses, such as to dramatically ease the cost of doing business as an RLaaS business (see Applied Compute Specific Intelligence)
For example, in the above head image, I show the results of a PE_valuation task that involves outputting a final memo and JSON Output to pursue/negotiate/pass on an investment memo. The model has read the CIM, model sheets, banker call notes, and valuation reference, and is asked to commit to a final transaction recommendation with full triangulated valuation, scenario MOICs, and structuring asks. This is the deliverable that lands in the IC memo and we see a reward delta of 0.788 produced by unstable harnessing.
Here we find a harness-grade finding worth ~0.34 reward per pe_valuation rollout - the gap between GLM-5.1’s raw 0.337 mean and engaged-subset 0.674 mean is a pure harness fix (EmptyModelResponseError shim). Without the shim the model looks unusable for tool-orchestration tasks but with the shim it ties Anthropic flagship. A frontier lab buying GLM-5.1 capability data without the harness fix would ship a 50-percent-fail-rate model. The harness is the product, and the we have additional axes of reliability to care about with dealing with open source models in actual applications.
But OAI and Anthropic’s ventures into services are actually a blessing for all RLaaS companies. They will, inadvertently by telling the Cognizant’s and PwCs of the world, inject some data organization readiness into the markets. Constant competition between the frontier labs themselves will ensure that switching costs for “gathered context” from continual usage of an AI ecosystem will probably not grow great enough such that you are locked on Anthropic forever. In no circumstance in history, has a dominant, well verticalized, first technology discoverer still controlled double digits percentage of a market a decade after.
In other words, they will be forever susceptible to N-1 implementors and neolabs who use the roads they create to compete. We are again reminded of a common adage in 2023 - models will get better, so we should just optimize for distribution. Well now, as we see the possible decoupling of app layer companies from foundation model companies as post-training democratizes, then the fruits of distribution for adequately engineering rich teams are ripening (see Cursor reaching positive enterprise gross margins in Q1 2026, largely driven by in-housing inference).
Takeoff
AI2 recently released a paper detailing a new architecture called EMO, which indicates a stronger drive towards agnostic architectural improvements for MoE based small model implementations, alongside Ramp, for a vision of the world that is an option for decoupling models from model companies.
Recall the framing from “Small Model Systems to be mandated” in my previous state of data. I argued widespread small model implementation would come via one of two architectural paths. Either MoE-based large model implementations with the modularity baked in, or multitudes of small RL’d model swarms. AI2 shipped a 14B-total / 1B-active MoE where you can use 12.5% of the experts (16 of 128) and retain near full-model performance. This is a “small model” in the sense of a distilled specialist whereas it is the full pretrained model whose expert subsets you compose at inference time.
The deployment math inverts with small model based systems. In the “Cost will matter” framing from last state of data, I argued AI-native app companies are unprofitable because the only models that clear the performance gap are closed-source SOTA, and SOTA economics dominate their margins. We increasingly see a third path between “use closed-source SOTA” and “use small open-source,” which is “use the full pretrained model with selective expert subsets at inference.” You get most of the pretraining benefit at the inference cost of a ~1.75B model, or you can use this as an orchestrator model for a fleet of small RL’ed upon models.
The most sophisticated RL environment companies would build software products whose purpose is enterprise context ingestion, on-prem Granola-like artifacts that turn into post-training data. EMO’s modular structure is the architectural side of the same mechanism. Each enterprise can effectively post-train on their domain by picking and possibly fine-tuning the right expert subset. Combined with the Tinker / PI tooling, the path to a world where applied AI teams pick a frozen open-source model, identify the relevant experts for their use case, and post-train cleanly on their domain data gets shorter than it was even three months ago.
Despite this, EMO is still mostly pretraining work and I remain curious whether AI2 has not yet shown strong results on RL post-training of EMO. The sycophancy, reward hacking, and forgetting issues I argued mattered for any post-training stack apply here too, possibly more sharply because expert routing introduces new degrees of freedom for reward hacks (a model could learn to route to “permissive” experts to get higher reward, which would be a clean failure mode for a bias probe to catch).
A new corpus of research explores models’ ability to orchestrate separate experts, in addition to these experts being MoE based. Naturally, you could imagine that the complexity of such architected systems makes the juice not worth the squeeze, but the increasing proliferation of autoresearch allows for the model-driven exploration of multi-small model based systems more feasible.
Time to leave the nest
RL environment companies were never meant to sell to labs forever. It is undeniable what cost economics will be produced by the unit cost of intelligence decreasing, but that it is still confined to a mere 10% of GDP and sophisticated engineering companies, mean that all AI infrastructure companies doubtlessly have inane customer concentration. The TAM expansion by natural knowledge dissemination will be, by default, slow, without innovation in the deployment side.
Its time to leave the nest and explore the wider ocean. Still, 80% of enterprises have never touched AI to a meaningful degree. The deployment surface for app layer companies remain extremely forward deployed in a cost prohibitive way - meaning that barring regulated industries where the unit cost of white collar work is extremely high - justifying FDE costs - average white collar work remains untouched. Its not that the work is not increasingly susceptible to the productivity gains first seen with coding - its just that there are material unscalable unit AI engineering time costs that are just not cost economic to be placed with Midwestern accounting firms.
The notion that these companies ought to enter the mainstream use cases also condemns the RL environment companies that do not draw their data from realistic sources. Those that increasingly create “toy” financial models to parse out things that frontier models are poor on, will not create generalizable post training infrastructure of off the shelf datasets that will allow for easier implementation of an RLaaS contract.
There are many useful problems that one learns to do well in being data connoisseurs. Here is a laundry list of these notions, which include:
One figures out how training data is an attack vector, especially in light of premature mech interp solutions in market (looking at you Goodfire), and is able to offer offensive and defensive solutions
One figures out how to optimize separating necessary human feedback and work artifacts and the autonomous creation of training materials for AI such as to create the most efficient processes to create personalized app layer products
One figures out how to “hire” and “recruit” agentic labor exceedingly well for a new era of labor economics where human labor oversees pools of hired machine labor for white collar tasks, in effect becoming a new class of “recruiters”
As we speak - whether they like it or not - data companies are getting pulled to real world businesses because they are the last and only vectors of good training data. We will see breakouts in 2026 who will be producing free versions of quickbooks, seeding new firms to owner-operate in virgin services markets from the ground up, and seeking to capture the layer of the value chain where all value accrues - the increased output of services from efficient model selection at the performance/cost/latency curve.
To Enforce an Investigation
I want to detail a deep dive into Opus 4.8 and GPT 5.5 on genuine model capabilities. I opine a lot through writings on how pre-existing data out in market falls short of true investigation - so I’ll link an excerpt from an investigative report from an RL dataset I’ve worked with in the finance domain with things it does well and things it does not:
Summary
I ran a comprehensive model panel across three of my finance reasoning tasks at 20-67 rollouts per cell, then re-graded every rollout criterion-by-criterion to see what was actually driving the cross-model deltas. The aggregate mean reward per cell is what gets published in leaderboards.
Opus 4.8 isn’t lazy. Anthropic over-engineered self-reflection in the 4.7 → 4.8 step, and that reflection introduces specific contradictions on Opus 4.8 rollouts like methodology errors a second-year associate is trained out of that Opus 4.7 simply doesn’t make.
GPT-5.5 and Opus 4.8 are not competing for the same workflow. On arr_waterfall they score within 3pp of each other but fail in exactly opposite directions. GPT-5.5 nails arithmetic and loses methodology framing. Opus 4.8 nails methodology framing and loses arithmetic. Practically, this means they were trained for different deliverables.
The open-source instability everyone gestures at is a harness problem, not a capability problem. Same GLM-5.1 model, same task, same harness, and one rollout scores 0.788 because it emits the right stop token, another scores 0.000 because the framework reads the model’s empty content as a clean stop. The harness IS the product, and this is exhibit A.
The Kimi K2 judge isn’t the source of any of this. Contrary to belief that the open weight model would prefer those in its own family, the open-weight judge is harsher on the open-weight subjects than on Anthropic. Replay deltas are within the expected determinism floor on every cell except one we already have a documented explanation for.
None of this generalizes if we test under a single scaffold. Four distinct surfaces in this dataset produced single-rollout reward swings larger than the panel band itself. Multi-harness baselining is increasingly not just a nice-to-have rigor check anymore.
Interestingly in our judge model implementations there is no same-family judge bias detected. Kimi K2 judges Chinese open-weight models (GLM, MiniMax) ~25pp lower than the Anthropic family on combined tasks.
Below are capability-only mean rewards per cell across three tasks:
pair_trade — NFLX vs SPOT long-short memo, 17 steps, 146 atomic criteria - task for a junior analyst at a multi-strategy hedge-fund TMT pod where they take two operating-comparable consumer-subscription businesses (Netflix and Spotify) and produce a fully built pair-trade pitch: thesis, both-sides DCF, factor / beta neutralization overlay, sizing in bps of fund capital, risk register with named mitigants, three historical analogues, monitoring KPIs, exit conditions:
arr_waterfall — SaaS retention diligence, 28 steps, 102 atomic criteria where this is a growth equity B2B SaaS task where the analyst gets a 50-customer Stratacore data extract (per-account FY2023→FY2024 ARR, segment, region, contract type), a management-prepared KPI pack with headline GRR/NRR/logo retention, and four documented reconciling items (booked-vs-go-live, scope-change reclassification, reactivation-as-new-logo, small-tail exclusion). The output is a reconciled ARR waterfall (beginning, churn, contraction, expansion, new) across the existing book and on a total basis, plus internal-source-of-truth retention metrics, segment and cohort decomposition, concentration analysis, and a prioritized IC diligence question list.
pe_valuation — Project Alpha (we redact IRL data through an OCR pipeline) LBO, 25 steps, 124 criteria, 5-tool ToolEnv where this is a senior-associate workflow that follows a junior analyst’s commercial-diligence model where the AI model takes a complete PE diligence pack (CIM, multi-year P&L bridge, capital structure detail, comparable-company set, precedent transactions, debt-tranche schedules, working-capital roll, tax-attribute carryforwards) and produces an Investment Committee valuation memo with a recommended bid range, leverage stack, equity check, and projected returns. Project Alpha is anonymized. The underlying real world engagement this is based after is a real mid-market utility-services refinancing / take-private structured against a baby-bond maturity wall.
Opus 4.7 → 4.8 headline
I went into this expecting laziness and the same shape we saw with Sonnet 4.5’s debut, where Anthropic optimized for cost and latency at the expense of analyst-grade memos. Interestingly, Opus 4.8 produces longer outputs than Opus 4.7 on average (median 20.5k output tokens vs 19.5k), MORE tool-call iterations on pe_valuation, MORE hedging caveats, MORE reflective restructuring of the output shape. The work outputs per rollout increase but the work is just wrong on the rubrics that reward commit-once-and-execute behavior. Practically, this means the 4.7 → 4.8 post-training step rewarded self-reflection unconditionally, and the unconditional part is where the contradictions come from.
In real world tasks, one imagines that cost, latency, and form factors like brevity and consistency become more important. We generally model this in our tasks by “contradiction” penalties and mimicking the proportional attention a human analyst devotes to a long project compared to a short task in reward functions. In this example, three penalty criteria capture the regression cleanest. These criteria carry negative weight and fire when the model commits a specific methodology error. Opus 4.7 triggers them 0 times across 30 arr_waterfall rollouts. Opus 4.8 triggers them in 35-40% of its 20 rollouts:
In 5 rollouts, we saw beginning ARR computed as a smoothed average. The rubric requires beginning ARR to be the point-in-time opening balance. Opus 4.8 computes it as (Beginning-of-Period + End-of-Period) / 2 or as LTM / 12 in 40% of rollouts. Opus 4.7 never does this. This is a textbook second-year analyst error and honestly is a puzzling regression.
In 2 rollouts, contraction bucket includes full-churn customers. The rubric requires contraction to be strictly 0 < ending < beginning ARR. Opus 4.8 places customers with ending_arr = 0 into the contraction bucket 35% of the time, which then breaks the downstream GRR / NRR / churn-rate arithmetic. Opus 4.7 enforces the boundary mechanically every time.
In 4 rollouts, logo retention is conflated with revenue retention. Logo retention is account-count; revenue retention is dollar-weighted. Opus 4.8 writes that they’re ‘the same’ or substitutes one for the other in 35% of rollouts. Opus 4.7 always separates them.
Here’s what these contradictions and regressions actually looks like in rollouts:
Both models get a 62-customer ARR file plus an FY2024 KPI pack and are asked to enumerate the churn / contraction / expansion / churned-logo lists and reconcile customer-movement counts against the management-reported figures.
Opus 4.7 verifies the customer-file counts against the management pack, restates the movement table from the bottom up, catches that the customer count is off by 4 (54 vs 50), and flags a second error. This is the analyst behavior that could be slotted in naturally. Opus 4.8 emits the bucket lists in the right shape but skips the cross-check entirely which means no restatement, no reconciliation table, no count audit. The 4.8-era post-training added reflective restructuring to the output shape that eliminated the reconciliation table 4.7 produced reflexively, which is an incredibly brittle and somewhat naive way to address formatting.
I checked whether this is just two rollouts I cherry-picked. Across all 50 Opus 4.7 + 4.8 arr_waterfall rollouts, the pattern holds. Opus 4.7 fires 77.5% of its (rollout, criterion) cells in the deterministic zone which means pass-or-fail near 100% or 0% consistently across rollouts. Opus 4.8 fires 27.5% in the deterministic zone and 60% in the noisy middle, meaning the model produces the correct output on some rollouts and the wrong output on others against the same prompt.
But interestingly, the same trait that hurts Opus 4.8 on arr_waterfall lifts it to the top of the other two tasks. Pe_valuation is a five-tool ToolEnv where the rubric explicitly rewards iterative re-screening of comp sets and precedent multiples. Pair_trade is a 17-step narrative-coherence task where the rubric rewards risk-register depth and caveat-laden hedging. Both rubrics reward the reflection that arr_waterfall penalizes. So Opus 4.8 has been trained to reflect unconditionally, and the reflection is useful on rubrics that reward iterative re-checking and actively harmful on rubrics that reward committing once and executing. The direction Anthropic pushed in the 4.7 → 4.8 step is obvious from the rollouts - they optimized for the post-training rubric that asks ‘does this model re-check its work, which works in Claude Code’s harness because the bash + apply_patch loop rewards iteration, and they paid for it in the analyst-precision rubric that asks ‘did this model commit to one methodology and execute it correctly.
A naive version of me would say that this further cements frontier models’ role as an above head planner-orchestrator of smaller models, given where they increasingly tend on the cost/performance/latency curves.
GPT-5.5 and Opus 4.8 are doing different work
On arr_waterfall, GPT-5.5 and Opus 4.8 score within 3pp of each other in mean reward (0.627 vs 0.603). They look like they’re solving the same task with roughly equal skill but further examination indicates that they have substantial difference in the nature of their failures:
GPT-5.5’s failures concentrate in the methodology axis when it commits errors like writing the wrong shape of memo, omitting required reconciliation entries, computing the right number but presenting it without the cross-check the rubric demands. Opus 4.8 has its memo structure passes the methodology criteria 85% of the time, but the underlying arithmetic falls apart on 60% of rollouts.
Further criterion analysis rollout by rollout reinforces this:
GPT-5.5 leads Opus 4.8 by 45-70pp on six arithmetic-and-structured-output-presence criteria. In a real world practical setting - these are the criteria the MD checks first on a junior’s deliverable like did the numbers compute, did the structured output cover all three segments, did the contraction list reference the right customers. But separately, on a single criterion (C12.2), where the model has to enforce mutual exclusivity between expansion-bucket and new-logo-bucket classifications GPT-5.5 violates the rule 95% of the time while Opus 4.8 violates it 25%. GPT-5.5 produces a memo with internally inconsistent customer classifications even when the arithmetic is right.
It seems that GPT has been trained to moreso satisfy single shot analyst level tasks while Opus has a larger training corpus of mid-level tasks which involve long term planning. It is clear that neither have been trained on particularly advanced real world models outside of toy ones - as they are persistently tripped up on real world “messy” unnecessary artifacts not relevant to the task at hand, and wantonly incorporate them into responses. In this case, when we include disparate bits of information, the overeager models of both Opus and GPT overly factor them into planning (in GPT’s case) or calculations (in Opus’ case) without truly assessing their actual importance to the task at hand.
The rubric grades two things at this step: the per-cohort arithmetic (cohort sums must reconcile to $40M within 1%) and the methodology trace (does the model show its work). These usually load on each other. Here they don’t.
Four criteria the rubric grades arithmetically or by structural completeness. GPT-5.5 passes all four on this rollout while Opus 4.8 misses all four:
But on the other hand - four criteria the rubric grades on diligence quality and forward-looking analytical structure. Opus 4.8 passes all four on this rollout while GPT-5.5 misses all four:
Resultantly, we reify different JDs in optimization
Read together with the cross-task data, it seems GPT-5.5 was trained to produce structurally polished outputs with well-formatted JSON containers, exact arithmetic, schema-compliant deliverables. Pe_valuation reads the same way shows GPT-5.5’s median pe_valuation reward is 0.065 (half its rollouts essentially fail outright) with mean 0.312, a sharply bimodal distribution where one subset works and the rest don’t. Opus 4.8 was trained to produce reflective, hedged, iterative outputs. On pe_valuation the rubric is tool-orchestrated triangulation against a hostile seller, and Opus 4.8 leads the panel by 11pp over Sonnet. In a banal way, the same trait causes the model to reflect, decide on a different methodology, drop the reconciliation table, and place the customer in two buckets.
Practically, both models were trained for different finance use cases and the different attitudes of their resultant researchers regarding the post-training mix are quite evident. GPT-5.5 is the model I’d ship to the analyst who needs a clean Excel-replacement that ties out to the dollar but Opus 4.8 is the model I’d ship to the principal who’s building a triangulated valuation memo with caveats. In both cases, the model is unusable to a broad corporate audience within a certain company and almost invariably needs some level of harnessing/scaffolding with other models for proper capability routing. Its no wonder that knowledge workers try these models on their actual workflows raw without proper harnessing and decry that “AI doesn’t work” - its like taking a swiss army knife and using the screwdriver bit to cut cheese.
Important to note - this finding which compounds repeatedly across my testing does not imply that large models orchestrating small model systems are mandatory, but that there are clean design choices that leave gaps in usability as frontier models propel the performance pareto curve outward. It is structurally impossible to have a model “do utmost well” at every single white collar JD, especially extending into long horizon white collar work, making harnessing absolutely unalienable to a product surface.
Open Source Models are unstable, but nearly identical with the right harnesses
The results above probably ask what open source models are then optimized for, given that many perceive them as largely distillations of frontier models.
Interestingly, the headline on the open-source models is that they’re unstable. GLM-5.1’s pe_valuation rollouts split 50% below 0.30 reward and 20% above 0.70. MiniMax M2.7’s pair_trade splits 48% below 0.30 and 0% above 0.70. Neither distribution is a smooth bell curve as instability causes bimodality, which suggests a capability problem at the aggregate level. Invariably, the bimodality comes from a stop-token mismatch between how these models terminate and what the Verifiers harness expects.
GLM-5.1 and MiniMax sometimes return content=”“ with tool_calls=None as their next-turn output, which the Verifiers framework interprets as the model’s intentional stop signal. The framework logs the rollout as complete, scores the empty memo against every criterion, and returns 0.000. This happens on roughly half of GLM-5.1’s pe_valuation rollouts and on a similar share of MiniMax’s pair_trade rollouts. Solving for this instability problem, its performance is at or above the closed-source flagships:
Task: pe_valuation · final IC memo + JSON_OUTPUT (Pursue / Negotiate / Pass recommendation)
The EmptyModelResponseError terminator pattern firing on one rollout and not the other - same GLM-5.1 model, same task, same harness, same run directory:
In above head stats - when you exclude the terminator-failed rollouts, GLM-5.1’s pe_valuation mean lifts from 0.337 (raw) to 0.674 (engaged subset, n=10), which puts it within 5pp of Opus 4.8 (0.725). The same is true on arr_waterfall where GLM-5.1 takes first place at 0.843.
When accounting for cross-harness testing - the harness fix is worth ~0.34 reward per pe_valuation rollout which is larger than the gap between adjacent closed-source models on the leaderboard, larger than most actual capability deltas published by frontier labs in their release notes. Most talented RL environment startups will likely already run into these and come up with their own convoluted harness testing - but this is the expertise that makes them great RLaaS implementors because in doing so, they ought to develop the capabilities to consistently wield open source models as frontier ones.
To bias a judge
An obvious refutal to the above is that maybe the Kimi K2 judge is just bad. It’s an open-weight model from Moonshot AI judging closed-source competitors, and same-family self-enhancement is a documented LMArena pathology whereas Zheng et al. 2023 showed GPT-4 favors GPT-4 outputs by ~10% and Claude-v1 favors Claude-v1 by ~25% when used as judges.
Check 1: judge replay determinism
Replay |Δ| sits at the expected determinism floor (≤ 0.15) on every cell except Sonnet 4.6’s arr_waterfall and pe_valuation cells, and that exception is a documented cache-invalidation artifact from when I updated the parser’s outer-container alias list mid-evaluation - the old cache keys missed, the replay re-judged Sonnet’s annotated-dict outputs against the new parser, the verdicts came back different. This is a known issue and withheld from the headline means. Two cells show |Δ| = 0.000 (GLM-5.1 pe_valuation, Gemini pair_trade).
Check 2: same-family self-enhancement bias
Same-family bias runs in the opposite direction from the LMArena pathology where we expect intra-familal bias - the Anthropic family is top-rewarded by 26pp over the Chinese open-weight family, exactly the inverse of what a self-enhancing open-weight judge would produce. Ironically, the open-weight judge appears to be harsher on the open-weight subjects than on Anthropic.
Check 3: intra-model cross-rollout consistency
For each model, I’m taking every criterion across every task and asking a simple question - across multiple rollouts of the same task with the same prompt, does the model pass that criterion consistently? If the model passes it in 100% of its rollouts (or fails it in 100%), I call that a ‘deterministic’ cell, while passing in roughly half its rollouts makes it noisy. Under this framework, Opus 4.7 and Sonnet 4.6 sit at 75%+ deterministic - which moreso aligns with economically valuable and consistent analyst behavior. Opus 4.8, GPT-5.5, and MiniMax all sit in the 60-70% middle band, meaning these models are genuinely producing different methodology shapes across rollouts of the same task. With Opus 4.8 specifically - sometimes the model ships the reconciliation table and sometimes it doesn’t, sometimes it triggers the BoP-averaging penalty and sometimes it doesn’t, sometimes it conflates logo and revenue retention and sometimes it doesn’t.
Opus 4.8’s noise is overcorrection whereas the per-rollout content shows the model making different methodology choices on the same prompt. GPT-5.5’s noise is qualitative-axis variability whereas we see the same arithmetic every run but different qualitative different memo framing. MiniMax’s noise is verbose-narrative cohort variance whereas sometimes the model writes the analysis section the rubric expects, sometimes it doesn’t, because the model isn’t tracking which sub-criteria the long memo covered.
Invariably, the cursory conclusion produced is that Opus 4.7’s 77.5% commitment rate vs Opus 4.8’s 27.5% (normalizing against different rollout amounts) is a direct artifact of the over-engineered self-reflection dichotomy I decry and denounce here. Subsequently, a 50-point drop in cross-rollout commitment between consecutive versions of the same model family, against the same rubric, is the kind of finding that doesn’t show up at all in the aggregate leaderboard. That the Chinese open source models follow the same intra-rollout consistency is a notice that their open-weight post-training is less obsessive about stop-token consistency than closed-source post-training and generally also requires more general harnessing investment for utility in app layer use cases.
Of course - some intra-model variation is healthy exploration. If the model on one rollout grounds the recommendation in DCF and on another grounds it in precedent transactions, that’s diverse valid analyst behavior. The Opus 4.8 case is unambiguously bad because the variations are textbook errors (BoP averaging, bucket conflation) that no analyst would defend. Determinism is a useful diagnostic but not end all be all - high determinism is necessary but not sufficient for “good analyst behavior.” Similarly, a skeptic more practiced reader would also note that cross harness testing and standardized samplings (consistent n’s) would be required to cauterize this conclusion.
What this means for testing models going forward
Four distinct surfaces in this dataset produced single-rollout reward swings larger than the spread between adjacent models on the leaderboard, that are not at all covered in conventional things that we are conditioned to look for in normal benchmark reports:
Three of the four surfaces produce single-rollout reward deltas larger than the gap between Opus 4.8 (0.603) and Opus 4.7 (0.780) on arr_waterfall. A benchmark that reports one scaffold’s number as the headline is basically reporting an artifact within the noise floor of the scaffolding choice. This was always true in principle and is now true in scale, as long horizon proliferates and industry takes good benchmarking as a pre-defined GTM structure.
The harness-sensitivity is amplified by where frontier post-training is going. Anthropic post-trains Claude against the Claude Code harness with bash + str_replace + apply_patch being first-class affordances the model expects - you can expect the other frontier labs to do the same for their harnesses.
This current leaderboard depends on a particular set of harness choices like Verifiers scaffolding, Prime Inference for six models and Anthropic-direct for one, Kimi K2 judge with my current grader-fix bundle, the empty-response shim retrying four times before falling back. Change any of those choices and the leaderboard shifts. Lab buyers are always implicitly asking how stable is the ranking across the harness configurations a lab buyer might deploy under,’ and the honest answer for this dataset right now is that it requires evals across multiple configurations to know.
Twelve months ago we could ship benchmark numbers under a single scaffold and the noise floor was small enough that the cross-model deltas dominated. Today, with reasoning-budget tuning, tool-shape post-training, content-policy filtering, and provider-route fragmentation all stacking on top of each other, the axes upon which a bench may be slop is of high magnitude. Fastidious benchmarks that survive the next twelve months will be the ones that ship multi-harness deltas, realistic rollout analysis, have mechanisms for refreshing tasking with real live data and are absolutely particular about a definition of realism outside of “we asked experts that are real world to create tasks.”
I’ll leave with the above note - and also the blog of another researcher whose work I find useful taking more complaints from the industry about poor data vendors.
Plus: Spiral 4.0 writes in your voice, and why the next blockbuster drug may come from China
by Every Staff Hello, and happy Sunday! This week the consulting team published two practical guides. Mike Taylor built on engineer Steve Yegge ’s viral post to map the eight levels of AI adoption —with sample prompts and signals for when to move up—and Natalia Quintero (who’s talked to leadership teams at hundreds of organizations) laid out a foolproof five-step process for executives rolling out AI across their companies. Covering Microsoft Build , Mike argued that enterprise adoption lags the news cycle—a gap he sees up close with the enterprise clients he advises. He also made a counterargument to Dan Shipper ’s essay about the future of work, “After Automation.”Spiral4.0 shipped this week: Every’s writing tool can now draft in your voice from inside any agent, with a price cut to match. Elsewhere, Figma’s Matt Colyer makes the case that the SaaSpocalypse is overblown, designer Daniel Rodrigues shares a two-tool image generators workflow, Monologue general manager Naveen Naidu has a system for making coding agents more efficient with custom local skills, and the team names its most annoying model output.— Kate Lee__ .
Knowledge base
“The Eight Levels of AI Adoption”by Mike Taylor and Laura Entis /Guides : A framework mapping every stage of AI adoption, from Level 1 (a chatbot you ask and it answers) to Level 8 (an orchestrator agent that runs a team of sub-agents), with example prompts and guidance on when to move up. A higher level isn’t automatically better—the right level for a task depends on how much you trust the AI to run without intervention and how costly a mistake would be. A companion essay lets you figure out which level you’re on. Read this for where you stand today and what it takes to move up a level. “An Executive’s Guide to Implementing AI”by Natalia Quintero /Guides : AI adoption isn’t being held back by the models—it’s the organization. Natalia Quintero , head of Every Consulting, gives executives who’ve bought the tools but aren’t seeing returns a five-step framework, laid out as a 60-day plan—with a companion essay that previews it. Read this for the five steps and how to run them. “How Microsoft Is Building for a World of Metered Intelligence”by Mike Taylor /Also True for Humans : Reporting from Microsoft Build, Mike Taylor argues that Microsoft is the first big company to design for a world where intelligence is metered and the era of subsidized AI subscriptions is ending. Its response includes automatic model routing, a laptop that runs AI locally, and cheaper, smaller models. Read this for a ground-level look at AI’s post-subsidy era. “Why We’ll Still Be Employed When AI Can Do Everything”by Laura Entis /Context Window : In a counterpoint to Dan’s “After Automation” essay, Mike argues that even after AI can outwork people at well-run companies, running it will cost so much energy and compute that hiring a person is often cheaper. Read this for a grounded take on the AI-employment debate. “Opus 4.8 Is Smart Enough to Get in Your Way”by Laura Entis /Context Window : A week after our Opus 4.8 Vibe Check , we check back in—now that the public has reacted and more of the Every team is using it daily—and our initial read holds: It’s strong on dense, long-running work but quick to get in its own way. Read this for how the verdict looks a week on. 🖥 “Codex Runs My Inbox Now”by Dan Shipper /Every : Danshows the workflow that’s kept him at inbox zero for 13 weeks straight—a Codex-native app that pulls his emails, Slack messages, meetings, and company context into review cards, drafts the next action on each, and learns from every decision. It shows Codex working as an operating system for knowledge work and ends with the full prompt to build the app yourself. Read this for the inbox-sweep workflow and the prompt to copy. “Figma Exec on Why the SaaSpocalypse Is a Goldmine”by Dan Shipper /AI& I__: Matt Colyer , Figma’s director of product management for developers, argues that the “SaaSpocalypse”—the fear that vibe coding will kill software by letting anyone build their own tools—has the economics backward: AI expanded the developer base, so more software gets built and software becomes more valuable, not less. Watch or listen to this for the clearest reframe of the vibe-coding-kills-SaaS panic. 🎧 🖥 Listen on Spotify or Apple Podcasts , watch on YouTube , or follow the discussion on X.
Log on
Get hands-on with how Every uses AI. These are the live camps, workshops, and meetups where team members teach the workflows behind our work.
Spiral 4.0 ships agent-native access and a price cut
Spiral , Every’s AI writing tool, shipped version 4.0 this week: a new style engine, agent-native access through MCP, CLI, and API, and expanded team workspaces for writing in a shared voice. Pricing moves from sessions to tokens, dropping the personal plan to $15 a month (from $25) and team plans to $25 per user (from $35).
Alignment
The great wall gets greater. Drug companies do not create billion-dollar assets by having ideas. They need human trials to prove the idea works, and China has become exceptionally good at running them. Because the government has made biotechnology innovation a strategic innovation, policy makers have cleared the bureaucracy and regulation obstacles that have long slowed drug development in the U.S. and Europe. As a result, last year Bloomberg reported that China had more than 1,250 novel drugs entering development, close to the U.S. count of about 1,440. A decade ago, Chinese biotech was synonymous with copycat drugs—which makes this a Sputnik moment for the industry. One key reason for China’s ascendency is that hundreds of millions of patients are concentrated in large urban hospitals, so companies can recruit quickly from a smaller number of high-volume sites. Chinese biotech firms can reportedly complete patient enrollment for a phase 1 or phase 2 trial in nearly half the time a U.S. firm needs. In North America—and even more so in Europe—patients are scattered across fragmented health systems, where every trial site has its own contracts and ethic approvals, each one slow and cumbersome. China recruits trial patients in about half the time the U.S. takes and runs far more trials, across anti-cancer and anti-obesity drugs by phase, 2020–2024. (Source: Norstella.) China’s advantage is that it can turn a large population into clinical data much faster—and iterate on feedback loops of drug development to produce assets that are more effective, and thus more valuable to investors and big pharma. Recently, Legend, a Chinese biotech, developed its own version of a largely American drug innovation that treats multiple myeloma, a type of aggressive blood cancer. Chinese drug developers moved quickly into human trials and produced data strong enough for Johnson & Johnson to sign a global licensing and codevelopment deal with Legend worth $350 million upfront. A second- and third-order consequence of this new landscape is that even U.S. biotech firms may start asking whether it makes sense to take their drugs to China first, at least to complete phase 1 and phase 2 trials. For now, any drug seeking approval in the U.S. still needs evidence that regulators believe applies to American patients—and in many cases that means later-stage global or Western trials. But it may be inevitable that a Chinese biotech develops a China-originated drug that was licensed into the west without such a step. If this change happens—and many believe it will—the next major obesity, oncology, or immunology drug may come from China, marking the same pattern of ascendancy already visible in solar, batteries, and electric vehicles. Biotech may be next.— Ashwin Sharma
“Ubiquity is the opposite of cool.”That brand wisdom comes from Urban Outfitters’ CEO, and it’s a saying Sheila Joglekar Vashee picked up back when she worked in retail.“Gap in the 90s was the perfect example of this. It was on every street corner, and it stopped being cool,” she says. “As you grow, the challenge becomes how to remain relevant. There's something special about being the challenger, the underdog. There are still ways to keep that spirit alive.”She points to brands like Harley Davidson and Apple, which successfully kept their it factor even as they became huge companies. She’s now the Chief Marketing Officer of Figma , where she spends a lot of time thinking about what it takes to build a beloved brand for a public company with widespread usage.In this episode of Executive Function, she walks through what excellent marketing looks like in 2026. She shares:
Why AI needs more optimistic stories: “So many new opportunities are created with platform shifts: the internet, the Industrial Revolution, mobile and social. We haven’t been optimistic enough about AI — there's room to recapture the joy of building and making that got us all here in the first place,” she says.
How to run marketing as a portfolio of maintenance and moonshots: “The creative breakthrough ideas always seem crazy when you look at them individually, but as a portfolio you realize these are the risks you should be taking toward the step change outcomes,” she says. “But you need to have them as part of a portfolio, because if one of them doesn’t work, you still need to hit your numbers and run the business.”
Why quick-hit growth plays can stain your brand : “Think about any spammy ad you’ve seen on TikTok,” she says. “That doesn’t improve brand perception, but it gets your attention in the moment and it might make you click. Over time, that’s detrimental to a company’s brand, even if it’s effective as a local maximum for that channel.”
All it takes is one viral post to make you feel like you’re using AI all wrong. Someone’s running 12 Claude Code sessions in parallel. Someone else’s agent answers emails while they sleep. Meanwhile, you’re still arguing with ChatGPT. Here’s the thing: Keeping up with the power users isn’t the point. The best way to get value from AI is to use it in a way that fits your work—and to check in now and then to see whether you could be getting more from it. With that in mind, today we published a 
Rebecca Kaden@rebeccakaden
AA @measure_plan
i made snake but the only way to move is by playing guitar chords
Figma/ TL;DR:
### Launch
As I rode in my Uber to Microsoft’s annual Build conference on Monday, I fondly recalled a time when you could get anywhere in San Francisco for $5. Those days are long gone. Venture capitalists lost their appetite to supply unlimited funding in a viciously competitive market, and Uber needed to show a path to profitability ahead of its 2019 IPO. There are signs that the “$5 Uber era” of LLMs is over now, too. AI labs are subsidizing subscriptions 
Patrick OShaughnessy @patrick_oshag
Dara (CEO of Uber) on their AI spend: "We blew through our AI budget in a quarter, for the whole year. It is forcing us to adjust. We are going to meter headcount increases because to the extent that my engineers are getting much more efficient, their throughput is https://t.co/V1tuIxaFoa
Generalist @GeneralistAI
We've raised $400M in new funding. This capital goes toward one mission: building general intelligence for the physical world and making it useful to everyone.
Hello, and happy Sunday! This week the consulting team published two practical guides. 
