Comments - Clouded Judgement 2.20.26 - The SSD / Memory Reckoning
Clouded Judgement
SubscribeSign in
Clouded Judgement 2.20.26 - The SSD / Memory…
Every week I’ll provide updates on the latest trends in cloud software companies. Read →
3 Comments
Top first
I don’t buy the argument that AI requires access to all stages of data storage for inference. I can understand why this would be required for training but the point of inference is to act on the synthesised learning from all the tiers and then apply that to the current problem being solved.
The problem is more likely that while training you need fast access to the entire data collection across all tiers and there’s not enough slack to continually move data from cold and backup storage to the fast storage without adding a huge amount more of NAND.
Unless you’re saying that enterprises lack the ability to train (or fine tune) and therefore they are are largely running generic pre trained models without that learning having happened and therefore need access to all data while inferencing as the models they are reliant on have never encountered the data stuck in cold storage or backup.
That is likely the case but the right solution is to fine tune or train models first (and once) rather than constantly access expensive storage forever.
Reply
Great article. What other constraints exist in the infra layer Main Street isn’t planning for?
Reply
Storage: It seems that every generational compute change involves a storage adjustment due to scale and scope. Mainframes totally controlled storage, which was both scarce and captive to the box. Minicomputers did the same. With the distributed processing of PCs, storage broke out. You had both local and departmental networked storage. The scale and scope of the internet pushed storage even further out to an independent layer for broader scale and access. AI being one more step, requires even broader scale and more random access than cloud workloads. It seems that every compute step breaks the old storage model in order to feed a more abstract and sophisticated task. This next step is exactly what history would predict.
Reply