Three Generations Down The Line

The first time you see a new technology, it often looks worse than the thing it's meant to replace. But what you're looking at isn't really the technology, but a single frame of a process that's still running. The previous baseline generation has nearly finished improving; the next generation has barely started. Comparing them as they stand tells you almost nothing, because the gap you're measuring is mostly a gap in time, not in potential. The question isn't how good it is now; it's how fast it's improving.

So before judging anything new, work out where it sits in its life cycle, then think about how it might perform a few generations down the line .

A useful rule of thumb follows. If something brand new is already roughly on par with something that has existed for a long time, that's worth paying attention to. The old thing likely took decades and a lot of effort to reach that level, while the new thing got there quickly, before much optimization had even started. Parity at the first generation usually means there's a lot of room left to grow.

I saw this at Meta. I made a lot of money for the company on the back of a model that was quite simple, essentially a version of a neural net, but the product idea it encoded was novel, and that was the point. Each improvement to the model produced a sharper rendition of that idea, and that translated into more revenue. The value wasn't in the model itself; it was in how much headroom the idea still had.