Nathan Labenz on AI pricing

I won’t double indent, these are all his words:

“I agree with your general take on pricing and expect prices to continue to fall, ultimately approaching marginal costs for common use cases over the next couple years.

A few recent data points to establish the trend, and why we should expect it to continue for at least a couple years…

  • StabilityAI has recently reduced prices on Stable Diffusion down to a base of $0.002 / image – now you get 500 images / dollar.  This is a >90% reduction from OpenAI’s original DALLE2 pricing.

Looking ahead…

  • the CarperAI “Open Instruct” project – also affiliated with (part of?) StabilityAI, aims to match OpenAI’s current production models with an open source model, expected in 2023
  • 8-bit and maybe even 4-bit inference – simply by rounding weights off to fewer significant digits, you save memory requirements and inference compute costs with minimal performance loss
  • mixture of experts techniques – another take on sparsity, allows you to compute only certain dedicated sub-blocks of the overall network, improving speed and cost
  • distillation – a technique by which larger, more capable models can be used to train smaller models to similar performance within certain domains – Replit has a great writeup on how they created their first release codegen model in just a few weeks this way!

And this is all assuming that the weights from a leading model never leak – that would be another way things could quickly get much cheaper… ”

TC again: All worth a ponder, I do not have personal views on these specific issues, of course we will see.  And here is Nathan on Twitter.



Add Comment