GitHub Copilot AI token charges go up 10×–100×

[ghostonthehalfshell@masto.ai]

@davidgerard

The screenshots you had the price differentials are eye watering.

I can’t wait for the other companies to follow suit. They probably won’t because these price changes would melt down most us usage.

The upside is they wouldn’t be burning token so much

[koos@mastodon.green]

@davidgerard

“My company is large. We were using GitHub copilot with several thousand users. With the change to token fees we are currently considering dropping ai agentic coding as a whole until at least the market stabilizes.”

Whyyy do such people get to run companies?! It’s always been public knowledge that LLM SaaS runs at a loss

[david_chisnall@infosec.exchange]

@davidgerard

The fun thing here is that training costs are very large, but mostly fixed NRE, whereas inference is OpEx that scales with userbase size. If you keep the prices low, you get more customers and so can amortise the NRE, but you can’t cover your OpEx, so you make a loss. If you put the prices up, you can cover the OpEx, but it pushes customers away and so reduces the amortisation of the NRE and so you can no longer cover it and make a loss.

Successful products have either low NRE or low OpEx to deliver. High NRE and low OpEx is ideal because it’s hard for competitors to enter the market but the early players can deliver cheaply. Low NRE and moderately high OpEx can work as long as the OpEx is less than customers are willing to pay, but usually precludes mass-market products unless economies of scale let you reduce the OpEx.

About the only places you find products with both high NRE and high OpEx are in the healthcare and defence sectors where demand is inelastic.

[davidgerard@circumstances.run]

@david_chisnall even there, the catch is training has to be ongoing too in practice

[david_chisnall@infosec.exchange]

@davidgerard Right, so you need a constant supply of customers to amortise it.

If it were a one-off, it wouldn't matter so much because you'll amortise it eventually.

This is what frustrates me about people to talk about using local models as if they're a viable alternative. Who do they think is going to pay to train an LLM if they can't get revenue from inference? And who wants to use, say, a coding LLM that was trained on an old language standard and a load of deprecated APIs?

[dysfun@social.treehouse.systems]

@david_chisnall @davidgerard i'm almost 2 weeks into trying to make a local model do the silly thing and i am very very unimpressed.

[etchedpixels@mastodon.social]

@koos @davidgerard It's also very normal for any other business to run at a loss whilst refining the processes, improving the technology and scaling.

So for senior mismanagement the fact XYZ is still loss making isn't often a concern so long as there's name and money behind it to assure people that it won't go pop (or you have a backup plan). They'd have to look a lot further behind the curtain to realize.

[david_chisnall@infosec.exchange]

@Gaelan @davidgerard

I think, if LLMs stop being trained, you'll see a short-term split in the industry: those that continue to evolve their languages, frameworks, and other tools and those that end up stuck on old versions. And then we can invert the 'left behind' narrative.

[davidgerard@circumstances.run]

@Gaelan @david_chisnall the eternal death of python 2.7

[davidgerard@circumstances.run]

@etchedpixels @koos yeah this github copilot price rise is a bit of a wakeup call

[bms48@mastodon.social]

@davidgerard @Gaelan @david_chisnall It is actually still possible to install Python 2.7 if you fetch the right .debs on certain platforms. Not that I'd recommend that in any way. I nearly had to resort to it for some things. For more trivial things like GraphML conversion, I just hand-ported the script: https://git.sr.ht/~bms/dottoxml

[bms48@mastodon.social]

@david_chisnall @Gaelan @davidgerard As far as LLM training goes: "Cripple the motherf**er before it gets *heavy*." -- RA Wilson, "The Illuminatus! Trilogy!" (Fancy naming one of your characters "Markoff Chaney" way back then...) My GitHub status is now malicious compliace only due to the network effect, pretty much. Cloudflare's Content-Signal for robots.txt is a bit weak as I found out yesterday, and German court action with the EU Digital Single Market Directive makes it explicit opt-out.

[grayrattus@mastodon.social]

@davidgerard "Embrace, extend, and extinguish"

https://en.wikipedia.org/wiki/Embrace%2C_extend%2C_and_extinguish