User:Cook Me Plox/Exchange ramblings

Goal: get number of pages referencing GEPrices/data or GEVolumes/data below 1000

Why?

• 80-90% of total CPU time, purges, activity, etc on the wiki is still from Gaz GEBot aftermath. This is unique(ly bad) among wikis.
• We purge/re-index ~50k articles a day because that many articles use a GE price somewhere.
• In theory we could use this to show realtime GE prices (via RuneLite)

Methods:

• Cloudflare Workers post-processing on MediaWiki's generated HTML, combined with a JSON of prices that the worker will have, means we can insert prices (most of the time) after MediaWiki generates the content, which means we can remove the dependency on GEPrices/data most of the time, which means it won't trigger anything when Gaz GEBot updates the module.
  • Example:
    • You write `return exchange._fastprice("Rune scimitar")`
    • MediaWiki produces `<span class="fastge" data-formula="{price|Rune scimitar}"></span>`
    • Cloudflare sees this and looks up the price, and replaces the whole thing with `21513`
• More use of specialized bulk data modules for individual things we care about (Profitable alchemy, etc), updated by Gaz GEBot after a GE run

Does Lua have a way to create objects and override + - * / etc? Same way that in Python you can override __mul__ etc

TODO: look more at GEPrices/data usage

Probably doable:

• Infobox Item price
• Infobox Item volume
• DropsLine price
• Recipe price
• Coins template
• Exchange price, lastprice, volume, direction

Not feasible:

• Categories based on GE prices (profitable alchemy, etc) - maybe better handled by loading a bulk module anyway?
• One-off things that have distinct logic in a module depending on the GE price or a value computed from it partially. Exceptions for things like {{Coins}} which are so ubiquitous that we'd implement it from scratch in a worker

exchange._fastprice(item):

• Lua object that handles arithmetic operations and essentially combines them into a span with a data string when tostring() is called, e.g.

`tostring(_fastprice("Rune scimitar") * 20 + 1000000)` = <span class="fastge" data-formula="{price|Rune scimitar} * 20 + 1000000"></span>

Which would get replaced by Cloudflare as (e.g.) `1430260`

{{FastGEP|item}}: calls exchange._fastprice(item)

Dilemma: how to handle `{{Coins|{{FastGEP|item}} + 10}}`? Or any other case where a template (Recipe, Coins, etc) takes in an input directly that uses {{GEP}} or equivalent

• Have {{Coins}} and other callers handle it? Can the caller even see the tags or will they be interpreted as UNIQ...?
• Have Cloudflare handle it? May require some recursive handling, might need to put data in innerHTML rather than `data` attrs (maybe even do the lexing this way?)

Another dilemma: calculator responses?

• Run action=parse responses through the worker too?
• Somehow make all calculator templates not use this? Fine from a performance standpoint because these aren't "pages" that need to get purged

High level question: is this a good idea?

• "Yes" camp:
  • It's insane how much of our resources are tied up in updating all the pages due to GE price changes
  • Refactoring this away from MediaWiki (as much as possible) would make things way faster for users, reduce costs (probably by half or more), make other things in the job queue go faster
  • Could be the first step in doing some really cool stuff with edge-side includes
• "No" camp:
  • Significantly added complexity for something that is just "nice to have"
  • Would need to reimplement a decent amount of behavior ({{Coins}} etc)
  • Although we could do a decent job of making Cloudflare worker scripts more public, they are still going to be pretty opaque/black magic to most people
  • There will still be other things (like one-off edits to Module:Exchange) that will incur similar costs. We need to have the scale to handle those, regardless