Forum:PSA: Grand Exchange changes

From Meta Wiki
Jump to navigation Jump to search
Forums: Index > PSA: Grand Exchange changes
Archive
 This page or section is an archive.
Please do not edit the contents of this page.
This thread was archived on 13 December 2020 by Cook Me Plox.

Hi everyone!

Gaz, Jayden, Kitty and I are in the process of making some changes to how the wikis handle Grand Exchange prices. Please read User:Cook Me Plox/Exchange proposal for the full set of things taking place, but the overview is:

  • We are moving historical data (used for charts) off the wiki to api.weirdgloop.org, and changing the charts to hit this instead of transcluding the data on the wiki page.
  • We are removing Grand Exchange prices, last prices and volumes from the Module:Exchange/% pages, and making it so we only access them from bulk modules.
  • After this is all done, we will be permanently deleting old revisions of Exchange-related pages if they're part of a consecutive string of only bot price updates. (These will be backed up to cold storage, but not stored in the database any more.)

Again, read the proposal for the full plan and reasons why, but here are the main benefits:

  • We get to make GazBot only do ~12 edits per day instead of 18,000, which has huge performance impact for the job queue, webservers, and repeated caching.
  • Currently historical prices are hardcoded in the HTML of item pages, which can often be 70% or more of the total page size (seriously, take a look at the HTML of https://runescape.wiki/w/Prayer_potion). This will remove that, and decrease the amount of total prices as well.
  • We get to delete roughly 73% of all revisions on the site, including 66% of RSW, 84% of OSRS, and 95% of pt-RSW. This has a huge impact on the portability of the database, both for migrations and possible future ephemeral versions of the wiki.
  • We get exchange data on pt-RSW basically for free, instead of needing to sign up for another 12,000 edits a day.
  • Third-party scrapers will be able to use the new API (which is heavily cached) instead of hitting our wiki web servers.

Some small downsides:

  • Historical data will no longer be interactable without some way to fetch the data (mainly JavaScript). That means templates that use that data directly on-wiki (rsw:Template:GEPHist) will have to break. However this was only used on one mainspace page.

We are going forward with this plan imminently unless people have major concerns. Thanks! ʞooɔ 23:15, 26 November 2020 (UTC)

Discussion[edit source]

Support - Riblet15 (talk) 03:59, 27 November 2020 (UTC)

Support - yahoooooooooo - Towelcat (talk) 04:01, 27 November 2020 (UTC)

Comment - "Third-party scrapers will be able to use the new API (which is heavily cached) instead of hitting our wiki web servers." Does this caching mean prices provided by the wiki could be outdated? Or would it still be about the same as before where they're updated roughly every day? zTUG5mD.png Crow  12:07, 27 November 2020 (UTC)

The cache will be purged whenever an update happens. ʞooɔ 12:12, 27 November 2020 (UTC)

Will the impact to the job queue mean faster updating for other things such as WLH during when Gaz Bot would have normally been running? zTUG5mD.png Crow  12:07, 27 November 2020 (UTC)

Maybe. It certainly won't hurt. ʞooɔ 12:12, 27 November 2020 (UTC)

Support - So you're saying we get to hit milestone total edits again? Badassiel 15:29, 27 November 2020 (UTC)

Depends on if you're counting by max revision number, or total revision count. I think the magic words use max, but Special:Statistics uses count. ʞooɔ 18:49, 27 November 2020 (UTC)

Comment - Thanks for letting us know. --Legaia 2 Pla ᴛ · ʟ · ᴄ 20:57, 28 November 2020 (UTC)

Question - What do you mean by "possible future ephemeral versions of the wiki"? Would that be like if we spun up a mirror of the wiki to prep for a major update we have prior knowledge of? Or are you referring to something else? (Sorry if ephemeral has a tech meaning I'm unaware of.) BigDiesel2m (talk) 22:52, 28 November 2020 (UTC)

Yeah, this would be sandbox versions of the wiki, either for embargoed update prep or technical testing. There may be situations where it's easier to have a sandbox wiki than sandboxing large parts of it. ʞooɔ 05:22, 6 December 2020 (UTC)

Closed - This has been launched, and announced in a wider setting. Previous revisions have been deleted. ʞooɔ 12:26, 13 December 2020 (UTC)

Retrieved from ‘https://meta.weirdgloop.org/w/Forum:PSA:_Grand_Exchange_changes?oldid=5004’