Brickipedia
Brickipedia
avatar 54.159.14.44

Brickipedia:Proposal to move Brickipedia to ShoutWiki/The Move

From Brickipedia, the LEGO Wiki

Process[edit]

The following section is an unfinalized version of the planned move process.

  • Read-only mode
    Prevents data loss during migration caused by new data being put in an already moved chunk on old server.
  • Move images
    We'll do this first as there's a lot of stuff.
  • Stop PHP to not get OOMs when dumping tables.
    Server throws OOMs regardless if PHP is still running. Can't dump databases if the servers not up.
  • Dump directly to ShoutWiki server using MySQL dump.
  • Put read-only site back up (start PHP)
  • Import dumps on ShoutWiki
  • Uncomment things so we've got all the tables in use.
  • Run the migration script
  • Update.php
  • Switch Varnish backend for Brickimedia to point to Bellow.
  • Clean remaining data off Brickimedia server and decommission it.

Updates[edit]

27th Feb[edit]

We'd have like to have moved Brickimedia in already, but it's been an insanely busy month for the team. Unfortunately, ShoutWiki is nobody's day job. I've been informed by Ajraddatz that the current server has been renewed for a month, which throws us a lifeline we needed as our working plan to get the move done today is falling behind schedule.

The DNS is now managed to us, and all site traffic will begin to go through our frontend cache as opposed to CloudFlare as soon as the DNS changes propagate. This will allow to switch the Brickimedia traffic straight to the ShoutWiki server when the time comes for everyone within the matter of milliseconds.

2nd March[edit]

We've started to conduct testing on our scripts to combine the user databases.

Please ensure that if you currently have an account at ShoutWiki.com that you change your email addresses in your preferences on both Brickimedia and ShoutWiki so that they match. This will help us combine accounts and attribute edits and revisions correctly.

31st March[edit]

Having completed some successful tests, we've worked out a strategy for moving the files from meta so that can be closed down. We'll make the move in a brief period (up to an hour, should be a lot shorter but depends on processing time) over the weekend, and then files will work in a similar way to InstantCommons - as in GBC can use images on Bricki but upload locally. We've done it this way as there are far more Bricki images on Meta than there are for GBC.

We can use the same method at a later date to return any and all of GBCs images to their site if they can provide us a list of this, as long as they don't upload a local copy. There's no easy way to pull a list of images they use at current.

2nd April[edit]

So having worked on this all day, it's now done as far as I can tell. Took a lot longer than expected. An underestimated was made on how slow the File: page import would be on Brickimedia's horrible little server.

In lieu of a full explanation, the wikis now have their own upload bases. Brickimedia got all the images, as 99% belong here. Meta is on its way out, it's been left up with uploads disabled so that people can pull content they want over (or build a list and Lcawte will do it).

GBC has access to Bricki's images like wikis use InstantCommons. Images exclusively for GBC can be moved back providing they are not reuploaded locally. Please export and import with full history, but please avoid overwriting existing local content. Please build a list and submit it to Lcawte.

A few botched Meta revisions and pages have made their way into Brickipedia due to script problems. Please clean those up where possible. Due to Meta's status as a co-ordination/image wiki, there should not be too many conflicts.

Each wiki now has it's own avatar directory, however, existing avatars have been copied to work on both sites.

Config changes have been made locally, and have not yet been committed back to the git repository.

Please report any issues to Lcawte immediately, preferably by email to lcawte (at) shoutwiki.com.

Comments[edit]

  • I changed my email on ShoutWiki to match the email here and am still waiting on a confirmation email to be sent. --ToaMeiko (talk) 17:43, 6 March 2017 (UTC)
    • Apparently my reply didn't save... it was something like this - Request it resent (Special:Prefs). When we fixed Bricki's email (I mean, within reason, it still seems to struggle on the server) we fixed a number of things, including a few we spotted at ShoutWiki. Google seems to be messing about with its email acceptance algorithms again so a lot of mail was being rejected. This should all be fixed now on ShoutWiki. --Lewis Cawte (talk) 09:43, 9 March 2017 (UTC)
  • Hi, just wondering if there's a vague estimate as to when we might be moving? From what I understand, the extra month with CloudFlare's just about up, so was just curious about what was happening since there hasn't been an update in the last few weeks :S NovaHawk 09:55, 27 March 2017 (UTC)
    • RamNode* I've emailed Adrian asking him to renew it. I've been rather busy at work this month, but we believe we've ironed out all the problems with the script. We've now looked back through our old notes and realised we've got to sort the Meta/Images mess out before we do the move because otherwise we'll be dealing with a whole mess of Bricki accounts and ShoutWiki accounts. Unfortunately, the way images are currently set up isn't the easiest of things to reverse. With help from the developer community, we think we've worked out how to do it, but I need time to test it. I'll also need to take the site down again (because there's no memory for MySQL to give us the data currently) so I can grab the relevant backups I need to test that. I'm free today so I'll try and start that today. --Lewis Cawte (talk) 10:18, 27 March 2017 (UTC)
      • Thanks for the updates Lcawte :) (I don't know how I missed this first one since there's hardly anything happening in the RC...) With an import happening over the weekend are there any times we should't be uploading images or editing? NovaHawk 22:59, 31 March 2017 (UTC)
        • When I'm doing any sort of migration, you won't be able to do any uploading or editing (whichever is relevant). There are MediaWiki variables which stop these things, because you can't help spambots not following the rules and ruining things. These should be announced in advance, as far as I'm aware, the meta close down and the actual migration should be the only times. You can carry on editing where that's technically possible. --Lewis Cawte (talk) 23:44, 31 March 2017 (UTC)
  • How do we plan to reconfigure things like the abuse filter and interwiki table? 174.192.25.86 21:14, 5 April 2017 (UTC)
    • Who do you mean by we? ShoutWiki supports a hybrid of local (editable by ... well, local admins) and global interwiki prefixes (editable by staff). Your local prefixes will be maintained, as will your abuse filters. --Lewis Cawte (talk) 21:51, 5 April 2017 (UTC)
  • So... it's been another month, any updates? NovaHawk 13:32, 30 April 2017 (UTC)
    • @Lcawte: ^ 24.39.1.139 16:04, 8 May 2017 (UTC)
      • So April was a pretty terrible month. Without going into details, there've been many personal issues that our team members have had to deal with. In addition to that, the directors (myself and Cook879) have had to file our first-year company accounts because being landed with a disqualification, prison sentence and/or big fines is not helpful to ShoutWiki in the long term. Saying that, we're a lot closer with the script. I don't have an abundance of free time this week, but I've been working on ShoutWiki and the Brickimedia migration again a little this week which is a step in the right direction. I've got four days free next week, and while I'm not promising anything, I'm somewhat confident there should finally be some news. Should know more in the next day or two.
      • As an additional incentive to make us hurry up, this is the second month where we're now reimbursing Ajraddatz for this horrific VPS, which is made even better by the terrible exchange rates we're getting. --Lewis Cawte (talk) 09:19, 11 May 2017 (UTC)