Brickipedia:Proposal to move Brickipedia to ShoutWiki/The Move

Process

The following section is an unfinalized version of the planned move process.

• Read-only mode

  Prevents data loss during migration caused by new data being put in an already moved chunk on old server.

• Move images

  We'll do this first as there's a lot of stuff.

• Stop PHP to not get OOMs when dumping tables.

  Server throws OOMs regardless if PHP is still running. Can't dump databases if the servers not up.

• Dump directly to ShoutWiki server using MySQL dump.
• Put read-only site back up (start PHP)
• Import dumps on ShoutWiki
• Uncomment things so we've got all the tables in use.
• Run the migration script
• Update.php
• Switch Varnish backend for Brickimedia to point to Bellow.
• Clean remaining data off Brickimedia server and decommission it.

Updates

27th Feb

We'd have like to have moved Brickimedia in already, but it's been an insanely busy month for the team. Unfortunately, ShoutWiki is nobody's day job. I've been informed by Ajraddatz that the current server has been renewed for a month, which throws us a lifeline we needed as our working plan to get the move done today is falling behind schedule.

The DNS is now managed to us, and all site traffic will begin to go through our frontend cache as opposed to CloudFlare as soon as the DNS changes propagate. This will allow to switch the Brickimedia traffic straight to the ShoutWiki server when the time comes for everyone within the matter of milliseconds.

2nd March

We've started to conduct testing on our scripts to combine the user databases.

Please ensure that if you currently have an account at ShoutWiki.com that you change your email addresses in your preferences on both Brickimedia and ShoutWiki so that they match. This will help us combine accounts and attribute edits and revisions correctly.

31st March

Having completed some successful tests, we've worked out a strategy for moving the files from meta so that can be closed down. We'll make the move in a brief period (up to an hour, should be a lot shorter but depends on processing time) over the weekend, and then files will work in a similar way to InstantCommons - as in GBC can use images on Bricki but upload locally. We've done it this way as there are far more Bricki images on Meta than there are for GBC.

We can use the same method at a later date to return any and all of GBCs images to their site if they can provide us a list of this, as long as they don't upload a local copy. There's no easy way to pull a list of images they use at current.

2nd April

So having worked on this all day, it's now done as far as I can tell. Took a lot longer than expected. An underestimated was made on how slow the File: page import would be on Brickimedia's horrible little server.

In lieu of a full explanation, the wikis now have their own upload bases. Brickimedia got all the images, as 99% belong here. Meta is on its way out, it's been left up with uploads disabled so that people can pull content they want over (or build a list and Lcawte will do it).

GBC has access to Bricki's images like wikis use InstantCommons. Images exclusively for GBC can be moved back providing they are not reuploaded locally. Please export and import with full history, but please avoid overwriting existing local content. Please build a list and submit it to Lcawte.

A few botched Meta revisions and pages have made their way into Brickipedia due to script problems. Please clean those up where possible. Due to Meta's status as a co-ordination/image wiki, there should not be too many conflicts.

Each wiki now has it's own avatar directory, however, existing avatars have been copied to work on both sites.

Config changes have been made locally, and have not yet been committed back to the git repository.

Please report any issues to Lcawte immediately, preferably by email to lcawte (at) shoutwiki.com.

30th May

We'll be putting the site in to read-only mode this evening and taking it down temporarily to allow us to produce a backup of the databases. We're going to give the move a go - we've done as much testing as we can, and now we'll give it a shot. The site will remain in read-only the entire time because otherwise they'll be data loss between the two copies. If all goes well, you'll be logging in to an account on ShoutWiki.com shortly, and the slight change you may notice is a bump to MediaWiki 1.28.2 (it says one, but there was no patch for the version number) and a Powered by ShoutWiki button in the footer. If not, we'll just re-enable editing here, and we'll try again after some further debugging/testing/programming.

Comments

• I changed my email on ShoutWiki to match the email here and am still waiting on a confirmation email to be sent. --ToaMeiko (talk) 17:43, 6 March 2017 (UTC)
  • Apparently my reply didn't save... it was something like this - Request it resent (Special:Prefs). When we fixed Bricki's email (I mean, within reason, it still seems to struggle on the server) we fixed a number of things, including a few we spotted at ShoutWiki. Google seems to be messing about with its email acceptance algorithms again so a lot of mail was being rejected. This should all be fixed now on ShoutWiki. --Lewis Cawte (talk) 09:43, 9 March 2017 (UTC)
• Hi, just wondering if there's a vague estimate as to when we might be moving? From what I understand, the extra month with CloudFlare's just about up, so was just curious about what was happening since there hasn't been an update in the last few weeks :S NovaHawk 09:55, 27 March 2017 (UTC)
  • RamNode* I've emailed Adrian asking him to renew it. I've been rather busy at work this month, but we believe we've ironed out all the problems with the script. We've now looked back through our old notes and realised we've got to sort the Meta/Images mess out before we do the move because otherwise we'll be dealing with a whole mess of Bricki accounts and ShoutWiki accounts. Unfortunately, the way images are currently set up isn't the easiest of things to reverse. With help from the developer community, we think we've worked out how to do it, but I need time to test it. I'll also need to take the site down again (because there's no memory for MySQL to give us the data currently) so I can grab the relevant backups I need to test that. I'm free today so I'll try and start that today. --Lewis Cawte (talk) 10:18, 27 March 2017 (UTC)
    • Thanks for the updates Lcawte :) (I don't know how I missed this first one since there's hardly anything happening in the RC...) With an import happening over the weekend are there any times we should't be uploading images or editing? NovaHawk 22:59, 31 March 2017 (UTC)
      • When I'm doing any sort of migration, you won't be able to do any uploading or editing (whichever is relevant). There are MediaWiki variables which stop these things, because you can't help spambots not following the rules and ruining things. These should be announced in advance, as far as I'm aware, the meta close down and the actual migration should be the only times. You can carry on editing where that's technically possible. --Lewis Cawte (talk) 23:44, 31 March 2017 (UTC)
• How do we plan to reconfigure things like the abuse filter and interwiki table? 174.192.25.86 21:14, 5 April 2017 (UTC)
  • Who do you mean by we? ShoutWiki supports a hybrid of local (editable by ... well, local admins) and global interwiki prefixes (editable by staff). Your local prefixes will be maintained, as will your abuse filters. --Lewis Cawte (talk) 21:51, 5 April 2017 (UTC)
• So... it's been another month, any updates? NovaHawk 13:32, 30 April 2017 (UTC)
  • @Lcawte: ^ 24.39.1.139 16:04, 8 May 2017 (UTC)
    • So April was a pretty terrible month. Without going into details, there've been many personal issues that our team members have had to deal with. In addition to that, the directors (myself and Cook879) have had to file our first-year company accounts because being landed with a disqualification, prison sentence and/or big fines is not helpful to ShoutWiki in the long term. Saying that, we're a lot closer with the script. I don't have an abundance of free time this week, but I've been working on ShoutWiki and the Brickimedia migration again a little this week which is a step in the right direction. I've got four days free next week, and while I'm not promising anything, I'm somewhat confident there should finally be some news. Should know more in the next day or two.
    • As an additional incentive to make us hurry up, this is the second month where we're now reimbursing Ajraddatz for this horrific VPS, which is made even better by the terrible exchange rates we're getting. --Lewis Cawte (talk) 09:19, 11 May 2017 (UTC)
• @30th May update: Good luck with move, hope everything goes smoothly for you :) NovaHawk 07:34, 30 May 2017 (UTC)
• Hey, I noticed the site was unlocked- just wondering if the move was complete or am I being annoying? :P (if it is done, got some problems with logging in to any accounts that existed prior to the move- they accept the password when it's correct but don't actually log you in) NovaHawk (test 3) (talk) 05:46, 3 June 2017 (UTC)
  • Have a look at Brickipedia:Welcome to ShoutWiki. --Lewis Cawte (talk) 18:18, 3 June 2017 (UTC)
    • Cleared my cookies and emptied my cache before posting this, just tried it again and still no luck. If it's just me who's having problems it's no big deal (I can make another account), just more worried about everyone else who may not have had a ShoutWiki account before NovaHawk (test 3) (talk) 23:51, 3 June 2017 (UTC)
      • This is Latenightguy. I too have found difficulty in trying to login. While I have cleared my cookies, I did not have a ShoutWiki account, thus leading me to have a similar issue to NovaHawk. 66.41.228.162 00:27, 4 June 2017 (UTC)
        • Berry too. Lcawte said to contact him or Shoutwiki support via email (not that I've done that yet... :P ). 72.230.198.60 02:18, 4 June 2017 (UTC)