WotC Forums Migration Tool

You most probably know by now that WotC is closing its D&D, M:tG and other community forums on October 29th. Already, hundreds of WotC forum members have begun migrating over here to EN World, and as part of that process they are transferring content that would otherwise be lost. The content is being placed in the Emergency WotC Evacuation Lifeboat forum, the Character Builds & Optimizations forum, and the D&D Adventurer's League forum. The Emergency WotC Evacuation Lifeboat Forum is a general storage area for any and all transferred content, regardless of its nature, until we get chance to sort through and move it all to where it needs to be. EN World and WotC member MerricB has spent over 12 hours coding a tool which makes it incredibly easy and quick to transfer your content from WotC's forums to here. It preserves formatting, and copies entire threads, making transferring your content take a matter of seconds rather than hours.


[h=4]Original Post[/h]
This tool isn't complete yet, but it should get most of the formatting from a series of posts on a page on the Wizards forums, convert it to BB Code, which you can copy and post on EN World.

https://onedrive.live.com/redir?res...00&authkey=!ABKffCWJONMiKjI&ithint=folder,zip

It's a Windows program, written in C# on my Windows 10 machine - I have no idea if it'll work on previous versions of Windows. I know it won't work on Macs. Let me know if it works for you.

The program is pretty simple:
* put the URL of the page you wish to convert in the text box at the top of the page.
* press "Convert"
* Wait patiently for the page to load on the left-hand pane, and then for the BBCode to appear in the right-hand pane.
* Press "Copy" to copy the text to your clipboard.

At that point, you can do anything you like with it.

The left-hand pane is a simple browser, so you could navigate using that; it'll convert a Wizards forum page automatically when you get to the next. If you spot any types of formatting it is not bringing over, let me know. (It removes any orders to turn the text white or black, and it doesn't understand certain types of colour codes at the moment.)

The browser doesn't quite understand the Wizards log-in system on my computer. It might on yours.

A row of post numbers will appear down the bottom of the page as well.
* Press on a number to just cause that post to appear on the right-hand side; it also copies itself to your clipboard, ready for re-posting.


Cheers!

 

[MerricB]

This tool isn't complete yet, but it should get most of the formatting from a series of posts on a page on the Wizards forums, convert it to BB Code, which you can copy and post on EN World.

https://onedrive.live.com/redir?res...00&authkey=!ABKffCWJONMiKjI&ithint=folder,zip

It's a Windows program, written in C# on my Windows 10 machine - I have no idea if it'll work on previous versions of Windows. I know it won't work on Macs. Let me know if it works for you.

The program is pretty simple:
* put the URL of the page you wish to convert in the text box at the top of the page.
* press "Convert"
* Wait patiently for the page to load on the left-hand pane, and then for the BBCode to appear in the right-hand pane.
* Press "Copy" to copy the text to your clipboard.

At that point, you can do anything you like with it.

The left-hand pane is a simple browser, so you could navigate using that; it'll convert a Wizards forum page automatically when you get to the next. If you spot any types of formatting it is not bringing over, let me know. (It removes any orders to turn the text white or black, and it doesn't understand certain types of colour codes at the moment.)

The browser doesn't quite understand the Wizards log-in system on my computer. It might on yours.

A row of post numbers will appear down the bottom of the page as well.
* Press on a number to just cause that post to appear on the right-hand side; it also copies itself to your clipboard, ready for re-posting.


Cheers!

 

[MerricB]

Okay, I've updated the program.

Now, a row of post numbers will appear down the bottom of the application when the forum page is loaded. Clicking on any of those numbers will display just that post in the right-hand side and copy it to the clipboard. Should make it easier to parse out the big posts and leave out the chatter.

Cheers!

 

[Morrus]

Wow. That is incredible, Merric!

 

[MerricB]

Thanks! It's the sort of thing I really enjoy doing - making small programs to solve a task - and it helps when it mostly works.

I'm just trying to work out where all the newlines are coming from in a particular set of pages. It's really frustrating - the html the Wizards forums create is bizarre and seems to be generated by three separate algorithms. Solving one problem breaks some of the others...

Cheers!

 

[Zaister]

You should be careful with reposting content from the WotC forum, they probably have a policy that everything posted there becomes their intellectual property, and reposting content could violate their rights.

 

[BoldItalic]

This is great, it will save a lot of work.

I've pasted a sample into your test thread that exhibits some spurious line breaks. It's strange, because they aren't visible in the converted text, only when it is pasted into a reply box here. Can you make any sense of what is happening?

<edit> I see from your post above that you are looking at it </edit>

Images on the WotC server will need to be pulled down and uploaded to ENWorld to preserve them, then the IMG links changed in the pages. We might have to do that manually, unless you can think of some magic solution? The thread I want to bring across is only 8 pages but contains several hundred images. I could do it manually, but I'd rather not have to. Any thoughts?

 

[MerricB]

I know why the line breaks are appearing, but getting them to disappear is proving harder than I think it should be.

Images are going to be an absolute pain. I have no easy solution for that. (Can I write an utility to pull them down and put them back on a site? Probably. Do I want to? No. I really don't have the bandwidth to test it, and where would they be stored?)

Cheers!

 

[BoldItalic]

You should be careful with reposting content from the WotC forum, they probably have a policy that everything posted there becomes their intellectual property, and reposting content could violate their rights.

I think we are okay. They have given us blanket permission:

We encourage past and current users to retrieve any information you want to retain from the Community Forums for both Dungeons & Dragons and Magic: The Gathering. The shutdown will occur on October 29, 2015, at 10:00 a.m. PT. We want to provide enough time for our forum members to move their content, and we recognize that given our forum's vibrant user base and extensive history, this may take time. Any information still on the forums on the cut-off date will be deleted.

 

[Morrus]

Using this tool, I think we'll be able to move tons of content very quickly. I think @merric has created a gamechanger here.

Merric, quick question -- how does it handle those threads with indexes full of links to other posts in the same thread? Is it able to convert those, or do we need to recreate links manually?

A suggestion -- if copying someone else's content, maybe automatically insert a little disclaimer at the top? Maybe disabled via a checkbox if it's our own content. It could say something like:

"This content was originally posted by X on the now-closed WotC forums. If you are the content creator, you can claim ownership of this thread by asking an admin to transfer it to you."

That would make life much easier, and help those who stumble across it later.