WotC Forums Migration Tool

You most probably know by now that WotC is closing its D&D, M:tG and other community forums on October 29th. Already, hundreds of WotC forum members have begun migrating over here to EN World, and as part of that process they are transferring content that would otherwise be lost. The content is being placed in the Emergency WotC Evacuation Lifeboat forum, the Character Builds & Optimizations forum, and the D&D Adventurer's League forum. The Emergency WotC Evacuation Lifeboat Forum is a general storage area for any and all transferred content, regardless of its nature, until we get chance to sort through and move it all to where it needs to be. EN World and WotC member MerricB has spent over 12 hours coding a tool which makes it incredibly easy and quick to transfer your content from WotC's forums to here. It preserves formatting, and copies entire threads, making transferring your content take a matter of seconds rather than hours.


Original Post


This tool isn't complete yet, but it should get most of the formatting from a series of posts on a page on the Wizards forums, convert it to BB Code, which you can copy and post on EN World.

https://onedrive.live.com/redir?resid=88B4DEB9530F6609!26300&authkey=!ABKffCWJONMiKjI&ithint=folder,zip

It's a Windows program, written in C# on my Windows 10 machine - I have no idea if it'll work on previous versions of Windows. I know it won't work on Macs. Let me know if it works for you.

The program is pretty simple:
* put the URL of the page you wish to convert in the text box at the top of the page.
* press "Convert"
* Wait patiently for the page to load on the left-hand pane, and then for the BBCode to appear in the right-hand pane.
* Press "Copy" to copy the text to your clipboard.

At that point, you can do anything you like with it.

The left-hand pane is a simple browser, so you could navigate using that; it'll convert a Wizards forum page automatically when you get to the next. If you spot any types of formatting it is not bringing over, let me know. (It removes any orders to turn the text white or black, and it doesn't understand certain types of colour codes at the moment.)

The browser doesn't quite understand the Wizards log-in system on my computer. It might on yours.

A row of post numbers will appear down the bottom of the page as well.
* Press on a number to just cause that post to appear on the right-hand side; it also copies itself to your clipboard, ready for re-posting.


Cheers!
 
Last edited by a moderator:

Comments

MerricB

Eternal Optimist
This tool isn't complete yet, but it should get most of the formatting from a series of posts on a page on the Wizards forums, convert it to BB Code, which you can copy and post on EN World.

https://onedrive.live.com/redir?resid=88B4DEB9530F6609!26300&authkey=!ABKffCWJONMiKjI&ithint=folder,zip

It's a Windows program, written in C# on my Windows 10 machine - I have no idea if it'll work on previous versions of Windows. I know it won't work on Macs. Let me know if it works for you.

The program is pretty simple:
* put the URL of the page you wish to convert in the text box at the top of the page.
* press "Convert"
* Wait patiently for the page to load on the left-hand pane, and then for the BBCode to appear in the right-hand pane.
* Press "Copy" to copy the text to your clipboard.

At that point, you can do anything you like with it.

The left-hand pane is a simple browser, so you could navigate using that; it'll convert a Wizards forum page automatically when you get to the next. If you spot any types of formatting it is not bringing over, let me know. (It removes any orders to turn the text white or black, and it doesn't understand certain types of colour codes at the moment.)

The browser doesn't quite understand the Wizards log-in system on my computer. It might on yours.

A row of post numbers will appear down the bottom of the page as well.
* Press on a number to just cause that post to appear on the right-hand side; it also copies itself to your clipboard, ready for re-posting.


Cheers!
 
Last edited by a moderator:

MerricB

Eternal Optimist
Okay, I've updated the program.

Now, a row of post numbers will appear down the bottom of the application when the forum page is loaded. Clicking on any of those numbers will display just that post in the right-hand side and copy it to the clipboard. Should make it easier to parse out the big posts and leave out the chatter.

Cheers!
 

MerricB

Eternal Optimist
Thanks! It's the sort of thing I really enjoy doing - making small programs to solve a task - and it helps when it mostly works.

I'm just trying to work out where all the newlines are coming from in a particular set of pages. It's really frustrating - the html the Wizards forums create is bizarre and seems to be generated by three separate algorithms. Solving one problem breaks some of the others...

Cheers!
 

Zaister

Visitor
You should be careful with reposting content from the WotC forum, they probably have a policy that everything posted there becomes their intellectual property, and reposting content could violate their rights.
 

BoldItalic

Visitor
This is great, it will save a lot of work.

I've pasted a sample into your test thread that exhibits some spurious line breaks. It's strange, because they aren't visible in the converted text, only when it is pasted into a reply box here. Can you make any sense of what is happening?

<edit> I see from your post above that you are looking at it :) </edit>

Images on the WotC server will need to be pulled down and uploaded to ENWorld to preserve them, then the IMG links changed in the pages. We might have to do that manually, unless you can think of some magic solution? The thread I want to bring across is only 8 pages but contains several hundred images. I could do it manually, but I'd rather not have to. Any thoughts?
 

MerricB

Eternal Optimist
I know why the line breaks are appearing, but getting them to disappear is proving harder than I think it should be. :)

Images are going to be an absolute pain. I have no easy solution for that. (Can I write an utility to pull them down and put them back on a site? Probably. Do I want to? No. I really don't have the bandwidth to test it, and where would they be stored?)

Cheers!
 

BoldItalic

Visitor
You should be careful with reposting content from the WotC forum, they probably have a policy that everything posted there becomes their intellectual property, and reposting content could violate their rights.
I think we are okay. They have given us blanket permission:

Trevor Kidd said:
We encourage past and current users to retrieve any information you want to retain from the Community Forums for both Dungeons & Dragons and Magic: The Gathering. The shutdown will occur on October 29, 2015, at 10:00 a.m. PT. We want to provide enough time for our forum members to move their content, and we recognize that given our forum's vibrant user base and extensive history, this may take time. Any information still on the forums on the cut-off date will be deleted.
 

Morrus

Well, that was fun
Staff member
Using this tool, I think we'll be able to move tons of content very quickly. I think @merric has created a gamechanger here.

Merric, quick question -- how does it handle those threads with indexes full of links to other posts in the same thread? Is it able to convert those, or do we need to recreate links manually?

A suggestion -- if copying someone else's content, maybe automatically insert a little disclaimer at the top? Maybe disabled via a checkbox if it's our own content. It could say something like:

"This content was originally posted by X on the now-closed WotC forums. If you are the content creator, you can claim ownership of this thread by asking an admin to transfer it to you."

That would make life much easier, and help those who stumble across it later.
 
Last edited by a moderator:

MerricB

Eternal Optimist
Strictly speaking, we should only be moving content *we* own, not content made by someone else. The license we've granted to Wizards does not remove our own rights to our own material; however, we haven't magically gained access to someone else's right by virtue of its being on the Wizards forums.

In many cases, you should be able to gain permission to move content from its creator. (Move? Duplicate is what we're doing).

Cheers!
 

MerricB

Eternal Optimist
New version up - should deal with the line breaks in a more sensitive way. I've also reduced (slightly) the size of the header font to fit on EN World pages better.

Let me know if you notice any odd behaviour... although I'm likely to be in bed. I've been working on this the last 10+ hours without much of a break.

Cheers!
 

Morrus

Well, that was fun
Staff member
I've stickied this, Merric. I think a lot of people are going to owe you some major thanks!
 

MerricB

Eternal Optimist
One of the reasons there are oddities in the conversion is because there are oddities in the formatting. I mean, how do you deal with this line of html?

<p style="margin-right:10px; margin-bottom:10px; color:rgb(68, 68, 68)>

Really? You want a little space below the paragraph and on the side, and the colour to be some sort of grey?

You can have a newline and that's it!

:)

Cheers!
 

Ferghis

Visitor
Serious kudos MerricB. You've always been a great poster, ever since I saw you for the first time over a decade ago on Dragonsfoot. Thanks for doing this.
 

koga305

Visitor
This looks fantastic. For those of us with Macs, would other folks be willing to run a few threads through the program and send us the BBCode? I'd love to repost my Fighter Guide on ENWorld and this looks like the perfect tool to do it, but I don't have access to a Windows computer.
 
Thank you very much, Merric. Copying over the coloured text is probably the most important bit; other HTML-to-BBCode converters I've found don't have that capability. I've added your tool to my "how-to" in the GitP index of 3.X material, here:

giantitp.com/forums/showthread.php?444041-Threads-from-the-Wizards-forums

If anyone is moving 3.X stuff over, please PM me a link or make a note of it in the thread, and the mirrored thread (from any site) will be added to the index.
 

Morrus

Well, that was fun
Staff member

MerricB

Eternal Optimist
Hmmm... I was converting along, no problem, and then I hit this page: http://community.wizards.com/forum/dd-encounters-season-15-murder-baldurs-gate/threads/3889061

When I try to "Convert" it gives me the "You are not authorized to access this page" page. I'm still logged in. I can see it just fine on my browser...
Ah, but you're not logged in using the program's internal browser, which is running in IE7 mode, because it feels like it. And IE7 won't log-in to the Wizards site. There's meant to be a registry entry that can fix that, but it's proving very troublesome to fix. :(

Cheers!
 

In Our Store!

Advertisement

Latest threads

Advertisement

Top