I have built a new site using 3.2 with new features, functionality as well as a better UX - all this has ultimately come down to the new database having a completely different style from the existing one. Now that all the necessary page types have been created I am looking to transfer the data over from the old database to the new one.

What would be the best way to do this?

Also, if I have PageType & PageType_Live would I need to migrate the data into both of those or could I just import into PageType and let SilverStripe set it into '_Live'?

Depends on a lot of things,, the number of pages, and if you are using different page types and/or DataObjects... So I don't think there actually is a default 'best' solution, especially since your data model setup is so different...

If the number of pages isn't too large, I would probably play it safe and take some time generating them by hand and copying the contents.

If you have a hughe number of pages, and you want to automate things, and your (original) page setup was fairly simple (basically just MenuTitle, Title, Content) you could still try copying the database (or maybe just SiteTree, SiteTree_Live and File), your assets folder and then do a dev/build - that would maybe give you back the default site structure plus just maybe the images in tinyMCE (not certain about the latter though). You'd still lose any custom pagetype settings though... you could add the two tables for pagetypes, dataobjects that have more or less the same dbfields,

As to your question: yes you can leave out the _Live tables, but then you'd have to publish all pages afterwards.

And if it is just to different, nothing left to do but some major database scripting...

There were scripts around for updating to v3, but they would probably only work on sites that kept their general structure sort of...

But maybe others have some great ideas?

I think it was mostly wishful thinking that someone had a magical solution but unfortunately it looks like I will have to script the old database into the new one. There's over 40,000 pages and 2.3m rows of data so manually doing anything is out the question lol

The approach I'm taking is fairly simple in concept, just taking the data I need and slotting it into the new database while making sure to transform the class names and preserve the hierarchy. Taking it step by step at the moment to make sure I'm not just pumping data in but keeping it usable in the CMS

Is there an automated way I can publish everything? The less things I have to migrate the less error-y it'll be I reckon (databases aren't really my strong point)

Wow that's a hughe undertaking!!!

You could always try Stack Overflow or the irc channel to see if there are others with better tricks then I can come up with. The forums don't always have everyones attention...

I'm kind of curious how you got SilverStripe to perform with 40.000 pages, that's a lót! Scripting that enormous database is also not to be thought lightly of... Anyway I wish you the best of luck with the migration!

In the cms, when you go to the pages section, there's a selectbox 'multi-selection' If you check it, you'll see an actions dropdown that will let you batch- publish a couple of pages at the time. but if you would perform that action on Too many pages at once that would probably crash things... And publishing 40.000 pages 20 at the time, eh...

An idea would maybe be to write a task that selects a fixed number of unpublished pages and publishes them. Then use cron to repeat that task... Or maybe a module like this: silverstripe-queuedjobs (I haven't tested it)

Thanks, need all the luck I can get to finish this ahha

SO is usually my go-to place but at the time of writing this question I probably felt these forums were more suited as this question is more 'what do you think' rather than 'I need a specific answer'

I read somewhere here were some guy used Excel/phpMyAdmin to edit the data in order to suit his new database and migrated it like that, that was my first approach because I thought it'd be easy and half way through the day I seriously considered throwing my computer out the window - there's just way to much data here to manually do anything and trying to do it like that drove me up the wall, theres simply too many things to take into account, it's not just columns but row data such as ClassName etc that needs amending or placed elsewhere. That's why I took the decision to script it in, if I map everything out into where things need to go I only need to think about what goes where and reading the script is quicker than dotting around the database getting confused or wondering if you remembered to change anything everywhere.

In terms of performing, I don't actually know to be honest but it's managing to handle it - I joined the company as this was already running so it was like this when I came in. We got that much data because everyone in the company has access to the CMS and can publish whatever they want. It's not that confusing if each team falls into a category where they publish things and maintain that section, overall it is gigantic which is why migrating is a pain in the ass but as far as users are concerned "it's not that big".

Ah I forgot there's multi-selection functionality, but I think you're totally right, the weight of the task will cripple it; probably would crash my browser before I even get to hit the publish button. I think doing it server-side would be alright though.
There's a module on the 2.4 one also called queuedjobs but I haven't looked into what it actually does, all I know for sure is it regularly crashes because sometimes it's processing so much (processing what, I don't actually know) so just that is putting me off using it.
I remember there's a method like doPublish() or something similar - would that be suitable for what I need?

Hi Vlad,

I've not fully read this thread (or will subscribe to it) however I thought I'd mention https://github.com/unclecheese/silverstripe-db-surgeon
This might help with migration, best of luck!

Thank Swaiba, I've already begun the script and it's going well so far, I think I'll stick to moving forward with that.

Just a quick question, so I've copied a blank database with only the structure included and moved the data in Page and SiteTree; I've edited the SS_Env file to point at the new database and ran a build and flush but it's still showing the old content?

Why does it remain the same? -- Found out I needed to change the mysite/_config.php file instead of SS_env...

Now I've got the database changed but none of my templates are working... why is that?
Say in the admin panel I go on a page, it'll load up the content but on the default silverstripe layout, the blue and grey one...