Automatic data migration between branches when renaming tables / columns

Hi Ling Wu,

It is not intended to work like this I think. Why would you create a new branch everytime a new version is released?

The Software Factory is temporal, meaning all changes are stored in history tables. At anytime you are able to "go back” and create a branch from a specific point in time. You can simply release from MAIN for example and let de SF create a model version for you with the Production version name.

When we release a new platform version, the MAIN is generated with the version name; e.g. 2024.3. This will in turn create a model version with that name. If at any time we need to go back for some reason, we can create a branch from this model version.

Did you use re-apply intermediate mapping? By default it sets the columns to new/removed. After using re-apply intermediate mapping it should fix the data migration. 

I can reproduce this with the following situation (in SF 2024.2):

1. Database is in sync with Main

2. In Main: rename table/col

3. Create branch from main
    
4. Generate upgrade script for the database to the branch. 
   
   Re-apply intermediate mappings have no effect:

Even in the proposed branching strategy (Manage branches with Thinkwise | Thinkwise Community (thinkwisesoftware.com)) this could happen when working with a hotfix branch at production, when creating feature branches and possibly when using a production database to make a new acceptance (release branch) database with fresh data.

This seems very risky for potential data loss, so I hope a solution can be found. At the very least there could be a validation message for objects that have status ‘removed’ by data migration so that a developer has a trigger to manually update the data migration (even though there could be false-positives). 

Hey @PeterKeeris,

This scenario should also work I believe. Which platform version are you running on?

In our latest release I tried your reproduction but I do get a mapping:

Edit: I worked with a branch and a branch of that branch, that might be why it's different; checking...

@Ling Wu Reading your issue, I can confirm I also have encountered a few occasions where the renamed table was seen as new/remove after the merge. No idea what exactly causes it to detect it correctly one situation, but not in another.

Seeing you mention you did create manual data migration scripts, I wanted to let you know you can change the “new” table to let it know it has an old “remove” table to get its data from. Use the task “Update table mapping” in the Data migration and let give it the old table as “From table”.
 

Afterwards it will show your table as “Changed”.

Perhaps this is what you meant by manual scripts, but I’d better let you know in case it is now :-) Hope it helps for the times you need it.

Thanks for the answers.

We rarely change table names or columns. In future upgrades we plan to resolve this by creating conversion scripts for the data migration that are executed during the upgrade to make sure no data is lossed and to rename only if absolutely necessary.

Renaming tables or columns should not be a scary thing to do. The data migration is a vital part of upgrading. Simply verify it before upgrading and link/unlink or adjust default values accordingly.