One of our strategic goals is to completing the DEVOPS spirit of working @Acto in an Agile way of thinking. This means that most of the complete development cycle has to be automated
Currently one of our projects is working on automating and speeding up the biggest part of the generation of the deploy package, one could say that these are the steps between Build and Test, which currently takes about 3 to 6 hours for each cycle (without testing, just generating). Our goal is to have several cycles a day if not many more, which of course can only be achieved when (almost) fully automated
We want everything that can be automated to be automated. We want it to run mainly in the background.
It would be helpful if the whole thing could be automated by Thinkwise – but until then, there are things we can do ourselves, but for certain parts we are unable to automate them in the process of creating a deploy package.
The process consists out of the following steps:
- CMD: Make a backup of several databases
- SF: Set the Previous EVO (trunk) project version to the last released version (EVO is our release name)
- SF: Generate the definitions
- SF: Start the validation check
- Stop process if fail: Manually solve issues
- SF: Activate/Apply conversion scripts (We maintain the list of conversion scripts somewhere else)
- SF: (re)Generate the definitions
- SF: (re)Start the validation check
- Stop process if fail: Manually solve issues
- SF: Generate source code "fully" and write code files to disk (provide location parameter)
- CMD: Create new "TEST_database" for testing purposes
- SF: Execute source code on previously created "TEST_database" - full without upgrade=clean installation
- SF: Start "Synchronization to IAM"
- IAM: Copy the EVO application (trunk) with alias 'evo' to a new application "TEST_application", set the DB to the previously created "TEST_database"
- IAM: Change the new application alias from 'evo' to 'deploy_test'
- IAM: Activate the new application
- IAM: Change the translation to Dutch for the new application
- Do some basic automated tests to make sure one can create a user with standard credentials within the application
- IAM: Remove "TEST_application" from IAM
- CMD: remove "TEST_database" from dbserver
- CMD: Create backup of existing "DEPLOY_database" and "DEMO_database"
- SF: Execute source code on existing "DEPLOY_database" - with upgrade - not full installation and check whether successfull
- SF: Execute source code on existing "DEMO_database" - with upgrade - not full installation and check whether successfull
- Handover to the test team.
- Manually solve issues
- SF: Start Create deployment package with parameter location where to drop the files
- CMD: Copy all scripts from step 8 to parameter location +install from step 23
- CMD: Remove 020_upgrade.sql from parameter location +install from step 23
- CMD: Copy all scripts from step 8 to parameter location +upgrade from step 23
- CMD: Remove 010_db.sql and 012_create.sql from parameter location +upgrade from step 23
- CMD: Create new deployment location for the deployment department
- CMD: Copy all from step 8 location to the new deployment location and then some other stuff to complete the whole installation package
We need help automating the bold italic process steps, we weren't able to find a solution to do this automated in a background process.
Best answer by Anne Buit
At Thinkwise, we use an auxiliary CI database to perform some of these tasks. Attached scripts are placed on this database and can be used as inspiration. Be sure to update the references in the script to the correct Software Factory database.
Step 8 can be done by executing the create_installation and write_installation procedure in the attachment.
Step 10 can be done via the deployer CLI using the command
twdeployer app-install, however a database will also be provisioned. Use
twdeployer help app-install for more info. Ensure a manifest is present, see the docs for more info on this.
Step 11 is not yet possible, we have no autonomous way available just yet to initiate synchronziation. Please create a topic for this in the ideation section. However, if synchronization scripts have been written to the MetaModel folder used by the deployer in advance, the deployer CLI will execute the synchronization on IAM and perform all the extra steps (12-17) in the post-sync code introduced in 2020.2. This will be done with
twdeployer app-install as part of step 10.
Step 20 and 21 can be done with the deployer CLI via
twdeployer app-upgrade and step 23 is in a way the same as step 8.
Alternatively, it is also possible to use sqlcmd to execute scripts that perform these tasks from commandline/powershell in your CI/CD environment instead of using the twdeployer.