release notes

Release notes Thinkwise Deployment Center (2.0.0)

  • 22 November 2021
  • 4 replies
  • 149 views
Release notes Thinkwise Deployment Center (2.0.0)
Userlevel 6
Badge +3

 

Hello everyone,

It has been a while since the Thinkwise Deployment Center (or 'Deployer' for short) has received new features, but over the past year, we have been gradually reworking the code to support some of the most frequently requested ones.

There have been a couple of breaking changes compared to the last version, which we will highlight in this post.

Download the Thinkwise Deployment Center 2.0.2 here.

 

 

Deployer GUI and CLI

 

New

 

Support for specifying manifests as YAML

In addition to the current JSON manifests:

{
"schema": 2,
"products": [
{
"type": "IAM",
"version": "2021.3",
"projectFolder": "T:\\Product Innovation\\Applications",
"dependencies": [
"CompatibilityLevel130"
],
"packages": [
{
"type": "Install",
"path": "IAM/Install",
"defaultDatabaseName": "THINKWISE_IAM"
},
{
"type": "Upgrade",
"path": "IAM/Upgrade",
"supportedVersions": [
{
"version": "2021.1",
"upgradesTo": "2021.2",
"files": [
"2021.2\\020_Upgrade.sql",
"2021.2\\030_Checks.sql",
"2021.2\\040_Constraints.sql",
"2021.2\\050_Indexes.sql",
"2021.2\\060_Functions.sql",
"2021.2\\065_Table_valued_functions.sql",
"2021.2\\070_Views.sql",
"2021.2\\080_Procedures.sql",
"2021.2\\090_Instead_of_triggers.sql",
"2021.2\\100_Triggers.sql",
"2021.2\\110_Tasks.sql",
"2021.2\\120_Defaults.sql",
"2021.2\\130_Layouts.sql",
"2021.2\\140_Contexts.sql",
"2021.2\\145_Badges.sql",
"2021.2\\150_Processes.sql",
"2021.2\\999_Apply_roles_on_database.sql",
"2021.2\\999_Manual.sql"
]
},
{
"version": "2021.2",
"upgradesTo": "2021.3",
"files": [
"2021.3\\020_Upgrade.sql",
"2021.3\\030_Checks.sql",
"2021.3\\040_Constraints.sql",
"2021.3\\050_Indexes.sql",
"2021.3\\060_Functions.sql",
"2021.3\\065_Table_valued_functions.sql",
"2021.3\\070_Views.sql",
"2021.3\\080_Procedures.sql",
"2021.3\\090_Instead_of_triggers.sql",
"2021.3\\100_Triggers.sql",
"2021.3\\110_Tasks.sql",
"2021.3\\120_Defaults.sql",
"2021.3\\140_Contexts.sql",
"2021.3\\150_Processes.sql",
"2021.3\\999_Apply_roles_on_database.sql",
"2021.3\\999_Manual.sql"
]
}
]
},
{
"type": "Hotfix",
"path": "IAM/Hotfixes"
}
]
}
]
}
YAML can now also be used:
schema: 2
products:
- type: "IAM"
version: "2021.3"
projectFolder: "T:\\Product Innovation\\Applications"
dependencies:
- "CompatibilityLevel130"
packages:
- type: "Install"
path: "IAM/Install"
defaultDatabaseName: "THINKWISE_IAM"
- type: "Upgrade"
path: "IAM/Upgrade"
supportedVersions:
- version: "2021.1"
upgradesTo: "2021.2"
files:
- "2021.2\\020_Upgrade.sql"
- "2021.2\\030_Checks.sql"
- "2021.2\\040_Constraints.sql"
- "2021.2\\050_Indexes.sql"
- "2021.2\\060_Functions.sql",
- "2021.2\\065_Table_valued_functions.sql"
- "2021.2\\070_Views.sql"
- "2021.2\\080_Procedures.sql"
- "2021.2\\090_Instead_of_triggers.sql"
- "2021.2\\100_Triggers.sql"
- "2021.2\\110_Tasks.sql"
- "2021.2\\120_Defaults.sql"
- "2021.2\\130_Layouts.sql"
- "2021.2\\140_Contexts.sql"
- "2021.2\\145_Badges.sql"
- "2021.2\\150_Processes.sql"
- "2021.2\\999_Apply_roles_on_database.sql"
- "2021.2\\999_Manual.sql"
- version: "2021.2"
upgradesTo: "2021.3"
files:
- "2021.3\\020_Upgrade.sql"
- "2021.3\\030_Checks.sql"
- "2021.3\\040_Constraints.sql"
- "2021.3\\050_Indexes.sql"
- "2021.3\\060_Functions.sql",
- "2021.3\\065_Table_valued_functions.sql"
- "2021.3\\070_Views.sql"
- "2021.3\\080_Procedures.sql"
- "2021.3\\090_Instead_of_triggers.sql"
- "2021.3\\100_Triggers.sql"
- "2021.3\\110_Tasks.sql"
- "2021.3\\120_Defaults.sql"
- "2021.3\\130_Layouts.sql"
- "2021.3\\140_Contexts.sql"
- "2021.3\\145_Badges.sql"
- "2021.3\\150_Processes.sql"
- "2021.3\\999_Apply_roles_on_database.sql"
- "2021.3\\999_Manual.sql"
- type: "Hotfix"
path: "IAM/Hotfixes"

 

Deployer GUI

 

BREAKING CHANGES

 

No more GUI and Indicium deployment during IAM installation and upgrade

In the old Deployer, when installing or upgrading IAM, you would be asked if you liked to install an Indicium or any of the GUIs.

However, since new deployment options for those products have been introduced, configuring them during the IAM flow has become quite complicated. For this reason, we have decided to remove the ability to deploy other products during the IAM flow.

The Deployer was always meant as a helper tool for deploying our products, and thus we expect someone using it to at least know the basic cohesion between our products. So, as a user, you should know which products you need to deploy and in which order to make your environment work. For more information, we recommend reading the Thinkwise Platform Overview section on our documentation site.

 

No more assistance for selecting a target meta source for Indicium and GUIs

Previously, the deployment flows of Indicium and the Windows/Web GUI had steps for selecting a target IAM to use as the meta source.

Due to the new way we configure these products in this release, we had to cut these steps from the flow.

We are looking into the possibility of adding additional buttons to the configuration steps that have a Simple mode to once again select a meta source with the assistance of UI dialogs.

 

New

 

Landing page rework

Due to adding more options and splitting existing ones to accommodate the deployment process better, the number of items on the landing page got rather long.

To solve this problem, we have reworked the landing page to group the deployment options per product. You can find them in a drop-down menu on the left side of the landing page.

Product menu on the landing page

 

Universal GUI deployment

We added the ability to install or upgrade the Universal GUI on an IIS server.

The landing page of the Universal GUI

 

Installing a Universal GUI

The process to install a Universal GUI has been broken down into:

  • Select the site on IIS where to deploy the Universal GUI.
  • Select an empty path relative to the selected site.
  • Specify the contents for the Universal GUI's config.json configuration file.

The configuration step supports two modes: Simple and Advanced.

In the Simple mode, you can only change the basic options for running a Universal GUI:

Universal GUI Simple configuration

To configure an option not listed in the Simple mode, you can use the Advanced mode instead.

In the Advanced mode, the Universal GUI will only check if the serviceUrl property has been specified correctly. Other options are used as-is.

Universal GUI Advanced configuration

 

Upgrading a Universal GUI

The process to upgrade a Universal GUI has been broken down into:

  • Select the IIS site containing the Universal GUI to upgrade.
  • Select the path relative to the site that contains the Universal GUI installation.
  • Optionally change contents for the Universal GUI's config.json configuration file.
    • Just as with the installation process, you can select either the Simple or the Advanced mode to upload a new config.json file.
    • You can choose to skip this step altogether and keep the current config.json in the installation directory.
  • Optionally, mark any additional files that contain settings for changing how the Universal GUI works.
Universal GUI: files/folders of the current installation to preserve

 

Local IIS support

An important feature request we received was the ability to deploy products to an IIS instance running on the current server without the need for the Microsoft IIS Administration API.

After a considerable effort in re-implementing the entire IIS deployment code, this has been made possible.

Products that deploy to IIS now have both a Local and an IIS API option on the landing page:

IIS API deployment options on the landing page

This is available for the following products:

  • Indicium
  • Indicium for Mobile GUI
  • Universal GUI
  • Web GUI.

 

Indicium deployment reworked

The deployment process for Indicium and Indicium (Mobile) has been reworked.

Previously, there was a single deployment option for both installation and upgrading. This made the logic for the deployment more convoluted than necessary, and it made it harder to add new features to the process.

For this release, we have split the single deployment process into separate install and upgrade flows.

Just like with the Universal GUI process, there is now a dedicated step to configure Indicium's appsettings.json. Both the Simple and Advanced modes are included, and the step can be skipped during the upgrade process.

Indicium: Simple mode for configuring the appsettings.json

Additionally, there is an option to help configure the application initialization feature for Indicium. When running Indicium as an agent of an IAM, e.g., to generate projects in the Software Factory 2021.3 or run scheduled system flows, we recommended turning on this feature.

Additional Indicium settings

Lastly, just like with the Universal GUI process, it is now possible to retain certain files as-is inside the installation that is being upgraded.

Indicium: files/folders of the current installation to preserve

 

Windows GUI deployment reworked

The deployment process for the Windows GUI has been reworked.

Just like with Indicium, there was previously only one deployment process for the Windows GUI. This process has also been split into separate installation and upgrade flows. Additionally, we have added an option to only create a new startup configuration file (.ini) by selecting an existing installation.

Windows GUI deployment options on the landing page

When creating a startup configuration for the installation, there are three options to choose from:

  • Create new
  • Link existing
  • Skip.

By choosing Create new, you can load a preset template based on the meta source connection you are going to use and then modify it with the required values.

When using the Indicium template, make sure that the Windows GUI you are using supports it.

Create a new configuration file for the Windows GUI

If you choose Link existing, e.g., to create a shortcut later in the deployment process, simply enter the path to the desired ini file.

Windows GUI: link an existing startup config to this installation

If you select the Use ClickStart option, any shortcuts made during this deployment will target TSF_CS.exe instead of TSF_dotNET.exe. It is possible to modify the clickstart.ini to configure the behavior of TSF_CS.exe.

Windows GUI: configuring the ClickStart ini

Lastly, you can either Skip creating or create a new shortcut that links the selected startup configuration to the Windows GUI installation. Remember that selecting Use ClickStart in the Startup configuration step will make this shortcut to TS_CS.exe instead of TSF_dotNET.exe.

Windows GUI: create a shortcut for the startup configuration

 

Web GUI deployment reworked

Just like in the Indicium and Windows GUI deployment, the Web GUI deployment has been split into separate installation and upgrade flows.

This rework is not as broad as the other two, but we have enabled the options to enter your own settings.ini and to preserve files/folders when upgrading.

Configure the settings.ini of the Web GUI

 

Known issues

 

Indicium application initialization not configured properly when using IIS API

Currently, the Microsoft IIS Administration API does not expose the properties we need for configuring the application initialization feature properly for Indicium.

We have made pull requests to the official repository on GitHub to try and fix this, but there will seemingly only be one more update by Microsoft to this repository to upgrade the software to .NET 6.

We are currently discussing whether it is worth it to continue support for this way of deploying our products.

 

Deployer CLI

The command-line version of the Deployer has also received some updates. Some of these updates are breaking changes compared to the previous version.

 

BREAKING CHANGES

 

Changed command-line library

The previous version of the Deployer used a command-line argument parser that did not support sub-commands. Due to this, several command names were appended together instead, e.g., iam-install/iam-upgrade/iam-hotfix. This made adding more commands to the CLI difficult since the help would soon be a lot more cluttered, as was the case for the landing page in the Deployer's GUI.

Because of this, we have changed the underlying command-line library to a different one that does support sub-commands.

This is a breaking change. Not only because, for example, iam-install must now be called as iam install but also because some subjects, like argument parsing, might be handled differently than before.

We recommend using this version to thoroughly test the scripts that use the Deployer CLI program before updating any previous executables currently used in production.

As part of adding sub-commands, the commands to auto-install, upgrade, or hotfix an IAM, Software Factory, or application have been moved from

  • twdeployer.exe iam (OPTIONS)
  • twdeployer.exe sf (OPTIONS)
  • twdeployer.exe app (OPTIONS)

to the following commands:

  • twdeployer.exe iam auto (OPTIONS)
  • twdeployer.exe sf auto (OPTIONS)
  • twdeployer.exe app auto (OPTIONS)

 

New

 

Indicium commands

We added commands to install, upgrade or reconfigure Indicium on a local IIS instance.

We also added a command to create an application pool with recommended settings for Indicium.

These commands can be called using twdeployer.exe indicium iis.

The commands must be run from an elevated prompt because modifying IIS requires admin privileges.

Indicium CLI commands

 

Universal GUI commands

We added commands to install, upgrade or reconfigure the Universal GUI on a local IIS instance.

These commands can be called using twdeployer.exe universal iis.

The commands must be run from an elevated prompt because modifying IIS requires admin privileges.

Universal GUI CLI commands

 

Generic IIS commands

Some commands were added to the Deployer CLI to control application pools on the local IIS instance.

You can now:

  • List information about every or specific pools
  • Create new pools
  • Edit the properties of one or more pools
  • Start one or more pools
  • Stop one or more pools
  • Delete one or more pools.

These commands can be called using twdeployer.exe iis pools.

These commands must be run from an elevated prompt because modifying IIS requires admin privileges.

IIS application pool commands

 

Next release

Even though Thinkwise has no dedicated Deployer team (like there are dedicated teams for the GUIs and Indicium), we would still like to ship the following changes before the 2022.1 Thinkwise Platform release:

  • Dedicated support for deploying the Thinkwise Upcycler.
  • Move Deployer CLI to .NET 6
    • While the commands that target IIS might not work on anything other than Windows, users would still be able to deploy IAM, the Software Factory, or application products on other operating systems.
    • Depending on the amount of work, we might have the Deployer GUI run on .NET 6 (Windows only) as well.
  • Add options to install an application without syncing the model to an IAM.
  • Add a separate flow that only syncs the application model to IAM.
  • Check access rights on the target directory when installing Indicium to a local IIS.
    • One of the most common causes we encounter where users cannot start their Indicium installations is that the user the application pool is running on does not have write access to Indicium's installation directory. We plan to add a deployment action that checks if this is configured correctly.

Would like to have:

  • A simple configuration mode for Windows GUI flows to make it easier to get a basic installation up and running.
  • A UI-assisted meta source selection for simple configuration modes during Indicium and Windows GUI flows.

 


4 replies

Userlevel 4
Badge +1
  • Add options to install an application without syncing the model to an IAM.
  • Add a separate flow that only syncs the application model to IAM.

We are testing with the Thinkwise Deployment Center GUI 2.0.0 to see if we could use it for Production deployments of our Custom Application, which are performed by our IT Operations department.

Above additions would be valuable, since we would like to do the following:

  • Sync to IAM + Post synchronization code which (among others) triggers the Custom Application Database backup
  • Wait until the Backup is finished → this is currently not possible, since the deployer runs both IAM and Database scripts right after each other
  • Deploy Custom Application

In addition, it is important that the use of the twdeployerGUI is as easy as possible and minimizes the risk of human error. This means that we simply want to write the Deployment Package to a certain folder, and do no changes to the manifest.json or any other file. Therefore, a few suggestions:

  • Our Custom Application Deployment Package is written in a Subfolder of the Folder where the twdeployerGUI is stored, meaning the needed manifest.json file also resides in this subfolder.
    • If a (SF/IAM) manifest.json exists in the same folder as the twdeployerGUI, it does load that manifest.json and we have no way to change this manually to the Custom Application manifest.json
    • If no (SF/IAM) manifest.json exists in the same folder as the twdeployerGUI, the Load Manifest screen is pointing to C:/Documents, instead of the twdeployerGUI folder itself

→ Could the next Release set this default Manifest folder to the twdeployerGUI itself, so the only thing we need to do is press File icon > open Custom Application subfolder > select manifest.json there? 

  • It would be great if we could set some defaults for the Server Connection screen regarding Host / Integrated Security / User / Password

→ Could set or save (checkbox ‘Remember f.e.) defaults functionality be added for the Server Connection screens?

  • The IAM/Application Database Selection screen throws nice errors if a Database is selected that does not match the expected project. 

→ Could this screen be improved by filtering the Name dropdown on Databases that match the expected project (so validation upfront, instead of after selection)?

 

Even though Thinkwise has no dedicated Deployer team (like there are dedicated teams for the GUIs and Indicium), we would still like to ship the following changes before the 2022.1 Thinkwise Platform release:

@Diana Kuipers I understand that 2022.1 Thinkwise Platform release is planned for end of the month, when can a new Thinkwise Deployment Center version be expected?

Userlevel 2
Badge

Hi @Arie V.

Thank you for sharing your experiences/feedback about the deployer.

I understand that 2022.1 Thinkwise Platform release is planned for end of the month, when can a new Thinkwise Deployment Center version be expected?

I am not sure I will be able to get the next deployer version out by the end of the month but if not then it should be released in early February.

I do not think I will be able to get every point stated in the blog post into the release but skipping the model import during an install/upgrade and the separate import model deployment option are almost done.

There will be a minor limitation in that you will not be able to use any other variables in the post-sync code that the IAM/SF/Upcycler scripts do not already do. This means for this release you will only be able to modify the <@db_name,,> template string. I am still thinking how this can be made more flexible for use with custom applications. This will probably require some changes/additions to the manifest schema that warrant a schema version increase, which increases the impact of the change to other products such as the SF generation code.

→ Could the next Release set this default Manifest folder to the twdeployerGUI itself, so the only thing we need to do is press File icon > open Custom Application subfolder > select manifest.json there? 

I could do that but I think other users will come and ask why they keep having to navigate back to the folder they were just in after closing the file picker window. Because the default behavior for the file picker dialog is to open in the location that was last used to pick a file. Which in your case seems to be C:/Documents.

I have been thinking about putting in some program startup options so a user can create a shortcut and specify multiple manifests to load when the program is starting.

E.g.:

twdeployerGUI.exe -m "D:/Manifests/Manifest1.json" -m "D:/Manifests/Manifest2.json"

Would that be an acceptable solution to your set-up as well?

It would be great if we could set some defaults for the Server Connection screen regarding Host / Integrated Security / User / Password

Would just one default connection to use when starting be enough or would you want this to be configurable per product type (e.g. IAM, SF or your custom app)?

→ Could set or save (checkbox ‘Remember f.e.) defaults functionality be added for the Server Connection screens?

I am not sure what you mean with this question, could you clarify further?

→ Could this screen be improved by filtering the Name dropdown on Databases that match the expected project (so validation upfront, instead of after selection)?

It used to work that way actually but users were unhappy with this on SQL Server instances with large amounts of projects because to check, for example, if a database has the right project ID the deployer cannot get around connecting to the target database and querying the sf_product_info table. Which takes longer the more databases are present on the server.

Userlevel 4
Badge +1

I am not sure I will be able to get the next deployer version out by the end of the month but if not then it should be released in early February.

I do not think I will be able to get every point stated in the blog post into the release but skipping the model import during an install/upgrade and the separate import model deployment option are almost done.

Sounds great to me!

There will be a minor limitation in that you will not be able to use any other variables in the post-sync code that the IAM/SF/Upcycler scripts do not already do. 

To be clear: is this a limitation to any Post synchronization code used already (meaning the script can only target the IAM database)?

Because the default behavior for the file picker dialog is to open in the location that was last used to pick a file.

The behavior is as follows:

  • If the Deployment Package folder of the previous deployment still exists, indeed the previously used location is used again. Then for Custom Application deployments the slight risk is that the IT Operator might select the old, wrong Manifest.json that resides there.
  • If the Deployment Package folder of the previous deployment is removed, the twdeployerGUI (in our Windows setup) defaults to C:/Documents.
  • Both scenario's are not ideal to me. And also taking into account the fact that the twdeployerGUI automatically opens the (SF/IAM) Manifest.json if that exists in the same folder as the twdeployerGUI, I would prefer a slightly more flexible solution.

If you slightly adjust your suggestion, I'd be happy (see below your quote):

I have been thinking about putting in some program startup options so a user can create a shortcut and specify multiple manifests to load when the program is starting.

E.g.:

twdeployerGUI.exe -m "D:/Manifests/Manifest1.json" -m "D:/Manifests/Manifest2.json"

Would that be an acceptable solution to your set-up as well?

Since in the case of Custom Application deployments the target folder in which the Manifest.json resides is not known upfront (the Deployment Package with Manifest.json only exists after it is created), this solution wouldn't do entirely. Instead of that, would this work for you?

  • Keep the existing behavior of the current twdeployerGUI.exe, unless specified otherwise. Then allow for setting a folder and/or a Manifest.json. In our case we would set it to below, so it always opens the root folder and never the location with the last used Manifest.json  
twdeployerGUI.exe -m "D:/Deployment Packages"

Would just one default connection to use when starting be enough or would you want this to be configurable per product type (e.g. IAM, SF or your custom app)?

One is good enough for me. To be clear: don't allow for storing the password. P.s. the question was a rephrase of the above statement, intended to make it a request with a question mark :)

It used to work that way actually but users were unhappy with this on SQL Server instances with large amounts of projects because to check, for example, if a database has the right project ID the deployer cannot get around connecting to the target database and querying the sf_product_info table. Which takes longer the more databases are present on the server.

Got it, makes sense not to do this then!

Userlevel 2
Badge

To be clear: is this a limitation to any Post synchronization code used already (meaning the script can only target the IAM database)?

The post-sync code/scripts are being run in the same step as the sync code/script itself (basically any script found in the MetaModel directory) so unless we establish some kind of standard to run scripts on another database you are indeed currently limited to having them target the IAM database you have selected to sync to.

  • Keep the existing behavior of the current twdeployerGUI.exe, unless specified otherwise. Then allow for setting a folder and/or a Manifest.json. In our case we would set it to below, so it always opens the root folder and never the location with the last used Manifest.json 

I have quickly/temporarily added this to the next release as follows:

twdeployerGUI.exe --manifest-selection-dir "D:/Deployment Packages"

It will probably not be mentioned in the release blog since I am not sure if I want to keep supporting it like this. I made an issue to add some kind of settings file for the deployer GUI so users can also specify a default database connection etc. and I might move this option there. Or I might support both if it is easy enough to do so. I will make a mention of it inside the changelog window though.

Reply