Although the 2022.2 version of the Thinkwise Platform has only just been released, we already have big plans for a major change for the 2023.1. We are planning to store only the latest version of every project and move all old data to history tables.

The main goal of this change is performance improvement by separating the operational data in the Software Factory from the non-operational data. A secondary goal is preventing massive amounts of unnecessary data from being stored.

In the 2022.2 and earlier versions of the Software Factory the version control was handled by making a copy of a complete model in the shape of a project version. This resulted in largely the same data being saved multiple numbers of times, whenever a new project version or branch was created. Since the changes only make up a small percentage of the whole model, this unnecessary data caused the Software Factory databases to grow over time and becoming slower due to their size.

The change that is intended is to start handling version control through the use of temporal tables. These tables, also known as system-versioned temporal tables, are a database feature that brings built-in support for providing historical data. It allows access to data stored in the table at any point in time, rather than only that data that is present at the current point in time.

With this change it will no longer be necessary to copy data as many times because modifications in the data will only be stored in the history tables once. This historical data can then be accessed whenever it is required by requesting the historical data at a specific point in time in the past. This results in less operational data that needs to be called upon when executing certain tasks in the Software Factory such as copying a project or the process of branching and merging. An additional benefit of this whole operation is that there will be less risk that a developer accidentally modifies an earlier version of the model, because the developer will always be working in the most recent version.

Changing version control

Let's start off this change with an explanation concerning the replacement of a couple of well-known terms such as project and project version. The reason behind these renames and what the new terms are will be discussed first in the following paragraphs.

Project becomes model

A model is the blueprint of any application. Within the scope of the Software Factory, a project is similar to a model. However, outside the Software Factory, a project can be something much broader than just the application or part of an application that is being built. Simply put, a project is a series of tasks that need to be completed to reach a specific outcome. The actual development of the application is only a subset of this series of tasks.

Since the Thinkwise platform is based on model driven software development, we plan to replace the term Project with Model, which we feel covers the core of the Software Factory better.

Project version becomes obsolete, introducing the new interpretation of branches

Implementation of the system-versioned temporal tables means that the use of project versions will become obsolete. Through the existence of the temporal tables, it will no longer be required to copy all the model data into a new project version, with the goal of saving a certain point in time. However, we do still want to make use of the functionality of branching and merging as we know it.

This means that project versions as they used to be known will cease to exist in the 2023.1. Instead we will start using branches, which are technically versions of the same model that can be different from each other. Every model will have at least a MAIN branch and the possibility of containing one or more additional branches that can be created based on any point in time in the past.

Model versions (point in time)

Within a branch the developers have the possibility to mark the situation at any point in time as a model version. This can be any point in time that is not in the future, as the situation in the future is not yet known. This is like the branching model used in GIT, where branches are effectively a pointer to a snapshot of your changes. When necessary, a comparison can then be made between two model versions, such as is required for an upgrade.

A model version can also be marked with a name, which can be quite helpful in the communication about versions of the application.

Branching and merging

In the old situation, when using branching and merging the following steps would be executed:

1. A branch was created. This resulted in:
   1. The selected project/project version being copied as a base.
   2. A new project being created as a branch project.
   3. The project version being copied into the new project as a branch project version.
2. A merge session was started:
   1. Delta actions would be determined based on a comparison of the new project version against its original base.
3. The merge session was executed:
   1. The base would be copied.
   2. Delta actions were executed over the base.

In the new situation, branching and merging will consist of the following steps:

1. A branch is created. This results in:
   1. The base being tagged with the current point in time.
   2. The branch will be copied from that point in time.
2. A merge session is started:
   1. Delta actions will be determined based on a comparison of the situation at the base point in time vs. the current point in time, which will be the moment the merge session is started.
3. The merge session is executed:
   1. Only selected delta actions will be executed.

In the old situation the project version would be copied three times during one full cycle of branching and merging. In the new situation it will only be necessary to copy the branch (prev. project version) once. The main benefit of this being performance gain as well as preventing the creation of unnecessary amounts of historical data.

MAIN branch

Due to the transition to temporal versioning all the historical data from previous project versions will be migrated to the history tables in the appropriate order. By doing this a timeline of all the history will be created within the context of one main model. The most recent project version will become the active model branch after the migration.

Recap

As a recap of everything that we have said in this blog, let's take a look at the image below. The circles in the image represent data in the Software Factory database. The grey background in the circles is a representation of historical data, and the green background represents operational data. 

The top half shows the situation preceding the 2023.1. As one can see the branch circle grows exponentially, depending on the amount of versions the branch contains. This is why branch A is a larger circle than branch B, it contains four versions instead of two. The same rule could be applied to the project versions.

The bottom half of the image presents the intended situation for the 2023.1 and onward. In this situation the amount of operational data remains relatively equal even when changes are made, because only one version of the model is saved in a branch at a time. This is why all four branches are of a similar size, regardless of the amount of model data they contain. The model versions also won't noticeably grow in size when there are multiple model versions, since only a point in time is saved, that can then be used to build the model at that point in time.

The only thing that can grow in size in the new situation are the history tables. This is why the history table in the MAIN branch and branch A are larger, than the history tables in branch B and: more changes have already been executed which have been stored in the history tables. However, because the history tables are only called upon in very specific processes, this does not create an obstacle for the performance of the Software Factory.

What has changed for the developer?

A few things will change within the use of the Software Factory due to the implementation of the temporal tables and subsequent revision of project and project version.

Details on these changes will be discussed in a follow-up implementation blog of the temporal versioning of the Software Factory in the 2023.1.