Release Notes

Release notes Indicium (2022.1.13)

Related products: Indicium Service Tier
Release notes Indicium (2022.1.13)

Hello everyone,

In this sprint, we added various performance metrics for requests using Server-Timing response headers. We also added the Agent ID to the Indicium landing page, and Indicium now automatically retries when encountering a database connection failure on startup.

You can read more about Indicium's features in the Indicium user manual.

We will keep you updated regularly about Indicium's progress.

Contents of this release

Breaking Changes

Thinkwise Platform support release 2020.1 ends

Indicium support for Thinkwise Platform release 2020.1 is ending. Please upgrade to at least release 2020.2 to use this release of Indicium.

 

Indicium Universal

Measuring Server-Timing responses

In this release of Indicium Universal, we have added various performance metrics for requests using Server-Timing response headers. Browser developer tools can interpret these headers and visualize these performance metrics. The goal of these features is to give more detailed insight into performance and help solve potential performance issues, which are almost always caused by specific queries, layout procedures, file storage operations, etc. that can be optimized fairly easily. We have already added dozens of these performance metrics, which are listed below, and we are looking into adding more of them in the future.

CRUD operations

  • All SELECT queries for requested subjects of an application (tables and views).
  • When the $count operation requests the total number of records, this will be measured and returned separately.
  • When the $expand operation joins a lookup or a detail subject, these will each be measured and returned separately.
  • Insert statements.
  • Update statements.
  • Delete statements.
  • Expression queries while adding or editing a record.
  • Default value queries when entering edit mode to add a new record or start a task or report.
  • Lookup validations when changing a lookup value or a lookup filter value.

Application logic

  • Tasks
  • Subroutines
  • Default procedures.
  • Layout procedures.
  • Context procedures.
  • Badge procedures.
  • Change detection procedures (the new logic concept).

File storage

  • Fetching the database record for a file to be downloaded.
  • Calculating the hash of a file.
  • File access (this can be considered as obtaining a file connection, but does not include the entire download process).
  • Writing files (this does include the entire upload process).

Miscellaneous

  • Fetching records by primary key. This is, for instance, done when starting a task in the context of a record. This record will first need to be fetched.
  • Loading the application model. After restarting Indicium or changing the application model, the model needs to be loaded, which may take multiple seconds on top of the request performed.

Once you have installed this version of Indicium Universal, you can see these metrics under Server timings in your browser's developer tools.

Some examples (partially in Dutch) are shown below.

The first request to an application
After changing a column value - 1
After changing a column value - 2
Downloading a file

 Examples of Servertiming metrics

Retry start-up on connection failure

When Indicium encountered a database connection failure during start-up, it would not automatically retry. As of this version of Indicium, a retry occurs every two minutes for up to 10 minutes. The cause of the connection failure could be a temporary issue with the database server or network that resolves itself.

Agent ID on the Indicium landing page

Indicium's landing page will now show the Agent ID of the Indicium instance. This Agent ID corresponds to the IDs that you will find on the 'Agent check in' screen in IAM's advanced menu. It can help you troubleshoot issues regarding system flows. You can also use the Agent ID to test if your load balancer's sticky sessions configuration is working by refreshing the landing page a few times and checking whether the Agent ID stays the same.

Indicium Landing page with Agent ID

 

Language of translated messages by application connector

If a tsf_send_message procedure sends a message to the application connector, the message is now translated into the language of the user who initiated the process flow.

If the application connector is called from a system flow, then the message is translated into the application fallback language.

Messages filtered out in log files

The following messages are now filtered out in the log files because they often occur and can quickly fill the log files, making them difficult to search for errors:

  • schedule_not_found
  • system_flow_already_running
  • system_flow_still_running

Minor fixes and tasks

  • In the database event log, SQL queries and messages did not contain newline characters, which made the statements poorly readable and could cause problems with single-line comments. This has been fixed. 

    Example of a SQL query with newlines

     

Nice, good QOL update!


Messages filtered out in log files

The following messages are now filtered out in the log files because they often occur and can quickly fill the log files, making them difficult to search for errors:

  • schedule_not_found
  • system_flow_already_running
  • system_flow_still_running

Does this mean these errors are not displayed at all, or only once? We indeed see the system_flow_already_running error for each scheduled interval in our Load Balanced environment.


And on another topic (asking here since the Release Notes do mention something about it, while no other existing Community topic deals with it): the required sticky session configuration isn't helping for our ambition of having Zero Downtime Deployments.

We can deploy Indicium and Universal GUI without the environment being unavailable, thanks to Auto Scaling and Rolling Deployments on Elastic Beanstalk. However, it will cause existing sessions to be interrupted when the particular Instance, on which a client runs the session, is taken down for their Deployment. Recommended solutions for AWS refer to ElastiCache, is this supported with Indicium and/or are you working on alternative solutions to support customers with Zero Downtime Deployments?


Hello Arie,

Those three error messages should not appear in the error log at all anymore. Are you sure that the log entries that you are seeing are not old ones? Or that some of your load-balanced instances of Indicium Universal are not running the most recent version?

As for sticky sessions, it is also our ambition to make it possible to horizontally scale Indicium without the need for sticky sessions. This would also enable zero downtime deployments by means of some form of blue-green deployment, such as AWS’s Rolling Deployments. We are currently working on this and we are treating the AWS and Azure platforms as first class citizens with regards to supporting this. We have already begun to incrementally release pieces of the implementation of this feature in our last few releases, but it will take some additional months for this to be fully supported.

I hope this answers your questions.


@Vincent Doppenberg Sorry for the confusion, but I was referring to an older version of Indicium that currently shows these errors. I was just wondering if they will completely disappear (your answer is yes) and if this solution is the most elegant.

I understand it's the easiest and probably makes sense for Load Balanced / horizontally scaled environments. But the system_flow_already_running did help me realize at some point that I had a locally hosted Indicium talking to IAM, which shouldn’t actually be running as an Agent.

And the system_flow_still_running might be a very valuable indicator that a certain system flow is running longer than anticipated (why else does it interfere with the next scheduled flow) or that the setup should be updated to allow for multiple running instances.

I don't recall a situation in which I saw the schedule_not_found error, so can't comment on that one.

Are there other ways to infer the above information?

As for sticky sessions, it is also our ambition to make it possible to horizontally scale Indicium without the need for sticky sessions. This would also enable zero downtime deployments by means of some form of blue-green deployment, such as AWS’s Rolling Deployments. We are currently working on this and we are treating the AWS and Azure platforms as first class citizens with regards to supporting this. We have already begun to incrementally release pieces of the implementation of this feature in our last few releases, but it will take some additional months for this to be fully supported.

Great to hear this! Then I'll leave the heavy lifting to you guys and await your solution :grin:


Hello Arie,

I understand your use for this information in the error log, but we feel that the error log is not the appropriate location for this information. It causes a lot of noise with regard to troubleshooting actual errors, many false reports of these messages as issues and ever-growing error logs that can end up becoming very big depending on the frequency of the schedule.

All of this information should be available to you in IAM as well. In the advanced menu of IAM, you can find several screens that are useful for managing your Indiciums.

For instance, if you want to know which Indiciums are active as agents on any given IAM, then you can find this information on the Agent check in screen. Every Indicium that is active as an agent will report itself to IAM every minute. If you see multiple GUIDs here, then you could consider disabling the agent for all but one of these Indiciums. The Extra information column will tell you where the given Indicium is located. Every Indicium will also display its own GUID on its landing page.

 

If you want to know whether or not a schedule needs to be adjusted due to a system flow being faster or slower than expected, then you can open the Schedule log screen and look at the Start time and End time values of a system flow. If the duration of a system flow is 10 seconds, then it might not be very useful to have it run every second.

 

It is our intention that IAM becomes the hub for all such information. If any useful information is missing here or if it is not easy enough to consume, then please create an idea for the improvements that you would like to see.

I hope this helps.


For instance, if you want to know which Indiciums are active as agents on any given IAM, then you can find this information on the Agent check in screen. 

Good to know, tried finding this screen and realized it apparently has never been added to the IAM Universal menu (we only use the Universal GUI for TEST/PROD…). Raised TCP 2982S to get that resolved.


It is indeed not possible to reach this screen in the Universal GUI at this moment. It is located in the advanced menu of IAM and the Universal GUI does not yet support switching between menu's.

I think we should also consider not hiding these screens in the advanced menu as they are quite useful and people should know about them and be able to find them. An ‘Advanced’ menu group inside of the default menu might be better, because the menu search option would be able to find them then. I will raise this topic internally for the 2022.2 release.


@Vincent Doppenberg In the Universal GUI it is indeed already setup as you suggest, showing the Advanced menu as a standard List bar group. However, your friends of the IAM team tend to forget about Universal GUI setup changes when developing new IAM functionality. The ‘Tenant’ menu item has also been missing for a whole release, for instance :)

 


Due to an oversight, the last release of Indicium Universal (2022.1.13) broke backward compatibility for platform versions 2021.2 and below (Indicium would not start). This has been fixed in version 2022.1.14, which is currently available in TCP.

No action is required if:

  • You are on platform version 2021.3 or newer OR
  • You are on Indicium Universal version 2022.1.12 or older.

Our apologies for any inconvenience caused by this issue.