Skip to main content

I'm having quite some disconnects with latest Indicium (2025.1.10) did anything change? 

I'm able to generate, validate, generate source code..  and with execute it sometimes already happens with the checks and most frequent with the indexes. 

---> Microsoft.Data.SqlClient.SqlException (0x80131904): A transport-level error has occurred when sending the request to the server. (provider: TCP Provider, error: 35 - An internal exception was caught)

       ---> System.IO.IOException: Unable to write data to the transport connection: Connection reset by peer.

       ---> System.Net.Sockets.SocketException (104): Connection reset by peer

 

Update:

I just validated, by re-deploying another environment still on 2024.3 and this one doesn't have the disconnects. while it runs on the same specs (SQL version 2022 CU16). 

 

Treatment of disconnects:

And can the platform handle these better? Now it's stuck.. you have to manually cancel the job.. it should detect this and retry or stop..  

Hello Freddy,

There have been no changes to how Indicium handles database connections. The error message indicates that the peer (i.e. the database server) closed the connection. It would seem that the cause of these connectivity issues is the database server or perhaps the network connection, rather than Indicium.

I agree that such disconnects could probably be handled more gracefully by the jobs in the SF. Can you raise a ticket for this in TCP for the SF?


I'm not exactly sure we can detect these issues if the system flow stops abruptly. The job would be in a state where it's considered active, but not actually actively execution anything. But we can look into finding other clever ways of handling such cases.


I'm not exactly sure we can detect these issues if the system flow stops abruptly. The job would be in a state where it's considered active, but not actually actively execution anything. But we can look into finding other clever ways of handling such cases.

We'll indicium knows and could inform..  what I don't get is that Universal works, only with the SF I have these issues…  


Hello Freddy,

There have been no changes to how Indicium handles database connections. The error message indicates that the peer (i.e. the database server) closed the connection. It would seem that the cause of these connectivity issues is the database server or perhaps the network connection, rather than Indicium.

I agree that such disconnects could probably be handled more gracefully by the jobs in the SF. Can you raise a ticket for this in TCP for the SF?

Done 11182S


@Vincent Doppenberg ​@Leon Kroon are you guys sure really nothing changed? I keep getting the connection reset by peer. 

---> Microsoft.Data.SqlClient.SqlException (0x80131904): A transport-level error has occurred when sending the request to the server. (provider: TCP Provider, error: 35 - An internal exception was caught)

       ---> System.IO.IOException: Unable to write data to the transport connection: Connection reset by peer.

       ---> System.Net.Sockets.SocketException (104): Connection reset by peer

 

From the same container I'm trying to find out what happens..  I've added FreeTDS to the container to test connections with TLS and without.. and all work without issues..  but for some reason all process flows end with this same error message. 

There were no updates to any library of such?  I'm lost to what has changed and is causing this problems. I don't have these disconnects for example from SMSS. 

 

Is there perhaps a cipher mismatch with the .NET client? There is a know issue with sql-server on Linux that prevents using all ciphers..  

2025-03-05 20:05:58.74 Server      Successfully initialized the TLS configuration. Allowed TLS protocol versions are '1.2']. Allowed TLS ciphers are ]'ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256'].

 


Hi ​@Freddy , as far as I can see, the only difference between the 2024.3.15 and 2025.1.10 images is .NET 8.0.13 instead of 8.0.12, the port, and of course Indicium.

The Thinkwise demo environment also runs on 2025.1, using the container images from our registry. None of these environments are experiencing connection issues. 

Maybe there's a firewall or network policy between the container instance and the database server? 


Hi ​@Freddy , as far as I can see, the only difference between the 2024.3.15 and 2025.1.10 images is .NET 8.0.13 instead of 8.0.12, the port, and of course Indicium.

The Thinkwise demo environment also runs on 2025.1, using the container images from our registry. None of these environments are experiencing connection issues. 

Maybe there's a firewall or network policy between the container instance and the database server? 

It's not the firewall. I deactivated the firewall. I even ran a TCP dump, there I see the disconnect (RST) but no indication of what is causing it in terms of SQL server vs the container networking backend. It's frustrating at the least...


@Mark Jongeling ​@Leon Kroon ​@Vincent Doppenberg do any of you know if there is anything going on with the store.thinkwisesoftware.com? 

I'm not sure what caused my issues before, but I got to state that things start to look working stable again, but for somer reason the connections to the store don't work. 

System.Net.Http.HttpRequestException: Connection refused (store.thinkwisesoftware.com:443)
 ---> System.Net.Sockets.SocketException (111): Connection refused


The Thinkstore has been running flawlessly for the weekend. I don’t see any errors in the Indicium log too. So I'm not sure what causes this error to occur for you. Perhaps our Infrastructure team can see in a log why the connection was refused. Feel free to create a ticket for them.


Hi,

This morning, my SF Indicium failed unexpectedly. Upon investigating, I discovered a 35MB log file from the weekend containing multiple issues related to connecting to the IAM database and the Thinkstore. I've never encountered this problem before and can’t explain it at the moment. It seems something might have gone wrong…
After some resetting of things Indicium is now working, but not sure what caused this. We haven’t made any changes, and had no other issues with systems. 

2025-03-10T02:10:35.2914183+00:00  3ERR] Error in process procedure of process action "check_queued_download" in process flow "system_flow_thinkstore_download" of application 25. (b2ae20de)
2025-03-10T02:10:35.2980881+00:00  8ERR] Error scheduling system flow '"system_flow_thinkstore_download"' for application 25. (b95a9b29)


That is indeed strange as we didn't change any part of this system flow for quite a while. Also any code in the process action "check_queued_download" in process flow "system_flow_thinkstore_download" should not cause an error to occur.

But with the limited amount of information, it is difficult to determine what went wrong there. Glad a restart of Indicium resolved the issue. I don’t think this is related with the issue Freddy is describing.


The Thinkstore has been running flawlessly for the weekend. I don’t see any errors in the Indicium log too. So I'm not sure what causes this error to occur for you. Perhaps our Infrastructure team can see in a log why the connection was refused. Feel free to create a ticket for them.

Created: 11261S 


The errors followed, to my best guess, an openssl update on Redhat which also broke the rhell container image from microsoft. Finally fixed it by downgrading to CU16 image and coupling it with an RPM install of FTS. All environments are running stable now. We are restarting the updating towards 2025.1


Reply