+11
Since the new version the generation process is done by Indicium. However, often this message is displayed;
It usually disappears/appears a few times and the generation does not fail, but still, It does not look good to me.
What is the cause of this message?
Thanks!
Best answer by pauloportu
View original
+21
Hi,
This message is given when Indicium does not pick up new system flows for at least 61 seconds. This usually means that Indicium is not running anymore and should be reactivated.
+14
Hi,
This message is given when Indicium does not pick up new system flows for at least 61 seconds. This usually means that Indicium is not running anymore and should be reactivated.
Hi
How can we tweak this setting. My partner here in Brazil encounters this message more than I do. How does it exactly work? We have the Indicium running in the cloud. When he gets the message more than I , does this directly relate to internet speeds? And is there a way to tweak with settings?
Because I have been monitoring the Indicium and it hasn't gone offline yet, but still we receive these messages that: 'we stopped hearing from Indicium'.
+21
When going into IAM, navigate to Applications, select the SF application, navigate to detail tab Scheduled system flows. In here, the system flow "system_flow_agent_check_in” is the one I'm interested in. It should have a log record every 30 seconds like this; is that also the case for you both or is it different?:
(Ignore the new available Task, that's 2022.1 functionality 
)
+21
This message is given when Indicium does not pick up new system flows for at least 61 seconds.
Originally it was 61 seconds but later we changed it to 65 seconds I just remembered, so we did increase it a bit. Looking forward seeing the schedule log 
+21
It should still update the agent check-in datetime every 30 seconds so the 61 or 65 second-check is not per se the issue here.
I advise you to create a TCP ticket in which we can take a deeper look at it. What we need is a couple of screenshots of the Schedule log, the “We stopped hearing from Indicium” text appearing, the Indicium error log of the day when the texts appear and the result of the following SQL query run on the SF database:
select * from last_agent_check_in
probably redundant info, but same here:
After restart indicium, no satisfactory results ubfortunatly.
please advise,
thank you,
Paul
+21
probably redundant info, but same here:
After restart indicium, no satisfactory results ubfortunatly.
please advise,
thank you,
Paul
Hi
We do need more information to help you with this. In normal circumstances this functionality should function just fine. Does Indicium show any errors in the error log? Does the system flow system_flow_agent_check_in have an active schedule?
Every case can be different, so I would advise you all to create a TCP ticket for this so we can investigate this.
I advise you to create a TCP ticket in which we can take a deeper look at it. What we need is a couple of screenshots of the Schedule log, the “We stopped hearing from Indicium” text appearing, the Indicium error log of the day when the texts appear and the result of the following SQL query run on the SF database:
select * from last_agent_check_in
after checking our logs we found the following:
2021-12-13T14:27:30.0377288+01:00 [ERR] Error scheduling system flow 'system_flow_agent_check_in' for application 14. (34d69cfe)
Microsoft.Data.SqlClient.SqlException (0x80131904): <msg id="system_flow_still_running"></msg>
at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
at Microsoft.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)
at Microsoft.Data.SqlClient.SqlCommand.CompleteAsyncExecuteReader(Boolean isInternal, Boolean forDescribeParameterEncryption)
at Microsoft.Data.SqlClient.SqlCommand.InternalEndExecuteNonQuery(IAsyncResult asyncResult, Boolean isInternal, String endMethod)
at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryInternal(IAsyncResult asyncResult)
at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryAsync(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location ---
at Indicium.BackgroundServices.SystemFlowSchedulerBase.RunSystemFlow(Int32 guiApplID, TSFApplication application, FullProcessFlow systemFlow, DateTime scheduledTime) in C:\azp\agent\_work\1\s\src\Indicium\BackgroundServices\SystemFlowSchedulerBase.cs:line 112
at Indicium.BackgroundServices.SystemFlowScheduler.RunSystemFlow(Int32 guiApplID, TSFApplication application, FullProcessFlow systemFlow, DateTime scheduledTime) in C:\azp\agent\_work\1\s\src\Indicium\BackgroundServices\SystemFlowScheduler.cs:line 90
at Indicium.BackgroundServices.SystemFlowSchedulerBase.ScheduleSystemFlow(Int32 guiApplID, String systemFlowID, DateTime scheduledTime) in C:\azp\agent\_work\1\s\src\Indicium\BackgroundServices\SystemFlowSchedulerBase.cs:line 84
ClientConnectionId:5f93a48f-65ed-41ac-a260-7e708c204d01
Error Number:50000,State:10,Class:16
------------
maybe some recognizes this error? Or maybe TW team can work with the info.
thank you,
Paul
+21
+11
But you’re going to fix it anyway right 
Aside from that, happy to see i’m not the only one with these issues. Perhaps there is more at the handje 
then we think!
+21
But you’re going to fix it anyway right
Aside from that, happy to see i’m not the only one with these issues. Perhaps there is more at the handje then we think!
Internally we haven't heard any of these issues and personally only encounter this problem when (indeed) Indicium is not running. I'm also surprised as the logic behind it is pretty simple. And of course we want to solve the issue as soon as we can 
Every 30 seconds, the System flow will update a datetime field in the Software Factory to determine if Indicium is still picking up and running new system flows. If the difference between current datetime and the logged datetime is greater than 65 seconds, we can safely assume Indicium is not picking up new system flows.
But it seems like your Indicium is picking up system flows selectively, or not updating the datetime field for some reason. ← That is what we need to find out 
+11
Sure, sounds good. Just to check, is TW going to look into this now, or do you need more from us? Just so we’re not waiting on each other.
I don’t think there is anything we can do right now to be honest.
+21
If you all could create a ticket in TCP, that would be great 
I cannot help much here sadly as each of your cases may need a different solution. That requires deeper investigation and that is something the Community website is not suited for.
+11
If you all could create a ticket in TCP, that would be great
I cannot help much here sadly as each of your cases may need a different solution. That requires deeper investigation and that is something the Community website is not suited for.
Done. Hopefully we can find it.
+14
To add to this discussion still. The log files show a log file entry every 30 seconds even with the text that we stopped hearing from indicium. What works is to go to indicium site and login. When I go back to SF the message disappears (after a refresh)…
+21
To add to this discussion still. The log files show a log file entry every 30 seconds even with the text that we stopped hearing from indicium. What works is to go to indicium site and login. When I go back to SF the message disappears (after a refresh)…
Again, I recommend creating a ticket for this in TCP so we can investigate your case.
Time out is probably the problem:
In IIS, ensure in the application pool's Advanced Settings that the Idle Time-out (minutes) is set to 0 (disabled) instead of 20, so scheduled process flows will continue running even if there is no user activity.
+11
Time out is probably the problem:
In IIS, ensure in the application pool's Advanced Settings that the Idle Time-out (minutes) is set to 0 (disabled) instead of 20, so scheduled process flows will continue running even if there is no user activity.
This was already the case in my IIS.
Although I still saw the error, I have a feelings it's less often. I'll keep an eye out and after today check the logs.
It should still update the agent check-in datetime every 30 seconds so the 61 or 65 second-check is not per se the issue here.
I advise you to create a TCP ticket in which we can take a deeper look at it. What we need is a couple of screenshots of the Schedule log, the “We stopped hearing from Indicium” text appearing, the Indicium error log of the day when the texts appear and the result of the following SQL query run on the SF database:
select * from last_agent_check_in
2021-12-16T09:06:08.2865241+00:00 [ERR] Error scheduling system flow 'system_flow_generate_definition' for application 11. (0c778fc7)
Microsoft.Data.SqlClient.SqlException (0x80131904): Transaction (Process ID 209) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
at Microsoft.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)
at Microsoft.Data.SqlClient.SqlCommand.CompleteAsyncExecuteReader(Boolean isInternal, Boolean forDescribeParameterEncryption)
at Microsoft.Data.SqlClient.SqlCommand.InternalEndExecuteNonQuery(IAsyncResult asyncResult, Boolean isInternal, String endMethod)
at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryInternal(IAsyncResult asyncResult)
at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryAsync(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location ---
at Indicium.BackgroundServices.SystemFlowSchedulerBase.RunSystemFlow(Int32 guiApplID, TSFApplication application, FullProcessFlow systemFlow, DateTime scheduledTime) in C:\azp\agent\_work\1\s\src\Indicium\BackgroundServices\SystemFlowSchedulerBase.cs:line 112
at Indicium.BackgroundServices.SystemFlowScheduler.RunSystemFlow(Int32 guiApplID, TSFApplication application, FullProcessFlow systemFlow, DateTime scheduledTime) in C:\azp\agent\_work\1\s\src\Indicium\BackgroundServices\SystemFlowScheduler.cs:line 90
at Indicium.BackgroundServices.SystemFlowSchedulerBase.ScheduleSystemFlow(Int32 guiApplID, String systemFlowID, DateTime scheduledTime) in C:\azp\agent\_work\1\s\src\Indicium\BackgroundServices\SystemFlowSchedulerBase.cs:line 84
ClientConnectionId:cebd2f0d-f4be-42c7-b3e5-a33b7b1c34ad
Error Number:1205,State:52,Class:13
+11
+21
We are getting the exact same message in SF after upgrading from 2021.1 to 2022.1 yesterday. This rendered the SF almost unusable but I also don’t really want to downgrade after putting all the time in the upgrade process. I don’t get why this happens. The deployment tool displayed green arrows after upgrading all databases, indicium and the windows GUI.
Online documentation doesn’t really cover how to deal with this issue. Also how does SF know where to contact Indicium in the first place? I can’t find any setting for this.
Indicium is definitely running. What is going on here?
+21
Hi
Can you check the Indicium log? Maybe there's an error hindering this process. Did you upgrade to 2022.1 with the latest installation package; when did you download it?
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.
