Skip to main content
Solved

SQL blocking caused by System Flow

Harm Horstman
Superhero
Forum|alt.badge.img+21

Task is running smooth when started from inside the application, but blocking the database when used in a system flow.

What can be the reason?

See log below:

2023-05-01T00:15:00.0191596+02:00  [ERR] Error scheduling system flow '"sf_send_notifications"' for application 9. (b95a9b29)
System.Exception: An exception occurred while trying to log the start of the system flow.
 ---> Microsoft.Data.SqlClient.SqlException (0x80131904): <msg id="system_flow_still_running"></msg>
   at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at Microsoft.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)
   at Microsoft.Data.SqlClient.SqlCommand.CompleteAsyncExecuteReader(Boolean isInternal, Boolean forDescribeParameterEncryption)
   at Microsoft.Data.SqlClient.SqlCommand.InternalEndExecuteNonQuery(IAsyncResult asyncResult, Boolean isInternal, String endMethod)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryInternal(IAsyncResult asyncResult)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryAsync(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location ---
   at Indicium.BackgroundServices.SystemFlowScheduler.<>c__DisplayClass21_0.<<runSystemFlow>b__0>d.MoveNext() in C:\azp\agent\_work\1\s\src\Indicium\BackgroundServices\SystemFlowScheduler.cs:line 191
--- End of stack trace from previous location ---
   at Indicium.BackgroundServices.SystemFlowScheduler.withLogging(String logMessage, Func`1 task) in C:\azp\agent\_work\1\s\src\Indicium\BackgroundServices\SystemFlowScheduler.cs:line 293
ClientConnectionId:12f4a183-4ed3-4c71-9b2a-64d6492df283
Error Number:50000,State:10,Class:16
ClientConnectionId before routing:86a37381-2abf-4a8a-97f6-68dca8401563
Routing Destination:cc4668c3ced6.tr2382.westeurope1-a.worker.database.windows.net,11051
   --- End of inner exception stack trace ---
   at Indicium.BackgroundServices.SystemFlowScheduler.withLogging(String logMessage, Func`1 task) in C:\azp\agent\_work\1\s\src\Indicium\BackgroundServices\SystemFlowScheduler.cs:line 293
   at Indicium.BackgroundServices.SystemFlowScheduler.runSystemFlow(Int32 guiApplID, TSFApplication application, FullProcessFlow systemFlow, DateTime scheduledTime) in C:\azp\agent\_work\1\s\src\Indicium\BackgroundServices\SystemFlowScheduler.cs:line 218
   at Indicium.BackgroundServices.SystemFlowScheduler.scheduleSystemFlow(Int32 guiApplID, String systemFlowID, DateTime scheduledTime) in C:\azp\agent\_work\1\s\src\Indicium\BackgroundServices\SystemFlowScheduler.cs:line 165
2023-05-01T01:15:00.2037387+02:00  [ERR] Error scheduling system flow '"sf_send_notifications"' for application 9. (b95a9b29)
System.Exception: An exception occurred while trying to log the start of the system flow.
 ---> Microsoft.Data.SqlClient.SqlException (0x80131904): <msg id="system_flow_still_running"></msg>
   at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at Microsoft.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)
   at Microsoft.Data.SqlClient.SqlCommand.CompleteAsyncExecuteReader(Boolean isInternal, Boolean forDescribeParameterEncryption)
   at Microsoft.Data.SqlClient.SqlCommand.InternalEndExecuteNonQuery(IAsyncResult asyncResult, Boolean isInternal, String endMethod)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryInternal(IAsyncResult asyncResult)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryAsync(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location ---

 

Best answer by Vincent Doppenberg

Hello Harm,

The errors that you are seeing in the log can typically be ignored. Indicium should not log these errors, but there was an issue recently that caused these errors to show up in the log again. This was fixed in version 2023.1.13.1.

The error log aside, is the process flow actually hanging? You said that you started the process flow with Postman, does the request run endlessly/until it reaches a timeout? If so, then it seems to have nothing to do with system flows, because it occurs when calling it as a regular process flow as well. This makes me think that it's simply an error in the process flow. Can you try to isolate the issue, for example by:

  • Turning parts of the process flow on or off, including process procedures
  • Calling parts of the process flow individually with Postman (like tasks)
  • Putting a task or some other kind of trigger in front of the process flow that allows you to test it with the Windows GUI, with the process flow monitor
  • Checking if certain parts of the process flow could take very long if certain input parameters are not set properly. For example, you wrote that a task is fast on its own but blocking inside of a system flow. Are the task parameters being set correctly in the process flow?

I hope that you provide some more information given these suggestions.

View original
Logic will get you from A to B. Imagination will take you everywhere (Albert Einstein)
Marius Korff
1 person likes this
  • Marius Korff
    Marius Korff
Did this topic help you find an answer to your question?
This topic has been closed for comments

5 replies

Mark Jongeling
Administrator
Forum|alt.badge.img+23

Hi Harm, I can see the <msg id="system_flow_still_running"></msg> message. Can you see if any log items for system flow sf_send_notifications in application 9 has not ended?

In IAM > Applications > Scheduled system flows > Log, I suspect there is one or more items without an end date. You can safely update the log item to set the end date and abandoned to 1.

That should solve it I believe😄

"Wake me up before you low-code"
Harm Horstman
Superhero
Forum|alt.badge.img+21

Hello Mark,

I did not found any line with an empty End Date. 

I see the System Flow is hanging when starting it via a request in Postman

Logic will get you from A to B. Imagination will take you everywhere (Albert Einstein)
Mark Jongeling
Administrator
Forum|alt.badge.img+23

Does Indicium have rights on the IAM database? Seems unlikely but I'm guessing at this point. This error should be suppressed as far as I'm aware of.

"Wake me up before you low-code"
Harm Horstman
Superhero
Forum|alt.badge.img+21

Indicium runs with a “tsf_admin” account with a “db_owner” role on both the END and IAM database.

The WebGUI and Universal GUI are running fine when logging in as “tsf_admin”

Logic will get you from A to B. Imagination will take you everywhere (Albert Einstein)
V
Moderator
Forum|alt.badge.img+4

Hello Harm,

The errors that you are seeing in the log can typically be ignored. Indicium should not log these errors, but there was an issue recently that caused these errors to show up in the log again. This was fixed in version 2023.1.13.1.

The error log aside, is the process flow actually hanging? You said that you started the process flow with Postman, does the request run endlessly/until it reaches a timeout? If so, then it seems to have nothing to do with system flows, because it occurs when calling it as a regular process flow as well. This makes me think that it's simply an error in the process flow. Can you try to isolate the issue, for example by:

  • Turning parts of the process flow on or off, including process procedures
  • Calling parts of the process flow individually with Postman (like tasks)
  • Putting a task or some other kind of trigger in front of the process flow that allows you to test it with the Windows GUI, with the process flow monitor
  • Checking if certain parts of the process flow could take very long if certain input parameters are not set properly. For example, you wrote that a task is fast on its own but blocking inside of a system flow. Are the task parameters being set correctly in the process flow?

I hope that you provide some more information given these suggestions.

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings