Services Dashboard

SyrahTrader.Services.Scheduler.Tasks.CacheSecurities

// Id: #14802
using SyrahTrader.Scheduler;

var jobsConfigurator = Activate<JobsConfigurator>();
await jobsConfigurator.ExecuteTaskAsync(
    FromJson<Type>("\"SyrahTrader.Services.Scheduler.Tasks.CacheSecurities, SyrahTrader.Services.Scheduler.Tasks, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null\""));

Parameters

RecurringJobId 
"CacheSecurities"
Time 
1776413711
CurrentCulture 
"en-US"
CurrentUICulture 
"en-US"
RetryCount 
4

State

4/17/2026 8:52:25 AM (+22m 41.222s) Succeeded

Latency:
14m 33.294s
Duration:
22m 41.174s

+2.676s Processing

Server:
rd2016427ee395:4156
Worker:
c8c1aeb4

+2m 27.415s Enqueued

Triggered by DelayedJobScheduler

+5ms Scheduled

Retry attempt 4 of 10: Cache refresh failed - count mismatch. Expected: …

Enqueue at:
4/17/2026 8:29:38 AM

+1m 53.805s Failed

An exception occurred during performance of the job.

System.InvalidOperationException (rd2016427ee395:4156)

Cache refresh failed - count mismatch. Expected: 15497, Actual: 15511

System.InvalidOperationException: Cache refresh failed - count mismatch. Expected: 15497, Actual: 15511
   at SyrahTrader.DAL.AzureCache.SecuritiesCacheCoordinator.<StoreAllSecuritiesInRedisAsync>d__31.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at SyrahTrader.DAL.AzureCache.SecuritiesCacheCoordinator.<StoreAllSecuritiesInRedisAsync>d__31.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.DAL.AzureCache.SecuritiesCacheCoordinator.<RefreshCacheWithSecuritiesAsync>d__25.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.DAL.AzureCache.Repositories.CachedSecuritiesRepository.<ForceRefreshCacheAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.MarketData.SecuritiesService.<ForceRefreshCacheAsync>d__87.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.Services.Scheduler.Tasks.CacheSecurities.<DoActionAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.Services.Scheduler.Tasks.SyrahTraderTaskBase.<ExecuteAsync>d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.Services.Scheduler.Tasks.SyrahTraderTaskRunner.<ExecuteTaskAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.Scheduler.JobsConfigurator.<ExecuteTaskAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()

+9.044s Processing

Server:
rd2016427ee395:4156
Worker:
c8c1aeb4

+2m 6.068s Enqueued

Triggered by DelayedJobScheduler

+5ms Scheduled

Retry attempt 3 of 10: Cache refresh failed - count mismatch. Expected: …

Enqueue at:
4/17/2026 8:24:57 AM

+1m 54.121s Failed

An exception occurred during performance of the job.

System.InvalidOperationException (rd2016427ee395:4156)

Cache refresh failed - count mismatch. Expected: 15497, Actual: 15511

System.InvalidOperationException: Cache refresh failed - count mismatch. Expected: 15497, Actual: 15511
   at SyrahTrader.DAL.AzureCache.SecuritiesCacheCoordinator.<StoreAllSecuritiesInRedisAsync>d__31.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at SyrahTrader.DAL.AzureCache.SecuritiesCacheCoordinator.<StoreAllSecuritiesInRedisAsync>d__31.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.DAL.AzureCache.SecuritiesCacheCoordinator.<RefreshCacheWithSecuritiesAsync>d__25.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.DAL.AzureCache.Repositories.CachedSecuritiesRepository.<ForceRefreshCacheAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.MarketData.SecuritiesService.<ForceRefreshCacheAsync>d__87.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.Services.Scheduler.Tasks.CacheSecurities.<DoActionAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.Services.Scheduler.Tasks.SyrahTraderTaskBase.<ExecuteAsync>d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.Services.Scheduler.Tasks.SyrahTraderTaskRunner.<ExecuteTaskAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.Scheduler.JobsConfigurator.<ExecuteTaskAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()

+123ms Processing

Server:
rd2016427ee395:4156
Worker:
c8c1aeb4

+59.928s Enqueued

Triggered by DelayedJobScheduler

+7ms Scheduled

Retry attempt 2 of 10: Cache refresh failed - count mismatch. Expected: …

Enqueue at:
4/17/2026 8:21:01 AM

+1m 53.219s Failed

An exception occurred during performance of the job.

System.InvalidOperationException (rd2016427ee395:4156)

Cache refresh failed - count mismatch. Expected: 15497, Actual: 15511

System.InvalidOperationException: Cache refresh failed - count mismatch. Expected: 15497, Actual: 15511
   at SyrahTrader.DAL.AzureCache.SecuritiesCacheCoordinator.<StoreAllSecuritiesInRedisAsync>d__31.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at SyrahTrader.DAL.AzureCache.SecuritiesCacheCoordinator.<StoreAllSecuritiesInRedisAsync>d__31.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.DAL.AzureCache.SecuritiesCacheCoordinator.<RefreshCacheWithSecuritiesAsync>d__25.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.DAL.AzureCache.Repositories.CachedSecuritiesRepository.<ForceRefreshCacheAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.MarketData.SecuritiesService.<ForceRefreshCacheAsync>d__87.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.Services.Scheduler.Tasks.CacheSecurities.<DoActionAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.Services.Scheduler.Tasks.SyrahTraderTaskBase.<ExecuteAsync>d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.Services.Scheduler.Tasks.SyrahTraderTaskRunner.<ExecuteTaskAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.Scheduler.JobsConfigurator.<ExecuteTaskAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()

+7.052s Processing

Server:
rd2016427ee395:4156
Worker:
c8c1aeb4

+37.979s Enqueued

Triggered by DelayedJobScheduler

+8ms Scheduled

Retry attempt 1 of 10: Cache refresh failed - count mismatch. Expected: …

Enqueue at:
4/17/2026 8:18:00 AM

+2m 16.650s Failed

An exception occurred during performance of the job.

System.InvalidOperationException (rd2016427ee395:4156)

Cache refresh failed - count mismatch. Expected: 15497, Actual: 15511

System.InvalidOperationException: Cache refresh failed - count mismatch. Expected: 15497, Actual: 15511
   at SyrahTrader.DAL.AzureCache.SecuritiesCacheCoordinator.<StoreAllSecuritiesInRedisAsync>d__31.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at SyrahTrader.DAL.AzureCache.SecuritiesCacheCoordinator.<StoreAllSecuritiesInRedisAsync>d__31.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.DAL.AzureCache.SecuritiesCacheCoordinator.<RefreshCacheWithSecuritiesAsync>d__25.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.DAL.AzureCache.Repositories.CachedSecuritiesRepository.<ForceRefreshCacheAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.MarketData.SecuritiesService.<ForceRefreshCacheAsync>d__87.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.Services.Scheduler.Tasks.CacheSecurities.<DoActionAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.Services.Scheduler.Tasks.SyrahTraderTaskBase.<ExecuteAsync>d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.Services.Scheduler.Tasks.SyrahTraderTaskRunner.<ExecuteTaskAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SyrahTrader.Scheduler.JobsConfigurator.<ExecuteTaskAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()

+5.164s Processing

Server:
rd2016427ee395:4156
Worker:
c8c1aeb4

+20ms Enqueued

Triggered by recurring job scheduler

4/17/2026 8:15:11 AM Created