errorcoreDemo

Error Context Layer / billing-worker / production / inc_2d91ab

fatal

Invoice settlement lock timeout

QueryTimeoutError invoice settlement query exceeded 8000ms

Status
open
Priority
P0
Events
42
Capture
partial

Context Timeline

Request entry, outbound I/O, local state records, and the throw frame.

request -> I/O -> local state -> throw
1
Request entered the service

JOB queue:settle-invoices reached billing-worker in iad1.

requestId = job_settle_8842capturedtraceId = 9c04b7a2e851d6f30b27c49ae15f8d62captured
queue:settle-invoices
2
Worker started batch
batchSize = 500capturedjob.data.cursor = "2026-05-01T00:00:00Z"captured
3
Overdue batch resolved
invoiceIds = [184 ids truncated]truncated
4
Lock query ran past the deadline
transaction = opencaptured
Invoice lock query / postgres.invoices / timeout / 8420ms /
5
Timeout while waiting for invoice locks
await invoiceRepo.lockOverdueBatch(cursor, batchSize);
batchSize500captured
src/jobs/settle.ts:133:19

Runtime evidence

Values, bound inputs, I/O records, and throw frame metadata.

partial
Captured locals
batchSize500captured
batchSize = 500 / invoiceIds = [184 ids truncated]
Origin I/O
Invoice lock querypostgres.invoicestimeout / 8420ms / timeout
Bound inputsjob.id = "settle_8842" / job.data.cursor = "2026-05-01T00:00:00Z"
Capture completenesstruncated locals / captured args / truncated I/O68 KB package / encrypted yes
Trace pathJOB queue:settle-invoicesInvoice lock query: timeoutbatchSize = 500src/jobs/settle.ts:133request / I/O / local / stack

Stack trace entry point

The stack identifies where Errorcore attached the context above; it is the entry point, not the full answer.

settleInvoiceBatchsrc/jobs/settle.ts:133:19
runJobsrc/jobs/worker.ts:57:7

Request context

LineJOB queue:settle-invoicesRequest IDjob_settle_8842Status500Regioniad1Routequeue:settle-invoicesBodyjob keys capturedTrace9c04b7a2e851d6f30b27c49ae15f8d62Span4e7b08c25f9a13d6

Runtime and I/O

Nodev22.14.0Hostiad-worker-02Release2026.05.08-4f17c2SDK0.9.0Memoryrss 512 MB, heap 166 MBLoop lag17 ms
Invoice lock query

timeout

postgres.invoices
timeout8420ms

Capture completeness

QualitypartialLocalstruncatedArgscapturedI/OtruncatedEncryptedyesPayload68 KB
truncatedlocals.invoiceIdsarray exceeded configured payload limitmissingio.redisHeartbeatring buffer kept the database event
policy appliedpayload 68 KB