Terra - Delayed webhook deliveries – Incident details

All systems operational

Delayed webhook deliveries

Resolved
Degraded performance
Started 3 months agoLasted 2 days

Affected

Terra Infrastructure

Operational from 4:00 AM to 6:30 AM, Degraded performance from 6:30 AM to 6:00 AM

Event Delivery

Operational from 4:00 AM to 6:30 AM, Degraded performance from 6:30 AM to 6:00 AM

Updates
  • Resolved
    UTC
    Resolved
    The backlog has been fully delivered. Data during this incident was delayed, not lost.
  • Monitoring
    UTC
    Monitoring
    Delivery of new events is back to normal speed. We are draining the remaining backlog; older events will arrive over the next hours.
  • Identified
    UTC
    Identified
    We identified a large delivery backlog caused by a queueing issue. New events are being prioritised while we work through the backlog.
  • Investigating
    UTC
    Investigating
    We are investigating delayed webhook deliveries. Incoming data is being received and stored; delivery to your endpoints is running behind.