Our data ingestion pipeline has kept up with the queue since our last update, and our process to gradually backfill data that may have been missing yesterday has been completed ahead of schedule. This incident is resolved. We will post more details after we complete our RCA.
Posted Sep 03, 2019 - 10:34 EDT
Update
We are continuing to monitor for any further issues.
Posted Sep 02, 2019 - 21:42 EDT
Monitoring
We unfortunately needed to drop some work from the queue to catch up. We are no longer lagging behind, but due to this there is data from earlier today that has still not been synced. We will gradually backfilling the missing data from the past day over the next day or so, but data for Monday may be incomplete until late Tuesday or early Wednesday.
We will be conducting an RCA tomorrow and we will be posting the outcome later this week.
Posted Sep 02, 2019 - 21:22 EDT
Update
We are still behind with processing records, but have introduced several modifications to work through the backlog faster over the course of the day. We are still investigating opportunities to clear the backlog faster.
Posted Sep 02, 2019 - 17:13 EDT
Identified
We are seeing delays in our data ingestion pipeline for Velocity. New data may take longer than expected to appear while this incident is ongoing. We apologize for the disruption, and are working on remediations.