Author

Pro6pp

Pro6pp Team

Root Cause Analysis: Pro6pp Service Outage - September 30, 2025

2025-10-01

Image title

Executive Summary

On September 30, 2025, Pro6pp experienced a service outage lasting 2 hours and 10 minutes, from 9:30 AM to 11:40 AM CEST. Publishing the analysis took a while as we were verifying the cause with our hosting provider, which added extra detail to the analysis with insights on the AMD vs Intel differences. The outage was caused by a server rescaling to an infrastructure with inadequate single-core performance for our web traffic workload.

Incident Timeline

  • September 30, 2:00 AM: Server rescaled.
  • September 30, 9:30 AM: Service became unresponsive; outage began.
  • September 30, 11:40 AM: Service restored after rescaling back to the original server type.
  • Total outage duration: 2 hours 10 minutes.

Root Cause

The primary root cause was CPU architecture incompatibility with our workload requirements. The new server's processors resulted in degraded single-core performance, which is critical for our web server workload.

Resolution

Service was restored by rescaling back to the original server configuration, immediately resolving the performance issues.

Preventive Measures

We are taking several steps to prevent similar incidents in the future, including:

  • Documenting emergency troubleshooting procedures.
  • Improving our alert routing and response coordination.
  • Relocating our monitoring tools to a separate server.
  • Thoroughly benchmarking any future server configuration changes before production deployment.

Conclusion

We sincerely apologize for the service disruption. We are committed to preventing similar incidents in the future and appreciate your patience and understanding.