Postmortem -
Read details
Jun 30, 15:30 UTC
Resolved -
After being placed under extended monitoring, the incident has now been successfully resolved. Aeris engineers have confirmed that full system stability has been restored and redundancy has been fully normalized. Please see the details below for additional information
Description: Connectivity issue for US PGW
Incident case number: 177393
If you have any additional questions, please contact support@aeris.net and reference the case number above.
Jun 25, 17:52 UTC
Monitoring -
UPDATE 28: This is to provide an update: the Aeris team has taken corrective action and has been observing service recovery since 20:17 UTC. Aeris will continue to monitor the situation and provide further updates as more information becomes available.
Description: Connectivity issue for US PGW
Incident case number: 177393
If you have any additional questions, please contact support@aeris.net and reference the case number above.
Jun 23, 20:43 UTC
Update -
UPDATE 27: This is an update regarding the earlier reported incident. Aeris Engineering continues to closely monitor service conditions while actively working to resolve an issue impacting Fusion Global and Global GSM SIM data session traffic. Further updates will be provided as new information becomes available
Description: Connectivity issue for US PGW
Incident case number: 177393
If you have any additional questions, please contact support@aeris.net and reference the case number above.
Jun 23, 19:28 UTC
Update -
UPDATE 26: This message provides a status update on the previously reported incident. Aeris Engineering is actively engaged in monitoring service health and addressing an issue affecting Fusion Global and Global GSM SIM data session traffic. Additional updates will be shared as more details become available.
Description: Connectivity issue for US PGW
Incident case number: 177393
If you have any additional questions, please contact support@aeris.net and reference the case number above.
Jun 23, 18:27 UTC
Update -
UPDATE 25: This is to provide an update on the incident reported earlier. Aeris Engineering continues to monitor and validate all services. Aeris Engineering has observed an issue impacting Fusion Global and Global GSM SIM data sessions. The Aeris team continues to investigate, and we will provide further updates as more information becomes available.
Description: Connectivity issue for US PGW
Incident case number: 177393
If you have any additional questions, please contact support@aeris.net and reference the case number above.
Jun 23, 17:41 UTC
Identified -
UPDATE 24: This is to provide an update on the incident reported earlier. Aeris Engineering continues to monitor and validate all services. We will provide further updates as more information becomes available.
Description: Connectivity issue for US PGW
Incident case number: 177393
If you have any additional questions, please contact support@aeris.net and reference the case number above.
Jun 23, 16:37 UTC
Update -
UPDATE [23]: All 6 VMs are up, with AAA traffic passing normally. Entering a 3-hours monitoring period to verify complete service restoration to previous baselines. Health checks are ongoing.
Description: [Connectivity issue for US PGW]
Incident case number: [177393]
Jun 23, 15:20 UTC
Update -
UPDATE 22: Aeris engineers are currently addressing an authentication and routing issue specifically affecting data sessions on a specific partner. Our team has implemented a partial fix by performing a targeted restart of our authentication servers, which has successfully allowed some affected devices to establish sessions and resume data traffic. To fully restore connectivity, we are now proceeding with a sequential, rolling restart of the remaining authentication servers to clear any stale sessions.
Please see below for the impacted services:
Description: Connectivity issue for US PGW
Incident case number: 177393
Jun 23, 14:01 UTC
Update -
UPDATE 21: Aeris team continue to troubleshoot the remaining issues, Currently we can observe gradual recovery for the session count and traffic across multiple partners. Further updates will be provided once they become available.
Please see below for the impacted services:
Description: Connectivity issue for US PGW
Incident case number: 177393
Jun 23, 13:07 UTC
Update -
UPDATE 21: Aeris team performed the reboot at 11:25 AM UTC having partially restored services, we are still looking into the remaining issues, new updates will be provided as soon as they become available.
Please see below for the impacted services:
Description: Connectivity issue for US PGW
Incident case number: 177393
Jun 23, 12:17 UTC
Update -
UPDATE 20: This is to inform you that Aeris will shortly begin the process of controlled restart of the core routing component. More updates will be provided as soon as they become available.
Please see below for the impacted services:
Description: Connectivity issue for US PGW
Incident case number: 177393
Jun 23, 11:19 UTC
Update -
UPDATE 19: The Aeris team continues to evaluate the network environment and is preparing for a necessary controlled restart of a core routing component to potentially resolve this issue. We will share further updates as soon as they become available.
Please see below for the impacted services:
Description: Connectivity issue for US PGW
Incident case number: 177393
Jun 23, 11:08 UTC
Update -
UPDATE 18: In our continued efforts to fully stabilize the network, Aeris engineers have determined that a controlled restart of a core routing component is necessary to resolve the intermittent packet drops observed under high traffic loads. To prevent further service degradation, we will consider to have this controlled restart within the next hour. Further updates will be provided once the assessment is completed.
Please see below for the impacted services:
Description: Connectivity issue for US PGW
Incident case number: 177393
Jun 23, 10:13 UTC
Monitoring -
UPDATE 17: Aeris engineers continue to make progress, and we are seeing a positive recovery trend for affected services. Core network routing is currently operational; however, our teams are observing some intermittent packet drops as traffic levels increase. We are actively running load simulations and fine-tuning network parameters to ensure complete stability under normal traffic loads.
Please see below for the impacted services:
Description: Connectivity issue for US PGW
Incident case number: 177393
Jun 23, 09:11 UTC
Update -
UPDATE 16: Aeris engineers have successfully stabilized the core network routing infrastructure. While this has restored baseline physical connectivity, we are currently investigating a secondary issue related to device authentication and session setup. Recent internal testing shows positive results with sessions now establishing successfully. We are actively monitoring traffic flow and working with partners to confirm full service recovery. No ETR is available at this time. Further updates will be provided once they become available.
Please see below for the impacted services:
Description: Connectivity issue for US PGW
Incident case number: 177393
Jun 23, 08:28 UTC
Update -
UPDATE 15: Aeris engineers are actively working to identify the root cause of the issue. Based on current observations, no packet loss or packet drop errors have been detected. The team is also evaluating potential networking workarounds to help minimize service disruption while a permanent resolution is being developed. At this time, there is no estimated time for resolution (ETR) available. We will continue to provide updates as more information becomes available.
Please see below for the impacted services:
Description: Connectivity issue for US PGW
Incident case number: 177393
Jun 23, 07:35 UTC
Update -
UPDATE 14: Aeris engineers are actively working on identifying the root cause and evaluating possible networking workarounds to minimize service disruption while a permanent resolution is developed. No ETR is available at this time. Please see below for the services that are impacted.
Description: Connectivity issue for US PGW
Incident case number: 177393
Jun 23, 06:46 UTC
Update -
UPDATE 13: Aeris engineers are actively working on identifying the root cause and evaluating possible networking workarounds to minimize service disruption while a permanent resolution is developed. No ETR is available at this time. Please see below for the services that are impacted.
Description: Connectivity issue for US PGW
Incident case number: 177393
Jun 23, 05:30 UTC
Update -
UPDATE 12: Aeris engineers are actively working on identifying the root cause and evaluating possible networking workarounds to minimize service disruption while a permanent resolution is developed. No ETR is available at this time. Please see below for the services that are impacted.
Description: Connectivity issue for US PGW
Incident case number: 177393
Jun 23, 04:26 UTC
Update -
UPDATE 11: Aeris engineers are actively working on identifying the root cause and evaluating possible networking workarounds to minimize service disruption while a permanent resolution is developed. No ETR is available at this time. Please see below for the services that are impacted.
Description: Connectivity issue for US PGW
Incident case number: 177393
Jun 23, 03:30 UTC
Update -
UPDATE 10: Aeris engineers are actively working on identifying the root cause and evaluating possible networking workarounds to minimize service disruption while a permanent resolution is developed. No ETR is available at this time. Please see below for the services that are impacted.
Description: Connectivity issue for US PGW
Incident case number: 177393
Jun 23, 02:26 UTC
Update -
UPDATE 9: Aeris engineers continue to troubleshoot the issue. Additional escalation to senior management team was already engaged. No ETR is available at this time. Please see below for the services that are impacted. Our engineering team is actively working with our hardware vendor and on-site data center technicians to isolate the physical connectivity issue within our US core infrastructure and work towards a resolution.
Description: Connectivity issue for US PGW
Incident case number: 177393
If you have any additional questions, please contact support@aeris.net and reference the case number above.
Jun 23, 01:23 UTC
Update -
UPDATE 8: Aeris engineers continue to troubleshoot the issue. Additional escalation to senior management team was already engaged. No ETR is available at this time. Please see below for the services that are impacted.
Description: Our engineering team is actively working with our hardware vendor and on-site data center technicians to isolate the physical connectivity issue within our US core infrastructure and work towards a resolution.
Incident case number: 177393
If you have any additional questions, please contact support@aeris.net and reference the case number above.
Jun 23, 00:26 UTC
Update -
UPDATE 7: Aeris engineers continue to troubleshoot the issue. Additional escalation to senior management team was already engaged. No ETR is available at this time. Please see below for the services that are impacted.
Description: Our engineering team is actively working with our hardware vendor and on-site data center technicians to isolate the physical connectivity issue within our US core infrastructure and work towards a resolution.
Incident case number: 177393
If you have any additional questions, please contact support@aeris.net and reference the case number above.
Jun 22, 23:24 UTC
Update -
UPDATE 6: Aeris engineers continue to troubleshoot the issue. Additional escalation to senior management team was already engaged. No ETR is available at this time. Please see below for the services that are impacted.
Description: Our engineering team is actively working with our hardware vendor and on-site data center technicians to isolate the physical connectivity issue within our US core infrastructure and work towards a resolution.
Incident case number: 177393
If you have any additional questions, please contact support@aeris.net and reference the case number above.
Jun 22, 22:21 UTC
Identified -
UPDATE 5: Aeris engineers continue to troubleshoot the issue. Additional escalation to senior management team was already engaged. No ETR is available at this time. Please see below for the services that are impacted.
Description: Our engineering team is actively working with our hardware vendor and on-site data center technicians to isolate the physical connectivity issue within our US core infrastructure and work towards a resolution.
Incident case number: 177393
If you have any additional questions, please contact support@aeris.net and reference the case number above.
Jun 22, 21:36 UTC
Update -
UPDATE 4: Aeris engineers continue to troubleshoot the issue. Additional escalation to senior management team was already engaged. No ETR is available at this time. Please see below for the services that are impacted.
Description: Aeris has escalated this incident to our highest internal severity level. Initial remote troubleshooting, including controlled equipment restarts, did not successfully restore the physical connectivity state within our US core infrastructure. On-site data center technicians are currently en route. Upon arrival, they will physically inspect the equipment and provide further details regarding the state of it.
Incident case number: 177393
If you have any additional questions, please contact support@aeris.net and reference the case number above.
Jun 22, 19:55 UTC
Update -
UPDATE 3: Aeris engineers continue to troubleshoot the issue. Additional escalation to senior management team was already engaged. No ETR is available at this time. Please see below for the services that are impacted.
Description: Our engineering team is actively working with our hardware vendor and on-site data center technicians to isolate the physical connectivity issue within our US core infrastructure and work towards a resolution.
Incident case number: 177393
If you have any additional questions, please contact support@aeris.net and reference the case number above.
Jun 22, 19:07 UTC
Update -
UPDATE 2: Aeris engineers continue to troubleshoot the issue. Additional escalation measures have been taken. No ETR is available at this time. Please see below for the services that are impacted.
Description: Aeris engineers have identified a hardware and routing issue within the US core infrastructure causing the current service disruption. Consequently, the incident severity has been upgraded to Critical. Our teams are actively engaged with our hardware vendors to perform diagnostics and replace the affected physical components if needed to restore services. No ETR is available at this time. Please see below for the services that are impacted.
Incident case number: 177393
If you have any additional questions, please contact support@aeris.net and reference the case number above.
Jun 22, 18:17 UTC
Investigating -
UPDATE 1: Aeris engineers continue to troubleshoot the issue. Additional escalation measures have been taken. No ETR is available at this time. Please see below for the services that are impacted.
Description: Aeris is currently troubleshooting an issue regarding the US PGW. More details will be provided once they become available.
Incident case number: 177393
If you have any additional questions, please contact support@aeris.net and reference the case number above.
Jun 22, 17:10 UTC
Identified -
UPDATE 1: Aeris engineers continue to troubleshoot the issue. Additional escalation measures have been taken. No ETR is available at this time. Please see below for the services that are impacted.
Description: Aeris is currently troubleshooting an issue regarding the US PGW. More details will be provided once they become available.
Incident case number: 177393
If you have any additional questions, please contact support@aeris.net and reference the case number above.
Jun 22, 16:58 UTC
Investigating -
Customer Reported at 14:33 UTC
Description: Connectivity issue for US PGW
Connectivity issue for US PGW. The Aeris technical team is actively troubleshooting to restore services. Further updates will be provided as soon as it become available.
Incident case number: 177393
Jun 22, 15:21 UTC