Rackspace Hosted Exchange Failure Charge to Security Incident

Posted by

Rackspace hosted Exchange suffered a devastating blackout beginning December 2, 2022 and is still ongoing as of 12:37 AM December 4th. At first described as connection and login concerns, the assistance was ultimately upgraded to reveal that they were handling a security incident.

Rackspace Hosted Exchange Issues

The Rackspace system went down in the morning hours of December 2, 2022. At first there was no word from Rackspace about what the problem was, much less an ETA of when it would be dealt with.

Customers on Buy Twitter Verification reported that Rackspace was not reacting to support emails.

A Rackspace consumer independently messaged me over social networks on Friday to relate their experience:

“All hosted Exchange clients down over the previous 16 hours.

Not exactly sure the number of companies that is, but it’s substantial.

They’re serving a 554 long delay bounce so individuals emailing in aren’t familiar with the bounce for numerous hours.”

The main Rackspace status page offered a running upgrade of the interruption however the initial posts had no info besides there was an interruption and it was being examined.

The very first official upgrade was on December 2nd at 2:49 AM:

“We are examining an issue that is impacting our Hosted Exchange environments. More information will be published as they appear.”

Thirteen minutes later Rackspace began calling it a “connectivity concern.”

“We are investigating reports of connectivity problems to our Exchange environments.

Users might experience a mistake upon accessing the Outlook Web App (Webmail) and syncing their email customer(s).”

By 6:36 AM the Rackspace updates explained the ongoing problem as “connectivity and login issues” then later on that afternoon at 1:54 PM Rackspace revealed they were still in the “investigation phase” of the outage, still trying to determine what failed.

And they were still calling it “connection and login problems” in their Cloud Workplace environments at 4:51 PM that afternoon.

Rackspace Recommends Moving to Microsoft 365

4 hours later Rackspace described the scenario as a “considerable failure”and started offering their consumers totally free Microsoft Exchange Plan 1 licenses on Microsoft 365 as a workaround up until they understood the issue and might bring the system back online.

The main guidance stated:

“We experienced a substantial failure in our Hosted Exchange environment. We proactively closed down the environment to avoid any more concerns while we continue work to bring back service. As we continue to overcome the source of the concern, we have an alternate option that will re-activate your ability to send out and receive emails.

At no charge to you, we will be offering you access to Microsoft Exchange Plan 1 licenses on Microsoft 365 until further notification.”

Rackspace Hosted Exchange Security Occurrence

It was not up until nearly 24 hours later on at 1:57 AM on December 3rd that Rackspace formally announced that their hosted Exchange service was suffering from a security occurrence.

The announcement further revealed that the Rackspace specialists had powered down and detached the Exchange environment.

Rackspace published:

“After more analysis, we have actually identified that this is a security occurrence.

The recognized impact is separated to a part of our Hosted Exchange platform. We are taking essential actions to assess and protect our environments.”

Twelve hours later on that afternoon they upgraded the status page with more details that their security group and outside professionals were still working on solving the failure.

Was Rackspace Service Affected by a Vulnerability?

Rackspace has not released details of the security event.

A security occasion usually involves a vulnerability and there are two extreme vulnerabilities presently in the wile that were covered in November 2022.

These are the 2 most current vulnerabilities:

  • CVE-2022-41040
    Microsoft Exchange Server Server-Side Request Forgery (SSRF) Vulnerability
    A Server Side Request Forgery (SSRF) attack permits a hacker to read and alter information on the server.
  • CVE-2022-41082
    Microsoft Exchange Server Remote Code Execution Vulnerability
    A Remote Code Execution Vulnerability is one in which an aggressor is able to run destructive code on a server.

An advisory published in October 2022 described the effect of the vulnerabilities:

“A confirmed remote enemy can perform SSRF attacks to escalate advantages and carry out arbtirary PowerShell code on vulnerable Microsoft Exchange servers.

As the attack is targeted against Microsoft Exchange Mailbox server, the attacker can potentially access to other resources via lateral motion into Exchange and Active Directory site environments.”

The Rackspace outage updates have actually not indicated what the specific issue was, only that it was a security event.

The most current status update as of December 4th specified that the service is still down and clients are encouraged to migrate to the Microsoft 365 service.

Rackspace posted the following on December 4, 2022 at 12:37 AM:

“We continue to make progress in addressing the event. The availability of your service and security of your data is of high importance.

We have actually committed substantial internal resources and engaged world-class external knowledge in our efforts to minimize unfavorable impacts to clients.”

It’s possible that the above noted vulnerabilities are related to the security occurrence affecting the Rackspace Hosted Exchange service.

There has actually been no statement of whether consumer information has been compromised. This occasion is still ongoing.

Included image by Best SMM Panel/Orn Rin