Albion Network Status

A network event that occurred on January 12th 2012 at 10:02 GMT lasting approximately forty eight minutes, affecting power to some customers and network equipment in our 20 Mastmaker Court facility located in London, UK.

At 08:37 AM GMT, an isolated hardware failure has been detected, affecting a single PDU, labeled as “B”, at the datacenter power supply system. The rest of power infrastructure kept on working properly. This power outage caused customers with only “B” side power in our datacenter to experience an interruption on his service. An engineer was dispatched immediately to bring back on line the impacted customers.

During the troubleshooting of this issue, and due to an unfortunate and isolated human error, at 10:02 AM GMT, a technician has accidentally tripped a protection mechanism intended to prevent voltage overloads for the whole power system, and this has lead to a complete power outage for our network core devices at this datacenter during 48 minutes, which also might have caused routing disturbances to some customers.

We apologize for any impact you may have experienced. Please contact us, if you have any additional question concerning this event.

Part 2 of the upgrade will take place between 10am and 5pm on the 5th January. Updates will be applied to the backup server software. Access to the online servers will be intermittent whiles these updates are installed.

There was 70%+ packet loss on our core network caused by a failing firewall. This has now been replaced and packet loss has stopped. The issue started at 11.02am on Sunday the 18th Dec and resolved at 7.30pm

Scheduled maintenance will be performed on the CN32 environment on Tuesday December 13th from 2:30 PM – 3:30 PM EDT. Users may experience downtime during this timeframe.

We are getting reports of customers who have BT internet connections not being able to connect Albion Servers (e-mail etc)

This a BT issue, if you are experiencing issues please contact BT directly.

7/12/11 – 9am – We have been updated from BT and believe the BT issue has now been resolved

Our online backup servers experienced  and I/O error on the RAID Storage system yesterday evening (26/7/11), as a way of protecting the storage the storage changed to Read Only. Thus all the online backups that were scheduled for backup on the Tuesday 26th failed. We have now resolved the issue and backups should resume as normal tonight.

This issue has now been resolved

We had a number of reports of customers having issues accessing our network services (E-mail and Web sites)

All Albion Services are working normally.

The issues is with BT and they are having a routing issue in there Amsterdam POP. BT are aware of the issue. This will also affect any customers that connect via the BT network example an iPhone using 3G routes via the BT Network so these may also be affected. ALL Customers who are on Albion Internet Connections AREN’T Affected

UPDATE – 9.55am BT have confirmed they have resolved the issue.

The Spam Filter is currently offline. Engineers are investigating. 3rd Feb 2011 13:22

Issue now resolved 3rd Feb 2011