maintenance Maintenance information and updates

Scheduled Maintenance: March 1st 8am- 10am CST.

Description:

One of our upstream providers, nLayer, will be performing maintenance on their core routers. This maintenance will not result in any downtime for our customers since we will be failing over to our other provider before the maintenance begins and merging connections once the Nlayer maintenance is complete.

Customers affected: All SingleHop customers are at any one point in time riding on the nLayer connection, however, during the maintenance period we will slowly merge all connections to our other uplink provider, AboveNet, and expect no downtime from this maintenance.

+ show updates
Suscribe to updates for this post
  1. nLayer has rescheduled this maintenance to March 3rd, 02:00 CDT. As before, no customer impact is expected.

  2. This maintenance has been rescheduled for March 3rd, 02AM CST throgh 04AM CST.

  3. The maintenance has been completed successfully at this time.

Public notice:
SCHEDULED MAINTENANCE NOTICE

Maintenance Window:  03/01/10 2200 CST - 03/02/10 0200 CST
Expected Impact:  None expected
Maintenance Description:  Network staff will be performing a software upgrade of the redundant SnapShield shared firewalls in DC 2.  No downtime is expected from this software upgrade.
+ show updates
Suscribe to updates for this post
  1. The software upgrade has been completed at this time. No downtime was experienced.

Connectivity issues on our network- techs are aware and working to resolve it.

+ show updates
Suscribe to updates for this post
  1. There was an issue with smtp connections trying to perform lookups on outgoing mail and failing. This was due to one of our resolvers failing. Some may have had trouble with smtp alerts and using leap.

    The problem has been taken care of and everything is back in order.

Connectivity issues on our network- techs are aware and working to resolve it.

+ show updates
Suscribe to updates for this post
  1. An issue in an upstream network caused increased packetloss and latency to a portion of customers. Network staff temporarily disabled links to the upstream until they reported the issue resolved.

    An official RFO will be provided as soon as possible.

Maintenance Window:  01/27/10 2200 CST - 01/28/10 02000 CST

Expected Impact:  Phone support unavailable during maintenance window

Maintenance Description:  SingleHop will be upgrading its phonesystem in order to continue to provide high-quality phone support to customers as we grow.  During the window, customers will be unable to access support staff via our normal contact phone numbers.  LEAP will still be available for customers to contact support if necessary during the window.

+ show updates
Suscribe to updates for this post

Closed.

Connectivity issues at Datacenter 1- techs are aware and working to resolve it.

+ show updates
Suscribe to updates for this post
  1. The Access switch in Cab 207.25 has failed and is in the process of being replaced. Service will be restored shortly.

  2. Full service has been restored as of 2:36AM CST. If you continue to experience technical difficulties please let us know by opening a support ticket.

    An offical RFO will be provided as soon as possible.

    Thank you for your continued patience in this matter.

Connectivity issues on our network- techs are aware and working to resolve it.

+ show updates
Suscribe to updates for this post
  1. The issue is completely resolved and was associated with one of our upstream providers. Our network recovered as designed and contained any domino effects from taking place. We will provide more information as soon as we can get more details from the upstream provider.

    Thanks!

  2. All services have returned to normal operation at this time. We will update this post with an RFO shortly.

  3. SingleHop Connectivity Issues Report-

    We experienced network issues last night due to emergency maintenance being performed by one of our upstream service providers, Abovenet. The emergency maintenance was unexpected, and although our systems responded as designed by self stabilizing and re-routing traffic through our secondary service provider, there was some packet loss of about 2 minutes during the upstream providers unexpected maintenance.

    Thanks for your patience.

We are currently experiencing what appears to be a DDoS atack on Singlehop.com

Our engineers are aware of the situation and making sure that it gets corrected ASAP.

+ show updates
Suscribe to updates for this post
  1. Everything is back up again, thanks for your patience.

A fire has been reported in a part of the building where our Downtown Chicago Datacenter 1 is located. SingleHop’s Datacenter is not affected but the Chicago fire department is in the building. The Fire department has informed us that there is a possibility that they will cut power to the entire building and stop the diesel powered backup power system from turning on for fire safety reasons. This means that there is a possibility that SingleHop’s Datacenter 1 may go offline for an unknown period of time.

We will keep you update as more information as soon as we know more.

Thanks.

+ show updates
Suscribe to updates for this post
  1. The fire department has informed us that the fire has been contained, but they may still turn off power to the entire building- which would most probably mean that the diesel generator would not be allowed to go on and the datacenter would go offline temporarily.

  2. Fire containted, not affecting SingleHop operations. Fire occurred inside a non-SingleHop datacenter that is located in the same building as our datacenter. Fire department may still cut power but it is less probable now that the fire has been contained and ascertained as having started and stayed at the other non-SingleHop datacenter.

Maintenance Window:  12/29/09 2200 CST - 12/30/09 0000 CST
Expected Impact:  None expected

Maintenance Description:  We will be doing maintenance to some internal routers so our network can more quickly address issues like the inter-Datacenter connectivity issues we experienced last week.

Our network staff will be altering network configuration to speed recovery times in case of network equipment failure.  No customer impact is expected from this maintenance.

+ show updates
Suscribe to updates for this post
  1. All clear. Fire extinguished. Chicago Fire Department has completed their work in the building, and left the building. SingleHop datacenter 1 was not affected in the fire in any way. Slight possibility that power may be pulled to inspect electrical damage, but it should not affect SingleHop.

    Thanks for your patience.

    SingleHop Team.