All systems are operational

About This Site

Check for ongoing status updates. Here you can find all information about host systems, data centers, API interfaces and more. Always stay up to date.

Stickied Incidents

Sunday 20th October 2024

Core-Network To MT: Extensive customisation of our network

We will be carrying out extensive maintenance work on our network from 20 October 2024 from 00:30 to 02:30.

We are adapting the failover and VRRP principle in more detail, revising all of our routing configurations and commissioning additional backup solutions - in order to be able to provide improved options in the future.

This work is urgently needed due to the recent maintenance of our upstream.

During the maintenance process, there may be interruptions, timeouts and probable downtimes of the network infrastructure - as we also have to work on the failover principle and adapt and test all or optimal options.

--

Update from 20.10 - 03:57:

We have now applied our optimisations to our routers and tested VRRP. Our configurations are correct and have largely been implemented.

Apparently there is a NAT problem with our upstream, which means that the failover is not 100% effective. We are still waiting for feedback from our upstream in order to clarify the problem and possibly rectify it. An answer is pending.

Once we have received the response from our upstream, we will continue our work at night.

  • Our current upstream obviously doesn't want to identify and fix the problem.

    We will be changing our provider in the coming week and will schedule a new maintenance appointment.

  • Past Incidents

    Tuesday 23rd July 2024

    No incidents reported

    Monday 22nd July 2024

    No incidents reported

    Sunday 21st July 2024

    No incidents reported

    Saturday 20th July 2024

    No incidents reported

    Friday 19th July 2024

    No incidents reported

    Thursday 18th July 2024

    No incidents reported

    Wednesday 17th July 2024

    No incidents reported

    Tuesday 16th July 2024

    No incidents reported

    Monday 15th July 2024

    No incidents reported

    Sunday 14th July 2024

    No incidents reported

    Saturday 13th July 2024

    No incidents reported

    Friday 12th July 2024

    No incidents reported

    Thursday 11th July 2024

    No incidents reported

    Wednesday 10th July 2024

    No incidents reported

    Tuesday 9th July 2024

    Core-Network Relocation of our core routers

    Due to the persistent network problems, we have to leave the penultimate instance and migrate to a different and more stable cluster.

    This should resolve the persistent timeouts and connection problems; if this is not the case, we will endeavour to find further solutions.

  • Since the migration, we have not been able to detect any further errors within 12 hours.

    The only thing was a hardware defect on the core02, which was not noticed. This did not lead to an interruption in the network infrastructure.

    We will continue to monitor our network closely.

  • Monday 8th July 2024

    No incidents reported

    Sunday 7th July 2024

    No incidents reported

    Saturday 6th July 2024

    No incidents reported

    Friday 5th July 2024

    Webhosting Faulty image on our web servers

    After a few tests, we realised that our cloud init templates for our web servers were configured and adapted incorrectly.

    As a result, some network crashes occur, incorrect cache values are written to RAM and some Plesk settings are not applied to the server itself.

    To eliminate these problems, we cannot avoid a new installation. We will organise the process as well as possible as follows:

    We will migrate all Plesk web servers apart from the current ones, before that we will install and configure new servers. For the migration, we will route the current Plesk web server to a different IP address so that we already have correct IP addresses on the new servers before the migration - this will ensure the configuration and restoration of all sites. We will not be able to avoid interrupting the web display, and this should only last as long as the migration process continues. Subsequent error corrections still have to be taken into account.

    The whole process will start at 10:00 am today and is expected to last until 4:00 pm. The time does not apply to each web server as a whole, but to all of them - as we are talking about over 13 web server nodes here.

  • We have now successfully migrated eight nodes, but the downtime was quite long due to unexpected errors on the part of Plesk.

    It is conceivable that all nodes will be available again by 13:30.

  • Thursday 4th July 2024

    No incidents reported

    Wednesday 3rd July 2024

    No incidents reported

    Tuesday 2nd July 2024

    No incidents reported

    Monday 1st July 2024

    No incidents reported

    Sunday 30th June 2024

    No incidents reported

    Saturday 29th June 2024

    No incidents reported

    Friday 28th June 2024

    Upstream (BGP Sessions) Unavailability of our upstream provider

    We have just experienced a complete outage of our infrastructure, apparently caused by our upstream provider.

    As our BGP VMs are also unavailable.

  • We have now made further observations and found some small things that we still need to adjust. We are not talking about urgency here.

    In the meantime, we have tried to make further adjustments at intervals of around 24 hours to enable future improvements in the event of core router failures at the switch between all core routers.

    The incident is now completely closed for the time being. We will update further adjustments and configurations in further 24-hour intervals (usually 02:00 to 03:00).

  • The last 48 hours look much better compared to the days before. We only noticed one timeout on 30.06.24 at 12:12, which didn't last a minute.

    It seems that all our configurations are now correct and running 100%, thanks also to our upstream who provided us with a session on newer hardware. However, we will continue to monitor for another 72 hours so that we can react to any problems within a few minutes and be on the safe side.

  • We have just completed our maintenance work for the implementation of VRRP.

    In addition, a BGP session was moved to our upstream on a new router. We continue to keep our network under 24/7 observation in order to be able to react directly in the event of further outages, timeouts or other problems. We are working intensively on this issue in collaboration with our upstream in order to resolve past problems.

  • The problem is due to a faulty fuse on the switch. All traffic was redirected via DUS7 (myLoc) in order to ensure the continued availability of the network.

    In this case, as well as the timeouts, we will continue to wait 24 hours until our maintenance appointment tomorrow (Saturday) and monitor it extremely closely. If our network does not stabilize within the next seven days from tomorrow and cannot be operated without timeouts, we will consider other solutions for the operation of our infrastructure and, if necessary, put a new location into operation or relocate.

    Maintenance details at: https://status.schleyer-edv.de/#scheduled-26

  • Thursday 27th June 2024

    No incidents reported

    Wednesday 26th June 2024

    No incidents reported

    Tuesday 25th June 2024

    No incidents reported

    Monday 24th June 2024

    No incidents reported

    Sunday 23rd June 2024

    No incidents reported

    Saturday 22nd June 2024

    No incidents reported

    Friday 21st June 2024

    No incidents reported

    Thursday 20th June 2024

    Core-Network Router failure from core01.dus01

    Due to a failure of Router-01, there is currently a partial network failure, a redirection via Router-02 exists - but currently also results in up to 60% packet loss.

    We are working on the problem.

  • We have discovered that we obviously have a misconfiguration on core01.de-dus01.

    We are currently setting up a new Bird2 VM to redirect the traffic - so that we can set up both core routers again and then route the traffic via the core routers again.

    In the meantime, there may be interruptions to the network infrastructure.

  • The network is accessible again without any problems, and we will of course continue to monitor it closely for another 24 hours.

    Our vhost01.dus obviously has problems, which is why some services are still unavailable. We are in contact with the data center.

  • We were able to identify the problem more precisely. We are already working on a fix.

    We are also waiting for a response from our upstream-provider.

  • Wednesday 19th June 2024

    No incidents reported

    Tuesday 18th June 2024

    No incidents reported

    Monday 17th June 2024

    No incidents reported

    Sunday 16th June 2024

    No incidents reported

    Saturday 15th June 2024

    No incidents reported

    Friday 14th June 2024

    No incidents reported

    Thursday 13th June 2024

    No incidents reported

    Wednesday 12th June 2024

    No incidents reported

    Tuesday 11th June 2024

    No incidents reported

    Monday 10th June 2024

    No incidents reported

    Sunday 9th June 2024

    No incidents reported

    Saturday 8th June 2024

    No incidents reported

    Friday 7th June 2024

    No incidents reported

    Thursday 6th June 2024

    No incidents reported

    Wednesday 5th June 2024

    No incidents reported

    Tuesday 4th June 2024

    No incidents reported

    Monday 3rd June 2024

    No incidents reported

    Sunday 2nd June 2024

    No incidents reported

    Saturday 1st June 2024

    No incidents reported

    Friday 31st May 2024

    No incidents reported

    Thursday 30th May 2024

    No incidents reported

    Wednesday 29th May 2024

    No incidents reported

    Tuesday 28th May 2024

    No incidents reported

    Monday 27th May 2024

    No incidents reported

    Sunday 26th May 2024

    No incidents reported

    Saturday 25th May 2024

    No incidents reported