Some systems are experiencing issues

About This Site

Check for ongoing status updates. Here you can find all information about host systems, data centers, API interfaces and more. Always stay up to date.

Maintenance
Network card replacement

We will expand our server connections to larger capacities. As the current network cards are not designed for this, we cannot avoid replacing them.

We will be doing this on Saturday 28 December 24 between 12 noon and 5 pm. An interruption is to be expected, as the cards cannot be replaced during operation. We expect each host system to be unavailable for approx. 30 minutes.

Past Incidents

Sunday 11th August 2024

No incidents reported

Saturday 10th August 2024

No incidents reported

Friday 9th August 2024

No incidents reported

Thursday 8th August 2024

No incidents reported

Wednesday 7th August 2024

No incidents reported

Tuesday 6th August 2024

No incidents reported

Monday 5th August 2024

No incidents reported

Sunday 4th August 2024

No incidents reported

Saturday 3rd August 2024

No incidents reported

Friday 2nd August 2024

No incidents reported

Thursday 1st August 2024

No incidents reported

Wednesday 31st July 2024

No incidents reported

Tuesday 30th July 2024

No incidents reported

Monday 29th July 2024

No incidents reported

Sunday 28th July 2024

No incidents reported

Saturday 27th July 2024

Core-Network Multiple issues with IPv6 addresses

** Automatic incident report **

Our monitoring has detected various connection problems with various IPv6 addresses from our networks. External monitoring services support this incident.

This did not lead to a direct outage, but the NOC was informed. Until the cause has been or will be found, connection interruptions to our IPv6 networks must be expected.

  • All IPv6 addresses can still be reached. The incident has therefore been resolved on our part.

  • So far, all IPv6 addresses are running reliably and connectively. We will wait another 48 hours to make a final decision on the incident.

  • Multicast rules have just been applied to each Proxmox host to fix the IPv6 issues for the future. We will now continue to monitor this.

    After the rules have been set, we cannot avoid restarting the network interface - which has just caused a short timeout. Upstream & routes are not affected and run independently.

  • We could not find a direct error. All upstream connections and routes are still available.

    We will start further analyses within our business hours. We can rule out a misconfiguration of the images, so far everything is still green. The interruptions occur at irregular intervals and do not cause any direct outages.

  • Friday 26th July 2024

    No incidents reported

    Thursday 25th July 2024

    No incidents reported

    Wednesday 24th July 2024

    No incidents reported

    Tuesday 23rd July 2024

    No incidents reported

    Monday 22nd July 2024

    No incidents reported

    Sunday 21st July 2024

    No incidents reported

    Saturday 20th July 2024

    No incidents reported

    Friday 19th July 2024

    No incidents reported

    Thursday 18th July 2024

    No incidents reported

    Wednesday 17th July 2024

    No incidents reported

    Tuesday 16th July 2024

    No incidents reported

    Monday 15th July 2024

    No incidents reported

    Sunday 14th July 2024

    No incidents reported

    Saturday 13th July 2024

    No incidents reported

    Friday 12th July 2024

    No incidents reported

    Thursday 11th July 2024

    No incidents reported

    Wednesday 10th July 2024

    No incidents reported

    Tuesday 9th July 2024

    Core-Network Relocation of our core routers

    Due to the persistent network problems, we have to leave the penultimate instance and migrate to a different and more stable cluster.

    This should resolve the persistent timeouts and connection problems; if this is not the case, we will endeavour to find further solutions.

  • Since the migration, we have not been able to detect any further errors within 12 hours.

    The only thing was a hardware defect on the core02, which was not noticed. This did not lead to an interruption in the network infrastructure.

    We will continue to monitor our network closely.

  • Monday 8th July 2024

    No incidents reported

    Sunday 7th July 2024

    No incidents reported

    Saturday 6th July 2024

    No incidents reported

    Friday 5th July 2024

    Webhosting Faulty image on our web servers

    After a few tests, we realised that our cloud init templates for our web servers were configured and adapted incorrectly.

    As a result, some network crashes occur, incorrect cache values are written to RAM and some Plesk settings are not applied to the server itself.

    To eliminate these problems, we cannot avoid a new installation. We will organise the process as well as possible as follows:

    We will migrate all Plesk web servers apart from the current ones, before that we will install and configure new servers. For the migration, we will route the current Plesk web server to a different IP address so that we already have correct IP addresses on the new servers before the migration - this will ensure the configuration and restoration of all sites. We will not be able to avoid interrupting the web display, and this should only last as long as the migration process continues. Subsequent error corrections still have to be taken into account.

    The whole process will start at 10:00 am today and is expected to last until 4:00 pm. The time does not apply to each web server as a whole, but to all of them - as we are talking about over 13 web server nodes here.

  • We have now successfully migrated eight nodes, but the downtime was quite long due to unexpected errors on the part of Plesk.

    It is conceivable that all nodes will be available again by 13:30.

  • Thursday 4th July 2024

    No incidents reported

    Wednesday 3rd July 2024

    No incidents reported

    Tuesday 2nd July 2024

    No incidents reported

    Monday 1st July 2024

    No incidents reported

    Sunday 30th June 2024

    No incidents reported

    Saturday 29th June 2024

    No incidents reported

    Friday 28th June 2024

    Upstream (BGP Sessions) Unavailability of our upstream provider

    We have just experienced a complete outage of our infrastructure, apparently caused by our upstream provider.

    As our BGP VMs are also unavailable.

  • We have now made further observations and found some small things that we still need to adjust. We are not talking about urgency here.

    In the meantime, we have tried to make further adjustments at intervals of around 24 hours to enable future improvements in the event of core router failures at the switch between all core routers.

    The incident is now completely closed for the time being. We will update further adjustments and configurations in further 24-hour intervals (usually 02:00 to 03:00).

  • The last 48 hours look much better compared to the days before. We only noticed one timeout on 30.06.24 at 12:12, which didn't last a minute.

    It seems that all our configurations are now correct and running 100%, thanks also to our upstream who provided us with a session on newer hardware. However, we will continue to monitor for another 72 hours so that we can react to any problems within a few minutes and be on the safe side.

  • We have just completed our maintenance work for the implementation of VRRP.

    In addition, a BGP session was moved to our upstream on a new router. We continue to keep our network under 24/7 observation in order to be able to react directly in the event of further outages, timeouts or other problems. We are working intensively on this issue in collaboration with our upstream in order to resolve past problems.

  • The problem is due to a faulty fuse on the switch. All traffic was redirected via DUS7 (myLoc) in order to ensure the continued availability of the network.

    In this case, as well as the timeouts, we will continue to wait 24 hours until our maintenance appointment tomorrow (Saturday) and monitor it extremely closely. If our network does not stabilize within the next seven days from tomorrow and cannot be operated without timeouts, we will consider other solutions for the operation of our infrastructure and, if necessary, put a new location into operation or relocate.

    Maintenance details at: https://status.schleyer-edv.de/#scheduled-26

  • Thursday 27th June 2024

    No incidents reported

    Wednesday 26th June 2024

    No incidents reported

    Tuesday 25th June 2024

    No incidents reported

    Monday 24th June 2024

    No incidents reported

    Sunday 23rd June 2024

    No incidents reported

    Saturday 22nd June 2024

    No incidents reported

    Friday 21st June 2024

    No incidents reported

    Thursday 20th June 2024

    Core-Network Router failure from core01.dus01

    Due to a failure of Router-01, there is currently a partial network failure, a redirection via Router-02 exists - but currently also results in up to 60% packet loss.

    We are working on the problem.

  • We have discovered that we obviously have a misconfiguration on core01.de-dus01.

    We are currently setting up a new Bird2 VM to redirect the traffic - so that we can set up both core routers again and then route the traffic via the core routers again.

    In the meantime, there may be interruptions to the network infrastructure.

  • The network is accessible again without any problems, and we will of course continue to monitor it closely for another 24 hours.

    Our vhost01.dus obviously has problems, which is why some services are still unavailable. We are in contact with the data center.

  • We were able to identify the problem more precisely. We are already working on a fix.

    We are also waiting for a response from our upstream-provider.

  • Wednesday 19th June 2024

    No incidents reported

    Tuesday 18th June 2024

    No incidents reported

    Monday 17th June 2024

    No incidents reported

    Sunday 16th June 2024

    No incidents reported

    Saturday 15th June 2024

    No incidents reported

    Friday 14th June 2024

    No incidents reported

    Thursday 13th June 2024

    No incidents reported