April 23, Update: NOC Team at innofield posted an explanation of the Incident in the comments section below.
Starting today at 17:09 UTC our systems detected a large scale routing incident affecting hundreds of Autonomous systems. Many BGPmon users have received an email informing them of this change.
Our initial investigation shows that the scope of this incident is widespread and affected 576 Autonomous systems and 3431 prefixes. Amongst the networks affected are high traffic prefixes including those of Google, Amazon, Twitter, Apple, Akamai, Time Warner Cable Internet and more.
All these events have either AS200759 “innofield AG” or private AS 65021 as the origin AS. In the cases where AS65021 appears as the origin AS, AS200759 is again the next-hop AS.
AS200759 “innofield AG” is a provider based out of Switzerland and normally only announces one IPv4 and one IPv6 prefix.
These are 2 example events:
Prefix 66.220.152.0/21 Is normally announced by Facebook AS32934 and during this event was announced by AS200759 as a more specific /22
Detected prefix: 66.220.152.0/22
Example aspath: 4608 24130 7545 6939 200759
And AS origin: 65021 behind AS 200759
Detected prefix: 66.220.152.0/22
Example aspath: 133812 23948 4788 6939 200759 65021
We saw the announcements via the following peers of AS200759 “innofield AG”:
- 20634 “Telecom Liechtenstein AG”
- 6939 “Hurricane Electric, Inc.”
- 16265 “LeaseWeb Network B.V.”
Not surprisingly, as HE is a major provider most of our probes (BGPmon peers) detected this path via their provider HE (6939).
It appears things have been resolved as of 17:30 UTC.
This event affected the reachability of many high traffic destinations, some good examples are posted on the outages.org mailing list. In this example posted by Frank Bulk we see how in his case amazon.com (54.239.16.0/20) is unreachable. The traceroute posted demonstrates how his traffic is routed via HE (6939) to towards Zurich, Switzerland where it eventually stops.
Since AS200759 (innofield AG) is connected to the SwissIX it’s likely they announced these prefixes to the route server there and as a result other peers such as HE picked it up from there. Since these are more specific announcements they are preferred over the original route even if the AS path is longer.
Below you’ll find an example email and screenshot that BGPmon users would have received alerting them of the incident in near real time.
====================================================================
Possible Prefix Hijack (Code: 10)
====================================================================
Your prefix: 199.16.156.0/23:
Prefix Description: twitter
Update time: 2016-04-22 17:10 (UTC)
Detected by #peers: 19
Detected prefix: 199.16.156.0/24
Announced by: AS65021 (-Private Use AS-)
Upstream AS: AS200759 (innofield AG)
ASpath: 58786 9957 6939 200759 65021
