DDoS is one of those problems that organisations tend to put on the back burner until they find themselves under attack. Many ISPs have little or no mitigation in place and are put off by the extremely high price tag for DDoS mitigation services, or alternatively the appliances that cost tens of thousands of pounds/dollars to counter-attack these threats.
One thing is for sure, DDoS attacks are growing in numbers each year and becoming more and more sophisticated, bringing thousands of websites down in a matter of minutes and leaving companies dumbfounded as to what to do in order to regain full service again.
In the ISP world, you’re unusual if you haven’t been affected yet – either by contributing to a DDoS (with a compromised server), or directly under attack yourself.
So what can you do?
As an ISP, there are several layers of protection you can put in place which will thwart many DDoS attacks – or at least leave you standing long enough to be able to comprehend, and mitigate (in real time), an attack without being blown out of the water completely.
It’s difficult to understand the mind-set of a DDoS attacker, particularly as it’s not always clear to the attacker if a DDoS is successful or not. You get the sense that some attackers simply get bored from not yielding the results they were hoping for and eventually give up. Of course the longer a DDoS continues, the longer you have to gather evidence about the nature of the attack and capture and analyse traffic that might help uncover the original source.
Hopefully some of the tips and tricks here will help in the fight, but if your business is attracting frequent DDoS attacks – it may be time to bring in the professionals. Customers will tolerate the occasional issue, but it won’t be long before they are considering their options.
1) Stay informed
If a new threat is emerging then it’s better to know before the fact, rather than after. At least this buys you time to make the necessary changes to protect your company.
Google recently released the Digital Attack Map which provides a real time view of active DDoS attacks around the Globe. Data is sourced from over 270 ISPs (and growing). It’s still in a development phase but shows promise. DDoS attacks aren’t going to go away, so expect significantly more development of counter measures over the next few years.
In a similar vein, Prolexic has PLXpatrol providing real-time analysis of DDoS threats as well as attack types and country rankings. Prolexic will also email you DDoS alerts if you subscribe to their newsletter. Prolexic is probably the largest DDoS mitigation operator (at least in the US) if you do need large scale professional services.
Cisco has also developed the Security Intelligence Operations Center – it’s a great place to get up to the minute threat advisories and information. You are also able to report DDoS activity through the website or over the phone. Incidentally, Cisco recently published an “Event Response” to the NTP Amplication Attack.
Further to this, Cisco has also published a Service Provider Best Practices guide to mitigate from security threats, and more specifically Strategies to Protect Against DDoS Attacks.
2) Get the basics in place
There are a number of well-known strategies which all ISPs should have in place as a bare minimum
RFC 2827 – Unicast Reverse Path Forwarding
Otherwise known as BCP 38, RFC 2827 is a framework which was introduced in May 2000 to deal with the issue of forged source addresses and outlines an effective way to filter such traffic. In practice, this is implemented using Unicast Reverse Path Forwarding. Essentially this ensures that a packet must be received on the interface the router would use to forward the return packet. Sounds like a good idea, but in practice ISPs have multiple upstream transit connections and it’s not unusual for a packet to return to the source address through a different ingress interface (otherwise known as asysmetric routing) Consequently, a relaxed version of the command (loose mode) can be implemented, where the source address must at least exist in the routing table. An access-list can also be applied to deny certain routes.
An example of the commands of a Cisco router would be:
interface gi0/0 ip verify unicast source reachable-via any allow-self-ping
RFC1918 – Private Address Ranges
RFC1918 is the one RFC that all network engineers worth their salt should know about. It defines a private range of IP addresses that are prohibited on the Internet. It’s therefore good practice to null route this traffic on all your core routers and switches. DDoS attacks quite often use RFC1918 addresses to mask the source address of the attack.
ip route 10.0.0.0 255.0.0.0 Null0 name RFC1918-10 ip route 172.16.0.0 255.240.0.0 Null0 name RFC1918-172 ip route 192.168.0.0 255.255.0.0 Null0 name RFC1918-192
Bogons and Martians
Following on in the same vein; a Bogon is a prefix that should never appear on the internet. Bogons are defined as Martians (RFC 1918, RFC5735, and RFC 6598) and IP blocks that have not yet been assigned – and as such Bogons are an ever evolving list. As mentioned above it’s not unusual for DDoS attacks to use Bogons as the source address so it makes sense to filter these prefixes.
A full list of bogon can be found here
https://www.team-cymru.org/Services/Bogons/fullbogons-ipv4.txt
Basic bogons (which should be filtered as minimum) can be found here:
https://www.team-cymru.org/Services/Bogons/bogon-bn-nonagg.txt
An good reference for Bogons can be found here:
https://www.team-cymru.org/Services/Bogons/
RFC 3882 – BGP Black holing
RFC 3882 outlines how to configure BGP to block DDoS Attacks. In particular, sections 2 and 3 discuss the enhanced use of BGP communities to segregate DDoS identified traffic. Using various tools (see later) it is relatively easy to identify the target of a DDoS attack. This traffic can be marked with a BGP community and sent in a BGP update to an upstream peer, where is can be matched and processed according by an associated route-map.
For example – Level (3) communities include 3356:9999 as a blackhole (discard) traffic community
A typical configuration would be to first mark suspect traffic with a tag
ip route x.x.x.x x.x.x.x Null0 tag 666
This will then be matched in a blackhole route-map
redistribute static route-map blackhole route-map blackhole permit 10 match tag 666 set community :666
The route-map matches the 666 tag and then assigns a community to the route. The community is then matched on the BGP neighbour
neighbor level3 route-map level3-out out route-map level3-out permit 10 match community :666 set community 3356:9999
3. Develop Infrastructure Access Control Lists (iACLs)
Technology changes and the impact of an ACL on traffic flow might not be what you were expecting. Traditionally, ACLs on the transit border have been frowned upon due to two reasons; firstly ISPs don’t want to tell their customer that they are filtering anything at all. This can lead to awkward conversations with customers who are expecting pure unfettered transit. Secondly, ACLs are seen to impact router performance as each packet has to be essentially opened up and examined.
Whilst the first issue is purely a political one, advances in ACL’s means that some are now processed at wirespeed with no perceivable impact on performance.
The following white paper outlines the ACLs that are processed in hardware compared to software on perhaps Cisco’s flagship switching platform the 6500 series
http://www.cisco.com/en/US/products/hw/switches/ps708/products_white_paper09186a00800c9470.shtml
Taking this into account, it is possible to compose an access list that provides a lot of protection from DDoS.
Cisco has also put together an article on such an ACL.
http://www.cisco.com/c/en/us/support/docs/ip/access-lists/43920-iacl.html
The following is a typical example:
Extended IP access list DDOS-Protect 10 deny udp any eq ntp any eq 80 << Protects against the NTP Amplification Attack 20 deny ip host 0.0.0.0 any << Protects against malformed packets to host 0.0.0.0 30 deny ip 127.0.0.0 0.255.255.255 any << Protects from packets from a loopback source address 40 deny ip 192.0.2.0 0.0.0.255 any << protects from the "TEST NET" assigned range 40 deny ip 224.0.0.0 31.255.255.255 any << protects from the multicast assigned range 50 deny ip 10.0.0.0 0.255.255.255 any << RFC 1918 60 deny ip 172.16.0.0 0.15.255.255 any << RFC 1918 70 deny ip 192.168.0.0 0.0.255.255 any << RFC 1918 80 deny udp any host x.x.x.x fragments << protects against malformed sequences 100 permit ip any any
4. Control Plane Policing
When a DDoS occurs in an ISP environment it is usually a customer host somewhere on the network that is under attack rather than the ISP routing and switching infrastructure. However the act of attacking a customer host clearly can impact the infrastructure connecting these hosts, if this infrastructure is not properly protected. This is where Control Plan Policy (CoPP) is hugely beneficial.
CoPP concerns itself with the packets that are actually destined for the routers and switches, and have to acted on (as opposed to simply forwarded on). Examples being ARP, OSPF, BGP, EIGRP etc. These are the packets that bond the network together and ensure routes are available. During a DDoS it’s essential that processor resources are provided to the control plane in order to protect the router itself. Cisco define CoPP as a “Cisco IOS-wide route processor protection mechanism”
CoPP is implemented using the Cisco IOS Modular QoS CLI (MQC for short) – which is basically a flexible modular command framework; a class-map is used to define traffic, a policy-map is used to associate class-maps together which result in a service-policy – which can be applied to the control plane – blimey sounds difficult(!), but in practice it is fairly straightforward to implement.
Here is a simple example that controls NTP only allowing NTP traffic from host 10.1.1.1 to be accepted by the device.
First enable mls qos
switch(config)#mls qos
Create an access-list to allow NTP traffic from host 10.1.1.1
switch(config)#ip access-list extended NTP switch(config-ext-nacl)#permit udp host 10.1.1.1 any eq ntp
Create a class-map
switch(config)#class-map match-any NTP switch(config-cmap)#match access-group name NTP
Create a policy-map
switch(config)#policy-map copp-policy switch(config-pmap)#class NTP switch(config-cmap-c) police 10000000 confirm-action transmit exceed-action drop
Apply the policy-map (the service policy)
switch(config)#control-plane switch(config-cp)service-policy input copp-policy
Cisco provide an excellent document on Control Plane Policing Implementation Best Practices which can be found here: http://www.cisco.com/web/about/security/intelligence/coppwp_gs.html
5. NetFlow
All being well you have retained control of your infrastructure, so the next important factor in the fight against DDoS is to identify the nature of the attack. The old adage “knowledge is Power” really helps here, and the tool that can provide you with all the information to identify the nature of the attack and whom is under attack is NetFlow.
NetFlow (aka JFlow by Juniper) was developed by Cisco and essentially caches information regarding “flows” of traffic traversing a router or switch. This cached information only stores the header information, packet size, source/destination, protocol etc so is fairly lightweight, but can be tremendously useful in understanding the traffic passing through the router.
This information can be seen cached on the router, and used from the command line to identify DDoS quite easily or can be exported to an NetFlow collector. A popular open source example being NfSen.
For example to quickly check a switch for unusually high volume flows you can execute the following command:
switch#show ip cache flow | i K Gi5/1 76.249.209.132 Gi1/47 33.191.34.2 32 B49C 28CD 51K Gi5/1 117.41.42.70 Te4/4 57.220.104.116 32 6D50 92AB 16K Gi5/1 91.148.111.100 Te4/4 198.255.65.144 2F 0000 0000 18K Te4/4 27.220.106.66 Gi5/1 82.232.32.146 32 BE1C F541 25K Gi5/1 12.232.32.146 Te4/4 57.220.106.66 32 193A 2AC5 44383K Gi5/1 53.34.249.51 Te4/4 148.248.110.66 2F 0000 0000 20K Gi5/1 41.132.133.125 Te4/4 27.252.197.88 2F 0000 0000 29K
Here we can see a potential issue of concern between source 12.232.32.146 and destination 57.220.106.66 which warrants further investigation.
Final Thoughts
To summarise DDoS is not something that going to go away any time soon, and DDoS strategies are supposedly even developed by GCHQ as a first line of attack during a military strike. If you’re running an ISP then at some point you will be under attack so to quote the Scout Moto – Be Prepared!
The Traceroute Blog 