Tracking Spoofed IP Addresses Version 2.0

By Rob Thomas, robt@cymru.com, 08 FEB 2001

[ Documents ]       [ Home ]

Introduction

Tracking spoofed IP addresses back to the source can be quite a difficult task. For myriad reasons, such as limited router access, attacks of a short duration, and the manual nature of spoofed address tracking, finding the actual generator of the spoofed packets can be very difficult. For this reason, attackers often use the bogon address ranges, where a bogon address range is any unassigned and likely unrouted (by BGP4 in the Internet) netblock. This includes the RFC1918 addresses as well as a collection of other address spaces, such as 1/8, 169.254/16, and the like.

However, with a certain combination of features enabled on a Cisco router, it is possible to determine the source of the spoofed packets. Further, this can be done without the laborious and CPU intensive task of adding ACLs to filter the spoofed packets. The key features are CEF and NetFlow.

NOTE: Please be aware that router resources are never infinite. Both CEF and NetFlow require resources from the router, and therefore are not entirely immune to issues. NetFlow exports, in particular, may heavily load both the router and the export interface. Please take the time to test your configuration prior to deploying it in production.

While this document details a method for tracking the source of a DDoS attack that utilizes spoofed IP addresses, there are several other documents that detail methods of mitigating DDoS attacks. You can view my Secure IOS Template and my Secure BGP Template to enhance your router and peering security. There are also several efforts currently underway to block DDoS attacks. Here are a few informative links (thanks to John Kristoff for passing these along):

Pushback
http://www.research.att.com/~smb/talks/pushback-dodcert.pdf
http://www.aciri.org/floyd/talks/pushback-Nov00.pdf

Traceback
http://www.cs.washington.edu/homes/savage/traceback.html
http://www.research.att.com/lists/ietf-itrace/

CenterTrack
http://www.nanog.org/mtg-9910/ppt/robert/index.htm
http://www.us.uu.net/gfx/projects/security/centertrack.pdf

Router Configuration

Most high-end Cisco routers on the Internet run either CEF or dCEF. This is because of the large performance gains to be realized with CEF, which stands for Cisco Express Forwarding. CEF has many benefits over fast switching, including a more reliable and sturdy method for building the forwarding table. CEF also offers some security benefits, such as RPF (Reverse Path Forwarding). RPF provides a means of blocking packets that claim to originate from within your network, but present themselves on an external interface. Keep in mind that CEF can be a bit tricky to configure in an environment that has asymmetric data flows. You may wish to review the Cisco CEF White Paper. CEF is, therefore, a wise choice for reasons of performance and security. CEF is enabled on a global basis with the command ip cef. To enable dCEF (Distributed CEF), the global command is ip cef distributed.

NetFlow provides a means of mapping traffic flows through a router. This can be of great use for capacity planning, statistical analysis of traffic patterns, and security reviews. Here is a sample of the output from NetFlow:

router1#sh ip cache flow
IP packet size distribution (11319 total packets):
   1-32   64   96  128  160  192  224  256  288  320  352  384  416  448  480
   .000 .016 .002 .002 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

    512  544  576 1024 1536 2048 2560 3072 3584 4096 4608
   .000 .000 .000 .000 .976 .000 .000 .000 .000 .000 .000

IP Flow Switching Cache, 278544 bytes
  1 active, 4095 inactive, 19 added
  1909 ager polls, 0 flow alloc failures
  last clearing of statistics never
Protocol         Total    Flows   Packets Bytes  Packets Active(Sec) Idle(Sec)
--------         Flows     /Sec     /Flow  /Pkt     /Sec     /Flow     /Flow
TCP-Telnet           1      0.0       204    47      0.0      71.5       1.3
UDP-other            7      0.0         3   627      0.0       8.4      15.3
ICMP                10      0.0         5    91      0.0       4.1      15.4
Total:              18      0.0        15   103      0.0       9.5      14.6

SrcIf         SrcIPaddress    DstIf         DstIPaddress    Pr SrcP DstP  Pkts
Se1           192.168.88.5    Et0           192.168.77.2    11 0013 0007    31

From NetFlow, we can determine our packet distribution, protocol distribution, and current flows. Clearly this is valuable data. Enabling NetFlow is done on a per interface basis with the command: ip route-cache flow.

Once both CEF (or dCEF) and NetFlow are enabled on the router, we are ready to begin hunting the source of a spoofed IP address attack. It is recommended that the routers run Cisco IOS 12.0 or better.

Test Topology

In the example scenario, a malicious user on the host sweatpants, IP address 192.168.97.2/24, wishes to flood the host spanky, IP address 192.168.77.2/24, with copious amounts of bogus UDP traffic. To avoid being caught, the miscreant on sweatpants has decided to spoof his address to be 96.170.4.8. The miscreant knows that the entire 96/8 netblock is unassigned, and therefore any packets destined for this network will not arrive at an actual site. The spoofed packets are all destined for UDP port 7, the echo port. The source port is UDP port 19, the chargen port.

The network topology is:

       -----------------
      | spanky SPARC5   |
      | 192.168.77.2/24 |
       -----------------
             | Ethernet
             | e0
       -----------------
      | 192.168.77.1/24 |
      | router1         |
      | 10.10.10.1/30   |
       -----------------
             | s1
             | Serial 64Kb
             | s1
       -----------------
      | 10.10.10.2/30   |
      | router2         |
      | 172.17.50.2/30  |
       -----------------
             | s0
             | Serial 64Kb
             | s0                      Ethernet
       ---------------------------------     -------------------------
      | 172.17.50.1/30                  |   |                         |
      | router3          10.222.88.1/25 |---| 10.222.88.73/25 router5 |
      | 10.222.88.129/25                |   |                         |
       ---------------------------------     -------------------------
             | e1                      e0   e0
             | Ethernet
             | e0
       ------------------
      | 10.222.88.144/25 |
      | router4          |
      | 192.168.97.1/24  |
       -----------------
             | e1
             | Ethernet
       ------------------
      | 192.168.97.2/24  |
      | sweatpants Linux |
       ------------------

The routing is configured thusly:

spanky
    Default route, gateway 192.168.77.1 (router1)
router1
    Default route, gateway 10.10.10.2 (router2)
router2
    Static route 192.168.77.0/24, gateway 10.10.10.1 (router1)
    Static route 192.168.97.0/24, gateway 172.17.50.1 (router3)
router3
    Default route, gateway 172.17.50.2 (router2)
    Static route 192.168.97.0/24, gateway 10.222.88.144 (router4)
router4
    Default route, gateway 10.222.88.129 (router3)
router5
    Default route, gateway 10.222.88.1 (router3)
sweatpants
    Default route, gateway 192.168.97.1 (router4)

While static routing was used for this experiment, the experiment is not fundamentally changed by the use of a dynamic routing protocol, such as OSPF or EIGRP. While the responses, to the spoofed address, from spanky might be dropped sooner in the path, the result is the same -- the spoofed packets never make it back to sweatpants, the source of the malevolent data stream. It is not uncommon to find the use of default routes for networks that are singly attached to the Internet.

The Game Begins

Using a packet generator, the attack is launched from sweatpants against spanky. A steady stream of spoofed packets now present themselves on the network interface of spanky. Due to the interrupt saturation and higher than normal CPU load, the attack is detected by the system administrator. The use of the snoop tool (Solaris specific packet sniffer) determines the source IP of the attack, 96.170.4.8. The network and security teams are alerted. Once the source IP address (96.170.4.8) and source port (UDP 19) are noted from the output of snoop, the first step is to login to the border router, router1, and take a look.

In this topology, it may seem quite obvious that the source of the spoofed packets, from the perspective of router1, must be the serial interface leading to router2. However, it is wise to validate this assumption to ensure that the source of the spoofed attack is not a host within the same subnet as spanky. First, the NetFlow cache is queried thusly:

router1#sh ip cache flow | include 96.170
Se1           96.170.4.8      Et0           192.168.77.2    11 0013 0007   159

Here we see that the source interface of the flow, which is listed in column one, is serial1. So it has been determined that the source is somewhere beyond the border router. Next, CEF is queried. CEF inserts all active sources, on a per interface basis, in its tables.

router1#sh ip cef se1
Prefix              Next Hop             Interface
0.0.0.0/0           10.10.10.2           Serial1
10.10.10.0/30       attached             Serial1

Here it is seen that the only next hop, according to the CEF cache, is 10.10.10.2. Consulting the topology above, it is noted that the next hop IP address is router2. The search moves one hop further, to router2.

The process is repeated on router2. First, a check of the NetFlow cache:

router2#sh ip cache flow | include 96.170
Se0           96.170.4.8      Se1           192.168.77.2    11 0013 0007   299

The source interface of the flow is serial0. Now for a check of the CEF tables:

router2#sh ip cef se0
Prefix              Next Hop             Interface
172.17.50.0/30      attached             Serial0
192.168.97.0/24     172.17.50.1          Serial0

Once again, the topology is consulted and it is determined that the next hop listed in the CEF tables, 172.17.50.1, is router3.

On router3, the NetFlow tables are examined:

router3#sh ip cache flow | include 96.170
Et1           96.170.4.8      Se0           192.168.77.2    11 0013 0007  3235

Ah, perhaps the end is near! The source interface for the flow is Ethernet1. Is the source station directly attached to this router? A check of the CEF tables reveals:

router3#sh ip cef et1
Prefix              Next Hop             Interface
10.222.88.128/25    attached             Ethernet1
10.222.88.144/32    10.222.88.144        Ethernet1
192.168.97.0/24     10.222.88.144        Ethernet1
10.222.88.73/32     10.222.88.73         Ethernet1

This presents a bit of a conundrum; there are two possible sources. It may be necessary to check both IP addresses. First, a check of router5, 10.222.88.73.

router5#sh ip cache flow | include 96.170
router5#

This command returns nothing. After verifying that the attack is still underway, it is obvious that the attacker's data flow does not pass through this router. Moving on to router4 reveals:

router4#sh ip cache flow | include 96.170
Et1           96.170.4.8      Et0           192.168.77.2    11 0013 0007  6673

Ah, this looks promising. A quick check of the CEF tables finds:

router4#sh ip cef et1
Prefix              Next Hop             Interface
192.168.97.0/24     attached             Ethernet1
192.168.97.2/32     192.168.97.2         Ethernet1

So the only active IP address is 192.168.97.2. Since a quick check of either the MAC address (with sh arp) or other means reveals that this is not a Cisco router, this IP address begins to look more suspect. At this point, network sniffing can be performed to verify that the source IP of the attack, 96.170.4.8, is tied to the MAC address of 192.168.97.2. The source of the spoofed IP addresses has been found.

Limitations

While this method is fast and presents very little impact on the routers, it is not without certain limitations.

First, NetFlow must be running on the interfaces. NetFlow can be configured, in real-time, during an attack. The NetFlow data is the key to this method.

Second, router access must be available. This can be a hurdle both technically (no access to the routers) and politically (the routers are owned by another entity). However, this can be a coordinated effort, with multiple teams handing off the tracing as each autonomous system boundary is crossed. If the trace is done completely within a single AS, however, many of the political and technical issues may not exist.

Third, the attack must be of a duration that allows for a trace. Short, bursty attacks may not allow for a full trace. While a partial trace may help to narrow the scope of the search, it will not find the culprit.

Fourth, this method is obviously limited to the Cisco IOS platform. Other platforms, such as a Check Point FireWall-1 firewall, will provide similar tracing capabilities through the rule base or tools such as tcpdump, snoop, and iptrace. However, some platforms may provide no trace method at all.

Conclusion

Uncovering the source of a spoofed IP attack can assure that the attacking host is removed as a threat to all networks. With a few relatively simple and quick steps, the source of such an attack can be revealed.

[ Documents ]       [ Home ]

Rob Thomas, robt@cymru.com, http://www.cymru.com