Guidelines to handle Distributed Denial of Service incidents

IRM author: CERT SG / CERT SG / Vincent Ferran-Lacome
IRM version: 1.4

Preparation

1

Objective: Establish contacts, define procedures, and gather information to save time during an attack.

Internet Service Provider support

  • Contact your ISP to understand the DDoS mitigation services it offers (free and paid) and what process you should follow.
  • If possible, subscribe to a redundant Internet connection.
  • If possible, subscribe to an Anti-DDoS service provider.
  • Establish contacts with your ISP and law enforcement entities. Make sure that you have the possibility to use an out-of-band communication channel (e.g.: phone).

Inventory

  • Create a whitelist of the IP addresses and protocols you must allow if prioritizing traffic during an attack. Don’t forget to include your critical customers, key partners, etc.
  • Document your IT infrastructure details, including business owners, IP addresses and circuit IDs, routing settings (AS, etc); prepare a network topology diagram and an asset inventory.

Network infrastructure

  • Design a good network infrastructure without Single Point of Failure or bottleneck.
  • Distribute your DNS servers and other critical services (SMTP, etc) through different AS.
  • Harden the configuration of network, OS, and application components that may be targeted by DDoS.
  • Baseline your current infrastructure’s performance, so you can identify the attack faster and more accurately.
  • If your business is Internet dependent, consider purchasing specialized DDoS mitigation products or services.
  • Confirm DNS time-to-live (TTL) settings for the systems that might be attacked. Lower the TTLs, if necessary, to facilitate DNS redirection if the original IP addresses get attacked. 600 is a good TTL value.
  • Depending of the criticality of your services, consider setting-up a backup that you can switch on in case of issue.

Internal contacts

  • Establish contacts for your IDS, firewall, systems, and network teams.
  • Collaborate with the business lines to understand business implications (e.g., money loss) of likely DDoS attack scenarios.
  • Involve your BCP/DR planning team on DDoS incidents.

The “preparation” phase is to be considered as the most important element of a successful DDoS incident response.

Identification

2

Objective: Detect the incident, determine its scope, and involve the appropriate parties.

Analyze the attack

  • Understand the logical flow of the DDoS attack and identify the infrastructure components affected by it.

  • Understand if you are the target of the attack or a collateral victim

  • Review the load and log files of servers, routers, firewalls, applications, and other affected infrastructure.

  • Identify what aspects of the DDoS traffic differentiate it from benign traffic

    • Source IP addresses, AS, etc
    • Destination ports
    • URLs
    • Protocols flags

    Network analysis tools can be used to review the traffic -> Tcpdump, Tshark, Snort, Argus, Ntop, Aguri, MRTG

  • If possible, create a NIDS signature to focus to differentiate between benign and malicious traffic.

Involve internal and external actors

  • Contact your internal teams to learn about their visibility into the attack.
  • Contact your ISP to ask for help. Be specific about the traffic you’d like to control:
    • Network blocks involved
    • Source IP addresses
    • Protocols
  • Notify your company’s executive and legal teams.

Check the background

  • Find out whether the company received an extortion demand as a precursor to the attack.
  • Search if anyone would have any interest into threatening your company
    • Competitors
    • Ideologically-motivated groups (hacktivists)
    • Former employees

Containment

3

Objective: Mitigate the attack’s effects on the targeted environment.

  • If the bottleneck is a particular feature of an application, temporarily disable that feature.
  • Attempt to throttle or block DDoS traffic as close to the network’s “cloud” as possible via a router, firewall, load balancer, specialized device, etc.
  • Terminate unwanted connections or processes on servers and routers and tune their TCP/IP settings.
  • If possible, switch to alternate sites or networks using DNS or another mechanism. Blackhole DDoS traffic targeting the original IP addresses.
  • Set up an alternate communication channel between you and your users/customers (e.g.: web server, mail server, voice server, etc.)
  • If possible, route traffic through a traffic-scrubbing service or product via DNS or routing changes (e.g.: sinkhole routing)
  • Configure egress filters to block the traffic your systems may send in response to DDoS traffic (e.g.: backsquatter traffic), to avoid adding unnecessary packets to the network.
  • In case of an extortion attempt, try to buy time with the fraudster. For example, explain that you need more time in order to get management approval.

If the bottleneck is at the ISP’s side, only the ISP can take efficient actions. In that case, work closely with your ISP and make sure you share information efficiently.

Remediation

4

Objective: Take actions to stop the Denial of Service condition.

  • Contact your ISP and make sure that it enforces remediation measures. For information, here are some of the possible measures:
    • Filtering (if possible at level Tier1 or 2)
    • Traffic-scrubbing/Sinkhole/Clean-pipe
    • Blackhole Routing
  • If the DDoS sponsors have been identified, consider involving law enforcement. This should be performed upon the direction of your company’s executive and legal teams.

Technical remediation actions can mostly be enforced by your ISP.

Recovery

5

Objective: Come back to the previous functional state.

Assess the end of the DDoS condition

  • Ensure that the impacted services are reachable again.
  • Ensure that your infrastructure performance is back to your baseline performance.

Rollback the mitigation measures

  • Switch back traffic to your original network.
  • Restart stopped services.

Ensure that the recovery-related actions are decided in accordance with the network teams. Bringing up services could have unexpected side effects.

Aftermath

6

Objective: Document the incident’s details, discuss lessons learned, and adjust plans and defences.

  • Consider what preparation steps you could have taken to respond to the incident faster or more effectively.
  • If necessary, adjust assumptions that affected the decisions made during DDoS incident preparation.
  • Assess the effectiveness of your DDoS response process, involving people and communications.
  • Consider what relationships inside and outside your organizations could help you with future incidents.
  • Collaborate with legal teams if a legal action is in process.