Many POST requests to /xmlrpc.php from GoogleBot taking down server?

I have several hosted wordpress blogs, and I’ve been trying to visit them and they are really slow. I looked at my server logs and I found this

stanfordflipside.com:80 188.138.33.149 - - [17/Aug/2013:17:14:28 -0700] "POST /xmlrpc.php HTTP/1.1" 200 595 "-" "GoogleBot/1.0"
stanfordflipside.com:80 188.138.33.149 - - [17/Aug/2013:17:14:28 -0700] "POST /xmlrpc.php HTTP/1.1" 200 595 "-" "GoogleBot/1.0"
stanfordflipside.com:80 188.138.33.149 - - [17/Aug/2013:17:14:28 -0700] "POST /xmlrpc.php HTTP/1.1" 200 595 "-" "GoogleBot/1.0"
stanfordflipside.com:80 188.138.33.149 - - [17/Aug/2013:17:14:28 -0700] "POST /xmlrpc.php HTTP/1.1" 200 595 "-" "GoogleBot/1.0"
stanfordflipside.com:80 188.138.33.149 - - [17/Aug/2013:17:14:29 -0700] "POST /xmlrpc.php HTTP/1.1" 200 595 "-" "GoogleBot/1.0"
stanfordflipside.com:80 188.138.33.149 - - [17/Aug/2013:17:14:29 -0700] "POST /xmlrpc.php HTTP/1.1" 200 595 "-" "GoogleBot/1.0"
stanfordflipside.com:80 188.138.33.149 - - [17/Aug/2013:17:14:29 -0700] "POST /xmlrpc.php HTTP/1.1" 200 595 "-" "GoogleBot/1.0"
stanfordflipside.com:80 188.138.33.149 - - [17/Aug/2013:17:14:29 -0700] "POST /xmlrpc.php HTTP/1.1" 200 595 "-" "GoogleBot/1.0"
stanfordflipside.com:80 188.138.33.149 - - [17/Aug/2013:17:14:31 -0700] "POST /xmlrpc.php HTTP/1.1" 200 595 "-" "GoogleBot/1.0"
stanfordflipside.com:80 188.138.33.149 - - [17/Aug/2013:17:14:31 -0700] "POST /xmlrpc.php HTTP/1.1" 200 595 "-" "GoogleBot/1.0"
stanfordflipside.com:80 188.138.33.149 - - [17/Aug/2013:17:14:31 -0700] "POST /xmlrpc.php HTTP/1.1" 200 595 "-" "GoogleBot/1.0"

I am getting ~10 hits per second to the file /xmlrpc.php from the GoogleBot to several sites, and this seems to be slowing down the server. I am running

Read More
tail -f 

on the log file, and can just see these requests continuing. Does anyone know why this might be happening or what you could do to stop it?

Related posts

4 comments

  1. I would block the IP with iptables if it were me, and if you have that kind of server level access.

    You could also disable xmlrpc. Unfortunately, since 3.5 the admin screen option to disable that feature has been removed. A single line of code should disable it though: add_filter( 'xmlrpc_enabled', '__return_false' ); That might save some overhead from the requests, though it won’t eliminate all of it.

  2. “Googlebot” has no reason to access xmlrpc.php
    You could add this to the top of your xmlrpc.php

    // Block fake Googlebot
    if ( strpos($_SERVER['HTTP_USER_AGENT'], "Googlebot") === true ) { exit(); }
    

    I’m guessing it’s a core WordPress file. So it might be annoying to keep this updated. Would be nice if Automattic used Akismet to blacklist these IPs from all WP scripts, everywhere.

    Update: I ended up removing permission with chmod 0 xmlrpc.php (see my comments) after a DDoS started to tax my server. In other words, this conditional PHP code might not stop an aggressive attacker from temporarily disabling your blog. In any case, they usually give up pretty fast.

  3. block the IP with iptables:

    for ip in $(grep xmlrpc /var/log/apache2/access.log | cut -d' ' -f1 | sort | uniq -c | sort -rn | head -n8 | awk '{print $2}'); do 
    iptables -A INPUT -s $ip -j DROP; 
    done
    
  4. Had this happen recently and it was killing the server and we’re now using fail2ban to mitigate the issue.

    Added this config to jail.local:

    [apache-xmlrpc]
    
    enabled = true
    port = http,https
    filter = xmlrpc
    logpath = /var/log/apache2/*access.log
    maxretry = 30
    findtime = 300
    bantime = -1
    

    And create the filter in filter.d/apache-xmlrpc.conf:

    [Definition]
    failregex = ^<HOST> -.*"(GET|POST) .*xmlrpc.php
    ignoreregex =
    

    In my case the attacks weren’t always coming from googlebot so made the regex a bit more broad but for my purposes there’s hardly any good reason for any IP to be hitting xmlrpc 30+ times in 5 minutes.

Comments are closed.