Going static

A few years ago, I started this blog for two reasons. First, I wanted a place to share thoughts, ideas, tools, and research on web security. After seven posts, I consider it a success.

Second, I saw a few WordPress sites being hacked under “my watch”. This would typically translate to unauthorized changes in the form of defacement or including malware to promote its dissemination.

All the forensic analyses led to the same conclusion: the attack could be tracked to either an outdated WordPress core or an outdated plugin. This got me thinking about how hard it would be to keep a WordPress installation up-to-date and safe from being hacked. 

And that was the second reason: I wanted to experience first-hand the pain other site owners were having and hopefully reach some practical advice that would make them safer and I could sleep better.

So my first blog post, Hardening WordPress, was the start of that experience.

And today, eight years later, is the end of that experience. I give it to you that the experience effectively ended a few years ago, and in the last few years, this blog was abandoned and only used as a firing range for my current challenge (Probely).

I am proud to say the blog was never hacked, I think… At least, I never saw any unauthorized content or change, it was never reported as malicious, and the downtimes were only caused by me being cheap and using a t2.nano AWS instance.

Doing an analysis after all this time is hard, but let me try:

– my initial hardening had a significant impact, and I’m pretty sure most installations are not hardened at this level. Protecting the login and admin pages with strong Basic Auth credentials protected me against brute-force attacks. More importantly, my passwords were 30 chars long, randomly generated by a password manager using a large character space.

– having a minimal set of plugins and themes reduced my attack surface: I only installed a stats plugin, Akismet, to handle spam in the comments and the Sucuri Security Scanner to check for changes.

– having a plugin checking for filesystem changes and applying security best practices also helped.

– having 1 click updates and automatic updates was super cool and helped me stay up-to-date. But I remember it was a pain to set up if you wanted to minimize the amount of access WordPress had to the filesystem. I had WordPress SSHing to itself through localhost, with a dedicated account, with limited permissions. This prevented the code from changing the filesystem, but it was still able to run a specific command to update it. That broke frequently, so I later reverted to whatever setup WordPress recommended.

Today it auto-updates itself, updates the plugins and themes, and I get a friendly email saying what happened.

I know my setup was not as realistic as other installations running businesses on top of their installation, so I would often install some plugins my network used. And it was a nightmare because I could no longer update WordPress immediately, as some plugins would get disabled. If I were lucky, the plugin would get an update in a few days or weeks. Some never got updated.

So that was my main conclusion after a few months: it was trivial to let WordPress become out-of-date because you didn’t set up auto-update. Either inadvertently or on purpose to avoid breaking something. Or you could have set up auto-update, but it was trivial for the installation process to break with the slightest change. 

And if you don’t update immediately, you will likely not update in the following months.

Another issue was the dependence on no longer (or poorly) maintained plugins. Here the options were to stop using the plugin or modify it to fix the vulnerabilities. The former was the choice when the plugin had a secondary role, like providing access stats. The latter was reserved for plugins that supported the business, like e-commerce, and if the team had developers capable of modifying the plugin. Unfortunately, this would easily take weeks or months.

After a while, I no longer really care about WordPress :)

I must be honest; for the last 3 or 4 years, my biggest challenge was handling MySQL with 512 MB of RAM and being able to upgrade to newer releases of Linux with just 8 GB of disk. When I got tired of that and upgraded the machine resources, I decided that it made no sense to keep supporting this cost.

So more expensive and less fun, and it made me convert this to a static version. And it couldn’t be easier:

  • Install a plugin like Simply Static
  • Generate a zip with all the content
  • Commit it to a GitHub Pages repository
  • Update Cloudflare CNAMES to point to GitHub, fine-tune the TLS, redirects, and DNS to work with HTTPS, with and without www, and you are done.
  • Take down the WordPress VM

So that is the end of this WordPress experiment. I will keep it offline just in case I want to continue to post or do some experiments. But most likely not.

Testing uploads with Burp Intruder – Updated

Just updated my Intruder extension (described here) to provide two payload generators: the original one with the file contents and another one with the matching file name.

So, if you need both the content and file name in your Intruder attack, choose Pitchfork as the Attack type and use File as Payload for one Payload set and Filename as Payload for the other.

With this update you can synchronize the file name and its contents in the attack without much hassle, something that was not trivial in the previous version.

Testing uploads with Burp Intruder

While testing an file upload functionality with Burp’s Repeater I noticed the site response changed depending on the file being uploaded, which is normal if the server is validating the file headers.
After a few requests with different responses I got tired of doing this manually and decided to use Burp’s Intruder feature. I quickly noticed that there is no way of telling the Intruder to inject the contents of a list of files in the request.

So I wrote an extension that reads a folder files and feeds their content, one at a time, as a payload to use in the Intruder.
Suppose you have a folder with your best malicious files, zip and xml bombs, jpegs with other contents, php shells, etc. You point the extension to this folder and just use the Repeater normally, setting the payload source as being this extension.
If you need/want to synchronize the file contents with the file name, the extension tab shows the names of the files it just read so you can use them as payload for a second position in the request, using the pitchfork attack.

EDIT
There is a new version of the extension that provides two generators, one for the file name and other for the file contents, easing this synchronization process. See this update post.

Burp Intruder File Payload Extension tab

Choosing the input files

Configuring the Intruder

Configuring the Intruder

The extension is available here as a single jar. The code is available here.
I have submitted it to the BApp Store but it hasn’t been approved yet.

EDIT
Now available at the BApp Store, here :)

Enjoy.

OCSP stapling

Tired of seeing wrong OCSP stapling configurations, even those made by me, I’m writing this so I won’t forget how to do it right..and hopefully aid others. This is also an excuse to read more about the subject :)

OCSP stapling is good: you have one less connection to a server that has nothing to do with the intended purpose of the original connection and still get an OCSP response. Sure, you have a few more bytes on every connection but it is harder for the attacker to interrupt the OCSP process without interfering with the target server itself.

For nginx, you need to use a configuration at least with these directives:

ssl_stapling on;
ssl_stapling_verify on;
ssl_trusted_certificate /etc/nginx/certs/bundle_with_root_stapling.pem;

ssl_stapling on; enables stapling. To be honest, you can make stapling work only with this line.

ssl_stapling_verify on; this forces our server to check if the OCSP response is valid, meaning properly signed by the respective CA.

ssl_trusted_certificate /etc/nginx/certs/bundle_with_root_stapling.pem; path to the chain that validates the responses. If you enabled ssl_stapling_verify you need this directive. It must have all the intermediates CAs, including the root. Just concatenate every CA in the chain, PEM encoded, with the root in the end.

Then just issue sudo service nginx restart and test it. You can test it with echo | openssl s_client -connect mendo.pt:443 -status

You will likely get something like
OCSP response: no response sent

This happens because nginx will not prefetch a valid response until it gets a connection, so the first connection won’t receive an OCSP response. Try again after a few seconds and you will get

OCSP response:
======================================
OCSP Response Data:
OCSP Response Status: successful (0x0)
Response Type: Basic OCSP Response
Version: 1 (0x0)
Responder Id: C = IL, O = StartCom Ltd. (Start Commercial Limited), CN = StartCom Class 1 Server OCSP Signer
Produced At: May 8 20:48:21 2015 GMT
Responses:
Certificate ID:
Hash Algorithm: sha1
Issuer Name Hash: 6568874F40750F016A3475625E1F5C93E5A26D58
Issuer Key Hash: EB4234D098B0AB9FF41B6B08F7CC642EEF0E2C45
Serial Number: 05A143427C3754
Cert Status: good
This Update: May 8 20:48:21 2015 GMT
Next Update: May 10 20:48:21 2015 GMT

Signature Algorithm: sha1WithRSAEncryption
98:47:f1:39:4b:ef:f6:67:3d:46:d4:3c:93:fc:ab:95:20:a3:
e8:d0:2b:64:63:d8:46:d7:65:3f:00:5d:50:9c:7d:b7:2c:4c:
36:b1:58:86:9f:4b:95:d4:a4:fa:0f:bb:50:3b:a3:e8:d2:a6:
f3:5f:8c:c3:62:b1:75:90:8a:ab:50:cd:33:6a:cc:6c:ea:04:
a5:65:ff:09:15:41:0c:76:98:7a:79:93:f4:e1:8b:a5:a1:0c:
1e:90:04:25:05:98:91:2c:cc:62:0a:2b:06:a3:28:46:55:90:
f3:0f:09:3c:57:51:98:ed:9c:6f:03:f6:01:9e:d5:61:1b:cf:
f2:7a:0b:f2:9d:e1:52:ba:4a:34:ba:5c:95:7b:40:de:83:96:
55:d0:df:41:e7:14:58:28:9b:6a:67:67:fc:db:59:ab:8a:35:
90:a9:75:8b:9e:8b:46:51:0c:e9:8c:4e:43:60:34:f7:86:06:
09:0b:5f:03:16:b7:ab:15:49:ea:15:c4:f7:c6:d0:ec:22:01:
fd:8a:f1:d0:b9:f5:12:14:b7:13:5b:13:80:a6:60:be:11:bc:
28:c8:86:44:4b:1f:dd:fc:1e:5a:2e:a0:21:52:8d:c7:6c:e5:
72:c0:11:f4:15:16:54:96:14:f7:5f:5a:3f:cf:46:10:60:f5:
46:15:0a:8b

Simple.

EDIT
Because nginx will only resolve the OCSP hostname at startup and its IP can change, you might consider setting the following directive (pick you DNS resolver):

resolver 8.8.8.8;

Monitoring Pastebin

Suppose you want to monitor the Internet for certain keywords such as your username, email or some text pattern relevant for you or your company. A good starting point is to monitor Pastebin for those keywords, since that is where most of the interesting stuff gets dumped.

What I wanted was simple:

  • detect new pastes that match a given pattern
  • save matching pastes
  • do some processing on those pastes
  • notify me with the results

You can register alarms on Pastebin for up to 3 keywords and get notified upon matching, but you cannot use regular expressions or exclude keywords. Also, mail notifications are not the easiest to handle automatically.

The classical choice was to use Pastemon, maybe the first software of its kind. I had two problems with it: it would not save pastes to file and it was no longer maintained.
Searching for alternatives I found Pystemon, a newer and improved version of Pastemon.

So I ended up building the following setup:

1. setup Pystemon to download and store any pastes that match my rules.

  1. get (my version of) Pystemon git clone https://github.com/tmendo/pystemon.git
  2. edit pystemon.yaml
  3. set the archive dir
  4. enable and configure email alerts
  5. enable the use proxy option
  6. define some search patterns, for instance
  7.  - search: '[^a-zA-Z0-9]example.(com|org)'
       description: 'example domains'
     - search: 'tmendo'
       description: 'tmendo'
       exclude: 'tmendoza'
    

  8. run it with python pystemon.py -v -c pystemon.yaml

A few mistakes and Pastebin blocked my IP. Later they unblocked it and told me the threshold to stay safe, but in the meantime I have decided to proxy the requests through TOR. Since TOR only exposes a SOCKS proxy and Pystemon only supports HTTP proxies I used DeleGate.
Pystemon was originally designed to cycle through multiple proxies removing those that fail. I needed it to stick with the TOR proxy even if it eventually failed sometimes, so I forked the project, added support for a single proxy without removals, fixed some TODOs, included a few fixes from another committer, and did a pull request.

2. setup DeleGate to convert an SOCKS proxy to an HTTP proxy
Because TOR exposes a SOCKS proxy and Pystemon only supports HTTP proxy I am running DeleGate to pass requests between them.

  1. download the source
  2. tar zxvf delegate9.9.13.tar.gz
  3. make
  4. create dg.conf
    -P127.0.0.1:8080
    -Tx
    -fv
    SOCKS=127.0.0.1:9050
    DGROOT="/path/to/delegate/"
    SERVER=http
    REMITTABLE="http,https/443"
    CACHE=no
    TIMEOUT="shutout:30m"
    MAXIMA="randstack:32"
    MAXIMA="randenv:1024"
    MAXIMA="randfd:32"
    [email protected]
    

  5. execute with /path/to/delegated +=/path/to/delegate/dg.conf

3. setup TOR following these instructions.
This will install and start the daemon with the default configuration that generates new circuits every 10 minutes (you get a new IP every 10 minutes, at least). Pastebin does not discriminate TOR exit nodes, not sure about the other paste sites.

Test the DeleGate and TOR combination:

telnet localhost 8080
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET / HTTP/1.1
host: mendo.pt

HTTP/1.1 301 Moved Permanently
DeleGate-Ver: 9.9.13 (delay=9)
Server: nginx
Date: Fri, 27 Mar 2015 14:38:04 GMT
Content-Type: text/html
Location: https://mendo.pt/
Strict-Transport-Security: max-age=31536000; includeSubDomains
Via: 1.1 - (DeleGate/9.9.13)
Connection: close
Content-Length: 178


profit!

4. setup a script that parses the stored pastes and do something useful.
You can use inotifywait to detect new pastes downloaded by Pystemon and do whatever you like with them (run it in a sandbox, do some further string matching, etc.)

#!/bin/bash

is_dir_pattern=\bISDIR\b

inotifywait -r -m /path/to/pystemon/output -e close_write |
    while read path action file; do
        if [[ ! "$action" =~ $is_dir_pattern ]]; then
            echo "The file '$file' appeared in directory '$path' via '$action'"
	    # process $path/$file
            # notify me of some finding
        fi
    done

EDIT
Listening to create and moved_to events might result in you being warned of a new file before its contents are written so you would end up with an empty file to process. Use close_write instead.

And thats it. Put everything under DJB Daemontools and you are able to monitor some paste sites for whatever you like and whatever reason.

Not even God forgives passwords in clear text

I think this comic from CommitStrip says it all:
CommitStrip clear text passwords

Passwords stored in clear text are half of the recipe to scary headlines such as “thousands of accounts published online by hackers”. To complete the recipe just add a single SQL injection or a good path traversal/LFI mix.

Mature and trained developers rarely make such mistake, why? Because they had training and understand the consequences. The problem is the others, which need to be self-taught.

So let me be very clear:

  • you don’t need passwords in clear text to compare against the ones from login
  • you can compare representations of them, lets say an hash
  • never store or log passwords

Regarding the storing itself, there are way too many options [1]. I will recommend just one, which is adequate for most situations (if you feel this does not fits you, please contact the nearest security guy, or leave a comment):

function store(password)
    salt = secureRandom(32bits)
    hash = pbkdf2(salt, password, 10000)

    storeAtDB(salt;hash)
end

function validate(user, password)
    salt,hash = getUserFromDB(user).split(;)
    login_hash = pbkdf2(salt, password, 10000)

    if (login_hash XOR hash == 0 )
        return true
    else
        return false
end

I won’t explain these in detail, but the basic idea is to store an hash of the password plus a salt. No, it is not easy to find out the password given an hash + salt. No, it won’t slow down your application. Yes, you will be much, much safer.

[1] https://www.owasp.org/index.php/Password_Storage_Cheat_Sheet

B-Sides Lisbon is coming

The second edition of B-Sides Lisbon is coming! This security conference is happening at the 3rd of July in Lisbon and its a whole day event with two tracks of talks and some good discussions. And it is free!

Screen Shot 2015-01-29 at 21.39.40

The previous, and first edition, happened in 2013 and it was awesome. You can revisit the program here and search for the talks on Google (you will find some for sure).
With a quick search I found Bruno Morisson’s preso:


btw, he is also one of the organizers :)

I also did a presentation on how to start using Burp Suite, including a few tips to use it more efficiently:

Now that you know this years edition will be even better, go ahead and submit a talk here

Hardening WordPress

I think it makes sense that the very first post on my brand new WordPress blog would be about hardening WordPress itself. This makes even more sense if you consider it took me more time to harden this blog than to write this post.

For the installation just follow the official documentation found here to which I just want to make one remark about the database password strength:

GRANT ALL PRIVILEGES ON databasename.* TO "wordpressusername"@"hostname" IDENTIFIED BY "password";

Use a long password here, something like sK8ytB8mR5PtxqqTmCEYThEUJS5J6D which is 30 chars long. You won’t need to type this password often so there is no need to use a memorable one.
To be honest, a weak password here wouldn’t be that bad since it is unlikely that you server will fall because of this…unless you expose it beyond 127.0.0.1.

For the hardening, start here. There is some smalltalk but there are also some good advices regarding a few topics:

  • updates
  • file permissions
  • wp-admin
  • wp-includes
  • security plugins

updates
Keeping your installation up to date is paramount to ensure proper security. Before version 3.7 updates really sucked because the easiest way to (automatically) update was to have all files writable by the webserver process, so basically any malicious code executed by the webserver would be able to overwrite any file from our installation. The alternative was more secure but harder, so people wouldn’t update.
In the latest versions there is some magic that let you have automatic updates without the need to have files writable by the webserver. I think there is more than one way to achieve this: I went for the SSH one.
I followed this tutorial. In addition to the 5 defines added to the wp-config.php file, I also added define('FS_METHOD', 'ssh2');.
Since we are adding a new SSH user and we don’t want to increase your attack surface, disable password authentication for SSH at /etc/ssh/sshd_config with PasswordAuthentication no. Well…you should always disable password authentication for SSH.

file permissions
The important thing here is to prevent the webserver from writing files. There are some exceptions, as you might want to allow functionality such as image upload (so I can haz pretty potatoes pic).
You can have all files with owner and group wp-user, everything with 644 (files) or 755 (diretories).
The exception is wp-content/uploads that must be writable by the group (775) and have the group www-data set. You don’t need to allow webserver writing for the others since updates and installations are done through SSH and run with wp-user.

wp-admin
/wp-admin is a prime target for attackers because that is where you login to administrate your blog with your admin username and your weak password :)
Leaving /wp-admin world accessible exposes you to two problems: credential bruteforce/reuse/theft and direct access to files that may have all sort of vulnerabilities.
The most practical way to protect you from these is require some sort of authentication, say basic authentication. With nginx you just need to add this to your virtual host

location = /wp-admin {
auth_basic "Restricted";
auth_basic_user_file /path/to/your/htpasswd;
}

and define the username and password. Please use a strong password here and also for your /wp-admin accounts.

EDIT
Also protect the /wp-login.php since it is used to login to edit the blog.
I include the snippet below to demonstrate the care you must take with nginx locations while configuring the rules described in this blog. Without going into details, prefix matching has precedence over regexp matching and nested locations with regex must have only regex inside.
location = /wp-admin {
auth_basic "Restricted";
auth_basic_user_file /etc/nginx/htpasswd;
}

location ~ .php$ {
location ~* /wp-login.php {
auth_basic "Restricted login";
auth_basic_user_file /etc/nginx/htpasswd;
}

location ~* wp-config.php {
deny all;
}

security plugins
Having many plugins installed is asking for trouble since many have vulnerabilities and don’t even get fixed. So, going against the usual recommendation of not installing plugins, install the Sucuri Security – Auditing, Malware Scanner and Hardening plugin.
After installing, you get a Sucuri Security button on your dashboard side menu that lets you access the plugin dashboard
Sucuri Security menu

Some of the features are paid, but you get a few nice things for free. In the hardening tab you can verify if have the proper secure configurations, however some only work if you are using Apache. More specifically, “Protect uploads directory”, “Restrict wp-content access” and “Restrict wp-includes access” will remain red under nginx even if properly configured, because the plugin expects Apache specific configuration.
In the Settings tab you have another interesting feature: the scanner. The plugin can scan your wordpress installation for changes and notify you by email, like an HIDS. Go through the Settings tabs and activate/deactivate options as you please.

Additionally to the 5 previous topics, I recommend following this extra 2:

Delete themes
Regarding themes…you just use one so delete those you don’t use, as well as the pre-installed useless plugin Hello Dolly.

Limit PHP execution
Direct access to PHP files laying around in your installation are known to cause problem, so disable PHP execution wherever you can. If using nginx add the configuration below to your virtual host. For Apache just use the Sucuri plugin or use Google.

location ~* wp-config.php {
deny all;
}

location ~* wp-content/(.*).php$ {
deny all;
}

location ~* wp-includes/(.*).php$ {
deny all;
}

location ~* wp-includes/uploads/sucuri$ {
deny all;
}

And finally…HTTPS
I don’ need extra motivation to make my blog available only by HTTPS but you if you are thinking twice, just remember Google favors HTTPS sites in the search results and you can get certificates for free from StartSSL, for instance.
Configuring HTTPS for WordPress is just like doing it for any another virtual host. Just remember a few things:

  • change your site from http://example.com to https://example.com at Settings->General->WordPress Address and Site Address
  • force a 301 redirect from HTTP to HTTPS
  • send a HSTS header to ensure you are always under HTTPS
  • disable all SSL versions and enable only TLS
  • enable strong ciphers such as ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-RC4-SHA:RC4:HIGH:!MD5:!aNULL:!EDH
  • enable server cipher preference

Then, go to SSL Labs and test your configuration.

If you are paranoid, you can continue hardening your WordPress, webserver, PHP, and OS forever, but you have to balance that against the time you have available.

So..I guess I have an WordPress blog. Lets see how long it holds without being hacked…