TIL: Vagrant, Nginx and illegal token errors

Recently I have moved all my development to Vagrant virtual machines to avoid having to setup a dev environment on my local machine. Mark Drew has an excellent article on how to set this up. My setup is modeled very close to Mark’s setup however I setup Nginx as a reverse proxy back to Tomcat as this models my production environment.

I had everything setup but I ran into trouble pretty quickly any time I changed a javascript file it would completely wreck the file and I would get illegal token errors. It appears there is an issue with VirtualBox when serving static content through Nginx if you are using shared folders. The fix for this issue is turning sendfile off in the Nginx configuration. More information can be found here.

Hopefully this will help someone out because I know I was pulling my hair out on this one.


Nginx and locking down your WEB-INF folder

I was working on some URL rewriting rules on my server this evening when I was taking a break at work. Just out of curiosity I tried hitting some of the files that are available in the WEB-INF directory and was surprised to find out that I could easily view all my context logs and anything that was not a ColdFusion template. In retrospect, I should not have been surprised but I should have taken steps to prevent that from ever happening.

I have a drop.conf file that gets loaded on every site so blocking directory access on all my sites was just a matter of adding the directive to block access to the directory and restarting nginx.

In hindsight, I should have checked that a long time ago. Hopefully someone will learn from my mistake.

Nginx connector for Railo/ACF part deux

Edit: This post is now outdated. See this post: https://kisdigital.wordpress.com/2013/03/04/my-final-nginxrailo-connector/

Almost a year ago I posted about an improved connector for nginx proxy to Railo. While I consider myself fortunate moving from Apache over to nginx, it is rather unfortunate that I did not have the time to really test my server setup and understand what was happening under the hood. Then I posted it.

Six months or so ago I drifted away from personal development. My family and work demand much of my attention and I don’t have a lot of “extra” time, but lately I have been trying to spend more time learning new things. As a web developer if your server stack is not solid you are going to be in bad shape so I figured it would be a good time to go back and revisit this.

I thought it would be best to look around and see what folks a lot smarter than me were doing which brought me to KBeezie’s Nginx configuration examples and it was a lot of help. My new server template is based off of his examples, I also went and did some digging in the Railo user group. Special thanks to Serge Droganov in the Railo group  for pointing me the right direction getting the administrator locked down.

This is my default new site template. First we have the default handler that looks to see if the URI exists, if not it looks at the rewrite rules to see if there is a match. If not I want to just load the default document. Here the rewrite rules are enabled SES URLs for the server that will work both with and without the index.cfm in the url (i.e., http://example.com/cool/page or http://example.com/index.cfm/cool/page). My preference leans toward SES URLs because they are much cleaner, but you know what they say about opinions. Standard URLs will work just fine as well.

Second I am hiding the Railo Administrator as it just makes sense. Adding the “internal;” directive to /railo-context/admin/ tells Nginx that you do not want to world to get to it, only rewrites coming from another rewrite rule on the server should be allowed.

Next I am moving my Railo Administrator to http://example.com/secret/location/(server|web).cfm. I was going to try to get a little bit fancier with it, but it works well. Since this is an internal rewrite, we are allowed to pass through to the administrator.

Finally we have the @rewrites named location block that is used by our default handler. If the page being requested does not exist it will try to find a match among these rewrite rules and dispatch the request. These can be built upon later to do some interesting things, but by default it just handles serving the SES urls.

The rest of the directives here describe how to handle static content and what URIs you want dropped automatically such as configuration files, directories, etc which are described in the dropped.conf file. I also included the railo.conf at the bottom of the file which handles passing off ColdFusion files to the app server. It has not changed much from my original post, but I will include it here anyway.

The only changes here are moving the railo-context code out and just keeping it in the server configuration blocks.

I definitely suggest you check out KBeezies. A lot of it is targeted towards PHP, but quite a bit of it is applicable to A/CF and also the Nginx Wiki.

Improved connector for Nginx proxy to Railo

Edit: This post is outdated. Please see this post https://kisdigital.wordpress.com/2013/03/04/my-final-nginxrailo-connector/

I have been working with Nginx quite a bit in the last week and I have had a little time to fine-tune my configuration a bit. Here is my stock Railo configuration. I have saved the settings to their own file and include the connector in each separate server configuration that requires Railo. This also remaps the standard Railo administrator to a more secure location and optionally sets basic authentication.

 # /etc/nginx/railo_connector.conf
 # Block default Railo admin
 if ($request_uri ~* ^/railo-context){
  return 404;

 # Hide the Railo Administrator and optionally lock down with password
 location ~ ^/hardtoguesslocation/(.*)$ {
  #auth_basic $host;
  #auth_basic_user_file /path/to/htpasswd;
  if($request_uri ~^/railo-context/admin){
   return 404;
  location ~^/hardtoguesslocation/{
   rewrite ^/hardtoguesslocation/(.*)$ /railo-context/admin/$1 last;

 # Main Railo proxy handler
 location ~ \.(cfm|cfml|cfc|jsp|cfr)(.*)$ {
  proxy_redirect off;
  proxy_set_header Host $host;
  proxy_set_header X-Forwarded-Host $host;
  proxy_set_header X-Forwarded-Server $host;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header X-Real-IP $remote_addr;

Also, Nginx allows you to use variables in your server configuration so it allows you to easily create a “catch all” virtual host. You can quickly add a new website just by adding it to your server.xml in Railo with no additional configuration required unless you require domain specific rewrites etc. Here is an example server configuration with a default virtual host and a separate domain configured:

server {
 #Catchall vhost
 listen    80; ## listen for ipv4
 server_name _;
 root /var/www/$host;
 index index.cfm;
 access_log  /var/logs/nginx/$host-access.log;
 # Do not log missing favicon.ico errors
 location = /favicon.ico { access_log off; log_not_found off; }
 # Do not serve any .hidden files
 location ~ /\. { access_log off; log_not_found off; deny all; }
 include /etc/nginx/railo_connector.conf;
# End of catch-all Server Configuration

server {
 #A domain with custom handling
 listen    80; ## listen for ipv4
 server_name mydomain.com www.mydomain.com;
 root /var/www/mydomain.com;
 index index.cfm;
 access_log  /var/logs/nginx/mydomain.com-access.log;
 # Do not log missing favicon.ico errors
 location = /favicon.ico { access_log off; log_not_found off; }
 # Do not serve any .hidden files
 location ~ /\. { access_log off; log_not_found off; deny all; }
 # Handle FW/1 style SES urls (i.e. http:/domain.com/main/default/key/value)
 location /{
  try_files $uri $uri/ @ses;
 location @ses{
  rewrite ^/index.cfm/$uri last;
 include /etc/nginx/railo_connector.conf;
# End of custom Server Configuration

If you want the X-Real-IP reported correctly on the Railo server you will also need to add the RemoteIP valve to your Tomcat configuration.

Overall Nginx makes a very nice front-end to Railo and as always I welcome any comments or suggestions to make it better.

Retain remote address when proxying Railo with Nginx

I have been taking a long, hard look at Nginx recently.  First I was playing around with it as a load balancer and the ease of getting it setup really got my attention.  After playing around with my cluster for a while I needed something else to play with so I decided to remove Apache from my standard server configuration and added Nginx.

I quickly had everything setup and running, but I noticed the remote address in the CGI scope was coming back as which is not exactly what I was looking for.  Looking at the proxy settings in my Nginx config I had set all the right proxy headers, but Tomcat was ignoring the proxy headers.  Doing a few quick searches I have seen this was an issue, but you could read the real ip address by examining the headers and pulling the appropriate header field, etc.  That is great, however I am lazy and I would prefer to do it automagically.

So I decided to do a little more searching.  As it turns out, Tomcat version 6.0.24 added a way to translate the X-Real-IP header and allow Railo to use that without having to do any header-fu.  All you have to do is add one line to your server.xml under the <Engine> container:

<Valve className="org.apache.catalina.valves.RemoteIpValve"  />


Dumping out the CGI scope you should now see the remote address of the user instead of the address of the proxy server. Hopefully this will save someone some time because I know it sure drove me crazy for a long time last night.

Load balancing a Railo cluster using AWS

Lately I have been playing around with load balancing Railo in a cluster.  Before Amazon Web Services this would have been a fairly expensive proposition, but AWS makes managing instances mostly painless.  Here I will be using EC2 instances, but this setup will can apply to just about any configuration.

The minimal requirements for this exercise will be to have three instances running.  First I created two instances of 64-bit Amazon Linux running configured with a basic Railo setup using the VivioTech installers and Apache as the front end.  I already have an AMI image created so I can easily create identically configured Railo instances with the click of a button.

Next we need to configure an instance to handle load balancing to each of our Railo instances and for this I will be using Nginx.   First, create a clean, new instance of Amazon 64-bit linux and SSH into the instance.  Amazon Linux uses the yum package manager so installing Nginx is as easy as:

yum install nginx

Once Nginx is installed, we just need to edit the config file and add in a few lines.  You can find the configuration file in /etc/nginx/nginx.conf.

# HTTP Core Module
#   http://wiki.nginx.org/NginxHttpCoreModule

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on;

    upstream balancer {
        server 10.x.x.1;
        server 10.x.x.2;

    # The default server
    server {
        listen       80;
        server_name  _;

        location / {
            proxy_pass http://balancer;

        error_page  404              /404.html;
        location = /404.html {
            root   /usr/share/nginx/html;

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   /usr/share/nginx/html;

    # Load config files from the /etc/nginx/conf.d directory
    include /etc/nginx/conf.d/*.conf;


This is the complete listing for the nginx.conf file.  This configuration is pretty much stock, I have removed the comments, added the upstream directive and modified the default server to proxy requests to the upstream provider named balancer.

This approach will evenly distribute requests to both machines.  You can add the hash;  ip_hash; directive to the top of  your upstream block to make the requests “sticky” and send requests to the same server behind the load balancer each time.

I have just started to using this setup so I am sure things will probably change as I start seeing what it can do. If you are interested in setting up a similar system to mess around with I highly recommend AWS.

Some helpful documentation: http://wiki.nginx.org/HttpUpstreamModule

Installing Railo with Nginx web server on Windows

Update 3/3/2011: This information is now a little dated.  If you would like to see the updated content, cruise over the the Railo Wiki for the updated instructions.

Installing Railo and Nginx is fairly straight forward.  Here we will cover a quick and easy single host installation and setup the configuration files.

Continue reading “Installing Railo with Nginx web server on Windows”