Nginx + Tomcat + Redis + Spring for Loadbalance and session sharing

This blog describes how to use Nginx + Tomcat + Redis + Spring to setup a load balance and session sharing system.

• Setup environment

• Nginx

Download: http://nginx.org/en/download.html

Guide: http://nginx.org/en/docs/beginners_guide.html

Install Nginx as a windows service (Just an example):

Download NGinx (http://nginx.org/en/download.html) and uzip to C:\foobar\nginx

Download nssm (https://nssm.cc/)

Run "nssm install nginx" from the command line

In NSSM gui do the following:

On the application tab: set path to C:\foobar\nginx\nginx.exe, set startup directory to C:\foorbar\nginx

On the I/O tab type "start nginx" on the Input slow. Optionally set C:\foobar\nginx\logs\service.out.log and C:\foobar\nginx\logs\service.err.log in the output and error slots.

Click "install service". Go to services, start "nginx". Hit http://localhost:80 and you should get the nginx logon. Turn off the service, disable browser cache and refresh, screen should now fail to load.

(On Linux, we need a service file, and put it into /etc/init.d/)

• Tomcat

Download: https://tomcat.apache.org/download-80.cgi

• Redis

Download: https://redis.io/download

Download2: https://github.com/MicrosoftArchive/redis/releases

Guide: http://www.runoob.com/redis/redis-tutorial.html

Common commands:

GET key

SET key value

DEL key

KEYS *

EXISTS key

TTL key

...

A tool used to query redis db:

Keylord: https://protonail.com/keylord/download

• Configure Spring for session sharing

• Add Maven Dependencies

<properties>
    <!-- Spring Redis -->
    <spring.redis.version>1.2.2.RELEASE</spring.redis.version>
    <!-- jedis -->
    <jedis.version>2.9.0</jedis.version>
</properties>
<!-- Redis -->
<dependency>
    <groupId>org.springframework.session</groupId>
    <artifactId>spring-session-data-redis</artifactId>
    <version>${spring.redis.version}</version>
</dependency>
<dependency>
    <groupId>redis.clients</groupId>
    <artifactId>jedis</artifactId>
    <version>${jedis.version}</version>
</dependency>
           

• Update web.xml to add delegating filter

<!-- Spring Reids filter-->
<filter>
    <filter-name>springSessionRepositoryFilter</filter-name>
    <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
</filter>
<filter-mapping>
    <filter-name>springSessionRepositoryFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
           

Note: This filter should be the first filter in web.xml

• Define spring-session related beans

Create file: spring-redis.xml,

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:p="http://www.springframework.org/schema/p"
	xmlns:mvc="http://www.springframework.org/schema/mvc"
	xmlns:context="http://www.springframework.org/schema/context"
	xsi:schemaLocation="http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc.xsd
		http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
		http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">
	<context:annotation-config/>
	<bean id="redisHttpSessionConfiguration"
		class="org.springframework.session.data.redis.config.annotation.web.http.RedisHttpSessionConfiguration">
		<property name="maxInactiveIntervalInSeconds" value="600" />
	</bean>
	<bean id="jedisPoolConfig" class="redis.clients.jedis.JedisPoolConfig">
	    <property name="maxTotal" value="100" />
	    <property name="maxIdle" value="10" />
	</bean>
	<bean
		class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">
		<property name="hostName" value="localhost" />
		<property name="port" value="6379" />
	</bean>
</beans>
           

• Configure 3 tomcat

Update configuration file: conf/server.xml, change port to 8081, 8082, 8083,

<Server port="8181" shutdown="SHUTDOWN">
....
           

....
<Connector port="8081" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
...
           

Each tomcat should work well separately.

• Configure Nginx

Update configuration file: conf/nginx.conf,

#user  nobody;

# Worker process number, suggest equals to CPU cores
worker_processes  2;

# Global error log
error_log  logs/error.log  info;

# Process file
pid        logs/nginx.pid;

events {
	# Use epoll to increase performance on Linux
	# use epoll;
    worker_connections  1024;
}

http {
	# Extend name and file type map table, TODO: investigate it.
    include       mime.types;
	# Default type
    default_type  application/octet-stream;

	# Logs
    access_log  logs/access.log;
	error_log   logs/error.log;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    gzip  on;

	# Load balance servers
	upstream local_tomcat_awsauth {
		# Loadbalance method:
		# 1. Default, polling by request time, we can add weight parameter.
		# 2. ip_hash: Calculate hash by request IP
		# 3: url_hash: Calculate hash by request URL
		# 4: fair: Allocate by server's response time
		# ip_hash;
		# url_hash;
		# fair;
		server      localhost:8081 weight=1;
		server      localhost:8082 weight=1;
		server      localhost:8083 weight=1;
	}

    server {
        listen       8080;
        server_name  localhost;

		# Static files
		location ~ .*awsauth.*\.(js|css|png|jpg|jpeg|ico|gif)$ {
			root html;
			expires 3d;
		}

		# Dynamic requests
        location ~ .*awsauth.*$ {
			proxy_pass      http://local_tomcat_awsauth;
			# Following config can solve the issue that browser location may be changed
			proxy_set_header Host      $host:$server_port;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_redirect default ;
			# Following config can solve unstable performance issue
			proxy_connect_timeout 1;
			proxy_send_timeout 30;
			proxy_read_timeout 60;
        }

        error_page  404              /404.html;

        # redirect server error pages to the static page /50x.html
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}
           

For static resources, such as js, css, jpg, etc, create a folder awsauth under html folder, and copy related static resource into awsauth. Why create awsauth? Because this is to separate projects, each project should have a root r

For dynamic requests, all requests will be forward to tomcat servers.

• Q & A

• Are there any other mechanisms to share session?

• Yes, refer to:

  http://blog.51cto.com/youguofusi/2053919,

  https://blog.csdn.net/u012383839/article/details/79756368

  1: Use tomcat internal copy session method

  2: Store user status data on local Cookie

  3: Using DB to stare session

  4: Proxy server, such as Nginx, and use ip_hash load balance policy

  5: Use memcached-session-manager (MSM)

• How spring-session works?

• We configured a filter "org.springframework.web.filter.DelegatingFilterProxy" in web.xml, this filter will change HttpServletRequest object, and override getSession() method to return a HttpSession proxy object. When we use spring-session, there are no more code changes to handle session related logic, such session.setAttribute(), session.getAttribute(), etc. It's transparent for users.

• Where to find demo code?

• Please download demo code from https://github.com/wangxiaohui2015/awsauth.git, branch: redis-share-session