Unix Like, 操作系统, 服务器, 网络, 运营维护

Nginx ssl_preread传递客户端IP

ssl_preread是基于L4的反代方案，TLS SNI握手时客户端会提供域名，所以可以让Nginx在无需完成TLS握手的情况下，就根据域名进行后端服务器的选择。简单来讲，你只需要给后端服务器配置一个SSL证书，而提供反代功能的Nginx则无需配置。文档见此：http://nginx.org/en/docs/stream/ngx_stream_ssl_preread_module.html

因为这是L4反代，所以通过“proxy_set_header”传递客户端IP的方法是行不通的。其实传递客户端IP的解决办法跟上一篇文章类似：L4(传输层)IP透明反向代理的实现(传递客户端真实IP)，nginx也是实现了IP_TRANSPARENT的：https://www.nginx.com/blog/ip-transparency-direct-server-return-nginx-plus-transparent-proxy#ip-transparency，所以，在server里面加上下面一行：

proxy_bind $remote_addr transparent;

1	proxy_bind $remote_addr transparent;

然后按前一篇文章的思路，对路由表进行配置即可。

提醒一下，ssl_preread是配置在stream block下的，放在别的地方，会提示：

nginx: [emerg] "ssl_preread" directive is not allowed here in /etc/nginx/nginx.conf

1	nginx: [emerg] "ssl_preread" directive is not allowed here in /etc/nginx/nginx.conf

所以正确的nginx.conf结构应该如下：

...
http {
    ...
    server {
        ...
    }
    server {
        ...
    }
    ...
}
...
stream {
    map $ssl_preread_server_name $name {
        backend.example.com      backend;
        default                  backend2;
    }

    upstream backend {
        server 192.168.0.1:12345;
        server 192.168.0.2:12345;
    }

    upstream backend2 {
        server 192.168.0.3:12345;
        server 192.168.0.4:12345;
    }

    server {
        listen      12346;
        proxy_pass  $name;
        proxy_bind $remote_addr transparent;
        ssl_preread on;
    }
}
...

...

http {

...

server {

...

}

server {

...

}

...

}

...

stream {

map $ssl_preread_server_name $name {

backend.example.com backend;

default backend2;

}

upstream backend {

server 192.168.0.1:12345;

server 192.168.0.2:12345;

}

upstream backend2 {

server 192.168.0.3:12345;

server 192.168.0.4:12345;

}

server {

listen 12346;

proxy_pass $name;

proxy_bind $remote_addr transparent;

ssl_preread on;

}

...

另外，如果在stream中listen了443，http中就无法listen 443了，启动nginx会提示：

nginx: [emerg] bind() to 0.0.0.0:443 failed (98: Address already in use)

1	nginx: [emerg] bind() to 0.0.0.0:443 failed (98: Address already in use)

可以让http中的server listen loopback地址的另一个端口，让stream中的server proxy_pass这个loopback地址的端口，例如127.0.2.1:8443，然后通过127.0.2.1做策略路由：

...
http {
    ...
    server {
        ...
        listen 127.0.2.1:8443 ssl http2;

        port_in_redirect off; # 重要：阻止nginx重定向到此Server listen的端口
        ...
    }
    server {
        ...
    }
    ...
}
...
stream {
    map $ssl_preread_server_name $name {
        localbackend.com         local_backend; # 这里指定使用http中的server
        backend.example.com      backend;
        default                  backend2;
    }

    upstream backend {
        server 192.168.0.1:12345;
        server 192.168.0.2:12345;
    }

    upstream backend2 {
        server 192.168.0.3:12345;
        server 192.168.0.4:12345;
    }

    # 对应http中的server
    upstream local_backend {
        server 127.0.2.1:443;
    }

    server {
        listen      443;
        proxy_pass  $name;
        proxy_bind $remote_addr transparent; # 加了这个才能传递客户端IP
        ssl_preread on;
    }
}
...

...

http {

...

server {

...

listen 127.0.2.1:8443 ssl http2;

port_in_redirect off; # 重要：阻止nginx重定向到此Server listen的端口

...

}

server {

...

}

...

}

...

stream {

map $ssl_preread_server_name $name {

localbackend.com local_backend; # 这里指定使用http中的server

backend.example.com backend;

default backend2;

}

upstream backend {

server 192.168.0.1:12345;

server 192.168.0.2:12345;

}

upstream backend2 {

server 192.168.0.3:12345;

server 192.168.0.4:12345;

}

# 对应http中的server

upstream local_backend {

server 127.0.2.1:443;

}

server {

listen 443;

proxy_pass $name;

proxy_bind $remote_addr transparent; # 加了这个才能传递客户端IP

ssl_preread on;

}

...

然后配置策略路由：

root@nginx:~# ip rule add from 127.0.2.1 lookup 61
root@nginx:~# ip route add local 0.0.0.0/0 dev lo table 61

1 2	root@nginx:~# ip rule add from 127.0.2.1 lookup 61 root@nginx:~# ip route add local 0.0.0.0/0 dev lo table 61

Tags: nginx, Reverse Proxy, SNI, ssl_preread, stream, tls, 反向代理, 客户端IP

You might also like:

Unix Like

libvirtd: virNetTLSContextValidCertificate:994 : Unable to verify TLS peer: No certificate was found.

Jan 29 06:20:12 debian libvirtd[3363]: 2019-01-29 11:20:12.116+0000: 3363: error : virNetTLSContextValidCertificate:994 : Unable to verify TLS peer: No certificate was found.
Jan 29 06:20:12 debian libvirtd[3363]: 2019-01-29 11:20:12.116+0000: 3363: warning : virNetTLSContextCheckCertificate:1125 : Certificate check failed Unable to verify TLS peer: No certificate was found.
Jan 29 06:20:12 debian libvirtd[3363]: 2019-01-29 11:20:12.116+0000: 3363: error : virNetTLSContextCheckCertificate:1128 : authentication failed: Failed to verify peer's certificate

Jan 29 06:20:12 debian libvirtd[3363]: 2019-01-29 11:20:12.116+0000: 3363: error : virNetTLSContextValidCertificate:994 : Unable to verify TLS peer: No certificate was found.

Jan 29 06:20:12 debian libvirtd[3363]: 2019-01-29 11:20:12.116+0000: 3363: warning : virNetTLSContextCheckCertificate:1125 : Certificate check failed Unable to verify TLS peer: No certificate was found.

Jan 29 06:20:12 debian libvirtd[3363]: 2019-01-29 11:20:12.116+0000: 3363: error : virNetTLSContextCheckCertificate:1128 : authentication failed: Failed to verify peer's certificate

使用TLS连接libvirtd时，默认需要验证客户端是否拥有由CA签名的证书，详见libvirtd配置文件（/etc/libvirt/libvirtd.conf）：

# Flag to disable verification of client certificates
#
# Client certificate verification is the primary authentication mechanism.
# Any client which does not present a certificate signed by the CA
# will be rejected.
#
# Default is to always verify. Uncommenting this will disable
# verification - make sure an IP whitelist is set
#tls_no_verify_certificate = 1

# Flag to disable verification of client certificates

# Client certificate verification is the primary authentication mechanism.

# Any client which does not present a certificate signed by the CA

# will be rejected.

# Default is to always verify. Uncommenting this will disable

# verification - make sure an IP whitelist is set

#tls_no_verify_certificate = 1

取消此行的注释即可关闭此验证（当然你得启用其它身份认证途径）：

#tls_no_verify_certificate = 1

1	#tls_no_verify_certificate = 1

也可选择以让CA给客户端签发证书：

# 自行修改CA私钥与证书的路径
CA_KEY_FILE_PATH="/etc/pki/libvirt/private/cakey.pem"
CA_CERTIFICATE_FILE_PATH="/etc/pki/CA/cacert.pem"

# 客户端私钥，CSR，证书保存路径
PKI_PREFIX="/tmp/client_pki/"
CLIENT_PRIVATE_KEY_FILE_PATH="${PKI_PREFIX}/clientkey.pem"
CLIENT_CSR_FILE_PATH="${PKI_PREFIX}/clientcsr.pem"
CLIENT_CERTIFICATE_FILE_PATH="${PKI_PREFIX}/clientcert.pem"

oldUmask=$(umask)
umask 0077
mkdir -p ${PKI_PREFIX}

# 生成客户端私钥
openssl genrsa -out ${CLIENT_PRIVATE_KEY_FILE_PATH} 2048
# 生成客户端CSR，根据自己需要自行修改subjects
openssl req -new -key ${CLIENT_PRIVATE_KEY_FILE_PATH} -subj "/C=CN/ST=China/L=China/O=China/OU=China/CN=ClientCommonName" -out ${CLIENT_CSR_FILE_PATH}
# CA通过CSR签发证书，根据需要自行修改有效期
openssl x509 -req -in ${CLIENT_CSR_FILE_PATH} -CA ${CA_CERTIFICATE_FILE_PATH} -CAkey ${CA_KEY_FILE_PATH} -CAcreateserial -out ${CLIENT_CERTIFICATE_FILE_PATH} -days 3650
# 把CA证书放进去
cat ${CA_CERTIFICATE_FILE_PATH} > ${PKI_PREFIX}/cacert.pem

umask ${oldUmask}

# 把${PKI_PREFIX}传送到客户端
# scp -r ${PKI_PREFIX} username@host:~/libvirt-server-pki

# 自行修改CA私钥与证书的路径

CA_KEY_FILE_PATH="/etc/pki/libvirt/private/cakey.pem"

CA_CERTIFICATE_FILE_PATH="/etc/pki/CA/cacert.pem"

# 客户端私钥，CSR，证书保存路径

PKI_PREFIX="/tmp/client_pki/"

CLIENT_PRIVATE_KEY_FILE_PATH="${PKI_PREFIX}/clientkey.pem"

CLIENT_CSR_FILE_PATH="${PKI_PREFIX}/clientcsr.pem"

CLIENT_CERTIFICATE_FILE_PATH="${PKI_PREFIX}/clientcert.pem"

oldUmask=$(umask)

umask 0077

mkdir -p ${PKI_PREFIX}

# 生成客户端私钥

openssl genrsa -out ${CLIENT_PRIVATE_KEY_FILE_PATH} 2048

# 生成客户端CSR，根据自己需要自行修改subjects

openssl req -new -key ${CLIENT_PRIVATE_KEY_FILE_PATH} -subj "/C=CN/ST=China/L=China/O=China/OU=China/CN=ClientCommonName" -out ${CLIENT_CSR_FILE_PATH}

# CA通过CSR签发证书，根据需要自行修改有效期

openssl x509 -req -in ${CLIENT_CSR_FILE_PATH} -CA ${CA_CERTIFICATE_FILE_PATH} -CAkey ${CA_KEY_FILE_PATH} -CAcreateserial -out ${CLIENT_CERTIFICATE_FILE_PATH} -days 3650

# 把CA证书放进去

cat ${CA_CERTIFICATE_FILE_PATH} > ${PKI_PREFIX}/cacert.pem

umask ${oldUmask}

# 把${PKI_PREFIX}传送到客户端

# scp -r ${PKI_PREFIX} username@host:~/libvirt-server-pki

完成上面的操作后，把整个${PKI_PRE […]

服务器, 未分类, 运营维护

巧用Nginx的日志功能配合IPTABLES实现防范基础CC攻击(2013年11月24日8:24:13更新)

相信不少人都被CC过，我也不例外，因此一直以来想尽各种办法降低PHP所占的资源。 PHP所耗的资源算压下去了，带宽却满了…… 之前也看过有人使用Nginx的第三方模块limit req2实现防CC攻击，自己也弄过，但效果没想象中的好，只用了几十分钟我就抛弃了…… 昨天其中考试结束后，突然灵机一动，想到一个新的防CC方法。回家后经过多次调试，几经完善，已经基本能正常使用了。这其中利用了Nginx的日志功能，还有自己写的一个Shell Script。本文出自微宇宙(http://zhensheng.im)，转载者死一户口本。 Apache的请移步这里：Apache:只有访客IP的日志格式首先给 […]

使用Varnish Cache时，让Nginx获取访客真实IP

之前发表过一篇文章：Varnish(前)+Nginx(中)时,让Apache(后)获取用户真实IP(多重代理) 该方法能成功解决使用或者不使用CDN时，在Varnish前端，Nginx中端，Apache解析PHP文件的情况下让Apache获取访客真实IP。因为当时主要是利用Nginx进行缓存，没使用其他功能，旧没让Nginx也获取访客真实的IP。其中Varnish处理XFF的关键代码：

     if (req.restarts == 0) {
        if (req.http.x-forwarded-for) {
#           set req.http.X-Forwarded-For =
#               req.http.X-Forwarded-For + ", " + client.ip;
        } else {
            set req.http.X-Forwarded-For = client.ip;
        }
     }

if (req.restarts == 0) {

if (req.http.x-forwarded-for) {

# set req.http.X-Forwarded-For =

# req.http.X-Forwarded-For + ", " + client.ip;

} else {

set req.http.X-Forwarded-For = client.ip;

}

也就是把客户的IP赋值给XFF。那样后端处理就方便多了。昨天在为一台Varnish+Nginx+Apache的服务器添加并测试防轻量级CC的 […]

一个高频率的HTTP服务监控Shell Script

Nginx/Apache之类的长期运行，无法保证在遇到各种突发情况进程不会死掉。如果这时候又出门在外，一时间无法操作，那网站岂不是要宕机数个小时？明显我无法接受这事情。于是一个简简单单的监控HTTP服务的Shell Script就出来了。

#!/bin/bash
#函数restart
restart() {
#重启Nginx
service nginx restart
#重启Apache
service httpd restart
}
#循环的开始
while [ "$loop" = "" ]
do
#延迟五秒
sleep 5
#是否能下载，是则进入下一步循环，否则执行restart
wget -O /dev/null http://你的网站地址 -t 1 || restart
done

#!/bin/bash

#函数restart

restart() {

#重启Nginx

service nginx restart

#重启Apache

service httpd restart

}

#循环的开始

while [ "$loop" = "" ]

#延迟五秒

sleep 5

#是否能下载，是则进入下一步循环，否则执行restart

wget -O /dev/null http://你的网站地址 -t 1 || restart

done

真的简简单单。可见我并没有采用cron功能，毕竟这东西正常情况最低频率只能一分钟了。 restart那个function是http服务不正常后执行的，可以自行增添，删减，替换。sleep后面跟着的是秒数，在这里简单理解成监控频率吧。wget处，-t参数指的是尝试次数，这里我写了一次。执行 […]