不久前开了个香港Sunny Vsion的VPN,平时我一般都不使用VPN,今天才发现无法访问百度,而尝试其他网站却正常,遂探讨了一番。
我使用的都是学校的中国电信网络,本地连接VPN后,访问百度网站无任何回应,在服务器上可以成功地使用curl对百度发起请求,检查了iptables,未发现任何问题。
既然服务器能访问,那么理论上VPN也有权访问,但现在却与理论背道而驰。为了找出问题的根源,搬出抓包神器——tcpdump。
首先我在服务器上抓取本地分别透过VPN访问cPanel/WHM的208.74.0.0/16与百度的103.235.46.39/32的数据包,以作对比,查找错误。
cPanel/WHM:
1 2 3 4 5 6 7 8 9 10 |
13:27:14.541132 IP 192.168.80.101.37810 > 208.74.121.58.http: Flags [S], seq 105236783, win 29120, options [mss 1456,sackOK,TS val 646368 ecr 0,nop,wscale 7], length 0 13:27:14.734461 IP 208.74.121.58.http > 192.168.80.101.37810: Flags [S.], seq 314263993, ack 105236784, win 14480, options [mss 1460,sackOK,TS val 3961523628 ecr 646368,nop,wscale 7], length 0 13:27:14.744998 IP 192.168.80.101.37810 > 208.74.121.58.http: Flags [.], ack 1, win 228, options [nop,nop,TS val 646419 ecr 3961523628], length 0 13:27:14.745009 IP 192.168.80.101.37810 > 208.74.121.58.http: Flags [P.], seq 1:91, ack 1, win 228, options [nop,nop,TS val 646419 ecr 3961523628], length 90 13:27:14.938442 IP 208.74.121.58.http > 192.168.80.101.37810: Flags [.], ack 91, win 114, options [nop,nop,TS val 3961523832 ecr 646419], length 0 13:27:14.941765 IP 208.74.121.58.http > 192.168.80.101.37810: Flags [P.], seq 1:275, ack 91, win 114, options [nop,nop,TS val 3961523835 ecr 646419], length 274 13:27:14.953069 IP 192.168.80.101.37810 > 208.74.121.58.http: Flags [.], ack 275, win 236, options [nop,nop,TS val 646471 ecr 3961523835], length 0 13:27:14.954152 IP 192.168.80.101.37810 > 208.74.121.58.http: Flags [F.], seq 91, ack 275, win 236, options [nop,nop,TS val 646471 ecr 3961523835], length 0 13:27:15.147484 IP 208.74.121.58.http > 192.168.80.101.37810: Flags [F.], seq 275, ack 92, win 114, options [nop,nop,TS val 3961524041 ecr 646471], length 0 13:27:15.158090 IP 192.168.80.101.37810 > 208.74.121.58.http: Flags [.], ack 276, win 236, options [nop,nop,TS val 646523 ecr 3961524041], length 0 |
百度:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
13:26:02.822931 IP 192.168.80.101.38152 > 103.235.46.39.http: Flags [S], seq 4272637935, win 29120, options [mss 1456,sackOK,TS val 628437 ecr 0,nop,wscale 7], length 0 13:26:02.825570 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [S.], seq 705137582, ack 4272637936, win 8192, options [mss 1452,sackOK,nop,nop,nop,nop,nop,nop,nop,nop,nop,nop,nop,wscale 5], length 0 13:26:02.844031 IP 192.168.80.101.38152 > 103.235.46.39.http: Flags [.], ack 1, win 228, length 0 13:26:02.844037 IP 192.168.80.101.38152 > 103.235.46.39.http: Flags [P.], seq 1:78, ack 1, win 228, length 77 13:26:02.846621 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], ack 78, win 776, length 0 13:26:02.910170 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [P.], seq 1:1063, ack 78, win 776, length 1062 13:26:02.910255 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 1063:2515, ack 78, win 776, length 1452 13:26:02.910297 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 2515:3967, ack 78, win 776, length 1452 13:26:02.910330 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [P.], seq 3967:5383, ack 78, win 776, length 1416 13:26:02.910384 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 5383:6835, ack 78, win 776, length 1452 13:26:02.910421 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 6835:8287, ack 78, win 776, length 1452 13:26:02.910464 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 8287:9739, ack 78, win 776, length 1452 13:26:02.910508 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 9739:11191, ack 78, win 776, length 1452 13:26:02.910541 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 11191:12643, ack 78, win 776, length 1452 13:26:02.910581 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 12643:14095, ack 78, win 776, length 1452 13:26:02.910620 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [P.], seq 14095:15463, ack 78, win 776, length 1368 13:26:02.910691 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 15463:16915, ack 78, win 776, length 1452 13:26:02.910731 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 16915:18367, ack 78, win 776, length 1452 13:26:02.910769 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 18367:19819, ack 78, win 776, length 1452 13:26:02.910802 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 19819:21271, ack 78, win 776, length 1452 13:26:02.910842 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 21271:22723, ack 78, win 776, length 1452 13:26:02.910878 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [P.], seq 22723:24103, ack 78, win 776, length 1380 13:26:02.921894 IP 192.168.80.101.38152 > 103.235.46.39.http: Flags [.], ack 1063, win 245, length 0 13:26:02.924587 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 24103:25555, ack 78, win 776, length 1452 13:26:02.924654 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 25555:27007, ack 78, win 776, length 1452 13:26:02.924692 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 27007:28459, ack 78, win 776, length 1452 13:26:02.924725 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 28459:29911, ack 78, win 776, length 1452 13:26:02.924757 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 29911:31363, ack 78, win 776, length 1452 13:26:03.127164 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 1063:2515, ack 78, win 776, length 1452 13:26:03.223104 IP 192.168.80.101.38152 > 103.235.46.39.http: Flags [.], ack 1063, win 267, options [nop,nop,sack 1 {3967:5383}], length 0 13:26:03.223109 IP 192.168.80.101.38152 > 103.235.46.39.http: Flags [.], ack 1063, win 290, options [nop,nop,sack 2 {14095:15463}{3967:5383}], length 0 13:26:03.223116 IP 192.168.80.101.38152 > 103.235.46.39.http: Flags [.], ack 1063, win 313, options [nop,nop,sack 3 {22723:24103}{14095:15463}{3967:5383}], length 0 13:26:03.531227 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 1063:2515, ack 78, win 776, length 1452 13:26:04.339146 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 1063:2515, ack 78, win 776, length 1452 13:26:05.955117 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 1063:2515, ack 78, win 776, length 1452 13:26:09.187091 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 1063:2515, ack 78, win 776, length 1452 13:26:15.651057 IP 103.235.46.39.http > 192.168.80.101.38152: Flags [.], seq 1063:2515, ack 78, win 776, length 1452 |
首先,在两边的记录,都可以看到Flags [P.],这是开始传输内容的标记,因此可以肯定,我们本地与百度之间的TCP三次握手是顺利地完成了的,不然也就不会开始传输数据。
在与cPanel/WHM服务器通讯时,看到了Flags [F.],这是完成数据传输,结束TCP连接的标记。
但是在百度却没看到Flags [F.],可以看到103.235.46.39发送了很多类似seq 1:1063的数据包,但是我们本地仅返回了ack 1063给百度,根据TCP协议,ack的可以理解为“确认收到”,ack 1063,那就是说我们本地告诉百度的服务器确认收到了seq 1:1063的数据包。
TCP是一个可靠的协议,之所以说可靠,是因为服务器会重新发送客户端没有收到的数据包,可以保障数据的完整性,既然我们本地仅返回了ack 1063,百度服务器很自然就会重发我们没收到的数据包,也就对应了百度那边的抓包结果最后面的seq 1063:2515。
总算找到方向了:我们本地可能没有完整地收到百度发送的所有数据包!
为了确定我们本地是否真的没有收到数据包,我再次访问百度,并进行了双边抓包:
本地:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
14:22:59.656792 IP 192.168.80.101.40744 > 103.235.46.39.80: Flags [S], seq 1055253884, win 29120, options [mss 1456,sackOK,TS val 1482546 ecr 0,nop,wscale 7], length 0 14:22:59.763117 IP 103.235.46.39.80 > 192.168.80.101.40744: Flags [S.], seq 337176589, ack 1055253885, win 8192, options [mss 1452,sackOK,nop,nop,nop,nop,nop,nop,nop,nop,nop,nop,nop,wscale 5], length 0 14:22:59.763156 IP 192.168.80.101.40744 > 103.235.46.39.80: Flags [.], ack 1, win 228, length 0 14:22:59.763196 IP 192.168.80.101.40744 > 103.235.46.39.80: Flags [P.], seq 1:78, ack 1, win 228, length 77: HTTP: GET / HTTP/1.1 14:22:59.872895 IP 103.235.46.39.80 > 192.168.80.101.40744: Flags [.], ack 78, win 776, length 0 14:22:59.917880 IP 103.235.46.39.80 > 192.168.80.101.40744: Flags [P.], seq 1:1062, ack 78, win 776, length 1061: HTTP: HTTP/1.1 200 OK 14:22:59.917895 IP 192.168.80.101.40744 > 103.235.46.39.80: Flags [.], ack 1062, win 245, length 0 14:23:00.218830 IP 103.235.46.39.80 > 192.168.80.101.40744: Flags [P.], seq 6870:8262, ack 78, win 776, length 1392: HTTP 14:23:00.218881 IP 192.168.80.101.40744 > 103.235.46.39.80: Flags [.], ack 1062, win 267, options [nop,nop,sack 1 {6870:8262}], length 0 14:23:00.218859 IP 103.235.46.39.80 > 192.168.80.101.40744: Flags [P.], seq 18426:19782, ack 78, win 776, length 1356: HTTP 14:23:00.218915 IP 192.168.80.101.40744 > 103.235.46.39.80: Flags [.], ack 1062, win 290, options [nop,nop,sack 2 {18426:19782}{6870:8262}], length 0 14:23:07.123426 IP 192.168.80.101.40744 > 103.235.46.39.80: Flags [F.], seq 78, ack 1062, win 290, options [nop,nop,sack 2 {18426:19782}{6870:8262}], length 0 14:23:07.440603 IP 192.168.80.101.40744 > 103.235.46.39.80: Flags [F.], seq 78, ack 1062, win 290, options [nop,nop,sack 2 {18426:19782}{6870:8262}], length 0 14:23:07.459161 IP 103.235.46.39.80 > 192.168.80.101.40744: Flags [.], ack 79, win 776, length 0 14:23:07.482778 IP 103.235.46.39.80 > 192.168.80.101.40744: Flags [.], ack 79, win 776, options [nop,nop,sack 1 {78:79}], length 0 |
服务器:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
14:22:59.312939 IP 192.168.80.101.40744 > 103.235.46.39.http: Flags [S], seq 1055253884, win 29120, options [mss 1456,sackOK,TS val 1482546 ecr 0,nop,wscale 7], length 0 14:22:59.329289 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [S.], seq 337176589, ack 1055253885, win 8192, options [mss 1452,sackOK,nop,nop,nop,nop,nop,nop,nop,nop,nop,nop,nop,wscale 5], length 0 14:22:59.422096 IP 192.168.80.101.40744 > 103.235.46.39.http: Flags [.], ack 1, win 228, length 0 14:22:59.422107 IP 192.168.80.101.40744 > 103.235.46.39.http: Flags [P.], seq 1:78, ack 1, win 228, length 77 14:22:59.439733 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], ack 78, win 776, length 0 14:22:59.483966 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [P.], seq 1:1062, ack 78, win 776, length 1061 14:22:59.484047 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 1062:2514, ack 78, win 776, length 1452 14:22:59.484107 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 2514:3966, ack 78, win 776, length 1452 14:22:59.484153 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 3966:5418, ack 78, win 776, length 1452 14:22:59.484187 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 5418:6870, ack 78, win 776, length 1452 14:22:59.484224 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [P.], seq 6870:8262, ack 78, win 776, length 1392 14:22:59.484318 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 8262:9714, ack 78, win 776, length 1452 14:22:59.484363 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 9714:11166, ack 78, win 776, length 1452 14:22:59.484416 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 11166:12618, ack 78, win 776, length 1452 14:22:59.484474 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 12618:14070, ack 78, win 776, length 1452 14:22:59.484525 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 14070:15522, ack 78, win 776, length 1452 14:22:59.484567 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 15522:16974, ack 78, win 776, length 1452 14:22:59.484602 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 16974:18426, ack 78, win 776, length 1452 14:22:59.484637 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [P.], seq 18426:19782, ack 78, win 776, length 1356 14:22:59.484716 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 19782:21234, ack 78, win 776, length 1452 14:22:59.484773 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 21234:22686, ack 78, win 776, length 1452 14:22:59.484826 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 22686:24138, ack 78, win 776, length 1452 14:22:59.484863 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 24138:25590, ack 78, win 776, length 1452 14:22:59.484902 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 25590:27042, ack 78, win 776, length 1452 14:22:59.484937 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 27042:28494, ack 78, win 776, length 1452 14:22:59.581306 IP 192.168.80.101.40744 > 103.235.46.39.http: Flags [.], ack 1062, win 245, length 0 14:22:59.599105 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 28494:29946, ack 78, win 776, length 1452 14:22:59.599180 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 29946:31398, ack 78, win 776, length 1452 14:22:59.814748 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 1062:2514, ack 78, win 776, length 1452 14:22:59.881946 IP 192.168.80.101.40744 > 103.235.46.39.http: Flags [.], ack 1062, win 267, options [nop,nop,sack 1 {6870:8262}], length 0 14:22:59.881951 IP 192.168.80.101.40744 > 103.235.46.39.http: Flags [.], ack 1062, win 290, options [nop,nop,sack 2 {18426:19782}{6870:8262}], length 0 14:23:00.244698 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 1062:2514, ack 78, win 776, length 1452 14:23:01.104726 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 1062:2514, ack 78, win 776, length 1452 14:23:02.824626 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 1062:2514, ack 78, win 776, length 1452 14:23:06.265629 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], seq 1062:2514, ack 78, win 776, length 1452 14:23:06.700973 IP 192.168.80.101.40744 > 103.235.46.39.http: Flags [F.], seq 78, ack 1062, win 290, options [nop,nop,sack 2 {18426:19782}{6870:8262}], length 0 14:23:06.725437 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], ack 79, win 776, length 0 14:23:07.019342 IP 192.168.80.101.40744 > 103.235.46.39.http: Flags [F.], seq 78, ack 1062, win 290, options [nop,nop,sack 2 {18426:19782}{6870:8262}], length 0 14:23:07.049633 IP 103.235.46.39.http > 192.168.80.101.40744: Flags [.], ack 79, win 776, options [nop,nop,sack 1 {78:79}], length 0 |
服务器完完整整地收到了百度发送的数据包,我们本地收到了seq 1:1062,但下一个seq是6870:8262,再下一个居然是18426:19782!这丢包率真把我给吓坏了……
赶紧从本地MTR到服务器看看:
然后走VPN MTR到百度看看:
虽然MTR可以看到有少量丢包,不过这丢包率完全不符合我TCPDUMP的结果,这说明这丢包根本不是线路不佳造成的。
完全不服……
想起来之前在家研究广电机顶盒时,遇到过桥架网卡后因为其中一个网卡MTU非1500(桥接后,桥接网卡的MTU不能大于MTU最小的网卡的MTU)而导致PPPOE连接成功却打不开网页的情况。
MTU是什么不在此长篇大论,至于为什么要减掉一部分数值,是构建了虚拟隧道的问题,数据包进入虚拟隧道前,还是要封装成适合进入这个虚拟隧道的类型的,因此数据包体积会比原来大,具体细节,不在此长篇大论,各位有兴趣可以通过搜索引擎了解。
更好服务器上也用桥接网卡,于是看了下br0(桥接网卡), eth0(物理网卡)和服务器与我本地通讯的ppp1的MTU:
br0和eth0都是1500, ppp1是1496,我本地的ppp也是1496。
看起来是非常正常且毫无问题的,因为默认情况下VPN构建的ppp的MTU都是1496这个值……
不过既然是非线路问题导致的丢包,改改MTU也无妨,于是分别在服务器和本地都把VPN构建的ppp的MTU改成1400:
1 |
ifconfig 网卡设备名称 mtu 1400 |
再次curl百度:
是成功地连上了……
不过我之前都在未桥架的服务器上搭建VPN,为了探究清楚是桥架还是该服务器的网络环境的问题,我又在另一台使用了桥架的服务器搭建了VPN并访问百度:
还好是成功的,ppp的mtu同样是1496,这就说明了这个问题在这次与桥接不沾边,是这台Sunny Vision服务器的问题。
虽然最后成功解决了无法打开百度(现在应该是说比较大的网页)的问题,但最大的问题是,目前为止,对于是何导致了此服务器无法使用1496,我还没有任何思绪……
才疏学浅啊,若各位有见解,请告知一下……