Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sharing a modified Shadowsocks as well as our thoughts on the cat-and-mouse game #136

Open
gfw-report opened this issue Oct 15, 2022 · 70 comments

Comments

@gfw-report
Copy link
Contributor

gfw-report commented Oct 15, 2022

Authors: Anonymous, Anonymous
Date: Saturday, October 15, 2022
中文版:分享一个修改版的Shadowsocks,兼谈我们对猫鼠游戏的一点想法


In this post, we release and open source a modified version of Shadowsocks that can bypass the current GFW's detection and blocking. We first introduce the reason why this modified Shadowsocks can bypass the detection and blocking. We then share a simple tutorial on how to setup the client and server. We will also cover two other ways that help Shadowsocks and VMess bypass the current GFW's blocking. At the end of the article, we share some of our thoughts on the cat-and-mouse game of censorship. In particular, we start with answering why censor always starts the massive blocking a few days or weeks before politically sensitive period of time in China; we then argue that, comparing to the anti-censorship community, a fundamental weakness of censor is its inflexibility. We further discuss how to exploit censor's weaknesses to achieve better anti-censorship effect with limited resources.

Motivations

We release this modified Shadowsocks tool today for three reasons:

  1. First, we want to provide Chinese netizens with a (temporarily) viable solution to bypass censorship, mitigating the GFW's massive blocking of multiple censorship circumvention tools since October 3rd, 2022.

  2. Second, we would like to take this opportunity to start a discussion among anti-censorship researchers and developers. Our empirical research shows that the current GFW can already accurately identify Shadowsocks, VMess, and Obfs4 and many other full-encrypted protocols. We estimate that the GFW's current traffic detection algorithm has 0.6% false positives, while the false negatives are almost negligible. This finding urgently requires us to brainstorm and discuss how to improve the current protocols collectively.

  3. Finally, we would like to use this release as an experiment to observe both the censor and the anti-censorship community on how fast each side can react to a new (anti-)censorship event.

Why can this modified Shadowsocks circumvent the GFW's current detection and blocking?

We worked with other researchers to discover that the current GFW utilizes a number of different rules to identify fully encrypted protocols like Shadowsocks, VMesss, and Obfs4. One of these rules takes advantage of the fact that the ratio of 0 bit to 1 bit in these encrypted flows is close to 1:1. Therefore, if we add more 0s or 1s to the encrypted traffic and then rearrange the bit sequence, we can achieve the goal of changing the original ratio feature to bypass detection and blocking.

How do I use this modified Shadowsocks?

This modified version of Shadowsocks is based on Shadowsocks-rust, and we also make use of Shadowsocks-android to compile the apk files for Android users. All client and server side software can be found at this branch and this release.

Installing the server

The installation process is the same as installing any other Shadowsocks-rust server.

  1. First you login to your remote server, and then get the server binary with:
wget https://github.com/gfw-report/shadowsocks-rust/releases/download/v0.0.1-beta/shadowsocks-v1.15.0-alpha.9.x86_64-unknown-linux-gnu.tar.xz
tar xvf shadowsocks-v1.15.0-alpha.9.x86_64-unknown-linux-gnu.tar.xz
  1. Then you create a configuration file:

sudo nano server_config.json

Copy and past the following settings to the file. Note that you need to replace the password ExamplePassword with a much stronger one. A handy way to do this from your terminal is: openssl rand -base64 16. You may also want to change the server_port.

{
  "server": "0.0.0.0",
  "server_port": 8388,
  "password": "ExamplePassword",
  "method": "aes-256-gcm"
}

After finishing editing, you type Ctrl + x to exit. The text editor will ask "Save modified buffer?", and you can type y and then hit Enter.

  1. Now you can start running the binary with the configuration file, but to make it work even after you ended your SSH session, you may want to create a tmux session by:
tmux

You then do:

./ssserver -c ./server_config.json

Finally, type Ctrl + b and then type d to detach from the tmux session.

Firewall configuration

We use ufw to open ports for the Shadowsocks server.

To install ufw on a Debian-based server:

sudo apt update && sudo apt install -y ufw

Then open ports for ssh and Shadowsocks-rust. Note that if you set the server_port to a value different than 8388 in server_config.json, you need to change the value 8388 below accordingly:

sudo ufw allow ssh
sudo ufw allow 8388

Now enable ufw:

sudo ufw enable

If it prompts Command may disrupt existing ssh connections. Proceed with operation (y|n)?, type y and hit Enter.

Finally, run sudo ufw status, and the output should look like this:

Status: active

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       Anywhere
8388                       ALLOW       Anywhere
22/tcp (v6)                ALLOW       Anywhere (v6)
8388 (v6)                  ALLOW       Anywhere (v6)

Client side configuration

Below is a configuration file for Client. Remember to change the server value from ExampleServerIP to your remote server's public IP address. If you used andorid and installed from the apk file we provided, the usage should be the same as usual.

{
    "server": "ExampleServerIP",
    "server_port": 8388,
    "password": "ExamplePassword",
    "method": "aes-256-gcm",
    "local_address": "127.0.0.1",
    "local_port": 1080
}

Limitation of the current implementation

  • Since we made changes to the Shadowsocks protocol, it is not compatible with other Shadowsocks clients and servers for now. Users need to download the client and server we prepared.
  • Currently, the clients only support Windows, Linux, macOS, Android phone, and Android TV. It doesn't not support iOS. We welcome iOS developers who have the ability to implement this modified protocol. We also encourage iOS developers to contact us and we will share other ways to bypass the censorship with you.
  • In terms of encryption method, the modified version does not yet support Shadowsocks-2022. This is not because we intend not to support it, rather, it's just that we do not have enough time to do it yet. Our current recommended encryption method choices are: chacha20-ietf-poly1305 or aes-256-gcm.
  • In terms of implementation, we have made changes directly on the core code. It's not because we want to start maintaining a separate folk of the Shadowsocks protocol, rather, we just have got enough time to make the modified algorithm a user-selectable option yet. We will actively communicate with the Shadowsocks developers and hope to eventually merge the new protocol into Shadowsocks as a user-selectable option.

Are you aware of any other ways to circumvent the current blocking?

We know of two other currently viable options, all of which utilize a different traffic detection rule by the GFW.

If you are a V2Ray user, you can turn on the ExperimentReducedIvHeadEntropy option to avoid the GFW's detection and blocking. The advantage of this option is that you do not need to make any changes to your servers.

@database64128 also implemented another way to bypass the censorship. Because the protocol is changed, it requires updating both client and server to use it.

Our thoughts on this cat-and-mouse game

We'd like to take this opportunity to share some of our thoughts on the cat-and-mouse game of censorship. Our views are based on our observations and reflections on both the censor and the anti-censorship community; we have also been deeply influenced by klzgrad, David Fifield, and many other anti-censorship developers and researchers.

Censor's timing choice of new blocking techniques

As many long-time Internet censorship observers have noticed, Chinese censors always start using their new censorship weapons on a large scale a few days or weeks before a politically sensitive event. In fact, such timing choices are no coincident. So what are the specific reasons? We suspect there are at least three reasons, and we encourage everyone to share their thoughts on them.

  1. First, it is an important political task for censors to ensure that they have sufficient control over public opinion and information flow, at least during politically sensitive time periods. This task is often described in official parlance as "protecting the cybersecurity during such-and-such event".

  2. Second, censors are willing to tolerate more collateral damage caused by false positives in detection during sensitive times. This nature provides new censorship weapons a more permissive trial-and-error environment when they are first put into use. Tschantz et al. analyze and summarize a large number of censorship incidents and find that "real censors tend to use vulnerabilities that produce underblocking but not overblocking" (see recommendation 5). And this tendency shifts slightly during politically sensitive times: censors become more tolerant of the collateral damage caused by false positives in detection in trade of a tighter social control. By deploying a new censorship weapon during such period of time, the censor's mistake will be more tolerated even if the tool caused any overblocking accident due to bugs that were not tested out in the prior phases. klzgrad shares a similar view in this comment.

  3. Finally, and most importantly, a often overlooked reason is the fact that the GFW is actually trying to compensate for and cover up its fundamental weakness of inflexibility. In this cat-and-mouse game, the censors know that they are simply no match for the anti-censorship community in terms of responsiveness and flexibility. If they started deploying their new secret weapon long before the sensitive period begins, the anti-censorship community will have more time to study it and find new ways to bypass censorship. At that point, if they can't be flexible and fast enough to improve their censorship weapons, then their attempts to tighten their controls over the Internet during politically sensitive times will fall flat. David Fifield shared a similar point of view in this comment.

Censors have weaknesses

The censor's lack of flexibility is dictated by the nature of itself and the problems it faces. Specifically, it is itself part of a large bureaucracy, which inevitably leads to inefficient internal operations and rigid behaviors. And yet the problem it faces is as complex as monitoring and censoring network traffic on a national scale. It is not hard to imagine that a new censorship weapon always has to go through the procedures of early-phase research, grants application, more formal scientific research, product development, debugging, surveying on real-world traffic, experimental deployment, and then the final nationwide large-scale deployment and use. The length of the process can take quite a long time.

One may be wondering if it indeed takes such a long time for the censor to deploy a censorship weapon. Let's take this release as another experiment to observe how fast the censor and anti-censorship community can react. In particular, let's see how long it takes the censors to block our released tool that has many weaknesses.

How to exploit censor weaknesses?

Exploiting the censor's weakness, we came up with a few principles in hope they can make the anti-censorship efforts more effective.

  1. Be more tolerant of imperfect circumvention solutions and do not give up on an imperfect circumvention solution too soon. As mentioned in the previous section, the fact that the GFW is less flexible than the anti-censorship community is often overlooked. And because of that, many circumvention solutions have been dismissed and killed prematurely simply because they "have weaknesses". This is often because when anti-censorship developers and researchers envisioning themselves as censors, they tend to focus on the first step for censor -- "pre-research" -- and thinking that a circumvention solution would be easily blocked; however, they actually underestimated the long process of funding applications, formal scientific research, product development, debugging, surveying real-world traffic, experimental deployments, and finally, nationwide deployment, that the real censors have to face. In fact, if an anti-censorship developer spends an afternoon rolling out a new anti-censorship tool, but it takes the censor a large amount of time, energy, human, material and financial resources to block it in six months, we have to say this "imperfect" tool served as a great leverage.

  2. Increase the diversity of censorship circumvention solutions by letting a thousand flowers bloom. In many people's imagination, the GFW is a perfect censor because of its national-level resources; however, in reality, there are limited number of teams with the skills, ability, and resources to walk through all the procedures to make a censorship weapon from an idea to a real weapon deployed national-wide. Therefore, the more anti-censorship solutions the community can create, the less likely these limited resource censor teams will be able to block all tools in one go. And as long as there is one working circumvention solution left in one of these massive blocking event, the information is not disrupted completely.

  3. Actively report new censorship events, promptly measure and understand new censorship techniques, and share the viable circumvention strategies with the community. Achieving this will require communication, effort, and cooperation between Chinese netizens, researchers, and developers. Encouragingly, we are now seeing more and more people joining this collaboration and working together as a collective.

  4. Develop backup circumvention tools in advance. We have seen that the GFW makes up for its lack of flexibility by starting using its new secret weapons in large scale a few days before politically sensitive events. So can we use a similar strategy where we develop more backup plans in normal times, and then send them out just before sensitive times, like this release? This way, even if the technical staffs working for the GFW immediately spot any flaws in the new tool, and knew how to block the tool, it would still take them a long research and development cycle before actually being able to block it in real world.

Acknowledgment

We thank David Fifield for commenting on an earlier draft of this article.

Contact

We encourage you to share your thoughts, comments, user experiences publicly or privately. Our private contact information can be found at the footer of GFW Report.

@gfw-report
Copy link
Contributor Author

gfw-report commented Oct 15, 2022

分享一个修改版的Shadowsocks,兼谈我们对猫鼠游戏的一点想法

作者: Anonymous, Anonymous
日期: 2022年10月15日,星期六
English version: Sharing a modified Shadowsocks as well as our thoughts on the cat-and-mouse game


我们在这篇文章中发布和开源一个修改版的Shadowsocks。这个版本的Shadowsocks可以绕过当前GFW的检测以及封锁。我们首先介绍这个修改后的Shadowsocks的原理,再分享一个如何部署服务器和客户端的简单教程。我们还会介绍其他两种当前能够帮助Shadowsocks和VMess绕过当前GFW封锁的办法。在文章的最后,我们分享我们对审查这个猫鼠游戏的一些看法。我们将从审查者每次开始大规模封锁的时间点选择出发,探讨审查者相比反审查社区缺乏灵活性这一根本性的弱点,并附上我们对如何利用审查者的弱点来达到四两拨千斤的效果的一些建议。

动机

我们在此时发表这个版本有三个目的:

  1. 首先,我们想为中国网民提供一个(暂时)可行的翻墙方案。用增加翻墙协议多样性的方式,缓解10月3号以来的GFW对多种翻墙工具的大规模封锁。

  2. 其次,我们想抛砖引玉地引起研究者和开发者的讨论。我们实证性的研究显示,当前的GFW已经可以精准地识别Shadowsocks,VMess,以及Obfs4这类完全加密协议(full-encrypted protocol)。我们估算GFW当前的流量检测算法会误伤约0.6%的非翻墙链接,而假阴性则低得几乎可以忽略不计。这迫切的需要我们群力群策地改进当前的协议。

  3. 最后,我们想把这次发布当作一场实验,同时观察审查者和反审查社区在面对新的(反)审查事件时的反应速度。

为什么这个修改后的Shadowsocks可以规避GFW当前的检测和封锁?

我们与其他研究人员合作发现,当前的GFW会利用多种不同的规则来识别Shadowsocks,VMesss,以及Obfs4这类完全加密协议。其中一条规则就利用了这些加密流量的0比特与1比特的比例接近1:1的特性。因此,如果我们在加密流量中加入更多的0或1,再对比特序列进行重排,就可以达到改变原有比例特征,绕过检测和封锁的目的。

我怎么用这个修改版的Shadowsocks?

这个修改版的Shadowsocks基于Shadowsocks-rust,我们还利用Shadowsocks-android编译了apk文件供安卓用户使用。所有的客户端和服务端软件都可以在这个branch这个release找到。

安装服务端

安装修改版的服务器的过程与安装任何其他Shadowsocks-rust服务器无异。

  1. 首先你需要登陆你的远程服务器,然后下载编译好的文件
wget https://github.com/gfw-report/shadowsocks-rust/releases/download/v0.0.1-beta/shadowsocks-v1.15.0-alpha.9.x86_64-unknown-linux-gnu.tar.xz
tar xvf shadowsocks-v1.15.0-alpha.9.x86_64-unknown-linux-gnu.tar.xz
  1. 接着你需要创建一个配置文件:

sudo nano server_config.json

将下面的配置文件复制粘贴。注意,你需要把里面的ExamplePassword替换成一个更强的密码。你可以用以下命令在终端生成一个强密码:openssl rand -base64 16。你也应该考虑更改服务器端口8388

{
  "server": "0.0.0.0",
  "server_port": 8388,
  "password": "ExamplePassword",
  "method": "aes-256-gcm"
}

将上方替换过密码的配置信息复制粘贴到配置文件后, 按Ctrl + x退出。 退出时,文本编辑器将问你"Save modified buffer?",请输入y然后按回车键

  1. 你现在就可以运行二进制的服务器了。但是为了让它在你退出SSH后还能继续运行,你可以考虑建立一个tmux会话:
tmux

然后再运行服务器:

./ssserver -c ./server_config.json

最后,按Ctrl + b再按d就可以脱离tmux会话了。

防火墙配置

我们使用ufw来管理Shadowsocks服务器的防火墙。

在基于Debian的服务器上,可以通过如下命令安装ufw

sudo apt update && sudo apt install -y ufw

然后开放有关sshShadowsocks-rust的端口。 请注意,以下命令假设你在server_config.json中的server_port的值为8388。 如果你的server_port用了其他的值,请对以下命令作相应的修改:

sudo ufw allow ssh
sudo ufw allow 8388

现在我们启动ufw:

sudo ufw enable

启动时如果弹出Command may disrupt existing ssh connections. Proceed with operation (y|n)?,请输入y并按回车键。

最后,请用sudo ufw status检查一下你的配置是否和下面的一样:

Status: active

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       Anywhere
8388                       ALLOW       Anywhere
22/tcp (v6)                ALLOW       Anywhere (v6)
8388 (v6)                  ALLOW       Anywhere (v6)

客户端配置

下面是桌面版客户端的配置文件,记得server的值替换为你远程服务器的IP地址。如果你们是用了我们提供的安卓apk在手机上使用,那么配置就和往常的使用Shadowsocks-android的办法一样。

{
    "server": "ExampleServerIP",
    "server_port": 8388,
    "password": "ExamplePassword",
    "method": "aes-256-gcm",
    "local_address": "127.0.0.1",
    "local_port": 1080
}

不足

  • 因为我们对Shadowsocks协议做了修改,因此暂不兼容其他Shadowsocks客户端和服务端。用户需要下载我们准备的客户端和服务端。
  • 目前客户端只支持Windows,Linux,macOS,安卓手机,和安卓电视版。不支持iOS。我们欢迎有能力的iOS开发者们实现兼容这个修改的协议。也欢迎iOS开发者与我们联系,我们将分享其他绕过审查的办法供你们参考。
  • 在加密方式上,修改版还不支持Shadowsocks-2022。这并非我们刻意不支持Shadowsocks-2022,只是时间和精力有限,还没来得及支持。我们推荐选择:chacha20-ietf-poly1305或者aes-256-gcm
  • 在实现方式上,我们直接在核心代码上进行了修改。这并非我们想要另起炉灶,维护一个Shadowsocks协议的分支版本,只是时间和经历有限,还没有把修改后的算法做成一个可供用户选择的选项。我们将与Shadowsocks开发者积极沟通,希望最终能以提供给用户选项的方式,将新协议合并到Shadowsocks中。

你们还知道什么其他的可以规避当前封锁的方法吗?

我们还知道两种目前可行的方案,他们都利用了另一种不同的GFW流量检测规则:

如果你是V2Ray用户,你可以开启ExperimentReducedIvHeadEntropy选项来避免GFW的检测和封锁。这个方案的好处是你无须在服务器进行任何修改。

如果你是Shadowsocks用户,@database64128实现了另外一种绕过审查的办法。因为对协议做了修改,所以需要同时更新客户端和服务端。

分享我们对猫鼠游戏的一点看法

我们想借此机会分享一些我们对于审查与反审查这个猫鼠游戏的看法。这些看法既来自于我们对审查者和反审查社区两方的观察与思考,又同时受到klzgrad,David Fifield以及其他许多许多反审查者的深深影响。

审查时间点的选择

长期关注互联网审查的朋友们一定已经发现,中国的审查者总是在政治敏感事件来临前的数天或数周开始大规模地使用他们的新审查武器。实际上,这样的时间点选择绝非偶然,那具体的原因是什么呢?我们猜想至少有三点,也欢迎大家继续讨论补充:

  1. 首先,对审查者而言,保证至少在政治敏感的时间段内,对社会舆论和信息流通有着足够的控制,是一项重要的政治任务。这项任务通常会被官方话语描述为"为某某期间的网络安全护航"。

  2. 其次,审查者在政治敏感时期更能容忍审查造成的误伤,因此给新的审查武器投入使用提供了更宽松的试错环境。Tschantz等人对大量审查事件进行分析总结后发现,真实世界的审查者通常宁可少封,也要尽量避免错封带来的影响(见recommendation 5)。而这种倾向仅在政治敏感时期略有变化:审查者会变得更能容忍误伤的代价,以达到其所追求的社会管控。而在这时上线新的武器,即使有前期没有测试出来的错误造成误伤或事故,也更能被容忍。klzgrad这篇评论中也分享了类似的观点。

  3. 最后,也是最为重要,却容易被忽视的一点是,GFW其实是在极力地弥补和掩盖其缺乏灵活性的根本性弱点。在这场猫与鼠的游戏中,审查者自知其在反应速度、灵活性上根本不是反审查社区的对手。 如果他们在敏感时期开始前的很长一段时间就开始部署他们新的秘密武器,那么反审查社区就有更多的时间去研究它,并发现绕过审查的新方法。到那时,如果他们不能灵活快速地改进自己的审查武器,那他们在政治敏感时期收紧网络管控的企图就会落空。David Fifield在这篇评论中也分享了类似的观点。

审查者有弱点

审查者缺乏灵活性是由其自身和其所面对的问题的本质所决定的。具体而言,其自身是庞大官僚机构的一部分,这不可避免的导致其内部运行效率低下,行为僵化。而其所面临的问题却又是在全国范围内大规模地监控和审查复杂多变的网络流量。可以想象,一款新型审查武器总是要经历预先研究,经费申请,立项,科研,产品开发,调试,真实流量调研,实验性部署,才能到最后的全国范围内的大规模部署和使用。其过程的漫长程度可想而知

空口无凭。我们不妨把这次发布当作又一场观察(反)审查者反应速度的实验:看一看审查者要花多久才能封锁我们这次发布的这个有着诸多弱点的版本。

如何利用审查者的弱点?

针对审查者的弱点,我们想到了一些也许能让反审查工作在有限的资源下,事半功倍的原则:

  1. 要对不完美的翻墙方案更宽容,不要过早放弃一个不完美的翻墙方案。我们前面说GFW灵活性不如反审查社区这个弱点通常被人们忽视,是因为看到许多的翻墙方案仅仅因为“存在弱点”就被过早的否定和扼杀了。这通常是因为反审查人员在设想自己是审查者时,更多的只考虑了第一步“预先研究”,觉得某个翻墙方案很容易被封锁,而忽视了真正的审查者所要面对的还有漫长的经费申请,立项,科研,产品开发,调试,真实流量调研,实验性部署,才能到最后的全国范围内的大规模部署和使用。事实上,如果我们的反审查人员花一个下午做好的反审查方案,在消耗了审查者半年的时间、精力、人力、物力、财力后被封锁了,其四两拨千斤的作用也早已经达到了。

  2. 要增加翻墙方案的多样性,鼓励不同翻墙方案的百花齐放。在很多人的想象中,GFW因为拥有国家级的资源而成为一个完美的审查者,但实际上,有技术、能力、资源将上述审查武器研发流程一一走完的团队是有限的。我们反审查社区开发的翻墙方案越多种多样,这些有限的资源团队就越无法一网打尽所有的翻墙软件。而只要在某次大封锁中还剩下一种能用翻墙方案,信息就不至于完全阻塞。

  3. 积极的报告新的审查事件,当新的审查事件发生时,迅速地测量新的审查机器,并将测量结果和绕过审查的方案分享给社区。达成这一项需要网民,研究者以及开发者三方一起沟通、努力、和配合。令人鼓舞的是,我们现在看到越来越多的人加入到这个协作之中,相互的配合也越来越默契顺畅。

  4. 提前开发备用的翻墙方案。我们看到GFW为了弥补自身灵活性的不足而想到在政治敏感时期前几天才使用新的秘密武器,杀网民一个措手不及。那我们是不是也可以用相同的思路,在平时多开发几种备用的翻墙方案,然后像这次的发布一样,到了敏感时期之前才发出?这样,就算是审查者中的技术人员马上发现了新工具中的缺陷,知道了如何反制,也还是需要经过漫长的研发周期。

感谢

我们感谢David Fifield对文章初稿的反馈。

联系我们

正如前文所说的,我们发帖的目的就是想引起用户,研究人员和开发者们的讨论。因此我们欢迎您或公开地或私下地与我们分享您的使用体验或想法。我们私下的联系方式可见GFW Report的页脚。

@SekiBetu
Copy link

SekiBetu commented Oct 15, 2022

概括:以量取胜,使用大量不同的协议来遏制gfw的服务器资源

Overview: Quantity wins, using a large number of different protocols to contain gfw's server resources

@ghost
Copy link

ghost commented Oct 15, 2022

感谢!!不知对 TLS in TLS 的研究有任何结论或新的编译吗?

Thanks!!! I wonder if there are any conclusions or new compilations of the TLS in TLS research?

@gfw-report
Copy link
Contributor Author

感谢!!不知对 TLS in TLS 的研究有任何结论或新的编译吗?
Thanks!!! I wonder if there are any conclusions or new compilations of the TLS in TLS research?

You may want to take a look at this post and this thread.

@ghost
Copy link

ghost commented Oct 15, 2022

概括:以量取胜,使用大量不同的协议来遏制gfw的服务器资源

Overview: Quantity wins, using a large number of different protocols to contain gfw's server resources

是的,这样做本质还是捉迷藏,并不能长期逃避审查。毕竟 Shadowsocks 一直有 数据熵较高 的特征。

Yes, intrinsically this modification is a kind of hide-and-seek and it can't be used to gight against the censorship in long term. The Shadowsocks protocol has a feature known as high data entropy after all.

@shawokou123
Copy link

非常感谢,我使用这个方式在ipv6服务器上搭建完成,并可以使用。
只是在服务器端设置上,如果是纯ipv6服务器,需要将server设置改为:
"server": "::",

Thank you very much, I used this to build it on an ipv6 server and it works.
Just on the server side settings, if it is a pure ipv6 server, you need to change the server settings to
"server": "::",

@mckuhei
Copy link

mckuhei commented Oct 15, 2022

不懂就问,基于游戏流量(https://github.com/MCModuleStudio/Minecraft-tunnel-server )再加上加速器能否规避封杀

Ask if you don't understand, based on the game traffic (https://github.com/MCModuleStudio/Minecraft-tunnel-server ) plus accelerator can circumvent the blocking

@klzgrad
Copy link

klzgrad commented Oct 15, 2022

Specifically, it is itself part of a large bureaucracy, which inevitably leads to inefficient internal operations and rigid behaviors.

I'd say it has been more efficient than before. It was a bureaucracy, but market forces have been introduced into the problem domain. Independent vendors compete to solve engineering challenges and optimize performance.

It is not hard to imagine that a new censorship weapon always has to go through the procedures of early-phase research, grants application, more formal scientific research, product development, debugging, surveying on real-world traffic, experimental deployment, and then the final nationwide large-scale deployment and use. The length of the process can take quite a long time.

Not always, probably not even close. The issue with the scientific workflow of censorship research was that the academic project lifecycle is very long and the deliverables are papers, not actual censorship systems and large scale results, and if there are any practical results, their ROIs are very unpredictable in both expenses and project schedule. That paradigm of censorship R&D being unsustainable doesn't mean there isn't another business mode that can make this work.

This is where I argue this area is now much more streamlined and efficient, because much of the censorship problems are no longer academic problems but industry engineering problems. The new variants of v2ray are not an interesting and novel academic research topic (blockchains, other new types of anonymous distributed networks may qualify for academic novelty). The variants are by nature not suited for the academic research workflow. It's entirely logical for these to be solved at the engineering level where a plethora of companies can compete to deal with the large diversity.

To compare the characteristics of the academic and industrial censorship R&D workflows:

  • Academia: deals with a few very deep, very large, very complex systems, values novelty, cannot afford to handle large diversity which requires manpower (number of grad students is limited), delivers academic papers without guarantees for practical deployment or real world performance, project schedule is unpredictable, may fail and produce nothing.
  • Industry: deals with a large quantity of simple, improvised, easy systems (easy as in usability), values real world performance, does not value theoretical novelty, can afford to handle diversity with manpower, delivers working systems with measurable real world performance, agile project schedules with possible expedited delivery, failure has severe consequences (bankruptcy or market irrelevance).

Why is machine learning very popular in this area of problem. It is because machine learning is actually more efficient than human learning at dealing with diverse protocols. With human learning it requires a large number of experts with deep understanding in networks and protocols but the time and budget to deal with a particular circumvention tool is unpredictable. With machine learning there is no requirement for deep domain expertise, and the cost for good performance is linearly or sublinearly correlated to the amount of raw compute power and ML system tuning manpower invested. This makes a very good deal for decisionmakers because ROIs are quite measurable and predictable in this case compared to the academic workflow.

@klzgrad
Copy link

klzgrad commented Oct 15, 2022

Some relatively easy ideas I gathered to improve TLS in TLS tunnel survivability:

  • Modify payload ClientHello padding to include more random sized bytes to confuse traffic analysis. This requires no change to the tunnel protocol. (via Large scale blocking of TLS-based censorship circumvention tools in China #129 (comment))
  • Perform a real or realistic http request as the tunnel preamble to confuse traffic analysis systems that handles the starting part of a traffic flow (i.e. most of them). Issues: this adds connection latency, but can be amortized with multiplexing.
  • Use multiplexing.
  • Use simple random padding for the initial few round trips.
  • Use a full-blown model-parroting traffic shaper to confuscate packet size and timing, but only for the initial few round trips (which can make implementation easier due to opportunities for hard coding). To make this happen, also:
    • A traffic model collection tool (candidate: chrome://net-export with a parsing script)
    • Traffic fingerprint self audit tools (proxy tools provide tooling to record traffic flow metadata and visualize it so users can share and compare the results with known traffic profiles, or centrally compared at some equivalents of tlsfingerprint.io).

@lolyinseo
Copy link

概括:以量取胜,使用大量不同的协议来遏制gfw的服务器资源

Overview: Quantity wins, using a large number of different protocols to contain gfw's server resources

Wrong strategy. Then GFW enough to detect "valid" protocols and block all the others.

@uruwasyo
Copy link

After 3 hours of re-completing the manual,it still does not work,which is quite identical as before.

@database64128
Copy link

@database64128 also implemented another way to bypass the censorship. Because the protocol is changed, it requires updating both client and server to use it.

I wouldn't call it a protocol change, since anyone can easily implement the idea as an SIP003 plugin. This feature was added for convenience. It is not and probably will never be a part of the protocol.

Modify payload ClientHello padding to include more random sized bytes to confuse traffic analysis. This requires no change to the tunnel protocol.

I have recently made the change in database64128/shadowsocks-go@d98e94d to pad initial payload by default.

Use simple random padding for the initial few round trips.

Use a full-blown model-parroting traffic shaper to confuscate packet size and timing

To make these possible, payload chunks in Shadowsocks TCP need to support padding. I've made plans to add it for Shadowsocks 2023:

Shadowsocks 2022 and before:

+----------------+
|  length chunk  |
+----------------+
| u16 big-endian |
+----------------+

+---------------+
| payload chunk |
+---------------+
|   variable    |
+---------------+

Shadowsocks 2023 (draft):

Length chunk:
+----------------------+----------------+
| payload chunk length | payload length |
+----------------------+----------------+
|    u16 big-endian    | u16 big-endian |
+----------------------+----------------+

Payload chunk:
+----------+----------+
| payload  | padding  |
+----------+----------+
| variable | variable |
+----------+----------+

@diwenx
Copy link

diwenx commented Oct 16, 2022

My apologies as I may have missed some discussion about this earlier, but I was under the impression that the recent blocking targets only TLS-based circumvention tools. Is shadowsocks also affected?
I've two Outline servers that my friends in mainland have been using over 6 months and both of them survived the recent blocking. Is this patch preemptive?

@cross-hello
Copy link

cross-hello commented Oct 16, 2022 via email

@HezeCaoxian
Copy link

HezeCaoxian commented Oct 16, 2022

昨天晚上11:20按群友建议购买了2台lightsail,
在配置翻墙前测试了IP都是干净的,都正常,没有被墙。
第一台配置了上面的shadowsocks,端口20060,加密方式aes-256-gcm
第二台配置了上面的shadowsocks,端口8878,加密方式chacha20-ietf-poly1305

客户端是Windows虚拟机新安装的系统,
访问内容是telegram客户端和youtube,
第一台在10.16早上9:40被墙端口,使用流量4.61GB,换端口到50505后,10:14被墙IP,使用流量341MB
换到第二台使用,11:09被墙IP,使用流量455MB


Last night at 11:20 as suggested by the group purchased 2 lightsail.
In the configuration of the wall before the test IP are clean, are normal, not walled.
The first configured the above shadowsocks, port 20060, encryption method aes-256-gcm
The second configured the above shadowsocks, port 8878, encryption method chacha20-ietf-poly1305

The client is a new installation of the Windows virtual machine.
The access to the content is telegram client and youtube.
The first one was walled at 9:40am on 10.16, using 4.61GB of traffic, after changing the port to 50505, the IP was walled at 10:14, using 341MB of traffic.
After switching to the second one, at 11:09, the IP was walled, using 455MB of traffic.

@liujunjia1996
Copy link

liujunjia1996 commented Oct 16, 2022

我昨天中午部署的,使用流量1g左右,暂时没有被墙,但是感觉不怎么好用啊

I deployed it yesterday at noon, using about 1g of traffic, and it's not walled for now, but it doesn't feel like it's working very well

@SekiBetu
Copy link

概括:以量取胜,使用大量不同的协议来遏制gfw的服务器资源
Overview: Quantity wins, using a large number of different protocols to contain gfw's server resources

Wrong strategy. Then GFW enough to detect "valid" protocols and block all the others.

That's whitelisting, not in the scope of discussion

@klzgrad
Copy link

klzgrad commented Oct 16, 2022

So I will mark these screenshots as suspicious.

I'll take these as bug reports.

@gfw-report
Copy link
Contributor Author

gfw-report commented Oct 16, 2022

HezeCaoxian:

昨天晚上11:20按群友建议购买了2台lightsail,
在配置翻墙前测试了IP都是干净的,都正常,没有被墙。
第一台配置了上面的shadowsocks,端口20060,加密方式aes-256-gcm
第二台配置了上面的shadowsocks,端口8878,加密方式chacha20-ietf-poly1305

客户端是Windows虚拟机新安装的系统,
访问内容是telegram客户端和youtube,
第一台在10.16早上9:40被墙端口,使用流量4.61GB,换端口到50505后,10:14被墙IP,使用流量341MB
换到第二台使用,11:09被墙IP,使用流量455MB

非常感谢您的报告。这样的用户报告及时、准确、详实,帮我们排除了很多其他可能性。我们现在立刻开始调查。

Thank you very much for your report. User reports like this are timely, accurate, and detailed, and help us rule out many other possibilities. We are now starting our investigation immediately.

@gfw-report
Copy link
Contributor Author

My apologies as I may have missed some discussion about this earlier, but I was under the impression that the recent blocking targets only TLS-based circumvention tools. Is shadowsocks also affected?

Good question! The GFW has been able to accurately detect and block Shadowsocks, VMess, and Obfs4, and many other fully-encrypted protocols since November, 2022.

There are at least two ways the GFW blocks these protocols: immediate dynamic blocking of ports and relatively long term static IP or port blocking.

We know dynamic blocking has only been applied to connections from China to the IP subsets of many popular data centers like Amazon Lightsail, Digital Ocean and Alibaba cloud. They selectively deployed this technique for probably two reasons:

  1. The censor knows their detection algorithm has a relatively high false positive and is not willing to harm 0.6% non-circumvention traffic;
  2. Having a tight control over these VPS providers' IP subnet is sufficient to disrupt most users' connections. This is due to the nature that most of the Chinese users can only rent from a few VPS providers that accept payment methods they have.

How do you know if a certain IP address is under dynamic blocking? Try sending some random payload from China to an open port of that IP address:

head -c 100 /dev/urandom | nc -vn $ip $port

If after executing the command several times, your connections got timeout for 120 or 180 seconds, then you can confirm that the IP you tested is within the GFW's dynamic blocking IP subnets.

I've two Outline servers that my friends in mainland have been using over 6 months and both of them survived the recent blocking.

Thanks for your report. Are your servers located within one of those big name VPS providers? Or are you hosting them at some residential or institutional IP addresses that often do not have circumvention servers hosted?

@wkrp
Copy link
Member

wkrp commented Oct 16, 2022

To make these possible, payload chunks in Shadowsocks TCP need to support padding.

@database64128 maybe you know already, but it is possible to implement padding in Shadowsocks AEAD with no protocol changes.

#101 (comment)

Something similar is possible with Shadowsocks AEAD ciphers using encryptions of empty plaintexts, even though Shadowsocks doesn't have explicit support for padding. I heard this idea from @fortuna. When you need a source of bytes for padding, you can encrypt a zero-length plaintext, which gives you 34 bytes that decrypt to nothing: 2 bytes encrypted length field, 16 bytes length field authentication tag, 0 bytes encrypted payload, 16 bytes payload authentication tag. You can concatenate as many of these empty ciphertexts as you need to satisfy the traffic schedule.

Let's say you want to add npadding bytes of padding to an existing current_packet before sending. You encrypt enough empty plaintexts to make at least npadding bytes and append the first npadding bytes to the packet. The excess bytes become the beginning of the next packet to send.

padding = ""
while len(padding) < npadding:
    padding += shadowsocks_aead_encrypt("")
current_packet += padding[:npadding]
next_packet = padding[npadding:]
send(current_packet)

@diwenx
Copy link

diwenx commented Oct 16, 2022

Thanks for your report. Are your servers located within one of those big name VPS providers? Or are you hosting them at some residential or institutional IP addresses that often do not have circumvention servers hosted?

Thank you for the information. In fact, right after I posted yesterday they got IP blocked. I guess I run out of luck eventually. They were rented from profitserver.net and located in Netherlands and Russia, respectively.

I'll set up new servers following the instructions provided here. Is there any additional steps I should consider (e.g., collecting pcaps for suspicious probes) to help investigate how they do blocking?

@fortuna
Copy link

fortuna commented Oct 17, 2022

@gfw-report Could you clarify what the change you made to shadowsocks is?

@database64128
Copy link

@wkrp Thank you for letting me know. This workaround doesn't sound very flexible. The padding length is limited to multiples of 34. And it takes 2 read() calls for the server to process each 34-byte padding. My implementation also explicitly rejects empty payload chunks, which I do not plan to change.

I've seen more than a few developers in the Shadowsocks community with the sentiment that we must avoid protocol changes at all costs, that protocol changes won't be accepted by downstream developers or users. If anything, my experience with Shadowsocks 2022 has taught me otherwise. People want new protocols to experiment with. People are concerned about the fact that we keep introducing hacks and workarounds to "improve" Shadowsocks AEAD. The end result is a fragmented, divided community, where new developers start their own small circles, instead of joining existing ones, even though we all share the same goal of circumventing censorship.

Sometimes protocol changes are necessary. At this point, I consider recommending Shadowsocks AEAD to users to be irresponsible, because of the known vulnerabilities. Like @gfw-report said, it's a cat-and-mouse game. The protocol needs to rapidly iterate when faced with new challenges.

@database64128
Copy link

We are now starting our investigation immediately.

@gfw-report In case you didn't already know and it might be helpful to your investigation, shadowsocks-rust by default only enables Shadowsocks 2022's replay protection. The bloom filter for older AEAD and stream ciphers is disabled by default.

@gfw-report
Copy link
Contributor Author

diwenx:

I'll set up new servers following the instructions provided here. Is there any additional steps I should consider (e.g., collecting pcaps for suspicious probes) to help investigate how they do blocking?

Yes, that would be very helpful because the more eyes we have, the more likely and quickly we spot any new censorship actions.

@gfw-report
Copy link
Contributor Author

database64128:

shadowsocks-rust by default only enables Shadowsocks 2022's replay protection. The bloom filter for older AEAD and stream ciphers is disabled by default.

Thanks for the heads up. As soon as we support this encoding scheme on Shadowsocks 2022, we can start recommending users to use Shadowsocks-2022. Any help to implement it is more than welcome!

BTW, in case you are concerned about the active probing vulnerability, this modified Shadowsocks has been proved to be able to evade the current active probings because the GFW doesn't suspect its traffic in the first place.

@wkrp
Copy link
Member

wkrp commented Oct 17, 2022

This workaround doesn't sound very flexible. The padding length is limited to multiples of 34.

I can respect your development decisions, but I believe you are mistaken on this point. There is no requirement that TCP segment boundaries be aligned with AEAD ciphertext boundaries. For example, if you need 75 bytes of padding for the current segment, you can generate 102 bytes and only use the first 75 bytes of it for the current segment. The remaining 27 bytes become the beginning of the next segment. (In general, you keep a buffer of available data. If you want to send a segment that's smaller than the amount of data in the buffer, you break off the head of the buffer and only send that. If you want to send a segment that's larger than the buffer, you increase the size of the buffer with padding ciphertexts until it is sufficiently large.)

You are correct, however, that the receiver can only consume the padding 34 bytes at a time, which for large amounts of padding means many recv system calls and verify+decrypt operations.

@database64128
Copy link

For example, if you need 75 bytes of padding for the current segment, you can generate 102 bytes and only use the first 75 bytes of it for the current segment. The remaining 27 bytes become the beginning of the next segment.

Unless the proxy program runs its own TCP stack (which could create its own problems (e.g. fingerprinting of the TCP stack)), I don't think it's realistic to control TCP segment boundaries during mass payload transfers.

I can respect your development decisions, but I believe you are mistaken on this point.

I think you misunderstood what I meant. My understanding is that the proxy program might want to add padding between individual payload chunks to change the characteristics of outgoing TCP segment sizes. And that works best when you can add as many bytes of padding as you want. Your proposed method of doing smaller writes cannot guarantee smaller TCP segments, even if you enable TCP_NODELAY.

@Kimberlysz
Copy link

Kimberlysz commented Oct 17, 2022

I started using these tutorials to build up my VPS, then some my servers add v2ray applications (wss: websocket+tls+vmess)

tutorials in sequence

recently I found out using JARM

And this is the JARM result (server 1)
JARM: 15d3fd16d29d29d00042d43d000000fe02290512647416dcf0a400ccbc0b6b
After check at the https://fofa.info/, I found out a result of such.

JARM1

I had 2 servers using this method (wss: websocket+tls+vmess),
server 1:
blocked at 2022-10-03 3-4pm;
blocked at 2022-10-06 3-4pm;
blocked one more time between 2022-10-07 to 2022-10-10, and since then no block, same method, v2ray version ver5.10.0 (maybe, not sure)

server 2:
blocked at 2022-10-03 3-4pm;
blocked at 2022-10-06 3-4pm;
blocked several times, at least 2022-10-09, 2022-10-10, 2022-10-11, 2022-10-12, 2022-10-13,

Then at 2022-10-14 I changed to v2fly.org ws+tls+vmess (url: https://guide.v2fly.org/advanced/wss_and_web.html)
, and this is the JARM result:
JARM: 2ad2ad0002ad2ad22c42d42d000000baee6456a90c4706c37778b7ac8d25d4

JARM2

it has 9504 fit results, but since then this server never gets blocked,

I just wanna share my status now.

I have a question, according to some of the resources (https://youtu.be/F2VsjS6fybs, time at 14m10s), so with server 1 JARM results - a good JARM blend-in, server 1 still got blocked several times from 2022-10-03 to 2022-10-10.

So, what's the relationship between this JARM and tls in tls detection? Kindly if you could let me know, that would be great.

@patchescamerababy
Copy link

Can you add support for udp, including uot
能否增加对udp的支持,包括uot

@Hadi-1624
Copy link

Thank you guys for your help

When using proxies such as shadowsocks or v2ray, my upload speed is very very bad. example my download speed is at 1 mb/s, my upload speed goes down to 30 kb/s
Are there ways to solve this issue?

@ghost
Copy link

ghost commented Oct 23, 2022

When using proxies such as shadowsocks or v2ray, my upload speed is very very bad. example my download speed is at 1 mb/s, my upload speed goes down to 30 kb/s Are there ways to solve this issue?

There's nothing in the protocol itself that would make it that slow. You would have to do A/B testing to narrow down the cause of the problem, e.g. overloaded server, congested route, deliberate throttling by ISP, or whatever.

@opmaaadi
Copy link

opmaaadi commented Oct 27, 2022

Is there a specific reason that you use "pure" ShadowSocks (without plugins such as V2Ray, Xray, etc)? @gfw-report

@Kimberlysz
Copy link

Kimberlysz commented Oct 31, 2022

If you are a V2Ray user, you can turn on the ExperimentReducedIvHeadEntropy option to avoid the GFW's detection and blocking. The advantage of this option is that you do not need to make any changes to your servers.

Hello there, currently I am using wss (vmess + tls + nginx), I use LEMP, then I use certbot to issue ssl, finally I add wordpress.
I am wondering how to add this ExperimentReducedIvHeadEntropy option?

for clients I use,
windows: Clash for Windows;
android: v2rayNG, Clash for Android;
macOS: clashX;
iOS: shadowrocket.

what I learnt and found out mostly close tothe description are located here in v2fly.org (https://www.v2fly.org/config/protocols/vmess.html#userobject), then search "experiments: string".

So I took clash for windows as an example, and I add this in clash for windows yaml file:

  # 仅适用于设置 allow-lan 为 true 时
  # 192.168.122.11: 绑定单个 IPv4 地址
  # "[aaaa::a8aa:ff:fe09:57d8]": 绑定单个 IPv6 地址
  # bind-address: "*", "*": 绑定所有 IP 地址
  experimental:
    ignore-resolve-fail: true # 忽略 DNS 解析失败,默认值为true
    Experiment-Reduced-Iv-Head-Entropy: true # reduce entropy
    experiment-reduced-iv-head-entropy: true # reduce entropy
    ExperimentReducedIvHeadEntropy: true # reduce entropy
    experimentreducedivheadentropy: true # reduce entropy

and then at the server side I add this in ClientObject (https://www.v2fly.org/config/protocols/vmess.html#clientobject) under InboundConfigurationObject:

  {
    "email": "[email protected]",
    "id": "random uuid",
    "level": 0,
    "alterId": 0,
    "experiments": "ExperimentReducedIvHeadEntropy"
  },

Finally I also add maxEarlyData (https://www.v2fly.org/config/transport/websocket.html#websocketobject) in WebSocketObject.

  "wsSettings": {
    "path": "/x",
    "headers": {
      "Host": "example.com"
    },
    "maxEarlyData": 1024,
    "earlyDataHeaderName": "Sec-WebSocket-Protocol"
  },

I know these are probably wrong but all these echos back to the question, how to add 'ExperimentReducedIvHeadEntropy'. Although my servers get blocked daily but I think wss is still a very good method. Thank you in advance.

@fortuna
Copy link

fortuna commented Jan 12, 2023

The Outline Client now supports a prefix parameter that gets used as the first bytes of the connection salt to disguise it as an allowed protocol. The prefix is specified on the Access Key. Usage details and examples can be found at https://www.reddit.com/r/outlinevpn/wiki/index/prefixing/

Combined with Dynamic Access Keys, you can use different prefixes for different sessions.

One advantage of our approach is that downloading the client or inspecting its code doesn't give the censor knowledge to know what to block. Furthermore, it enables service providers to change prefixes easily and move faster than the censor.

@free-the-internet
Copy link

The Outline Client now supports a prefix parameter that gets used as the first bytes of the connection salt to disguise it as an allowed protocol. The prefix is specified on the Access Key. Usage details and examples can be found at https://www.reddit.com/r/outlinevpn/wiki/index/prefixing/

Combined with Dynamic Access Keys, you can use different prefixes for different sessions.

One advantage of our approach is that downloading the client or inspecting its code doesn't give the censor knowledge to know what to block. Furthermore, it enables service providers to change prefixes easily and move faster than the censor.

FYI: Tests have been done, users report that it does not working in Iran even with these modifications. However, it might be used in China.

@rezarms
Copy link

rezarms commented Jan 19, 2023

@gfw-report unfortunately doesn't work for Iran.

@KaraRyougi
Copy link

I just read the paper How the Great Firewall of China Detects and Blocks Fully Encrypted Traffic, which seems to be the research related to this modification. However, I also noticed that this research in 2021 has a contradictory conclusion regarding entropy. Did the blocking strategy change during the 1 year period?

@yoshyv
Copy link

yoshyv commented Apr 30, 2023

@gfw-report
How do you know if a certain IP address is under dynamic blocking? Try sending some random payload from China to an open port of that IP address:

head -c 100 /dev/urandom | nc -vn $ip $port

If after executing the command several times, your connections got timeout for 120 or 180 seconds, then you can confirm that the IP you tested is within the GFW's dynamic blocking IP subnets.

Sorry for necro but I did the test yesterday on a proxy server that I set up for my friend (only one user) and there was no timeout after reruning the command 200 time while the proxy server was blocked last year and unblocked few months later. It's an AWS Tokyo EC2 instance so I assume both the IP and ASN are red flags according to the latest research.

On the server end, I ran
nc -l -p 10000 -k -v
and my friend ran your command with for loop for 200 times.

Does it mean the block last year was just a false positive ? or did the GFW upgrade the algorithm?

BTW, do you have any insight of IPv6 traffic? According to some, IPv6 traffic is affected by neither active probing and passive analysis.

@gfw-report
Copy link
Contributor Author

gfw-report commented Apr 30, 2023

Hi @KaraRyougi,

I just read the paper How the Great Firewall of China Detects and Blocks Fully Encrypted Traffic, which seems to be the research related to this modification.

Thank you very much for your interest in the paper. And yes, you are right that this is the "modified Shadowsocks" we mentioned in the paper: https://gfw.report/publications/usenixsecurity23/en/#sec:responsible-disclosure

However, I also noticed that this research in 2021 has a contradictory conclusion regarding entropy. Did the blocking strategy change during the 1 year period?

Great question! Thank you for giving us the opportunity to clarify on that.

We did not observe any changes of the blocking strategy from November 2021 to the ceased blocking in March 2023. The contradicting statement was simply due to our lack of understanding of the system in 2021.

In particular, at that time in 2021, we did not fully understand the GFW's new censorship system and thus made the false/inaccurate statement in the preliminary report that "[b]locking is not based on entropy.... we find that entropy does not tell the whole story. Some payloads with very high entropy never trigger blocking, whereas other payloads with very low entropy can trigger blocking immediately".

We confirm that the GFW was indeed using the monobit testing as a measurement of randomness/entropy.

@gfw-report
Copy link
Contributor Author

gfw-report commented Apr 30, 2023

Hi @yoshyv,

We really appreciate your spirit of measuring the censorship system yourselves.

I did the test yesterday on a proxy server that I set up for my friend (only one user) and there was no timeout after reruning the command 200 time while the proxy server was blocked last year and unblocked few months later. It's an AWS Tokyo EC2 instance so I assume both the IP and ASN are red flags according to the latest research.
...
Does it mean the block last year was just a false positive ? or did the GFW upgrade the algorithm?

The reason why you couldn't observe the blocking anymore is likely because the GFW has stopped dynamically blocking fully encrypted traffic since sometime between Tuesday March 7, 2023 and Wednesday March 15, 2023.

It may be because the politically sensitive time has passed and the censor is thus less willing to tolerate the collateral damage. Specifically, there were two politically sensitive conferences in China ended on Monday March 13, 2023:

We documented this observation in this directory and README in the repo: https://github.com/gfw-report/usenixsecurity23-artifact/tree/master/artifacts/ceased-dynamic-blocking


BTW, do you have any insight of IPv6 traffic? According to some, IPv6 traffic is affected by neither active probing and passive analysis.

Great question. We do not have an answer to this question because we haven't done any testing over IPv6 yet.

@KaraRyougi
Copy link

So from my understanding, the current investigation is only focusing on "real-time TCP block", which seems to be only part of how GFW works now. I suspect there are also offline detections. Evidence includes the reports of blocking in this thread, and other reports of UDP-based protocols such as Hysteria being blocked.

@yoshyv
Copy link

yoshyv commented May 1, 2023

@gfw-report
We documented this observation in this directory and README in the repo: https://github.com/gfw-report/usenixsecurity23-artifact/tree/master/artifacts/ceased-dynamic-blocking

Thanks again for the heads up.

Are the affected AS and IP prefix the same for static blocking and dynamic blocking? Have you collected the data in this regard?
Since there's no way to find out the affected IPs in a short period of time via dynamic blocking test now, is there any other method to find out which IPs are more likely to be blocked?

Is the original shadowsocks (2017) still considered a viable protocol to bypass GFW now? How to take proactive approach to prevent proxy server being IP address blocked? In fear of IP blocking, I haven't used my residential network to set up shadowsocks proxy as my IP address is not dynamic and doesn't change over years.

@robot-dot-win
Copy link

robot-dot-win commented Jul 23, 2024

每年花50-200美元租【亚马逊/微软/Oracle/阿里/百度/腾讯】在【新加坡/北美】的VPS作为代理,随便用个什么加密协议只要数据不被破解,即可堂堂正正访问外网资源,速度极好,还不会被封。实在太担心,再加点洋葱佐料。简单、安全。

Spend $50-200 per year to rent [Amazon / Microsoft / Oracle / Ali / Baidu / Tencent] in [Singapore / North America] VPS as a proxy, casually use what encryption protocols as long as the data is not cracked, you can squarely access to the outside resources, the speed is excellent, but also will not be blocked. If you're too worried, add some onion spices. Simple and safe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests