使用 workerman 做TCP client 做服务器的压力测试

0

物联网业务单纯测试 TCP 连接数量已经没用意义,所以需要做压力测试的时候模拟真实业务,比如有5万台设备,每分钟内都上传一次 GPS 数据等。这时候测试的不再是 workerman 框架自身的性能,而是 Server 端整体的性能。


昨天开始尝试用 workerman 作为 TCP Client 与 server 对接(server 端也是 wokerman-gatewway),通信协议是基于 TCP 的自定义协议,下面是示例代码:


<?php

use Protocols\LindeProtocol;
use Workerman\Lib\Timer;
use \Workerman\Worker;
use \Workerman\Connection\AsyncTcpConnection;

require_once __DIR__ . '/Autoloader.php';

$deviceMaxNumber = 10;

for ($i = 0; $i < $deviceMaxNumber; $i++) {

$task = new Worker();

$deviceId = "99999888887777$i";

$task->onWorkerStart = function ($task) use ($deviceId) {

require_once __DIR__ . '/SmartLinkBusinessWorker.php';
$busWorker = new \Workerman\SmartLinkBusinessWorker($deviceId);
$time_interval = 1;
Timer::add($time_interval,
function () use ($busWorker) {
$busWorker->timerTick();
}
);

$connection = new AsyncTcpConnection('lindeProtocol://_._._._:*);
// 当连接建立成功时
$connection->onConnect = ;
$connection->onMessage = ;
$connection->onClose = ;
$connection->onError = ;
$connection->onWorkerStop = ;
$connection->connect();
};
}

// 运行worker
Worker::runAll();

目前是模拟10个客户端没有问题,如果想模拟一万个客户端,能否继续使用上面的方法(只把 deviceMaxNumber 从10改成10000)?


或者能否指点一下更优的方案?

已邀请:

keytehu

赞同来自: aidong_wang

http://wenda.workerman.net/?/question/1453
直接new 多个 AasyncTcpConnection,类似这样估计可行。

blogdaren - 乐于分享的PHP码农【http://www.blogdaren.com】

赞同来自: aidong_wang

个人认为作者这个方案不是很好,因为 deviceMaxNumber 的数量直接决定了启动的子进程数量,而且每个子进程只维持了一个AsyncTcpConnection连接,随着deviceMaxNumber 数量的增加,进程启动开销随之增加,其实可以将进程分组,每组进程内维持N个链接,类似负载均衡效果。

aidong_wang

赞同来自:

大家试过workerman/gateway-worker 做为 server 端,能保持多少 TCP 链接吗?
我的情况:


服务器:


Centos 6.8
4核心
16G 内存
已经完全按照 Linux 内核调优参数章节进行了调优


但成功建立 TCP 链接始终不超过两千,达到一千多以后,AsyncTcpConnection的 onError 函数频繁被触发:Error code:1 msg:connect 115.159.77.248:20073 fail after 127.281 seconds


    $task = new Worker();

$task->count = 50;

$task->onWorkerStart = function ($task) {

$deviceMaxNumber = 100;
for ($i = 0; $i < $deviceMaxNumber; $i ) {

$taskId = $task->id;

$deviceId = "99999$taskId" . "888887777$i";

require_once __DIR__ . '/SmartLinkBusinessWorker.php';
$busWorker = new \Workerman\SmartLinkBusinessWorker($deviceId);

$connection = new AsyncTcpConnection('lindeProtocol://_._._._:*');
// 当连接建立成功时
$connection->onConnect = ;
$connection->onMessage = ;
$connection->onClose = ;
$connection->onError = ;
$connection->onWorkerStop = ;
$connection->connect();

//sleep(1);

}
};

现在是开启50个进程,每个进程维护100个连接。


这是 “/etc/sysctl.conf” 配置


fs.file-max=6815744
net.ipv4.tcp_max_tw_buckets = 20000
net.ipv4.tcp_sack = 1
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_rmem = 4096 87380 4194304
net.ipv4.tcp_wmem = 4096 16384 4194304
net.ipv4.tcp_max_syn_backlog = 262144
net.core.netdev_max_backlog = 32768
net.core.somaxconn = 65535
net.core.wmem_default = 8388608
net.core.rmem_default = 8388608
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_fin_timeout = 20
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syn_retries = 2
net.ipv4.tcp_syncookies = 1
#net.ipv4.tcp_tw_len = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle=0
net.ipv4.tcp_mem = 94500000 915000000 927000000
net.ipv4.tcp_max_orphans = 3276800
net.ipv4.ip_local_port_range = 1024 65000
net.nf_conntrack_max = 6553500
net.netfilter.nf_conntrack_max = 6553500
net.netfilter.nf_conntrack_tcp_timeout_close_wait = 60
net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_established = 3600
net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.default.disable_ipv6 = 0
net.ipv6.conf.lo.disable_ipv6 = 0

客户端连接失败时,workerman/gateway-worker 端没有触发任何错误信息,看起来状态良好


----------------------------------------------GLOBAL STATUS----------------------------------------------------
Workerman version:3.5.11 PHP version:7.1.10
start time:2018-08-06 23:08:06 run 0 days 9 hours
load average: 0, 0, 0 event-loop:\Workerman\Events\Event
4 workers 83 processes
worker_name exit_status exit_count
linde_test_worker 0 0
linde_gateway 0 0
internalGateway 0 0
Register 0 0
----------------------------------------------PROCESS STATUS---------------------------------------------------
pid memory listening worker_name connections send_fail timers total_request qps status
13458 6M none linde_test_worker 53 0 0 5538 0
13459 6M none linde_test_worker 53 0 0 11943 0
13460 6M none linde_test_worker 53 0 0 10641 0
13461 6M none linde_test_worker 53 0 0 17033 0
13462 6M none linde_test_worker 53 0 0 14599 0
13463 6M none linde_test_worker 53 0 0 6763 0
13464 6M none linde_test_worker 53 0 0 13083 0
13466 6M none linde_test_worker 53 0 0 15920 0
13467 6M none linde_test_worker 53 0 0 15398 0
13468 6M none linde_test_worker 53 0 0 15713 0
13469 6M none linde_test_worker 53 0 0 11995 0
13470 6M none linde_test_worker 53 0 0 14709 0
13471 6M none linde_test_worker 53 0 0 18411 0
13472 6M none linde_test_worker 53 0 0 18084 0
13473 6M none linde_test_worker 53 0 0 13224 0
13474 6M none linde_test_worker 53 0 0 14478 0
13475 6M none linde_test_worker 53 0 0 10551 0
13476 6M none linde_test_worker 53 0 0 16148 0
13479 6M none linde_test_worker 53 0 0 26092 0
13480 6M none linde_test_worker 53 0 0 19470 0
13481 6M none linde_test_worker 53 0 0 26773 0
13482 6M none linde_test_worker 53 0 0 19910 0
13483 6M none linde_test_worker 53 0 0 14391 0
13484 6M none linde_test_worker 53 0 0 13781 0
13486 6M none linde_test_worker 53 0 0 7992 0
13487 6M none linde_test_worker 53 0 0 6793 0
13488 6M none linde_test_worker 53 0 0 11859 0
13489 6M none linde_test_worker 53 0 0 13144 0
13490 6M none linde_test_worker 53 0 0 12420 0
13491 6M none linde_test_worker 53 0 0 16900 0
13492 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1730 0
13493 4M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1740 0
13494 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1760 0
13495 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1740 0
13497 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 4302 0
13498 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 4330 0
13499 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 4318 0
13500 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1740 0
13501 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1745 0
13502 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1735 0
13503 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 6904 0
13505 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 6898 0
13506 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1735 0
13507 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1740 0
13509 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1735 0
13510 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1741 0
13511 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1735 0
13512 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1730 0
13513 4M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1755 0
13514 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1730 0
13515 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1746 0
13516 4M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 4322 0
13517 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 4309 0
13518 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1735 0
13521 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1750 0
13522 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1745 0
13523 4M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 14851 0
13524 4M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 4342 0
13525 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 4322 0
13526 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 4325 0
13527 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 4335 0
13528 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 4305 0
13529 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 1735 0
13530 2M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 4312 0
13531 4M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 12079 0
13532 4M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 6929 0
13533 4M lindeProtocol://0.0.0.0:20073 linde_gateway 33 0 0 43721 0
13534 4M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 9568 0
13535 4M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 19957 0
13536 4M lindeProtocol://0.0.0.0:20073 linde_gateway 33 0 0 38502 0
13537 4M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 17961 0
13538 4M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 66837 0
13539 4M lindeProtocol://0.0.0.0:20073 linde_gateway 32 0 0 33455 0
13540 4M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 85416 0
13541 4M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 32825 0
13542 4M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 12241 0
13543 4M lindeProtocol://0.0.0.0:20073 linde_gateway 32 0 0 151696 0
13544 4M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 95544 0
13545 4M lindeProtocol://0.0.0.0:20073 linde_gateway 32 0 0 193222 0
13546 4M lindeProtocol://0.0.0.0:20073 linde_gateway 31 0 0 17377 0
13547 2M text://127.0.0.1:7274 internalGateway 31 0 0 1730 0
13548 2M text://127.0.0.1:7274 internalGateway 31 0 0 1730 0
13549 4M text://0.0.0.0:1410 Register 82 0 0 82 0
----------------------------------------------PROCESS STATUS---------------------------------------------------
Summary 330M - - 3291 0 0 1385605 0

客户端维持每10秒发送一次心跳,烦请大家帮忙看看

walkor

赞同来自:

客户端所在服务器也要按照workerman手册调优内核,安装event扩展。

要回复问题请先登录注册