Swoft source code analysis-connection pool

  php, swoole

Author:bromine
Links:https://www.jianshu.com/p/1a7 …
Source: Simple Book
The copyright belongs to the author. This article has been reprinted with the authorization of the author and the original text has been rearranged.
Swoft Github:https://github.com/swoft-clou …

Why do you need to introduce connection pool?

For traditional php-web applications based on php-fpm, including but not limited to Mysql,Redis,RabbitMq, a unique set of connections must be created for each request, which directly brings some typical problems:

  1. Connection overhead: Connections are newly created with the arrival of http requests and destroyed with the return of requests. A large number of connections are newly created and destroyed, which is a waste of system resources.
  2. Too many connectionsEach request requires its own set of connections, and the number of system connections and concurrency will form a nearly linear relationship. If the system concurrency reaches 1w, 1w corresponding connections need to be established, which is a big load for backend services such as Mysql.
  3. Idle connection: Suppose we have an interface that uses a Mysql connection. After the interface conducts a sql query at the beginning, the subsequent operations are sql independent, so the idle connection occupied by the request is a waste of resources.

For asynchronous systems, this problem becomes more serious. A request processing process needs to perform concurrent operations on the same service, which means that the request needs to hold more than one connection of the same kind. This is undoubtedly an added insult to the system pressure, so connection pool is already a necessary mechanism for Swoole-based Web framework.

The life cycle and process model of Swoft connection pool.

Connection pool as aSCOPEForSINGLETONTypical ofBean,
An example of this will be at the earliestSwoft\Bean\BeanFactory::reload()The phase is initialized.

Worker/Task process

For RPC or HTTP requests, the processes that are most closely related are certainly Worker and Task processes.
For both, SwoftBeanBeanFactory::reload () will be called during the callback phase of swoole’s onWorkerStart event.

//Swoft\Bootstrap\Server\ServerTrait(HttpServer和RpcServer都使用了该性状)
/**
 * OnWorkerStart event callback
 *
 * @param Server $server server
 * @param int $workerId workerId
 * @throws \InvalidArgumentException
 */
public function onWorkerStart(Server $server, int $workerId)
{
    // Init Worker and TaskWorker
    $setting = $server->setting;
    $isWorker = false;

    if ($workerId >= $setting['worker_num']) {
        // TaskWorker
        ApplicationContext::setContext(ApplicationContext::TASK);
        ProcessHelper::setProcessTitle($this->serverSetting['pname'] . ' task process');
    } else {
        // Worker
        $isWorker = true;
        ApplicationContext::setContext(ApplicationContext::WORKER);
        ProcessHelper::setProcessTitle($this->serverSetting['pname'] . ' worker process');
    }

    $this->fireServerEvent(SwooleEvent::ON_WORKER_START, [$server, $workerId, $isWorker]);
    //beforeWorkerStart()内部会调用BeanFactory::reload();
    $this->beforeWorkerStart($server, $workerId, $isWorker);
}

This means that the life cycle of the connection pool object at this time isProcess global periodrather thanProgram global period
I think there are mainly three reasons for designing the process pool as a process global period instead of the process global period with the highest degree of sharing.

  1. The simultaneous reading and writing of a connection by multiple processes will lead to data transmission disorder., you need to ensure that connections are not accessed at the same time.
  2. When a Worker process writes to an object in the program global phase, it will cause copy-on-write and produce a copy of the process global phase, which is difficult to maintain.
  3. If you use the process global phase, you can use the existing Bean mechanism to manage objects and reduce the special encoding.

Connection Pool in Process

//Swoft\Process\ProcessBuilder.php
/**
     * After process
     *
     * @param string $processName
     * @param bool   $boot 该参数即Process 注解的boot属性
     */
    private static function beforeProcess(string $processName, $boot)
    {
        if ($boot) {
            BeanFactory::reload();
            $initApplicationContext = new InitApplicationContext();
            $initApplicationContext->init();
        }

        App::trigger(ProcessEvent::BEFORE_PROCESS, null, $processName);
    }
}

There are two kinds of Processes in Swoft: one is that the boot attribute that defines the process annotation is truePre-process, this kind of process starts with the system startup; The other is that the boot attribute that defines the Process annotation is falseUser defined processThis type of process requires the user to call it manually when needed.ProcessBuilder::create()Start.

But whatever it is, it will eventually be called in Process.beforeProcess()Initialize the subprocess. True for bootPre-processBecause the parent process did not initialize the bean container when it was started, the bean container will be initialized separately, while for others whose boot is falseUser defined processWhich directly inherits the Ioc container of the parent process.

Swoft basically abides by the rule that a process has a separate connection pool, so that connections in all processes are independent, ensuring connections
Will not be read and written at the same time. There is only one special case in Process. If you first use connection pool-dependent services, such as CRUD Mysql, then callProcessBuilder::create()Start upUser defined processDue toUser defined processThe Bean container of the parent process is directly inherited without resetting, and then the child process obtains the connection pool and connection in the parent process.

Command

/**
 * The adapter of command
 * @Bean()
 */
class HandlerAdapter
{
    /**
     * before command
     *
     * @param string $class
     * @param string $command
     * @param bool   $server
     */
    private function beforeCommand(string $class, string $command, bool $server)
    {
        if ($server) {
            return;
        }
        $this->bootstrap();
        BeanFactory::reload();

        // 初始化
        $spanId = 0;
        $logId = uniqid();

        $uri = $class . '->' . $command;
        $contextData = [
            'logid'       => $logId,
            'spanid'      => $spanId,
            'uri'         => $uri,
            'requestTime' => microtime(true),
        ];

        RequestContext::setContextData($contextData);
    }
}

Command line scripts have their own separate Bean containers, which are similar to and simpler than Process. They strictly follow one process and one connection pool, and will not be repeated here.

Swoft连接池
Assuming that the number of Worker is j, the number of tasks is k, the number of k,Processes is l, the number of l,Command is m, the maximum number of connections configured in each process pool is n, and the number of deployed machines is x, it is not difficult to see that the number of connections occupied by each swoft project is(j+k+l+m)*n*x.

Tianfeng himself mentioned another connection pool model based on Swoole.
Rango- < real PHP database connection pool based on swoole extension >

Rango曾提出的连接池方案
In this scheme, the number of connections occupied by the project is onlyk*x.
Except that each process of the Task process does not directly hold the connection pool, but submits instructions to the Task process (task(),sendMessage()) It requires at least one extra inter-process communication (Unix Socket by default) for the connection pool related services to be operated on its behalf.
Although this scheme can reuse connections and save connections better, it is not convenient to implement the mechanism. From another perspective, Swoft’s connection pool scheme is to solve the problem of the number of connections required for concurrent execution by a single process when Swoole is used. Range proposed the connection pool scheme to solve the pressure control problem of Mysql and other services under the super-large flow system. The two are suitable for different scenes, and their purposes and meanings coincide to some extent, but they are not exactly the same.

Implementation of Swoft connection pool

Container for pool

The connection pool selects an appropriate queue structure as the container of the connection according to whether the current coordination environment exists or not.

  1. \SplQueue:SplQueueIt is the data structure of PHP standard library, and the bottom layer is a double linked list. Under the special scenario of queue operation, its performance is much higher than the array () data structure realized by linked list+hash table.
  2. \Swoole\Coroutine\ChannelSwoole provides coordination-related data structures, which not only provide regular queue operations. In the coordination environment, when the queue length is switched from 0 to 1, the coordination control will be automatically relinquished and the corresponding producer or consumer will be awakened.

Acquisition of connections

\\Swoft\Pool\ConnectionPool.php
abstract class ConnectionPool implements PoolInterface {
    /**
     * Get connection
     *
     * @throws ConnectionException;
     * @return ConnectionInterface
     */
    public function getConnection():ConnectionInterface
    {
        //根据执行环境选择容器
        if (App::isCoContext()) {
            $connection = $this->getConnectionByChannel();
        } else {
            $connection = $this->getConnectionByQueue();
        }

        //连接使用前的检查和重新连接
        if ($connection->check() == false) {
            $connection->reconnect();
        }
        //加入到全局上下文中,事务处理和资源相关的监听事件会用到
        $this->addContextConnection($connection);
        return $connection;
    }
}
\\Swoft\Pool\ConnectionPool.php
/**
 * Get connection by queue
 *
 * @return ConnectionInterface
 * @throws ConnectionException
 */
private function getConnectionByQueue(): ConnectionInterface
{
    if($this->queue == null){
        $this->queue = new \SplQueue();
    }
    
    if (!$this->queue->isEmpty()) {
        //队列存在可用连接直接获取
        return $this->getEffectiveConnection($this->queue->count(), false);
    }
    //超出队列最大长度
    if ($this->currentCount >= $this->poolConfig->getMaxActive()) {
        throw new ConnectionException('Connection pool queue is full');
    }
    //向队列补充连接
    $connect = $this->createConnection();
    $this->currentCount++;

    return $connect;
}
\\Swoft\Pool\ConnectionPool.php
/**
 * Get effective connection
 *
 * @param int  $queueNum
 * @param bool $isChannel
 *
 * @return ConnectionInterface
 */
private function getEffectiveConnection(int $queueNum, bool $isChannel = true): ConnectionInterface
{
    $minActive = $this->poolConfig->getMinActive();
    //连接池中连接少于数量下限时直接获取
    if ($queueNum <= $minActive) {
        return $this->getOriginalConnection($isChannel);
    }

    $time        = time();
    $moreActive  = $queueNum - $minActive;
    $maxWaitTime = $this->poolConfig->getMaxWaitTime();
    //检查多余的连接,如等待时间过长,表示当前所持连接数暂时大于需求值,且易失效,直接释放
    for ($i = 0; $i < $moreActive; $i++) {
        /* @var ConnectionInterface $connection */
        $connection = $this->getOriginalConnection($isChannel);;
        $lastTime = $connection->getLastTime();
        if ($time - $lastTime < $maxWaitTime) {
            return $connection;
        }
        $this->currentCount--;
    }

    return $this->getOriginalConnection($isChannel);
}

It is very clear to add some comments and will not be repeated here.

Release of Connection

The release of connection has two different ambiguous usages, so we make the following definitions:
One is that the connection is no longer in use and can be closed, which we callDestruction of connections.
One is that the connection is temporarily no longer in use, its occupancy state is released, and it can be handed back to the idle queue from the user’s hand, which we callReturn of Connection.

Destruction of Links

Generally, unset variables or other means are used to clear all references of connection variables, waiting for Zend engine to clear link resources.
This is in the abovegetEffectiveConnection()It has appeared in. Execute to$this->currentCount--;By the time, the connection was out of line, and$connectionThe variable will be replaced as a loop variable in the next loop or cleared as a local variable when the method returns, and the reference to the connection resource will be cleared 0. The resource whose reference falls to 0 will be recycled in the next gc execution, so it is normal that you do not see the active connection release code.
If your code refers to this connection elsewhere and is not managed properly, it may lead to resource leakage.

Return of Link

/**
 * Class AbstractConnect
 */
abstract class AbstractConnection implements ConnectionInterface
{
    //Swoft\Pool\AbstractConnection.php
    /**
     * @param bool $release
     */
    public function release($release = false)
    {
        if ($this->isAutoRelease() || $release) {
            $this->pool->release($this);
        }
    }
}
//Swoft\Pool\ConnectionPool.php
/**
 * Class ConnectPool
 */
abstract class ConnectionPool implements PoolInterface
{
    /**
     * Release connection
     *
     * @param ConnectionInterface $connection
     */
    public function release(ConnectionInterface $connection)
    {
        $connectionId = $connection->getConnectionId();
        $connection->updateLastTime();
        $connection->setRecv(true);
        $connection->setAutoRelease(true);

        if (App::isCoContext()) {
            $this->releaseToChannel($connection);
        } else {
            $this->releaseToQueue($connection);
        }

        $this->removeContextConnection($connectionId);
    }
}

When the user finishes using a connection, such as executing a sql, the connected should be calledrelease()Methods.
The connection itself is the reverse connection that holds the connection poolConnectionInterface->release()Method, it does not destroy itself immediately, but cleans up its own tags and calls thePoolInterface->release()Add it back to the connection pool.

//Swoft\Event\Listeners\ResourceReleaseListener.php
/**
 * Resource release listener
 *
 * @Listener(AppEvent::RESOURCE_RELEASE)
 */
class ResourceReleaseListener implements EventHandlerInterface
{
    /**
     * @param \Swoft\Event\EventInterface $event
     * @throws \InvalidArgumentException
     */
    public function handle(EventInterface $event)
    {
        // Release system resources
        App::trigger(AppEvent::RESOURCE_RELEASE_BEFORE);

        $connectionKey = PoolHelper::getContextCntKey();
        $connections   = RequestContext::getContextDataByKey($connectionKey, []);
        if (empty($connections)) {
            return;
        }

        /* @var \Swoft\Pool\ConnectionInterface $connection */
        foreach ($connections as $connectionId => $connection) {
            if (!$connection->isRecv()) {
                Log::error(sprintf('%s connection is not received ,forget to getResult()', get_class($connection)));
                $connection->receive();
            }

            Log::error(sprintf('%s connection is not released ,forget to getResult()', get_class($connection)));
            $connection->release(true);
        }
    }
}

Considering that the user may not release the connection after use and cause connection leakage, Swoft will trigger one after Rpc/Http request or Task ends.Swoft.resourceReleaseEvents (note: Swoft is the prefix added by the author, which is convenient for readers to distinguish Swoole-related events from Swoft-related events), and the connection will be forcibly received and returned to the unit.

Swoft source code analysis series catalog:https://segmentfault.com/a/11 …