Multi-threaded programming in PHP using Pthreads Translation. Multi-threaded computing in PHP: pthreads Path of the Jedi - using the PCNTL extension
I recently tried pthreads and was pleasantly surprised - it's an extension that adds the ability to work with multiple real threads in PHP. No emulation, no magic, no fakes - everything is real.
I am considering such a task. There is a pool of tasks that need to be completed quickly. PHP has other tools for solving this problem, they are not mentioned here, the article is about pthreads.
What are pthreads
That's all! Well, almost everything. In fact, there is something that may upset an inquisitive reader. None of this works on standard PHP compiled with default options. To enjoy multithreading, you must have ZTS (Zend Thread Safety) enabled in your PHP.
PHP setup
Next, PHP with ZTS. Do not pay attention to such a large difference in execution time compared to PHP without ZTS (37.65 vs 265.05 seconds), I did not try to reduce it to a common denominator PHP settings. In the case without ZTS, I have XDebug enabled for example.
As you can see, when using 2 threads, the speed of program execution is approximately 1.5 times higher than in the case of linear code. When using 4 threads - 3 times.
You can note that even though the processor is 8-core, the execution time of the program remained almost unchanged if more than 4 threads were used. It seems that this is due to the fact that my processor has 4 physical cores. For clarity, I have depicted the plate in the form of a diagram.
Summary
In PHP, it is possible to work quite elegantly with multithreading using the pthreads extension. This gives a noticeable increase in productivity.
Tags: Add tags
Sometimes it becomes necessary to perform several actions simultaneously, for example, checking changes in one database table and making modifications to another. Moreover, if one of the operations (for example, checking changes) takes a lot of time, it is obvious that sequential execution will not ensure resource balancing.
To solve this kind of problem, programming uses multithreading - each operation is placed in a separate thread with an allocated amount of resources and works inside it. With this approach, all tasks will be completed separately and independently.
Although PHP does not support multithreading, there are several methods for emulating it, about which we'll talk below.
1. Running several copies of the script - one copy per operation
//woman.php if (!isset($_GET["thread"])) ( system("wget http://localhost/woman.php?thread=make_me_happy"); system("wget http://localhost/ woman.php?thread=make_me_rich"); ) elseif ($_GET["thread"] == "make_me_happy") ( make_her_happy(); ) elseif ($_GET["thread"] == "make_me_rich") ( find_another_one( ); )When we execute this script without parameters, it automatically runs two copies of itself, with operation IDs ("thread=make_me_happy" and "thread=make_me_rich"), which initiate the execution of the necessary functions.
This way we achieve the desired result - two operations are performed simultaneously - but this, of course, is not multithreading, but simply a crutch for performing tasks simultaneously.
2. Path of the Jedi - using the PCNTL extension
PCNTL is an extension that allows you to fully work with processes. In addition to management, it supports sending messages, checking status and setting priorities. This is what the previous script using PCNTL looks like:
$pid = pcntl_fork(); if ($pid == 0) ( make_her_happy(); ) elseif ($pid > 0) ( $pid2 = pcntl_fork(); if ($pid2 == 0) ( find_another_one(); ) )
It looks quite confusing, let's go through it line by line.
In the first line, we “fork” the current process (fork is copying a process while preserving the values of all variables), dividing it into two processes (current and child) running in parallel.
To understand where we are at this moment, in a child or mother process, the pcntl_fork function returns 0 for the child and the process ID for the mother. Therefore, in the second line, we look at $pid, if it is zero, then we are in the child process - we are executing the function, otherwise, we are in the mother (line 4), then we create another process and similarly perform the task.
Script execution process:
Thus, the script creates 2 more child processes, which are its copies and contain the same variables with similar values. And using the identifier returned by the pcntl_fork function, we find out which thread we are currently in and perform the necessary actions.
It seems, PHP developers parallelism is rarely used. I won’t talk about the simplicity of synchronous code; single-threaded programming is, of course, simpler and clearer, but sometimes a little use of parallelism can bring a noticeable increase in performance.
In this article, we'll take a look at how multithreading can be achieved in PHP using the pthreads extension. To do this, you will need the ZTS (Zend Thread Safety) version of PHP 7.x installed, along with installed extension pthreads v3. (At the time of writing, in PHP 7.1, users will need to install from the master branch in the pthreads repository - see third party extension.)
A small clarification: pthreads v2 is intended for PHP 5.x and is no longer supported, pthreads v3 is for PHP 7.x and is actively being developed.
After such a digression, let's get straight to the point!
Processing one-time tasks
Sometimes you want to process one-time tasks in a multi-threaded way (for example, executing some I/O-bound task). In such cases, you can use the Thread class to create a new thread and run some processing on a separate thread.
For example:
$task = new class extends Thread ( private $response; public function run() ( $content = file_get_contents("http://google.com"); preg_match("~
Here the run method is our processing, which will be executed inside a new thread. When Thread::start is called, a new thread is spawned and the run method is called. We then join the child thread back to the main thread by calling Thread::join , which will block until the child thread has finished executing. This ensures that the task finishes executing before we try to print the result (which is stored in $task->response).
It may not be desirable to pollute a class with additional responsibilities associated with flow logic (including the responsibility of defining a run method). We can distinguish such classes by inheriting them from the Threaded class. Then they can be run inside another thread:
Class Task extends Threaded ( public $response; public function someWork() ( $content = file_get_contents("http://google.com"); preg_match("~ (.+) ~", $content, $matches); $ this->response = $matches; ) ) $task = new Task; $thread = new class($task) extends Thread ( private $task; public function __construct(Threaded $task) ( $this->task = $task; ) public function run() ( $this->task->someWork( ); ) ); $thread->start() && $thread->join(); var_dump($task->response);
Any class that needs to be run in a separate thread must inherit from the Threaded class. This is because it provides the necessary capabilities to perform processing on different threads, as well as implicit security and useful interfaces (such as resource synchronization).
Let's take a look at the class hierarchy offered by the pthreads extension:
Threaded (implements Traversable, Collectable) Thread Worker Volatile Pool
We've already covered and learned the basics of the Thread and Threaded classes, now let's take a look at the other three (Worker, Volatile, and Pool).
Reusing Threads
Starting a new thread for each task that needs to be parallelized is quite expensive. This is because a common-nothing architecture must be implemented in pthreads to achieve multithreading within PHP. Which means that the entire execution context of the current instance of the PHP interpreter (including every class, interface, trait, and function) must be copied for each thread created. Because this has a noticeable performance impact, the stream should always be reused whenever possible. Threads can be reused in two ways: using Workers or using Pools.
The Worker class is used to perform a number of tasks synchronously within another thread. This is done by creating a new Worker instance (which creates a new thread), and then pushing tasks onto that separate thread's stack (using Worker::stack).
Here's a small example:
Class Task extends Threaded ( private $value; public function __construct(int $i) ( $this->value = $i; ) public function run() ( usleep(250000); echo "Task: ($this->value) \n"; ) ) $worker = new Worker(); $worker->start(); for ($i = 0; $i stack(new Task($i)); ) while ($worker->collect()); $worker->shutdown();
In the example above, 15 tasks for a new $worker object are pushed onto the stack via the Worker::stack method, and then they are processed in the order they were pushed. The Worker::collect method, as shown above, is used to clean up tasks as soon as they finish executing. With it, inside a while loop, we block the main thread until all tasks on the stack are completed and cleared - before we call Worker::shutdown . Terminating a worker early (i.e. while there are still tasks that need to be completed) will still block the main thread until all tasks have completed their execution, just that the tasks will not be garbage collected (which entails represent memory leaks).
The Worker class provides several other methods related to its task stack, including Worker::unstack for removing the last stacked task and Worker::getStacked for getting the number of tasks in the execution stack. A worker's stack contains only the tasks that need to be executed. Once a task on the stack has been completed, it is removed and placed on a separate (internal) stack for garbage collection (using the Worker::collect method).
Another way to reuse a thread across multiple tasks is to use a thread pool (via the Pool class). A thread pool uses a group of Workers to enable tasks to be executed simultaneously, in which the concurrency factor (the number of pool threads it operates with) is set when the pool is created.
Let's adapt the above example to use a pool of workers:
Class Task extends Threaded ( private $value; public function __construct(int $i) ( $this->value = $i; ) public function run() ( usleep(250000); echo "Task: ($this->value) \n"; ) ) $pool = new Pool(4); for ($i = 0; $i submit(new Task($i)); ) while ($pool->collect()); $pool->shutdown();
There are a few notable differences when using a pool as opposed to a worker. First, the pool does not need to be started manually; it starts executing tasks as soon as they become available. Secondly, we send tasks to the pool, not put them on a stack. Additionally, the Pool class does not inherit from Threaded and therefore cannot be passed to other threads (unlike Worker).
How good practice For workers and pools, you should always clean up their tasks as soon as they complete, and then manually terminate them themselves. Threads created using the Thread class must also be attached to the parent thread.
pthreads and (im)mutability
The last class we'll touch on is Volatile, a new addition to pthreads v3. Immutability has become an important concept in pthreads because without it, performance suffers significantly. Therefore, by default, properties of Threaded classes that are themselves Threaded objects are now immutable, and therefore they cannot be overwritten after their initial assignment. Explicit mutability for such properties is currently preferred, and can still be achieved using the new Volatile class.
Let's look at an example that will demonstrate the new immutability restrictions:
Class Task extends Threaded // a Threaded class ( public function __construct() ( $this->data = new Threaded(); // $this->data is not overwritable, since it is a Threaded property of a Threaded class ) ) $task = new class(new Task()) extends Thread ( // a Threaded class, since Thread extends Threaded public function __construct($tm) ( $this->threadedMember = $tm; var_dump($this->threadedMember-> data); // object(Threaded)#3 (0) () $this->threadedMember = new StdClass(); // invalid, since the property is a Threaded member of a Threaded class ) );
Threaded properties of Volatile classes, on the other hand, are mutable:
Class Task extends Volatile ( public function __construct() ( $this->data = new Threaded(); $this->data = new StdClass(); // valid, since we are in a volatile class ) ) $task = new class(new Task()) extends Thread ( public function __construct($vm) ( $this->volatileMember = $vm; var_dump($this->volatileMember->data); // object(stdClass)#4 (0) () // still invalid, since Volatile extends Threaded, so the property is still a Threaded member of a Threaded class $this->volatileMember = new StdClass(); ) );
We can see that the Volatile class overrides the immutability imposed by the parent Threaded class to provide the ability to change Threaded properties (as well as unset()).
There is another subject of discussion to cover the topic of variability and the Volatile class - arrays. In pthreads, arrays are automatically cast to Volatile objects when assigned to a property of the Threaded class. This is because it is simply not safe to manipulate an array of multiple PHP contexts.
Let's look at an example again to understand some things better:
$array = ; $task = new class($array) extends Thread ( private $data; public function __construct(array $array) ( $this->data = $array; ) public function run() ( $this->data = 4; $ this->data = 5; print_r($this->data); ) ); $task->start() && $task->join(); /* Output: Volatile Object ( => 1 => 2 => 3 => 4 => 5) */
We see that Volatile objects can be treated as if they were arrays because they support array operations such as (as shown above) the subset() operator. However, Volatile classes do not support basic array functions such as array_pop and array_shift. Instead, the Threaded class provides us with such operations as built-in methods.
As a demonstration:
$data = new class extends Volatile ( public $a = 1; public $b = 2; public $c = 3; ); var_dump($data); var_dump($data->pop()); var_dump($data->shift()); var_dump($data); /* Output: object(class@anonymous)#1 (3) ( ["a"]=> int(1) ["b"]=> int(2) ["c"]=> int(3) ) int(3) int(1) object(class@anonymous)#1 (1) ( ["b"]=> int(2) ) */
Other supported operations include Threaded::chunk and Threaded::merge .
Synchronization
In the last section of this article, we will look at synchronization in pthreads. Synchronization is a method that allows you to control access to shared resources.
For example, let's implement a simple counter:
$counter = new class extends Thread ( public $i = 0; public function run() ( for ($i = 0; $i i; ) ) ); $counter->start(); for ($i = 0; $i i; ) $counter->join(); var_dump($counter->i); // will print a number from 10 to 20
Without the use of synchronization, the output is not deterministic. Multiple threads write to the same variable without controlled access, which means updates will be lost.
Let's fix this so that we get the correct output of 20 by adding timing:
$counter = new class extends Thread ( public $i = 0; public function run() ( $this->synchronized(function () ( for ($i = 0; $i i; ) )); ) ); $counter->start(); $counter->synchronized(function ($counter) ( for ($i = 0; $i i; ) ), $counter); $counter->join(); var_dump($counter->i); // int(20)
Synchronized code blocks can also communicate with each other using the Threaded::wait and Threaded::notify (or Threaded::notifyAll) methods.
Here is an alternate increment in two synchronized while loops:
$counter = new class extends Thread ( public $cond = 1; public function run() ( $this->synchronized(function () ( for ($i = 0; $i notify(); if ($this->cond === 1) ( $this->cond = 2; $this->wait(); ) ) )); ) ); $counter->start(); $counter->synchronized(function ($counter) ( if ($counter->cond !== 2) ( $counter->wait(); // wait for the other to start first ) for ($i = 10; $i notify(); if ($counter->cond === 2) ( $counter->cond = 1; $counter->wait(); ) ) ), $counter); $counter->join(); /* Output: int(0) int(10) int(1) int(11) int(2) int(12) int(3) int(13) int(4) int(14) int(5) int( 15) int(6) int(16) int(7) int(17) int(8) int(18) int(9) int(19) */
You may notice additional conditions that have been placed around the call to Threaded::wait . These conditions are critical because they allow the synchronized callback to resume when it has received a notification and the specified condition is true . This is important because notifications can come from places other than when Threaded::notify is called. Thus, if calls to the Threaded::wait method were not enclosed in conditions, we will execute false wake up calls, which will lead to unpredictable code behavior.
Conclusion
We looked at the five classes of the pthreads package (Threaded, Thread, Worker, Volatile, and Pool) and how each class is used. We also took a look at the new concept of immutability in pthreads and gave a brief overview of the supported synchronization capabilities. With these basics in place, we can now start looking at how pthreads can be used in real world cases! This will be the topic of our next post.
If you are interested in the translation of the next post, let me know: comment on social media. networks, upvote and share the post with colleagues and friends.
I recently tried pthreads and was pleasantly surprised - it's an extension that adds the ability to work with multiple real threads in PHP. No emulation, no magic, no fakes - everything is real.
I am considering such a task. There is a pool of tasks that need to be completed quickly. PHP has other tools for solving this problem, they are not mentioned here, the article is about pthreads.
What are pthreads
That's all! Well, almost everything. In fact, there is something that may upset an inquisitive reader. None of this works on standard PHP compiled with default options. To enjoy multithreading, you must have ZTS (Zend Thread Safety) enabled in your PHP.
PHP setup
Next, PHP with ZTS. Don't mind the big difference in execution time compared to PHP without ZTS (37.65 vs 265.05 seconds), I didn't try to generalize the PHP setup. In the case without ZTS, I have XDebug enabled for example.
As you can see, when using 2 threads, the speed of program execution is approximately 1.5 times higher than in the case of linear code. When using 4 threads - 3 times.
You can note that even though the processor is 8-core, the execution time of the program remained almost unchanged if more than 4 threads were used. It seems that this is due to the fact that my processor has 4 physical cores. For clarity, I have depicted the plate in the form of a diagram.
Summary
In PHP, it is possible to work quite elegantly with multithreading using the pthreads extension. This gives a noticeable increase in productivity.
Tags:
- php
- pthreads
I recently tried pthreads and was pleasantly surprised - it's an extension that adds the ability to work with multiple real threads in PHP. No emulation, no magic, no fakes - everything is real.
I am considering such a task. There is a pool of tasks that need to be completed quickly. PHP has other tools for solving this problem, they are not mentioned here, the article is about pthreads.
What are pthreads
That's all! Well, almost everything. In fact, there is something that may upset an inquisitive reader. None of this works on standard PHP compiled with default options. To enjoy multithreading, you must have ZTS (Zend Thread Safety) enabled in your PHP.
PHP setup
Next, PHP with ZTS. Don't mind the big difference in execution time compared to PHP without ZTS (37.65 vs 265.05 seconds), I didn't try to generalize the PHP setup. In the case without ZTS, I have XDebug enabled for example.
As you can see, when using 2 threads, the speed of program execution is approximately 1.5 times higher than in the case of linear code. When using 4 threads - 3 times.
You can note that even though the processor is 8-core, the execution time of the program remained almost unchanged if more than 4 threads were used. It seems that this is due to the fact that my processor has 4 physical cores. For clarity, I have depicted the plate in the form of a diagram.
Summary
In PHP, it is possible to work quite elegantly with multithreading using the pthreads extension. This gives a noticeable increase in productivity.
