PHP — Stream Filters

If you have ever worked with files in PHP, you have probably used functions like fopen, fwrite, fread and fclose. Streams are a very powerful and flexible abstraction for working with files and other I/O resources.

Streams abstract the handling of input and output data, letting you read and write data from and to different sources, such as files, strings, network connections, etc.

For example, you can use streams to read data from a file, process it and write it to another file, without worrying about where the data comes from or where it goes.

Or you can use streams to read data from a network connection, process it and write it to a database. And so on.

Basic example

$stream = fopen('data.txt', 'r');
while (!feof($stream)) {
    $line = fgets($stream);
    echo $line;
}
fclose($stream);

In this example, we open a file called data.txt in read mode ('r'). Then we read the file line by line until the end (feof($stream)). Finally, we close the file with fclose($stream).

But what is a stream? A stream is a resource that represents a source or destination of data. In the example above, $stream is a stream that represents the file data.txt.

Let's look at a more advanced example.

Advanced example

$context = stream_context_create([
    'http' => [
        'method' => 'POST',
        'header' => 'Content-Type: application/json',
        'content' => json_encode(['key' => 'value']),
    ],
]);

$stream = fopen('http://example.com/data.csv', 'r', false, $context);
while (!feof($stream)) {
    $line = fgetcsv($stream);
    print_r($line);
}
fclose($stream);

In this example, we create a stream context with stream_context_create, an associative array of configuration options for the stream. Here we configure an HTTP stream with the POST method, the Content-Type: application/json header and a JSON request body.

Then we open a stream to the URL http://example.com/data.csv in read mode ('r') with the stream context we just created.

Next, we read the stream line by line with fgetcsv, which reads a line from the stream and turns it into an array of comma-separated values.

Finally, we close the stream with fclose.

But why use streams?

PHP was born at a time when file handling was the main way to interact with the file system and other I/O resources. That is why PHP's file functions are based on low-level operations such as opening, reading, writing and closing files: the famous fopen, fread, fwrite and fclose.

Over time, the need to interact with other kinds of I/O resources, such as network connections and databases, became more and more common. That is where streams came in.

Streams are essentially an abstraction layer over different kinds of I/O resources, letting you read and write data from and to them consistently, regardless of where the data comes from or goes to.

By I/O resources, I mean anything you can read data from or write data to, such as files, strings, console windows, network connections, sockets, pipes, etc.

And by not worrying about where the data comes from or goes to, I mean you don't need to know whether you are reading from a file, a network connection, a database, etc. You simply read the data from the stream and process it however you like. That is very powerful and flexible.

So how do I use streams?

To use streams in PHP, you need to understand a few basic concepts:

• Resource
• Context
• Wrapper
• Stream functions
• Stream filters

Note: this is just a basic overview of streams in PHP. To learn more, see the official documentation.

Resource

Let's dig a little deeper into the concept of a resource in PHP.

A stream is represented by a resource in PHP. A resource is a special variable that holds an internal reference to an external resource, such as a file or a network connection. You can create a resource with fopen and close it with fclose.

A resource in PHP is an internal identifier for external resources, and the get_resource_type function can be used to get the resource's type.

$stream = fopen('data.txt', 'r');
echo get_resource_type($stream);
fclose($stream);

In the console, you will see something like stream.

The resource always refers to the open file.

Context

A stream context is an associative array of configuration options for the stream. You can create one with stream_context_create and pass it as an argument to functions that open streams, such as fopen, file_get_contents, etc.

When you open a stream with a stream context, the context's options are applied to the stream. For example, you can configure an HTTP stream with the POST method, the Content-Type: application/json header, etc.

$context = stream_context_create([
    'http' => [
        'method' => 'POST',
        'header' => 'Content-Type: application/json',
        'content' => json_encode(['key' => 'value']),
    ],
]);

$stream = fopen('http://example.com/data.csv', 'r', false, $context);

Or you can use a stream context to configure an FTP stream with a username and password.

$context = stream_context_create([
    'ftp' => [
        'username' => 'user',
        'password' => 'pass',
    ],
]);

$stream = fopen('ftp://example.com/data.csv', 'r', false, $context);

Wrapper

A wrapper is a URL scheme that defines how PHP should open a stream. For example, the http wrapper is used to open HTTP streams, the ftp wrapper to open FTP streams, and so on.

When you open a stream with fopen, PHP uses the matching wrapper to open it. For example, if you open http://example.com/data.csv, PHP uses the http wrapper.

You can also use wrappers to open streams over other kinds of resources, such as strings, in-memory variables, etc.

$stream = fopen('data:text/plain;base64,SGVsbG8gV29ybGQ=', 'r');
echo stream_get_contents($stream);
fclose($stream);

In this example, we open a stream over a base64-encoded string with the data wrapper. The string's content is Hello World.

$stream = fopen('php://memory', 'r+');

In this example, we open an in-memory stream with the php wrapper. The stream is opened in read-write mode ('r+').

$stream = fopen('ogg://temp', 'r+');

OGG is a wrapper for handling audio files in the OGG format; this is an example of a custom wrapper.

You can create your own custom wrappers to open streams over other kinds of resources, such as databases, APIs, etc.

See the official documentation for more information about wrappers in PHP.

Stream functions

PHP provides several functions for working with streams, such as fopen, fclose, fread, fwrite, feof, fseek, ftell, fgetcsv, stream_get_contents, etc.

You can use these functions to open, close, read, write, seek, check for the end of a stream, and so on.

For example, you can use fopen to open a stream, fread to read from it, fwrite to write to it and fclose to close it.

$stream = fopen('data.txt', 'r');

while (!feof($stream)) {
    $line = fgets($stream);
    echo $line;
}

fwrite($stream, 'Hello World');
fclose($stream);

In this example, we open a file called data.txt in read mode ('r'). Then we read the file line by line until the end with fgets. Next, we write the string Hello World to the file with fwrite. Finally, we close the file with fclose.

You can use these functions with streams over different kinds of resources, such as curl, sockets, pipes, etc.

$url = 'http://example.com/data/page.html';
$data = curl_init($url);
curl_setopt($data, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($data);
curl_close($data);
$stream = fopen('php://memory', 'r+');
fwrite($stream, $response);
rewind($stream);
echo stream_get_contents($stream);
fclose($stream);

In this example, we make an HTTP request with curl to http://example.com/data/page.html. Then we open an in-memory stream with the php wrapper in read-write mode ('r+'). Next, we write the response to the stream with fwrite. Finally, we print the stream's contents with stream_get_contents and close it with fclose.

Stream filters

Here we reach one of the most powerful and flexible features of streams in PHP: stream filters.

Imagine the following scenario: you are reading data from a file and need to process it before printing it. With stream filters, you can create a filter that processes the data as needed and attach it to the read stream, without changing the code that reads the data.

For example, you can use a stream filter to convert a text file's contents to uppercase before printing it.

$stream = fopen('data.txt', 'r');
stream_filter_append($stream, 'string.toupper');
echo stream_get_contents($stream);
fclose($stream);

In this example, we open a file called data.txt in read mode ('r'). Then we attach the string.toupper filter to the stream with stream_filter_append, which converts the data to uppercase. Finally, we print the stream's contents with stream_get_contents and close the file with fclose.

The conversion is done automatically by the stream filter while the data is being read, and it does not affect the original file.

You can create your own stream filters to process data however you need.

See the official documentation for more information about stream filters in PHP.

FilterPHP

I prepared an example to help you better understand how a stream filter works.

In this example, we will create a stream filter that keeps only the lines containing the word “PHP”.

<?php

class FilterPHP extends php_user_filter
{
    public $stream;
    public string $filter;

    public function onCreate(): bool {
        $this->stream = fopen('php://temp', 'w+');
        return $this->stream !== false;
    }

    public function filter($in, $out, &$consumed, $closing): int {
        $this->filter = 'PHP';
        $out_data = '';
        while ($bucket = stream_bucket_make_writeable($in)) {
            $line = explode("\n", $bucket->data);
            foreach ($line as $l) {
                if (str_contains($l, $this->filter)) {
                    $out_data .= $l . "\n";
                }
            }
        }

        $bucket_out = stream_bucket_new($this->stream, $out_data);
        stream_bucket_append($out, $bucket_out);

        return PSFS_PASS_ON;
    }
}

Let's go through what the code above does, step by step.

1. We create a class called FilterPHP that extends php_user_filter. This class is used to create a custom stream filter.

If you have worked with classes in PHP, extending a class with such an odd name may look strange. That is because php_user_filter is an internal PHP class that predates the class naming conventions the community established over the years.

2. The FilterPHP class has two properties: $stream and $filter. $stream holds a temporary stream where the filtered data will be written. $filter holds the filter that will be applied to the data.
3. The onCreate method is called when the filter is created. In it, we open a temporary stream with fopen('php://temp', 'w+') and store the stream resource in $stream. If the stream cannot be opened, we return false.

The onCreate method acts as a constructor for the filter, and custom stream filters typically implement it to set up their resources.

But what is php://temp? php://temp is a wrapper for creating temporary streams in PHP. Temporary streams are kept in memory or in a temporary file, depending on the size of the data.

4. The filter method is called to filter the stream's data. It receives the following parameters:

• $in: the input bucket brigade, from which the original data is read. It is read-only and is passed to the filter.
• $out: the output bucket brigade, to which the filtered data is written. It is write-only and is passed to the filter.
• &$consumed: the amount of data consumed by the filter. The filter updates this value and it is passed back to PHP.
• $closing: indicates whether the stream is being closed. If it is true, the filter should release any resources it allocated.

Note: the & in front of a variable in PHP means it is passed by reference, so changes made inside the function are reflected outside it.

5. In the filter method, we read the input data with stream_bucket_make_writeable. Then we split it into lines with explode("\n", $bucket->data) and check whether each line contains the word "PHP" with str_contains($l, $this->filter).

6. If the line contains the word “PHP”, it is added to $out_data. Then we create a new output bucket with stream_bucket_new and append it to the output with stream_bucket_append.

7. Finally, we return PSFS_PASS_ON to indicate that the filter should keep passing data on to the next filter or to the output stream.

Now that we have created the stream filter, let's attach it to an input stream.

<?php
require_once './FilterPHP.php';
$json = 'https://raw.githubusercontent.com/sschonss/stream-php/main/data.json';
$fileContents = file_get_contents($json);

if ($fileContents === false) {
    die('Failed to fetch data from the endpoint.');
}

stream_filter_register('filterphp', 'FilterPHP') or die("Failed to register filter.");

$tempStream = fopen('php://temp', 'r+');
fwrite($tempStream, $fileContents);
rewind($tempStream);

$out_fp = fopen('php://stdout', 'w') or die("Failed to open output stream.");
stream_filter_append($tempStream, 'filterphp');

while ($data = fread($tempStream, 1024)) {
    $data = str_replace('"name": ', '', $data);
    $data = str_replace('"description": ', '', $data);
    echo $data;
}

fclose($tempStream);
fclose($out_fp);

In this example, we read a JSON file from a URL with file_get_contents and store its contents in a variable called $fileContents.

Then we register the FilterPHP stream filter with stream_filter_register. The first argument is the filter's name, used to attach it to the input stream. The second argument is the filter's class name.

It is important to register the stream filter before attaching it to the input stream, so PHP knows how to handle it.

Next, we open a temporary stream with fopen('php://temp', 'r+') and write the file's contents to it with fwrite. Then we rewind the stream with rewind.

Note: rewind moves the stream's internal pointer back to the beginning. This is needed because the pointer is left at the end of the stream after writing.

Note: a pointer in PHP works like a cursor that marks the current position in the stream.

Then we open an output stream with fopen('php://stdout', 'w') and attach the FilterPHP stream filter to the temporary stream with stream_filter_append.

Finally, we read the data from the temporary stream with fread and print it with echo. Before printing, we strip the name and description keys with str_replace.

At the end, we close the temporary stream and the output stream with fclose.

The output of the code above will look like this (the sample data is in Portuguese):

php-filter_1  |       "PHP",
php-filter_1  |        "PHP (Hypertext Preprocessor) é uma linguagem de script amplamente utilizada para desenvolvimento web e pode ser embutida em HTML."
php-filter_1  |        "Composer é um gerenciador de dependências para PHP, permitindo que você declare as bibliotecas que seu projeto depende e as instale."
php-filter_1  |        "Laravel é um framework web PHP, conhecido por sua sintaxe elegante e ferramentas robustas para desenvolvimento rápido de aplicativos."
php-filter_1  |        "Symfony é um conjunto de componentes PHP reutilizáveis e um framework web para criar aplicações e sites."
php-filter_1  |        "CodeIgniter é um framework PHP poderoso e simples de usar, construído para desenvolvedores que precisam de um toolkit elegante para criar aplicativos web completos."
php-filter_1  |       "CakePHP",
php-filter_1  |        "CakePHP é um framework PHP rápido de desenvolvimento que proporciona uma estrutura extensível para desenvolvedores criar aplicativos web."
php-filter_1  |        "Zend Framework é uma coleção de pacotes PHP profissionais com mais de 570 milhões de instalações."

If you want to try the code above, clone the stream-php repository and run docker-compose up to start the PHP container and test it.

Conclusion

Streams are a powerful and flexible abstraction for working with files and other I/O resources in PHP. With streams, you can read and write data from and to different sources, such as files, strings and network connections, consistently and regardless of where the data comes from or goes to.

You can also use stream contexts to configure streams with specific options, wrappers to open streams over different kinds of resources, stream functions to work with streams efficiently, and stream filters to process data as needed.

I hope this guide helped you better understand how to work with streams in PHP. If you have any questions, suggestions or corrections, feel free to reach out.

See you next time!