Skip to content

Commit

Permalink
Improve CallbackStreamFilter implementation
Browse files Browse the repository at this point in the history
  • Loading branch information
nyamsprod committed Jan 6, 2025
1 parent f242a29 commit c88d921
Show file tree
Hide file tree
Showing 14 changed files with 585 additions and 119 deletions.
6 changes: 5 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,14 @@ All Notable changes to `Csv` will be documented in this file

- Adding the `TabularDataReader::map` method.
- Adding `CallbackStreamFilter` class
- `AbstractCsv::appendStreamFilterOnRead`
- `AbstractCsv::appendStreamFilterOnWrite`
- `AbstractCsv::prependStreamFilterOnRead`
- `AbstractCsv::prependStreamFilterOnWrite`

### Deprecated

- None
- `AbstractCsv::addStreamFilter` use `AbstractCsv::appendStreamFilterOnRead` or `AbstractCsv::appendStreamFilterOnWrite` instead.

### Fixed

Expand Down
155 changes: 137 additions & 18 deletions docs/9.0/connections/callback-stream-filter.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,46 +5,165 @@ title: Dynamic Stream Filter

# Callback Stream Filter

<p class="message-info">Available since version <code>9.22.0</code></p>
<p class="message-info">Available since version <code>9.21.0</code></p>

Sometimes you may encounter a scenario where you need to create a specific stream filter
to resolve a specific issue. Instead of having to put up with the hassle of creating a
fully fledge stream filter, we are introducing a `CallbackStreamFilter`. This filter
is a PHP stream filter which enables applying a callable onto the stream prior to it
being actively consumed by the CSV process.

## Usage with CSV objects
## Registering the callbacks

Out of the box, to work, the feature requires a callback and its associated unique filter name.

```php
use League\Csv\CallbackStreamFilter;

CallbackStreamFilter::register('string.to.upper', strtoupper(...));
```

<p class="message-warning"><code>CallbackStreanFilter::register</code> register your callback
globally. So you only need to register it once. Preferably in your container definition if you
are using a framework.</p>

The callback signature is the following

```php
callable(string $bucket [, array $params]): string
```

- the `$bucket` parameter represents the chunk of the stream you will be operating on.
- the `$params` is an array of additional paraneters you will be passing onto the function when it is attached to a stream or a CSV object. This parameter is optional

Once registered you can re-use the filter via its `$filtername` with CSV documents or with a resource.

You can always check for the existence of your registered filter by calling the `CallbackStreamFilter::isRegistered` method.
The method will only return `true` for filter registered via the class. It will return `false` otherwise.

```php
CallbackStreamFilter::isRegistered('string.to.upper');
//returns true - exists; was registered in the previous example
CallbackStreamFilter::isRegistered('string.to.lower');
//returns false - does not exist; is not registered by CallbackStreamFilter
CallbackStreamFilter::isRegistered('string.tolower');
//returns false - exits, is registered by PHP itself not by CallbackStreamFilter
```

Last but not least you can always list all the registered filter names by calling the

```php
CallbackStreamFilter::registeredFilterNames(); // returns a list
```

Out of the box, the filter can not work, it requires a unique name and a callback to be usable.
Once registered you can re-use the filter with CSV documents or with a resource.
<p class="message-info">To avoid conflict with already registered stream filters a best
practice is to namespace your own filter by using a unique prefix. Instead of
naming it <code>string.to.lower</code> you should name it <code><strong>myapp.</strong>string.to.lower</code>
where <code>myaoo</code> is specific for your own codebase.</p>

let's imagine we have a CSV document with the return carrier character as the end of line character.
This type of document is parsable by the package but only if you enable the deprecated `auto_detect_line_endings`.
## Applying the callback

If you no longer want to rely on that feature since it emits a deprecation warning you can use the new
`CallbackStreamFilter` instead by swaping the offending character with a modern alternative.
Once registered you can use the following methods to attach your filter to your instance.

- `CallbackStreamFilter::appendOnReadTo`
- `CallbackStreamFilter::appendOnWriteTo`
- `CallbackStreamFilter::prependOnReadTo`
- `CallbackStreamFilter::prependOnWriteTo`

Those static public method will add the filter to the stream filter queue attached to the structure (CSV objects or PHP stream resource).
They all share the same signature and differ in when in the queue the filter is added
and in which mode. To illustrate their usage please check the two examples below with
League CSV classes and with PHP stream resources.

## Usage with CSV objects

Let's imagine we have a CSV document using the return carrier character (`\r`) as the end of line character.
This type of document is parsable by the package but only if you enable the deprecated `auto_detect_line_endings` ini setting.

If you no longer want to rely on that feature since it has been deprecated since PHP 8.1 and will be
removed from PHP once PHP9.0 is release, you can use the `CallbackStreamFilter` instead by
swaping the offending character with a supported alternative.

```php
use League\Csv\CallbackStreamFilter;
use League\Csv\Reader;

$csv = "title1,title2,title3\rcontent11,content12,content13\rcontent21,content22,content23\r";
$csv = "title1,title2,title3\r".
. "content11,content12,content13\r"
. "content21,content22,content23\r";

$document = Reader::createFromString($csv);
CallbackStreamFilter::addTo(
$document,
'swap.carrier.return',
$document->setHeaderOffset(0);

CallbackStreamFilter::register(
'replace.eol',
fn (string $bucket): string => str_replace("\r", "\n", $bucket)
);
$document->setHeaderOffset(0);
CallbackStreamFilter::appendOnReadTo($document, 'replace.eol');

return $document->first();
// returns ['title1' => 'content11', 'title2' => 'content12', 'title3' => 'content13']
// returns [
// 'title1' => 'content11',
// 'title2' => 'content12',
// 'title3' => 'content13',
// ]
```

The `addTo` method register the filter with the unique `swap.carrier.return` name and then attach
it to the CSV document object on read.
The `appendOnReadTo` method will check for the availability of the filter via its
name `replace.eol`. If it is not present a `LogicException` will be
thrown, otherwise it will attach the filter to the CSV document object at the
bottom of the stream filter queue. Since we are using the `Reader` class, the
filter is attached using the reader mode. If we were to use the `Writer` class,
the filter would be attached using the write mode only.

<p class="message-warning">On read, the CSV document content is <strong>never changed or replaced</strong>.
Conversely, the changes <strong>are persisted during writing</strong>.</p>
However, on write, the changes <strong>are persisted</strong> into the created document.</p>

## Usage with streams

Of course the `CallbackStreamFilter` can be use in other different scenario or with PHP stream resources.
<p class="message-notice">In the following example we will use the optional <code>$params</code> parameters
to add a specific behaviour to our callback</p>

```php
use League\Csv\CallbackStreamFilter;

$csv = <<<CSV
title1,title2,title3
content11,content12,content13
content21,content22,content23
CSV;

$stream = tmpfile();
fwrite($stream, $csv);

// We first check to see if the callback is not already registered
// without the check a LogicException would be thrown on
// usage or on callback registration
if (!CallbackStreamFilter::isRegistered('replace.word')) {
CallbackStreamFilter::register(
'replace.word',
function (string $bucket, array $params): string {
return str_replace($params['search'], $params['replace'], $bucket);
}
);
}

CallbackStreamFilter::appendOnReadTo(
$stream,
'replace.word',
[
'search' => ['content', '1', '2', '3'],
'replace' => ['contenu ', 'A', 'B', 'C'],
]
);
$data = [];

rewind($stream);
while (($record = fgetcsv($stream, 1000, ',')) !== false) {
$data[] = $record;
}
fclose($stream);

return $data[1]
//returns ['contenu AA', 'contenu AB', 'contenu AC']
```
35 changes: 26 additions & 9 deletions docs/9.0/connections/filters.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,16 +79,33 @@ Here's a table to quickly determine if PHP stream filters works depending on how

```php
public AbstractCsv::addStreamFilter(string $filtername, mixed $params = null): self
public AbstractCsv::appendStreamFilterOnRead(string $filtername, mixed $params = null): self
public AbstractCsv::prependStreamFilterOnRead(string $filtername, mixed $params = null): self
public AbstractCsv::appendStreamFilterOnWrite(string $filtername, mixed $params = null): self
public AbstractCsv::prependStreamFilterOnReadWrite(string $filtername, mixed $params = null): self
public AbstractCsv::hasStreamFilter(string $filtername): bool
```

The `AbstractCsv::addStreamFilter` method adds a stream filter to the connection.

- The `$filtername` parameter is a string that represents the filter as registered using php `stream_filter_register` function or one of [PHP internal stream filter](http://php.net/manual/en/filters.php).
<div class="message-notice">
<ul>
<li><code>addStreamFilter</code> is deprecated since version <code>9.21.0</code></li>
<li><code>appendStreamFilterOnRead</code> is available since <code>9.21.0</code></li>
<li><code>prependStreamFilterOnRead</code> is available since <code>9.21.0</code></li>
<li><code>appendStreamFilterOnWrite</code> is available since <code>9.21.0</code></li>
<li><code>prependStreamFilterOnWrite</code> is available since <code>9.21.0</code></li>
</ul>
</div>

- The `$filtername` parameter is a string that represents the filter as registered using php `stream_filter_register` function or one of [PHP internal stream filter](http://php.net/manual/en/filters.php).
- The `$params` : This filter will be added with the specified parameters to the end of the list.

<p class="message-warning">Each time your call <code>addStreamFilter</code> with the same argument the corresponding filter is registered again.</p>
The `appendStreamFilterOn*` methods add the stream filter at the bottom of the stream filter queue whereas
`prependStreamFilterOn*` methods add the stream filter on top of the queue. Both methods share the same
arguments and the same return type.

<p class="message-warning">Each time your call a method with the same argument the corresponding filter is attached again.</p>

The `AbstractCsv::hasStreamFilter` method tells whether a specific stream filter is already attached to the connection.

Expand All @@ -101,8 +118,8 @@ stream_filter_register('convert.utf8decode', Transcode::class);

$reader = Reader::createFromPath('/path/to/my/chinese.csv', 'r');
if ($reader->supportsStreamFilterOnRead()) {
$reader->addStreamFilter('convert.utf8decode');
$reader->addStreamFilter('string.toupper');
$reader->appendStreamFilterOnRead('convert.utf8decode');
$reader->appendStreamFilterOnRead('string.toupper');
}

$reader->hasStreamFilter('string.toupper'); //returns true
Expand All @@ -116,11 +133,11 @@ foreach ($reader as $row) {

## Stream filters removal

Stream filters attached **with** `addStreamFilter` are:
Stream filters attached **with** `addStreamFilter`, `appendStreamFilterOn*`, `prependStreamFilterOn*` are:

- removed on the CSV object destruction.

Conversely, stream filters added **without** `addStreamFilter` are:
Conversely, stream filters added **without** the feature are:

- not detected by the library.
- not removed on object destruction.
Expand All @@ -133,8 +150,8 @@ stream_filter_register('convert.utf8decode', Transcode::class);
$fp = fopen('/path/to/my/chines.csv', 'r');
stream_filter_append($fp, 'string.rot13'); //stream filter attached outside of League\Csv
$reader = Reader::createFromStream($fp);
$reader->addStreamFilter('convert.utf8decode');
$reader->addStreamFilter('string.toupper');
$reader->prependStreamFilterOnRead('convert.utf8decode');
$reader->prependStreamFilterOnRead('string.toupper');
$reader->hasStreamFilter('string.rot13'); //returns false
$reader = null;
// 'string.rot13' is still attached to `$fp`
Expand All @@ -148,4 +165,4 @@ The library comes bundled with the following stream filters:
- [RFC4180Field](/9.0/interoperability/rfc4180-field/) stream filter to read or write RFC4180 compliant CSV field;
- [CharsetConverter](/9.0/converter/charset/) stream filter to convert your CSV document content using the `mbstring` extension;
- [SkipBOMSequence](/9.0/connections/bom/) stream filter to skip your CSV document BOM sequence if present;
- [CallbackStramFilter](/9.0/connections/callback-strean-filter/) apply a callback via a stream filter.
- [CallbackStreamFilter](/9.0/connections/callback-stream-filter/) apply a callback via a stream filter.
2 changes: 1 addition & 1 deletion docs/9.0/interoperability/encoding.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ $reader = Reader::createFromPath('/path/to/my/file.csv', 'r');
//let's set the output BOM
$reader->setOutputBOM(Bom::Utf8);
//let's convert the incoming data from iso-88959-15 to utf-8
$reader->addStreamFilter('convert.iconv.ISO-8859-15/UTF-8');
$reader->appendStreamFilter('convert.iconv.ISO-8859-15/UTF-8');
//BOM detected and adjusted for the output
echo $reader->getContent();
```
Expand Down
Loading

0 comments on commit c88d921

Please sign in to comment.