★ A package to create personal data exports

One of the good things that GDPR brought us was the right to data portability. Shortly put, this means that an app should be able to export all data that it has for a user.

Because we have multiple apps at Spatie that need to create such an export, we decided to extract our solution to a package called laravel-personal-data-export. In this blog post, I'd like to introduce the package to you.

Creating an export

Creating a data export can be done anywhere in your app by dispatching the CreatePersonalDataExportJob job.

// somewhere in your app

use Spatie\PersonalDataExport\Jobs\CreatePersonalDataExportJob;

// ...

dispatch(new CreatePersonalDataExportJob(auth()->user());

The package will create a zip containing all personal data. When the zip has been created a link to it will be mailed to the user. By default, the zips are saved in a non-public location, and the user should be logged in to be able to download the zip.

You can configure what data will be exported in the selectPersonalData method on the user model.

// in your User model

public function selectPersonalData(PersonalDataSelection $personalDataSelection) {
    $personalDataSelection
        ->add('user.json', ['name' => $this->name, 'email' => $this->email])
        ->addFile(storage_path("avatars/{$this->id}.jpg");
        ->addFile('other-user-data.xml', 's3');
}

You can add new files to it or copy over existing files (even from remote filesystems). All of this will be done via streams so big files won't pose problems at all.

The package also offers a command named personal-data-export:clean to clean up old data exports.

Some cool code tidbits

The package is built relatively straightforward, but it contains a few cool pieces of code.

Before zipping it, we copy all selected data to a temporary directory. If we're copying files from a remote filesystem we are going to use streams. Here's how that looks like in code (taken from PersonalDataSelection):

$stream = Storage::disk($diskName)->readStream($pathOnDisk);

$pathInTemporaryDirectory = $this->temporaryDirectory->path($pathOnDisk);

file_put_contents($pathInTemporaryDirectory, stream_get_contents($stream), FILE_APPEND);

Here's how we use Laravels storage fakes to test that code (taken from PersonalDataSelectionTest.php)

/** @test */
public function it_can_copy_a_file_from_a_disk_to_the_personal_data_temporary_directory()
{
    $disk = Storage::fake('test-disk');
    $disk->put('my-file.txt', 'my content');

    $this->personalDataSelection->addFile('my-file.txt', 'test-disk');

    $this->assertFileContents($this->temporaryDirectory->path('my-file.txt'), 'my content');
}

Here's how we stream the zip to the browser (taken from ZipDownloadResponse). This code works for both local and remote filesystems. Notice that we can use the Content-Disposition header to determine which filename the users downloading the zip will get in the browser.

class ZipDownloadResponse extends StreamedResponse
{
    public function __construct(string $filename)
    {
        $disk = Storage::disk(config('personal-data-export.disk'));

        if (! $disk->exists($filename)) {
            abort(404);
        }

        $downloadFilename = auth()->user()
            ? auth()->user()->personalDataExportName()
            : $filename;

        $downloadHeaders = [
            'Cache-Control' => 'must-revalidate, post-check=0, pre-check=0',
            'Content-Type' => 'application/zip',
            'Content-Length' => $disk->size($filename),
            'Content-Disposition' => 'attachment; filename="'.$downloadFilename.'"',
            'Pragma' => 'public',
        ];

        parent::__construct(function () use ($filename, $disk) {
            $stream = $disk->readStream($filename);

            fpassthru($stream);

            if (is_resource($stream)) {
                fclose($stream);
            }
        }, Response::HTTP_OK, $downloadHeaders);
    }
}

In this package I also found a good use case for the void typehint. Let take a look at the ExportsPersonalData interface:

namespace Spatie\PersonalDataExport;

interface ExportsPersonalData
{
    public function selectPersonalData(PersonalDataSelection $personalDataSelection): void;

    public function personalDataExportName(): string;

    public function getKey();
}

That void typehint communicates to the user that nothing should be returned from selectPersonalData. This will make it more clear that the given $personalDataSelection is mutable and we except selection methods to be called on that instance.

In closing

Our package has a few more options not mentioned in this blog post. To learn them, head over to the readme of the package on GitHub.

I hope this package will come of us in your next project. Be sure also to take a look at the list of packages our team has created previously.