Configure LiipImagineBundle
to create thumbnails of images stored on Amazon S3 and save a cache version of them again on S3 may be really painful.
In this post I’ll try to guide you step by step in the configuration process, to have a full configured data flow to create thumbnails stored on Amazon S3 using LiipImagineBundle
.
Which is our goal
Our goal is to configure the following data flow:
- Get a source image file from AWS S3 (from the
[bucket_name]/images
folder); - Manipulate it in some ways with
LiipImagineFilters
(for example, thethumbnail
filter); - Save the manipulated image again to AWS S3 (into the
bucket_name/cache
folder).
As the configuration of the full data flow could be difficult and prone to errors, we will use a simple configuration to reduce to minimum possibilities of issues.
So we will try to get an already uploaded image from the images
folder in our bucket on S3 (hardcoding the path in the Twig template), resize it applying custom filters test_medium_thumb
and test_small_thumb
(that we will configure), then we will save again the resized image in the cache
folder into same bucket on S3.
Before starting with the real configuration of the LiipImagineBundle
we have to prepare our test bucket and our test route
to simplify things.
We will, in fact, create a dedicate action
in a controller
to test in the tiniest way our filters. This will help us to test in a simpler manner the funcionalities of LiipImagineBundle
.
Install the required bundles: LiipImagineBundle
and KnpGaufretteBundle
To make all things work we need to install and activate, besides LiipImagineBundle
, also KnpGaufretteBundle
.
So, install and activate LiipImagineBundle
and install and activate KnpGaufretteBundle
.
Gaufrette is a PHP5 library that provides a filesystem abstraction layer.
Gaufrette is required to make LiipImagineBundle
to get and save files from and to Amazon S3 buckets.
For the moment believe me and install the bundles: continuing in reading of this post you’ll better understand why we need also KnpGaufretteBundle
.
Prepare our app for the test
Now that we have installed and activated our required bundles, lets prepare our app to test the data flow:
- Create in your bucket the folder
images
; - Into this
images
folder upload a big image (I used this); - Create a
cache
folder in your bucket; - Create a
DefaultController::TestThumbAction
in yourDefaultController
(or in another controller: this action will be used to just render the test image and its thumbnail) that return the path of the just MANUALLY uploaded test image:
/**
* @Route("/test_thumb", name="thumb")
* @Template()
*/
public function testThumbAction()
{
$image = 'https://s3-'.
$this->getParameter('amazon.s3.region').
'.amazonaws.com/'.
$this->getParameter('amazon.s3.bucket').
'/images/symfony2.png';
return [
'image' => $image
];
}
- Create the template for the
DefaultController::TestThumbnailAction
:
<h1>Test resized with <code>test_small_thumb</code></h1>
{# NOTE THAT FOR THE MOMENT THE THUMBNAIL IS COMMENTED #}
{# <img src="{{ image | imagine_filter('test_small_thumb') }}" alt="" /> #}
<h1>Test resized with <code>test_medium_thumb</code></h1>
{# NOTE THAT FOR THE MOMENT THE THUMBNAIL IS COMMENTED #}
{# <img src="{{ image | imagine_filter('test_medium_thumb') }}" alt="" /> #}
<h1>Original Image</h1>
<img src="{{ image }}" alt="" />
Now go to http://127.0.0.1:8000/test_thumb
and check that the test image is loaded correctly.
If the test image is not loaded, may be you have to set it as public
on AWS S3 (select the image and in the Actions
dropdown select Make Public
).
Once you can see the test image in its original size, remove comments from the image with test_small_thumb
filter applied (only from this for the moment!) and reload the page.
You’ll see this Twig_Error_Runtime
exception
An exception has been thrown during the rendering of a template (“Could not find configuration for a filter: test_small_thumb”) in src/AppBundle/Resources/views/Default/testThumb.html.twig at line 2.
500 Internal Server Error – Twig_Error_Runtime
1 linked Exception: NonExistingFilterException »
Ok, we are now ready to start configuring LiipImagineBundle
to create our thumbnails of images stored on Amazon S3 and save the cached thumbnail again on Amazon S3. Lets start!
Understand the dataflow of LiipImagineBundle
The dataflow followed by LiipImagineBundle
is described in the documentation. I suggest you to read it to fully understand what we need to make the thumbnails being generated and saved to S3.
The short story is this:
- STEP 1:
LiipImagineBundle
retrieve the original image from S3 using aDataLoader
. OurDataLoader
is theStreamLoader
(that uses Gaufrette to work). - STEP 2: The source image is then processed with filters.
The filters we will configure are
test_medium_thumb
andtest_small_thumb
. - STEP 3: The resulting image file is then saved (cached) to S3 using a
CacheResolver
. TheCacheResolver
we’ll use is theAwsS3Resolver.
STEP 1: Get the image source file from AWS S3 with Gaufrette
So, as told, the first step is to get the source image from the images
folder in our bucket on AWS S3.
To do this, LiipImagnineBundle
requires a DataLoader
.
LiipImagnineBundle
comes with some bundled DataLoader
s. The default DataLoader
is the FilesystemLoader
as described also in the Configuration chapter of the documentation.
To load images from Amazon S3, we need another DataLoader
, the StreamLoader
:
The
LiipImagineBundle Documentation, Stream LoaderLiip\ImagineBundle\Binary\Loader\StreamLoader
allows to read images from any stream (http, ftp, and others…) registered thus allowing you to serve your images from literally anywhere.
The easier way to use this loader is configuring it to use Gaufrette.
So we have to first configure Gaufrette and then use it with the LiipImagineBundle
‘StreamLoader
.
Understand how Gaufrette works
At this point Gaufrette should be already installed as you should have already installed and activated KnpGaufretteBundle
.
But, before continuing, just a little bit of theory to understand how Gaufrette works.
As told before,
Gaufrette is a PHP 5.3+ library providing a filesystem abstraction layer. This abstraction layer allows you to develop applications without needing to know where all their media files will be stored or how.
In other words, you interact with Gaufrette’s API that emulates actions you can take on a filesystem to manage files, so, when you decide to switch from the local filesystem to a cloud environment, you have to simply change some parameters in the configuration. Clean, simple and easy!
If you like, read some more details on the Knp Labs’ blog.
Anyway, to make your application “storage agnostic”, Gaufrette uses two main concepts: Adapters
and Filesystems
.
Your application will ever communicate with a Filesystem
object. An Filesystem
object uses a Adapter
to communicate with the “storage” and perform the real actions on it.
Here the details about this simple process.
Introducing Gaufrette StreamWrapper
To make LiipImagineBundle
StreamLoader
work with images we need to understand another concept used by Gaufrette, the last one: StreamWrapper
.
As we don’t simply want to interact with the filesystem, but we want also to manipulate images, we need a StreamWrapper
that we’ll then use as DataLoader
in LiipImagineBundle
.
A StreamWrapper
permits to register a new stream wrapper so we’ll can get the contents of the image and use them to create a new image.
Configuring KnpGaufretteBundle
to work with AWS S3
So, after the theory, finally the practice! Lets configure KnpGaufretteBundle
.
So, we need to configure those three things:
- An
Adapter
: we’ll use theAwsS3Adapter
- A
Filesystem
that uses theAwsS3Adapter
: we’ll call itfilesystem_aws_S3_images
- A
StreamWrapper
(we don’t have to do anything more than writing a directive in ourconfig.yml
file to configure it)
Configure the AwsS3Adapter for KnpGaufretteBundle
The AwsS3Adapter
needs the credentials to access the bucket on Amazon AWS S3, so, we first configure these credentials, then we’ll configure the S3Client
(that uses the credentials):
shq.amazon.s3Credentials:
class: Aws\Credentials\Credentials
arguments: ["%amazon.s3.key%", "%amazon.s3.secret%"]
shq.amazon.s3:
class: Aws\S3\S3Client
arguments:
- version: %amazon.s3.version%
region: %amazon.s3.region%
credentials: "@shq.amazon.s3Credentials"
Now, we can configure the AwsS3Adapter
, the Filesystem
and the StreamWrapper
:
# config.yml
# KnpGaufretteBundle Configuration
knp_gaufrette:
adapters:
adapter_aws_s3_images:
aws_s3:
service_id: "shq.amazon.s3" # ! ! ! Without @ ! ! !
bucket_name: "%amazon.s3.bucket%"
options:
directory: 'images'
create: true
filesystems:
filesystem_aws_s3_images:
adapter: adapter_aws_s3_images
stream_wrapper: ~
As told in the documentation,
The
GaufretteBundle Documentation, Stream Wrappersstream_wrapper
settings allow you to register filesystems with a specified domain and then use as a stream wrapper anywhere in your code like:gaufrette://domain/file.txt
Testing that filesystem_aws_s3_images
works
Now that we have configured our Adapter
and our Filesystem
, we should test that it really works as expected.
Once configured, Filesystem
s are accessibile from the container, so your DefaultController::testThumbAction()
write the following code:
/**
* @Route("/test_thumb", name="thumb")
* @Template()
*/
public function testThumbAction()
{
$filesystem = $this->get('knp_gaufrette.filesystem_map')->get('filesystem_aws_s3_images');
$file = $filesystem->get('symfony2.png');
/*
* Note the use of the dump() function.
* If you don't have the VarDumperComponent installed, use var_dump().
* @see http://symfony.com/doc/current/components/var_dumper/introduction.html
*/
dump($file);die;
$image = 'https://s3-'.
...
}
If the dump()
shows you a Gaufrette\File
object, then… CONGRATULATIONS! Gaufrette is correctly configured!
As we know that Gaufrette works, we can now use it with LiipImgineBundle
!
Using Gaufrette as DataLoader
for LiipImagineBundle
As told, we need a DataLoader
to make LiipImagineBundle
get the real content of the file so it can create a thumbnails.
Now that we have a working Gaufrette, we need to use it to configure the StreamLoader
.
We have to alternatives to configure a DataLoader
: using the factory and using a defined service.
We’ll use the factory method, so we have all configuration in one place (I find this more comfortable, but you can use the method you like).
So, to define the DataLoader
using the factory method, put this in your config.yml
file:
liip_imagine:
loaders:
loader_aws_s3_images:
stream:
# This refers to knp_gaufrette filesystems configuration
wrapper: gaufrette://filesystem_aws_s3_images/
Perfect: we have done. Now we have to configure custom filters.
STEP 2: Define filters to resize images and create the thmbnails
Now that we can get images from AWS S3, it’s time to define our custom filters to manipulate them.
We will create two filters: test_medium_thumb
and test_small_thumb
.
To define filters we have to edit the config.yml
file:
liip_imagine:
loaders:
...
filter_sets:
test_medium_thumb:
data_loader: loader_aws_s3_images
# We don't yet have a cache resolver configured
cache: cache_resolver_aws_s3
quality: 75
filters:
thumbnail: { size: [400, 400], mode: outbound }
test_small_thumb:
data_loader: loader_aws_s3_images
# We don't yet have a cache resolver configured
cache: cache_resolver_aws_s3
quality: 75
filters:
thumbnail: { size: [100, 100], mode: outbound }
Done! Really simple, doesn’t it?
But, as you can see, we have the directive cache
in each of the two defined filters: these directives use the CacheResolver
called cache_resolver_aws_s3
.
So, as it may be yet clear, we have to define the cache_resolver_aws_s3
. We are fastly moving toward the STEP 3!
STEP 3: Define the CacheResolver
AwsS3Resolver
As we want to save our generated thumbnails to AWS S£, we need to use the CacheResolver
AwsS3Resolver
.
In this case, too, we can decide to define our resolver as a service or using the factory.
In this case, too, I’ll use the factory, but you can choose to use the method you like.
So, in the config.yml
file:
liip_imagine:
loaders:
...
resolvers:
cache_resolver_aws_s3:
aws_s3:
client_config:
credentials:
key: %amazon.s3.key%
secret: %amazon.s3.secret%
region: %amazon.s3.region%
version: %amazon.s3.version%
get_options:
Scheme: 'https'
put_options:
CacheControl: 'max-age=86400'
filter_sets:
...
Test that all works well
And now that we have all configured, it’s time to test!
Edit DefaultController::TestThumbAction
First: change the code into the action:
/**
* @Route("/test_thumb", name="thumb")
* @Template()
*/
public function testThumbAction()
{
//$filesystem = $this->get('knp_gaufrette.filesystem_map')->get('filesystem_aws_s3_images');
//$file = $filesystem->get('symfony2.png');
/*
* Note the use of the dump() function.
* If you don't have the VarDumperComponent installed, use var_dump().
* @see http://symfony.com/doc/current/components/var_dumper/introduction.html
*/
//dump($file);die;
/*$image = 'https://s3-'.
$this->getParameter('amazon.s3.region').
'.amazonaws.com/'.
$this->getParameter('amazon.s3.bucket').
'/images/symfony2.png';*/
$image = 'symfony2.png';
return [
'image' => $image
];
}
Activate filters in the template
Now activate the filters in the template removing comments:
<h1>Test resized with <code>test_small_thumb</code></h1>
<img src="{{ image | imagine_filter('test_small_thumb') }}" alt="" />
<h1>Test resized with <code>test_medium_thumb</code></h1>
<img src="{{ image | imagine_filter('test_medium_thumb') }}" alt="" />
<h1>Original Image</h1>
<img src="{{ image }}" alt="" />
Test!
Reload your page at http://127.0.0.1:8000/test_thumb
and, if you’ve done well all the steps, you’ll see something like this:
Obviously, “Original image” is no longer visible as the path
isn’t correct (we commented it in the controller!).
Done: now you have LiipImagineBundle
up and running. You can now save your images to Amazon AWS S3, get them and resize to thumbnail sizes and save them again on AWS S3!
Some things to note: what happened behind the scenes
The first thing you should note is the URLs of the thumbnails.
The first time you load the page the resulting URLs are those:
<h1>Test resized with <code>test_small_thumb</code></h1>
<img src="http://127.0.0.1:8000/media/cache/resolve/test_small_thumb/symfony2.png" alt="" />
<h1>Test resized with <code>test_medium_thumb</code></h1>
<img src="http://127.0.0.1:8000/media/cache/resolve/test_medium_thumb/symfony2.png" alt="" />
<h1>Original Image</h1>
<img src="symfony2.png" alt="" />
As you can note, they are in this form: media/cache/resolve/[filter_name]/
.
Try to reload the page.
The resulting URLs, this time, are those:
<h1>Test resized with <code>test_small_thumb</code></h1>
<img src="https://s3-eu-west-1.amazonaws.com/trustback-me-dev/test_small_thumb/symfony2.png" alt="" />
<h1>Test resized with <code>test_medium_thumb</code></h1>
<img src="https://s3-eu-west-1.amazonaws.com/trustback-me-dev/test_medium_thumb/symfony2.png" alt="" />
<h1>Original Image</h1>
<img src="symfony2.png" alt="" />
They refers to the public URL on AWS S3: https://s3-eu-west-1.amazonaws.com/[bucket_name]/[filter_name/
.
In fact, the first time you load the images, if LiipImagineBundle
cannot find a cached version, points to a controller defined in vendor/liip/imagine-bundle/Resources/config/routing.xml
The URL you see the first time points to the controller that applies the filters. Once the resulting image is cached, the next time LiipImagineBundle
uses directly the URL of the bucket on AWS S3.
The second thing you should note, is that LiipImagineBundle
automatically creates a folder for each filter in your bucket:
In practice, for each configured filter, the resulting images are saved into a dedicated folder.
Troubleshooting
And finally, some advices to debug in case of issues:
- Keep attention to differences between UPPERCASE and lowercase letters: maybe you can write
filesystem_aws_s3_images
in one place andfilesystem_aws_S3_images
in another place. Spot the difference! - As the bundle is usually usedfrom inside a template, you’ll don’t get exceptions or errors. If something doesn’t work, go to the logs! There there are all the errors: find them looking for strings like “gaufrette” or “liip_imagine” or “liip/imagine-bundle” or “request.ERROR” or “php.DEBUG” and you’ll find all the info you need to understand what went wrong.
And finally, before to close this long post, remember that in the Issues section on GitHub there are a lot of useful information to better understand how to use the bundle.
Here are a couple of interesting issues:
Remember to “Make. Ideas. Happen.”.
I wish you flocking users, see you soon!
L'articolo How to use LiipImagineBundle to manage thumbnails through Amazon S3 proviene da ÐΞV Experiences by Serendipity HQ.
Top comments (0)