The use case is wanting to render a particular stored_file at a certain size and optimized for those dimensions. And also done in a way that minimizes the overhead on a plugin developer and at the same time doing it the Right Way (tm) so that it isn't polluting the file area's with confusing duplicated resized images.

There are a bunch of use cases for this, many of them boil down to performance like the filter_imageopt plugin, but I think this is a common enough use case that it should be a core api.

Ideally I would like to see some form of image optimization like imageopt built into core so it has deep proper support and is available in all places eg from themes and not just from a filter. But that can come later.

The existing file thumbnail preview code is very close to what is needed so I think it is a natural place to extend into a fully fledged image resizing and serving on demand feature.

There are many standalone projects offer this kind of service which I think we can learn from, in particular I think a really great project to take inspiration from is Thumbor:

http://thumbor.org/

I will link to the relevant thumbor docs for comparison.

So the proposal:

1) We extend the preview 'modes' to accept an arbitrary 'command' which in most cases is just an instruction to resize an image:

For instance for an image url such as:

/pluginfile.php/247/mod_label/intro/catan.jpg

We can currently get a square preview via:

/pluginfile.php/247/mod_label/intro/catan.jpg?preview=thumb

and I want to get any arbitrary resize via a command something like

/pluginfile.php/247/mod_label/intro/catan.jpg?preview=800x600

Or if we want to set just it's width or height but not both we could do either of these and it would calculate the other dimension for us:

/pluginfile.php/247/mod_label/intro/catan.jpg?preview=800x

/pluginfile.php/247/mod_label/intro/catan.jpg?preview=x600

There would be subtly different variants on the command to specify how you want the image cropped or not.

https://thumbor.readthedocs.io/en/latest/getting_started.html#changing-its-size

2) Thumbor has a very rich set of commands that you can send to crop, resize and a host of other filters. I'm not suggesting we even attempt to implement a fraction of what thumbor offers but it would be good if we employed a similar syntax for the commands so we could in future offer more commands, such as brightening an image or flipping it etc.

So in some future world we could do something like:

/pluginfile.php/247/mod_label/intro/catan.jpg?preview=800x600/filters:brightness(10):contrast(30)

https://thumbor.readthedocs.io/en/latest/filters.html

3) Apart from resizing the next most useful candidate for an image command to implement would be to optimize it in some way to reduce bytes over the wire, eg

/pluginfile.php/247/mod_label/intro/catan.jpg?preview=800x600/filters:quality(70)

https://thumbor.readthedocs.io/en/latest/quality.html

4) This service so far would be easy to abuse, eg a cpu DOS attack making the server do a ton of resizing work. Thumbor has solved this by using signatures derived from both the 'command' and some secret in the backend. I haven't "hashed" out the exact details but I'm pretty sure we could do something involving the contenthash and maybe throw in the siteid as a salt. If the signature doesn't match then it fails fast. eg:

/pluginfile.php/247/mod_label/intro/catan.jpg?preview=800x600/filters:....&previewsignature=abcdefgh

https://thumbor.readthedocs.io/en/latest/security.html

5) By doing this it would turn the url api into something which can only really be used from a php api. You'd give it a command and it returns the pluginfile url including the signature. Ideally this can be created without actually needing to do the file resize so its faster, as we might want to do a whole batch of them, and it also means we do the resizing work just-in-time in another process only when the image is actually needed.

I think this covers all of the use cases I can think of but if there is later valid use cases from js land this can be wrapped in an ajax webservice which has some form of protection in there from abuse.

https://github.com/99designs/phumbor#usage

6) Under the hood the resulting derived images will be stored exactly the same way as preview files are in an internal filearea separate from the original plugin file. When the original file is replaced or deleted the derived file will be cleaned up automatically and they will not clog up backups with extra files. (also we'd do this in a way which is performant to clean up, see also MDL-72082)

Because of the signature we should be able to tweak the cache headers to work better with cdns.

7) Bonus points the internal resizing api could attempt to try to find the best format to store the file in, eg it could do some tests and decide that a given image is better stored as jpeg or png. Double bonus points, the file serving endpoint could look at the accepts header from the browser and optionally serve a different file such as webp instead of the jpeg / png.

https://thumbor.readthedocs.io/en/latest/configuration.html#auto-webp

Resources on adaptive images:

https://danburzo.ro/responsive-images-html/