EWWW Bulk optimise about windows-azure-storage

[jeffpaul]

@jorenvh looks like we were both commenting at the same time there, sorry that my prior one looks out-of-context to your second comment. Are you able to confirm whether you can upload an image with just this plugin to see if it's a compatibility issue with EWWWIO or not?

[nosilver4u]

@jeffpaul EWWW IO should indeed work with the Azure storage plugin, but I've not tested that very recently.
Once an image has been optimized, we simply run wp_update_attachment_metadata() to let other plugins, like this one, take whatever action necessary. The main compatibility layer for EWWW IO is in being able to fetch images from Azure if they've been deleted from disk. We don't do anything special otherwise after optimizing an image.

[rickalee]

Current:

WordPress 5.3 & Azure Blob Storage wp_update_attachment_metadata

• Doesn't generate image sub-sizes

With Keep Images Local

• 504 error on upload but when you navigate to Media Library image successfully uploaded with sub-sizes

With EWWW and Keep Images Local

• Optimizes Image and uploads optimized on Azure Blob Storage

With Classifai

• With Keep Image Local - Image tags are added but not smart cropped
• Image doesn't get tags or smart cropped

---

Proposed Fix:

Change Azure Blob Storage to use wp_generate_attachment_metadata instead of wp_update_attachment_metadata

With wp_generate_attachment_metadata

• Images generate sub-sizes

With wp_generate_attachment_metadata & Keep Images Local

• Images upload without error and generate sub-sizes

With wp_generate_attachment_metadata & EWWW

• Bulk optimization works but stores files locally
• Image upload generates sub-sizes, EWWW adds back local image and optimizes local image but not Azure remote image

With wp_generate_attachment_metadata & Classifai

• With Keep Image Local - Image tags are added but not smart cropped
• Image doesn't get tags or smart cropped

Next steps: Change hook for initially uploading images to Azure Blob Storage to wp_generate_attachment_metadata, find core or EWWW hook to re-upload optimized image and delete the local files again if applicable.

[jorenvh]

I dug a bit deeper and found that the plugin is trying to push the optimised images back to Azure, however it receives an error response.

Error code 409 (conflict) with message "The blob type is invalid for this operation"

[jorenvh]

@jeffpaul thanks for getting back to me. Regular uploads are working as expected, it's the bulk optimise feature of EWWW that is giving issues. This feature allows to re optimise all existing images in the media library.

I added log statements through the code in the Azure storage plugin and I can see it being triggered when EWWW Bulk optimise is running so thats good. The response return from the blob storage however is invalid causing the optimised image not being pushed back to Azure

[nosilver4u]

On a side note, WP core has changed the way new uploads are handled in regards to wp_update_attachment_metadata(), you may want to make sure your plugin is compatible with that before Nov 12: https://make.wordpress.org/core/2019/11/05/use-of-the-wp_update_attachment_metadata-filter-as-upload-is-complete-hook/
That's from the rc4 notes: https://wordpress.org/news/2019/11/wordpress-5-3-rc4/

[rickalee]

Hi team,

Can you replicate the following behavior in WordPress 5.3?

If disabled:
Settings > Microsoft Azure > Keep local files after uploading them to Azure Storage.

WordPress fails to generate sub-sizes. I believe it may be related to this functionality and/or change.
https://core.trac.wordpress.org/ticket/47872

Suggestion: We may need to schedule a cron to delete local file vs doing in upload process.

cc @jeffpaul @oscarssanchez

[jeffpaul]

@nosilver4u from the notes in the EWWWIO docs, it sounds like you state that it supports Azure storage. Can you confirm if that's up to a certain version of the Windows Azure Storage plugin so we can help @jorenvh continue to troubleshoot his issue with EWWWIO and Azure not working together?

[jeffpaul]

@nosilver4u thanks for confirming those details, much appreciated!

@jorenvh are you able to update a single image on Azure, or do you get that invalid blob type error in a non-bulk image optimization as well?

[jorenvh]

@jeffpaul I'm not seeing the error when an image is uploaded, which I find a bit strange as the process is the same. The original image is pushed to Azure & later when EWWW is ready the optimised image is pushed to Azure and overwrites the initial image.
I'm not sure if I can trigger re optimisation for a single image, @nosilver4u how would I do this if it's possible?

Regarding the WP 5.3 will this plugin be compatible? @nosilver4u EWWW will be I guess as you shared the details here?

[nosilver4u]

You can do individual images via the list mode in the media library: https://docs.ewww.io/article/62-power-user-options-in-list-mode

The changes in 5.3 aren't necessarily breaking, it just might result in images being pushed to Azure multiple times during upload, depending on how the Azure plugin handles it currently. EWWW IO is fully compatible with WP 5.3.

[jorenvh]

Ha list mode...the only place I wasn't looking 🤣 thanks 👍

@jeffpaul running some tests right now

ps. please note Monday is a public holiday in Belgium, so sorry if there is any delay in my responses during the weekend & on Monday.

[jorenvh]

@jeffpaul first test conclusions when re optimising a single image:

• Invalid blob type error for older images
• Working for newer images

Will have to dig a bit deeper I guess. I don't know that much about blobs & blob storages etc, but is it possible that things changed a bit over time resulting in this error?

EDIT:
Quite some images are also showing "Could not retrieve file path." so I'm not even able to optimise those cc: @nosilver4u

I will dive further into this on Tuesday morning next week

[rickalee]

@jorenvh Could you verify the BlobType and LeaseState of those images that are returning InvalidBlobType (409 error) via Azure Storage Explorer? AppendBlob is default BlobType for purpose of this plugin and you could be running into permissions issues if LeaseState isn't 'Available' in older Blobs.

https://docs.microsoft.com/en-us/rest/api/storageservices/append-block

Azure Storage Explorer:
https://azure.microsoft.com/en-us/features/storage-explorer/

[jorenvh]

@rickalee sure

A newer image looks like this

A few notes to make:

• The older images are from the period before we even used the blob storage, so they where uploaded manually
• The last modified date is not representative, it's the date I synced the production container to staging

[jeffpaul]

@nosilver4u note that we're working on a release for this Microsoft Azure Storage plugin to be compatible with the changes in WordPress 5.3, this will likely entail a parallel update to EWWWIO so that both plugins and core function properly together. We'll keep you posted as we prepare our release so that we can see how the compatibility across these areas work and what tweaks may be needed.

[nosilver4u]

@jeffpaul sounds good, let me know if you need anything on our end with EWWW IO.

[jorenvh]

Hi @jeffpaul I was wondering if you have an ETA for the 4.4.0 release

[jeffpaul]

@jorenvh we're working through a couple separate fixes for other issues and I don't expect to get a release for those until the new year, which puts the fix for this a bit behind that. So with a little luck, maybe Q1 for a fix on this? Though if you're able to work on a PR based on @rickalee's proposed fix above, then we could get that through review and into the next pending minor release (with those other fixes I noted).

[JadoJodo]

Are there any updates on this issue? I'm still seeing it with current versions of both plugins.

[jeffpaul]

@JadoJodo there's no PR crafted yet that might solve this, the team is primarily focused on #136 for now, and our core team are currently diverted well into Q4 on other projects so its unlikely a fix will be released for this until 2023 at the earliest.