thaiphan / magento2-s3 Goto Github PK

I have 2 stores on the Magento install and the base media URL is only working for one of the stores. Does this extension work with multiple stores? The main store serves up media files from the S3 bucket but the other store does not.

After deleting images will regenerate

HI Just want to know, the setting customer headers like expiry etc is very common.
how can I do that using this extension? If not, can you please guide me for the piece of the code that is uploading the file, I'll do necessary changes and submit a pull request.

Hi, I have installed the package but when I try to enable the storage via s3:storage:enable, it gives me the following error
Class 'Aws\S3\S3Client' not found in /var/www/html/app/code/Thai/S3/Console/Command/StorageEnableCommand.php:52

Hi Thai,
I have install module using given steps. Magento2 admin showing everything about S3 module like AccessKey etc. But while select storage Amazon S3 then clicking synchronise , nothing happen(We were unable to synchronize one or more media files. Please refer to the log file for details). And also images are not store in S3 bucket while adding product. it there anything to do.
Please help .

Hi Thai,
My installation done what next? because all images in S3 but when i checked my url in pointed to my local media.

thanks,

I have a magento (v2.2.3) website and using "thaiphan/magento2-s3" extension for uploading images to s3. Now I am facing an issue with configurable product images uploading.

For configurable variations, the image is only saving for only one size. I have attached the step by step with images. In the last step the images are hidden.

Step1 : Selected two attributes for color and size.

Step2 : Selected two color and two size options for variations

Step3 : Selected 'Apply unique images by attribute to each SKU' and uploaded images for color attribute for two variations

Step4: Two images are applied for 4 variations. ie two colors for each sizes. Everything is fine. Then clicked "Generate products"

Step5: Generated products are successfully showing with correct images and sizes. Then I clicked "Save"

Step6: Here is the issue. when I saved the product only two images are showing. That is the images are only available for first two sizes. That is for size 40.

I think it is actually saving but not showing. Because when I try to edit that configurable variation, that image place holder is there.

Any idea for this issue. It is working perfectly when upload images to the same server. Only s3 has this issue.

Hello,
I am using this https://github.com/thaiphan/magento2-s3 module and now i need to upload new custom folder images like pub/media/custom_folder_name to amazon so, What can i do?

Can you please explain me?

I have installed the extension and configured the configurations below.

1. Access Key
2. Secret Key
3. Bucket
4. Region

Also, I have also installed the AWS Library using composer require aws/aws-sdk-php and the version of AWS library is 3.0.

After configured the extension, I have tried to run the enable / export command but I am always getting the error below.

"The AWS credentials you provided did not work. Please review your details and try again. You can do so using our config script."

i have configured the s3 with magento but getting below error when running below command :
Command:
php bin/magento s3:storage:export

Error:
The AWS credentials you provided did not work. Please review your details and tr
y again. You can do so using our config script.

I installed the extension and configured it. No error messages come up and when I run php bin/magento s3:storage:export it appears to be uploading everything however nothing uploads to the S3 bucket and when it is enabled, the images are still brought in from within the media folder.

This extension doesn't work

The happened when I want to use web setup wizard to do module installation, the page stay infinity loading, and when you open up the inspector, it will show 500 server error:

angular.js:7715 GET http://somedomain.com/setup/index.php/componentGrid/components 500 (Internal Server Error)

If you click on the link it will show this error:

An error occurred during execution; please try again later.

Additional information:

RuntimeException

File:
/var/www/html/setup/src/Magento/Setup/Model/PackagesData.php:88
Message:
Couldn't get available versions for package thaiphan/magento2-s3
Stack trace:
#0 /var/www/html/setup/src/Magento/Setup/Controller/ComponentGrid.php(98): Magento\Setup\Model\PackagesData->syncPackagesData()
#1 /var/www/html/vendor/zendframework/zend-mvc/src/Controller/AbstractActionController.php(82): Magento\Setup\Controller\ComponentGrid->componentsAction()
#2 [internal function]: Zend\Mvc\Controller\AbstractActionController->onDispatch(Object(Zend\Mvc\MvcEvent))
#3 /var/www/html/vendor/zendframework/zend-eventmanager/src/EventManager.php(444): call_user_func(Array, Object(Zend\Mvc\MvcEvent))
#4 /var/www/html/vendor/zendframework/zend-eventmanager/src/EventManager.php(205): Zend\EventManager\EventManager->triggerListeners('dispatch', Object(Zend\Mvc\MvcEvent), Object(Closure))
#5 /var/www/html/vendor/zendframework/zend-mvc/src/Controller/AbstractController.php(118): Zend\EventManager\EventManager->trigger('dispatch', Object(Zend\Mvc\MvcEvent), Object(Closure))
#6 /var/www/html/vendor/zendframework/zend-mvc/src/DispatchListener.php(93): Zend\Mvc\Controller\AbstractController->dispatch(Object(Zend\Http\PhpEnvironment\Request), Object(Zend\Http\PhpEnvironment\Response))
#7 [internal function]: Zend\Mvc\DispatchListener->onDispatch(Object(Zend\Mvc\MvcEvent))
#8 /var/www/html/vendor/zendframework/zend-eventmanager/src/EventManager.php(444): call_user_func(Array, Object(Zend\Mvc\MvcEvent))
#9 /var/www/html/vendor/zendframework/zend-eventmanager/src/EventManager.php(205): Zend\EventManager\EventManager->triggerListeners('dispatch', Object(Zend\Mvc\MvcEvent), Object(Closure))
#10 /var/www/html/vendor/zendframework/zend-mvc/src/Application.php(314): Zend\EventManager\EventManager->trigger('dispatch', Object(Zend\Mvc\MvcEvent), Object(Closure))
#11 /var/www/html/setup/index.php(31): Zend\Mvc\Application->run()
#12 {main}

Any Help?

I follow all steps of installation.And create one product with images but images still upload in local pub/media folder not on s3 bucket.

Hi I got this error message when I try to upload a new image after the installation.

Kudos to you for this plugin.
I followed all your instructions and have been able to upload all the required files to my s3 server. I have also enable the storage for s3 in magento2.
Still, all my images are currently coming from my own server and not from S3.. although everything is synced.

How can i point all the images to come from S3 rather than my server

Thanks

You have not provided an S3 region. You can do so using our config script.

Getting the above error

Hi,

I'm having an issue uploading image files for swatch colours when using the plugin. Uploading the file doesn't seem to work / save.

Disabling the plugin allows me to upload the file, although I now have synch issues as it's looking for it at cdn.mysite.com/attribute/swatch

I've had the problem on Magento 2.1.7 - 9.

Thanks

Andrew

Hello,

I have installed the extension using your instructions. My configuration is correct and according to the output, I was able to connect using my credientals. My credentials have been tested with other tools (including a windows s3 browser) and work with no issues. I have my file synced.

I have selected the "Amazon S3" option under
Stores->Configuration->Storage Configuration for Media

Important: I understand that Stores->Configuration->General->Base Media URL needs to be changed to the s3 bucket path, making this change only changes the src on the img tag and is not a resolution to this issue. For explanation purposes, I have set the base media path to my local server storage. To confirm, I have tried the steps both ways, with no change in the outcome.

Please confirm that my expected outcomes are in sync with the functionality your plugin provides.

Expected Outcomes

1. Using the upload option within the WYSIWYG would upload the images directly to the S3 bucket
2. Any media file browser in the admin panel will show the directory structure of the S3 bucket

Steps to reproduce:

1. Open Content -> Pages
2. Select or create a CMS page
3. Navigate to the "content" area that displays the WYSIWYG
4. Click the "Insert/Edit Image" menu option (icon is picture of a tree)
5. In the popup, click the "Browse" icon next to the "Image URL" field

Within this window, you'll notice the two issues:

• The directory structure show is that of the local server storage and not the s3 bucket
  

• Using the upload function, uploads images to the local server storage and not the s3 bucket
  

Can give some example or which files I should refer if I want to recall the function in another module(e.g slider module)?

HI,

I installed your extension, and my media files are now also on the s3. It works real good but Magento still uses the local files. How do I make my installation use the images on the S3?

Hi Thai,

I guess you missed the installation guide for this module. it redirects to the same github page.

While deleting media from system (magento 2) and saving it, thaiphan extension doesn't remove it from amazon S3. Can you please provide solution for that.

Hi, First, I would like to give you my thanks for this great extension.

But I found an issue with Magento 2.1.10 version, and realized that is an issue with all Magento versions 2.1.*.

Step to reproduce:

1. Install this extension into Magento 2.1.* version
2. Run commands and on setup:di:compile you will get the error

[ReflectionException]
Class Magento\Framework\Serialize\Serializer\Json does not exist

Basically into the class Thai\S3\Model\MediaStorage\File\Storage\S3, is trying to use a class that does not exist on those Magento versions.

So, I added support for this, the fix is into the PR #46

Thank you very much.

Regards.-

Logo and Favicon not uploaded to S3

How can we check applied credential is valid or not. when I trying php bin/magento s3:config:set command it showing "s3:config:set [--access-key-id[="..."]] [--secret-key[="..."]] [--bucket[="..."]] [--region[="..."]]" but already I have configured through the admin panel.

As well I triggered php bin/magento s3:storage:export and php bin/magento s3:storage:enable command is not responding any kind of success message.

Hi @thaiphan, I have a problem when I create new configurable product in Magento2.1.12 , while I try to add simple product to the product and add image for all simple (Select option Apply single set of images to all SKUs) and save product. The images only show in one simple product and others don't show image. I found that the image path of simple product that don't show image, it has generated suffix (Example a_1, a_2) and the image with that path doesn't exist in S3.

How I can resolve this problem ?

Thank you.

hello, I have installed this module, and went through installation steps, 1, 2, 3.

At step 4, I am trying to add my bucket name, but it's not saving, i get the following error:
Invalid Base URL. Value must be a URL or one of placeholders: {{base_url}}

Magento version: 2.2.2
PHP version: 7.1

Any ideas ?

Hi,

I execute the bin/magento s3:storage:export
and got this error.
{"exception":"[object] (InvalidArgumentException(code: 0): Unable to unserialize value, string is corrupted. at /var/www/html/stg/vendor/magento/framework/Serialize/Serializer/Serialize.php:38)"} []

Hi,
Hope you are doing well.
I am using this extension for my client. This extension is working fine for catalog and product images. But at magento 2 customer registration form when I use captcha, captcha image path is taken from s3 but captcha image is creating at native file system.
Kindly advise me at this issue.
Looking forward for your response.
Regards,
Ijaz Ali

Hey,

I have been running into an issue with this plugin. It gets installed and everything is fine and alright. Problem is when i open a product page, i sometimes get an empty page with an image in between, specially on chrome browser.
i get the following in console.
Resource interpreted as Document but transferred with MIME type image/jpeg:

I tried disabling all the extension i have installed one by one.. and this is the one which was causing the issue

Can you please let me know where i went wrong. I am currently debugging it myself as well.

I will let you know if i come up with something

Thanks

Hi, i could integrate the media files with S3 and i saw on the frontend that the products images are loaded thought S3 but when i delete the web server pub/media folder the product move to show the placeholder default image of magento

Before delete pub/media:

After delete pub/media:

What am i doing wrong? i want to delete all web server local image storage

Hello,

I have properly setup policy for my IAM user according to this page: https://aws.amazon.com/blogs/security/writing-iam-policies-how-to-grant-access-to-an-amazon-s3-bucket/.

However, when I run export command, it all goes well, it prints all files to upload, e.g.
Uploading import/product.jpg to use S3.
But, I can't see any of them on AWS S3 console, neither I can GET any of them (from Postman) using same credentials.

Do you have a clue?

Hi @thaiphan , I had no problems installing the extension, in fact, the images are loaded in the bucket of s3, but also loaded on the server, something I want to avoid with this extension.
I installed something wrong? Or the extension works like this?
I upload the logs:
`ubuntu@magento:/var/www/html/nuevo$ sudo php bin/magento s3:storage:export
You are already using S3 as your media file storage backend!
ubuntu@magento:/var/www/html/nuevo$ sudo php bin/magento s3:storage:enable
You are already using S3 as your media file storage backend!
ubuntu@magento:/var/www/html/nuevo$ sudo /etc/init.d/apache restart
sudo: /etc/init.d/apache: command not found
ubuntu@magento:/var/www/html/nuevo$ sudo /etc/init.d/apache2 restart
[ ok ] Restarting apache2 (via systemctl): apache2.service.

but

What happens if I delete the theme and catalog folders from the ssh? How do I know if the images are loaded from s3?

I want Magento not to download files to the file system, how can I configure the static media URL to point to S3?

Using this extension if we directly use s3 bucket images file then why it will regenerate all images in local server means ( in current site's server ) ? On refresh the page all product images file re-created in my server , images file is generate on s3 bucket is okay but why its generate in server side also ?

Hi Thai,

I have setup S3 properly and image are auto syncing in to S3 bucket but images are also uploading in magento media folder which must not be .

kindly help for the same.

Hello,

I did the installation process (https://github.com/thaiphan/magento2-s3/wiki/Installation) and it went smooth, everything was uploaded on S3 and even if i upload a new image I see it on s3 but the images are not working from s3 on the frontend, it tries to load them from

get.php/catalog/product/cache/f073062f50e48eb0f0998593e568d857/w/h/wh09-purple_main.jpg

I've also tried to update Base URL for User Media Files poiting to s3 but it still loads from get.php

I tried cleaning all caches and everything.

What can I check more?

Thanks

Hi,
Thanks for making this module available. I installed it correctly. And, I was able to export media files to S3 using bin/magnento s3:storage:export I am using CDN to serve media (with S3 as backend) and I have made changes to media url accordingly in store configuration. Yet, I am facing a strange issue when I enable the module using bin/magento s3:storage:enable None of the media files are served from CDN, instead, magento is trying to look for media at this link http://local.magento.com/get.php/wysiwyg/home/home-main.jpg which is causing 404s. Expected link is http://<my_cdn_url>/wysiwyg/home/home-main.jpg

I am getting this error message "Fatal error: Uncaught Error: Class 'Aws\S3\S3Client' not found:" when I ran this command "php bin/magento s3:storage:export". From where I can get the Aws\S3\S3Client file and in Magento2 where I can deploy this file.

Thanks in advance
Inderjeet

Can I use AWS IAM instance profile instead of using access key and secrets? Also is it possible to set this value using env. I am trying to run magento using docker, looks like we need to store aws credentials in docker image itself.

Hi,

First of all thanks for creating the extension, it's amazing.

Want to know if there is a capability in the assign custom folder of S3 bucket to upload the images into.
For example, I have an S3 bucket name s3sharednewdev inside which I have a folder magento-stg in which I would like to see the images from pub/media get uploaded. So my upload path becomes: s3sharednewdev/magento-stg

Is there a configuration where I can do this ? or is it something the module does not support at this point of time?

Thanks

Hello,

When I installed the plugin via composer I got an exception.

Could not find package thaiphan/magento2-s3 at any version for your minimum-stability (alpha). Check the package spelling or your minimum-stability

Probably it's something wrong in configuration php package manager or installation instruction ;)

We (unfortunately) are not using composer to install modules. Is there a way to install easily by hand?

I followed the instructions on the wiki and tested product image by adding a new product. The image loads from my s3 bucket successfully. All other existing images appear to have worked.

My problem is regarding captcha. I'm using captcha on my user registration form and the image link appears broken at all times. I checked the source of an image that was broken and it appears in my pub/media/captcha/base folder on my server, however new images do not go to s3.
EG pub/media/captcha/base/c6c8cc5ca22a308a0a4429b1f4d93fc6.png appears, but only old captcha images that existed when I ran s3:export are in my s3 bucket.

I'm not sure how this is supposed to work but it seems to be broken to me right now. Any suggestions?

every thing has been configured but when i will upload any product image in s3 i will get the file content as like below:

what was this how i will configured it

0fb2c48877f209ff5adf2017a08d6ad947d032067f56a54fcc0611757fe5ccd5 nextdoorhub [15/Nov/2017:06:37:49 +0000] 111.93.168.150 0fb2c48877f209ff5adf2017a08d6ad947d032067f56a54fcc0611757fe5ccd5 2147406AC1F35F66 REST.HEAD.BUCKET - "HEAD /nextdoorhub HTTP/1.1" 200 - - - 16 15 "-" "S3Console/0.4, aws-internal/3" -
0fb2c48877f209ff5adf2017a08d6ad947d032067f56a54fcc0611757fe5ccd5 nextdoorhub [15/Nov/2017:06:37:49 +0000] 111.93.168.150 0fb2c48877f209ff5adf2017a08d6ad947d032067f56a54fcc0611757fe5ccd5 A2BCA2C6FD76E003 REST.GET.VERSIONING - "GET /nextdoorhub?versioning= HTTP/1.1" 200 - 162 - 7 - "-" "S3Console/0.4, aws-internal/3" -

Hi,
My product images are not loading from s3 bucket via cloud front url, instead it is loading from my pub media folder of orgin domain.

My product images are loading from htt://mydomain.com/pub/get.php/catalog/product/cache/75eed2686e01eb22cb4050b2f40ddf97/d/i/dish_1.png
I need to get from cloud front http://cloudfront.net/catalog/product/cache/75eed2686e01eb22cb4050b2f40ddf97/d/i/dish_1.png

Thanks in advance....

Hello Thaiphan,

new sizes of product images are not being uploaded, ie. folder 64x64 in resized folder is not made.

Bests,
Honza

Hi ,

1. many sites have some other static files(like product attachments, 360view html) ,not only images.
   Will you add the feature that user can decide which folder can be uploaded to S3?
2. Let user decide to delete files after files have been synced to S3. Sometimes we don't know which file has been uploaded ,which no.

Thank you.

After installing this plugin, the component manager is broken ( keeps waiting unlimited time )
When removing the upper layer ( inline edit style ) and click the sync button manualy, resluts in : Couldn't get available versions for package thaiphan/magento2-s3

Guess why that is why the component manager waits unlimited time with wait cursor....
Can it be fixed?

Magento 2.1.7

I have to admit that I am not the most experienced at using S3, but I have used it before this, and got things working on a totally different platform just fine. It's just not working here.

When I set the config, it works fine, but when I go to export, I get an error saying "The AWS credentials you provided did not work. Please review your details and try again. You can do so using our config script."

So, I ran php bin/magento s3:config:list, and everything appears fine. Since I had never used these particular credentials before, I tried connecting to another bucket that is currently in use on another site, with the credentials that are currently working for that site, and the same thing happens, so it isn't because the credentials are wrong. The only difference between one setup and the other, as far as credentials are concerned, is that on my other site, the region is not required, but I'm sure that I am using the correct region.

I don't know how to get more information about the error, or what I may be doing wrong.

Hi,

I have a problem with Magento 2.1.8 during the install of MAgento when I have this module installed.
Basically the module add some cli command but not properly and so the install doesn't work.
I had the same kind of issue with the table "store_website" and here is the result of my investigation :

it occurs when you have added any new CLI command that ask (directly or not) for Magento\Framework\Locale\ResolverInterface.
This Locale Resolver try in its constructor to init the local to use from the current website, but it makes a fall-back the default website if no website is already loaded.
So, if any class ask for this Locale Resolver, it will directly try to load the default website when using the Magento CLI.
But on the install, this default website does not exists yet => You will have the error.
In my case, I was asking for Magento\Customer\Api\CustomerRepositoryInterface.
So what you have to do, is to identify the asked class that indirectly ask for the Local Resolver, and to use its Factory instead.

Here is the diff I used on my issue, in order to help you :

diff --git a/app/code/Bug/Email/Console/Command/TestCommand.php b/app/code/Bug/Email/Console/Command/TestCommand.php
index 83aec04..22463fc 100644
--- a/app/code/Bug/Email/Console/Command/TestCommand.php
+++ b/app/code/Bug/Email/Console/Command/TestCommand.php
@@ -14,7 +14,9 @@ use Symfony\Component\Console\Input\InputInterface;
 use Symfony\Component\Console\Input\InputOption;
 use Symfony\Component\Console\Output\OutputInterface;
 use Magento\Framework\App\State as AppState;
+use Magento\Customer\Api\AccountManagementInterfaceFactory;
 use Magento\Customer\Api\AccountManagementInterface;
+use Magento\Customer\Api\CustomerRepositoryInterfaceFactory;
 use Magento\Customer\Api\CustomerRepositoryInterface;
 
 /**
@@ -26,33 +28,43 @@ use Magento\Customer\Api\CustomerRepositoryInterface;
 class TestCommand extends AbstractCommand
 {
     /**
+     * @var CustomerRepositoryInterfaceFactory
+     */
+    protected $customerRepositoryFactory;
+
+    /**
      * @var CustomerRepositoryInterface
      */
     protected $customerRepository;
 
     /**
+     * @var AccountManagementInterfaceFactory
+     */
+    protected $accountManagementFactory;
+
+    /**
      * @var AccountManagementInterface
      */
-    protected $customerAccountManagement;
+    protected $accountManagement;
 
     /**
      * TestCommand constructor.
      *
-     * @param AppState                    $appState
-     * @param CustomerRepositoryInterface $customerRepository
-     * @param AccountManagementInterface  $customerAccountManagement
-     * @param null                        $name
+     * @param AppState                           $appState
+     * @param CustomerRepositoryInterfaceFactory $customerRepositoryFactory
+     * @param AccountManagementInterfaceFactory  $accountManagementFactory
+     * @param null                               $name
      */
     public function __construct(
-        AppState                    $appState,
-        CustomerRepositoryInterface $customerRepository,
-        AccountManagementInterface  $customerAccountManagement,
+        AppState                           $appState,
+        CustomerRepositoryInterfaceFactory $customerRepositoryFactory,
+        AccountManagementInterfaceFactory  $accountManagementFactory,
         $name = null
     ) {
         parent::__construct($appState, $name);
 
-        $this->customerRepository        = $customerRepository;
-        $this->customerAccountManagement = $customerAccountManagement;
+        $this->customerRepositoryFactory = $customerRepositoryFactory;
+        $this->accountManagementFactory  = $accountManagementFactory;
     }
 
     /**
@@ -84,7 +96,6 @@ class TestCommand extends AbstractCommand
         parent::configure();
     }
 
-
     /**
      * Executes the current command.
      *
@@ -113,12 +124,12 @@ class TestCommand extends AbstractCommand
         $output->writeln('Customer Id ['.$customerId.']');
 
         // load the customer
-        $customer = $this->customerRepository->getById($customerId);
+        $customer = $this->getCustomerRepository()->getById($customerId);
         $output->writeln('Customer Email ['.$customer->getEmail().']');
 
         // send the email
         $output->writeln('Reset the Password');
-        $this->customerAccountManagement->initiatePasswordReset(
+        $this->getAccountManagement()->initiatePasswordReset(
             $customer->getEmail(),
             \Magento\Customer\Model\AccountManagement::EMAIL_REMINDER,
             $customer->getWebsiteId()
@@ -128,4 +139,32 @@ class TestCommand extends AbstractCommand
 
         return 0;
     }
+
+    /**
+     * Get the customer repository
+     *
+     * @return CustomerRepositoryInterface
+     */
+    protected function getCustomerRepository()
+    {
+        if ($this->customerRepository === null) {
+            $this->customerRepository = $this->customerRepositoryFactory->create();
+        }
+
+        return $this->customerRepository;
+    }
+
+    /**
+     * Get the account management
+     *
+     * @return AccountManagementInterface
+     */
+    protected function getAccountManagement()
+    {
+        if ($this->accountManagement === null) {
+            $this->accountManagement = $this->accountManagementFactory->create();
+        }
+
+        return $this->accountManagement;
+    }
 }

Concerning the stack trace, I have stopped my analysis where all the cli commands are initialised. I removed this module from composer and tried again and everything was working just fine after.
I don't know which one is the interface that induces the issue as the flag table is not used that much.

Best Regards,