GithubHelp home page GithubHelp logo

cob's People

Contributors

ajorg avatar henrysher avatar marcindulak avatar oliver006 avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar

Forkers

vadirajks tuapuikia prudhvikartheek dh-harald marcindulak shantanugadgil piyushr93 jnoxon syskill philgrayson pankeshgupta1994 telesofia malisettirammurthy ankeetc georgebunnclara

cob's Issues

Problems when deploying in new AWS account

Hullo! I've been using version 0.3.0 of your plugin for a few weeks and it's been working great. Today I deployed the same AMI with the plugin pre-installed onto a different AWS Account in the
same region, eu-west-1, and it was unable to read from the S3 repo I had defined:

[root@bastion-id4160830 ~]# yum -v makecache
Loading "cob" plugin
Loading "fastestmirror" plugin
Config time: 0.017
Yum version: 3.4.3
base                                                                                                                                                                         | 3.6 kB  00:00:00     
epel/x86_64/metalink                                                                                                                                                         |  27 kB  00:00:00     
extras                                                                                                                                                                       | 3.4 kB  00:00:00     
Calculating signature using v4 auth.
CanonicalRequest:
GET
/repos/puppet/repodata/repomd.xml

host:stage-yumbucket-8ynuyi1pfclm-s3bucket-1o6a82dodg7d9.s3.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20141204T151556Z
x-amz-security-token:AQoDYXdzEKf//////////wEa0APrnoe2ndekV4VWUO+L2P2djEF8976KUc4/TB+auB5oATRVXvd7DN3Ek7O6/hyHNj6GsFLjPJVdsaoKOQL+/4uP9achHV+NEM95ul6GbgLeqS9+9EmlGp5W9lDlyIf/bypAf0sp+iG2U9oqKSe6jBDbWnM0OLgGSs4JeFpxp+avX51E9R5c7WONasOdIkFDW6WuJic+3OaKdFaZj4b0k7jZutzH8iDIufeit9h8JcYuncUzXb3e8jkB8bcLPLZRZe5kFeoF7n1NVjplA5BaNwZOtLoznG2BAJvqTDOo4yPw5M7SXVJ2cVH1uOSSb21T6+mLjflUKMUfL/Yscck/J7FLSh4VLCY/nmnGQCXbR1D9Bd+CSN+YqtoUHnFGzNlUFLoNd5Fp+PAWw5iuU2R786ceUB/leFn+kMDSvPVaoqDSXUGOoFJ0y4D2ysnDCm+jrg+8rvODsdcJM0sGxcXf14MLn7IK+MQMi+/5afCpGY+kbDafY57X8WvWgpDlDwVeg6bQyrH8Nf8uab8omN1sBOBtCkvmNteIckj+6eumf+G/yTu1O9RQq6nOOpNAyMUXOCvnZjDAboz0litvimxNUoH35tHXdQaIqB/GM+XUHu/0riDr2oGkBQ==

host;x-amz-content-sha256;x-amz-date;x-amz-security-token
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

StringToSign:
AWS4-HMAC-SHA256
20141204T151556Z
20141204/eu-west-1/s3/aws4_request
b356d64147c233fa7dd8f5dbb6d3250c87176aa3cf68c16dc8bb511247618c22

Signature: 7a4611566c112616134c1950b0ab6a1a298492492bf0acad1f61bb4739d5a19d
https://stage-yumbucket-8ynuyi1pfclm-s3bucket-1o6a82dodg7d9.s3.amazonaws.com/repos/puppet/repodata/repomd.xml: [Errno 14] HTTPS Error 403 - Forbidden
Trying other mirror.
failure: repodata/repomd.xml from itvs3: [Errno 256] No more mirrors to try.
https://stage-yumbucket-8ynuyi1pfclm-s3bucket-1o6a82dodg7d9.s3.amazonaws.com/repos/puppet/repodata/repomd.xml: [Errno 14] HTTPS Error 403 - Forbidden
puppetlabs-deps                                                                                                                                                              | 2.5 kB  00:00:00     
puppetlabs-products                                                                                                                                                          | 2.5 kB  00:00:00     
updates                                                                                                                                                                      | 3.4 kB  00:00:00     
Loading mirror speeds from cached hostfile
 * base: centos.mirror.constant.com
 * epel: ftp.heanet.ie
 * extras: mirror.atlanticmetro.net
 * updates: mirror.solarvps.com
Metadata Cache Created

The instance definitely has an IAM EC2 role which allows it to access this bucket:

[root@bastion-id4160830 ~]# aws s3 cp s3://stage-yumbucket-8ynuyi1pfclm-s3bucket-1o6a82dodg7d9/repos/puppet/repodata/repomd.xml .
download: s3://stage-yumbucket-8ynuyi1pfclm-s3bucket-1o6a82dodg7d9/repos/puppet/repodata/repomd.xml to ./repomd.xml

The repo definition is this:

[itvs3]
name=itv-s3
baseurl=https://stage-yumbucket-8ynuyi1pfclm-s3bucket-1o6a82dodg7d9.s3.amazonaws.com/repos/puppet
metadata_expire=10s
enabled=1
gpgcheck=0

I found that if I embed the region name in the baseurl then it works, but I don't understand why that was not required before, and is still not required for the instances in the other AWS account.

Can you help? I did notice that resolving the hostname for each of the two buckets has a different output. From an instance in the original AWS account:

[root@bastion-i23f2e5c6 ~]# host sit-yumbucket-rbix7r3dlq8j-s3bucket-8trs8qdn31lj.s3.amazonaws.com
sit-yumbucket-rbix7r3dlq8j-s3bucket-8trs8qdn31lj.s3.amazonaws.com is an alias for s3-3-w.amazonaws.com.
s3-3-w.amazonaws.com has address 54.231.136.208

From an instance in the new account:

[root@bastion-id4160830 ~]# host stage-yumbucket-8ynuyi1pfclm-s3bucket-1o6a82dodg7d9.s3.amazonaws.com
stage-yumbucket-8ynuyi1pfclm-s3bucket-1o6a82dodg7d9.s3.amazonaws.com is an alias for s3-directional-w.amazonaws.com.
s3-directional-w.amazonaws.com is an alias for s3-directional-w.a-geo.amazonaws.com.
s3-directional-w.a-geo.amazonaws.com is an alias for s3-1-w.amazonaws.com.
s3-1-w.amazonaws.com has address 54.231.11.57

I was able to see with tcpdump that both instances were able to look up the availability-zone from the placement meta-data on 169.254.169.254 to eu-west-1.

Any ideas warmly welcomed!

Cheers,
Gavin.

does not work with IMDSv2

This plug-in does not work with IMDSv2. We have tried to modify the cob.py to include the http session token as call to metadata_server. The error goes away but yum does not see any enabled repos. any plans to include support for IMDSv2 ?

This plugin is literally unconfigurable

Due to the use of this anti-pattern:

metadata_server = "http://169.254.169.254"

...

def get_iam_role(url=metadata_server, version="latest",
                 params="meta-data/iam/security-credentials/"):

...

        global timeout, retries, metadata_server
        timeout = self.conduit.confInt('aws', 'timeout', default=timeout)
        retries = self.conduit.confInt('aws', 'retries', default=retries)
        metadata_server = self.conduit.confString('aws',
                                                  'metadata_server',
                                                  default=metadata_server)

...

        iam_role = get_iam_role()

Python binds the default argument values when the function is defined, and so the later override has no effect, as you can see below:

Python 2.7.18 (default, Jun 10 2021, 00:11:02)
[GCC 7.3.1 20180712 (Red Hat 7.3.1-13)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> globalvar = 'Hello'
>>> def foo(a=globalvar):
...     print a
...
>>> globalvar = 'Goodbye'
>>> foo()
Hello

Consequently the metadata_server, retries, and timeout settings in the config file are not used, if different from the default values.

Plugin malfunction with python-2.7.5-92.el7_9

Hello, the plugin has become malfunction with the (currently) latest python-2.7.5-92.el7_9.x86_64 in CentOS7. Could you please check it? No error is thrown by the plugin, it "just" doesn't work with static AWS credentials anymore. Thank you very much in advance for check&fix, Michal.

No license file

It would be helpful if you had a license file of some sort so people know how they can (re)use the code here.

Disabled repos are ignored, even if they are enabled with the --enablerepo flag

I have a repository that I prefer to leave disabled until I explicitly enable it, but cob doesn't support this scenario.

The following seems to do the trick, but I'd rather you patch it (knowing the yum codebase a bit better than myself):

In init_hook, instead of

if isinstance(repo, YumRepository) and repo.enabled:

just

if isinstance(repo, YumRepository):

And in S3Repository.__init__:

if repo.enabled:
    self.enable()

s3:// style URLs?

Would it be possible to support URLs of the format s3://<bucket>/<prefix>? These are used in the AWS CLI and would be a nice way to differentiate S3 from HTTP.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.