Radosgw docs are missing something about ceph-container

[Ulexus]

Interesting. I definitely ran it successfully before upgrading to giant. Can you show me your docker run options?

[leseb]

Weird, you've probably pulled an old image.

[bobrik]

I run it with ansible, here's my docker inspect:

[{
    "AppArmorProfile": "",
    "Args": [],
    "Config": {
        "AttachStderr": false,
        "AttachStdin": false,
        "AttachStdout": false,
        "Cmd": null,
        "CpuShares": 0,
        "Cpuset": "",
        "Domainname": "",
        "Entrypoint": [
            "/entrypoint.sh"
        ],
        "Env": [
            "RGW_NAME=something",
            "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
            "CEPH_VERSION=firefly",
            "ETCDCTL_VERSION=v2.0.4",
            "ETCDCTL_ARCH=linux-amd64"
        ],
        "ExposedPorts": {
            "80/tcp": {}
        },
        "Hostname": "958cfdfeba97",
        "Image": "ceph/radosgw:firefly",
        "MacAddress": "",
        "Memory": 0,
        "MemorySwap": 0,
        "NetworkDisabled": false,
        "OnBuild": null,
        "OpenStdin": false,
        "PortSpecs": null,
        "StdinOnce": false,
        "Tty": false,
        "User": "",
        "Volumes": {
            "/etc/ceph": {}
        },
        "WorkingDir": ""
    },
    "Created": "2015-03-12T19:52:36.760926072Z",
    "Driver": "devicemapper",
    "ExecDriver": "native-0.2",
    "HostConfig": {
        "Binds": [
            "/var/lib/ceph-photos/etc:/etc/ceph:rw"
        ],
        "CapAdd": null,
        "CapDrop": null,
        "ContainerIDFile": "",
        "Devices": null,
        "Dns": null,
        "DnsSearch": null,
        "ExtraHosts": null,
        "IpcMode": "",
        "Links": null,
        "LxcConf": null,
        "NetworkMode": "",
        "PortBindings": {
            "80/tcp": [
                {
                    "HostIp": "192.168.2.89",
                    "HostPort": "80"
                }
            ]
        },
        "Privileged": false,
        "PublishAllPorts": false,
        "RestartPolicy": {
            "MaximumRetryCount": 0,
            "Name": "always"
        },
        "SecurityOpt": null,
        "VolumesFrom": null
    },
    "HostnamePath": "/var/lib/docker/containers/958cfdfeba97cea71dfd161a8f6ea1f3c0d61cbac0889bdb6c842864a9401c32/hostname",
    "HostsPath": "/var/lib/docker/containers/958cfdfeba97cea71dfd161a8f6ea1f3c0d61cbac0889bdb6c842864a9401c32/hosts",
    "Id": "958cfdfeba97cea71dfd161a8f6ea1f3c0d61cbac0889bdb6c842864a9401c32",
    "Image": "33ffbf82e2efb6e7a4c5ac9c95e29884a87d19e23da989346fdf5a61d76b1328",
    "MountLabel": "",
    "Name": "/ceph.radosgw0",
    "NetworkSettings": {
        "Bridge": "docker0",
        "Gateway": "172.17.42.1",
        "IPAddress": "172.17.0.18",
        "IPPrefixLen": 16,
        "MacAddress": "02:42:ac:11:00:12",
        "PortMapping": null,
        "Ports": {
            "80/tcp": [
                {
                    "HostIp": "192.168.2.89",
                    "HostPort": "80"
                }
            ]
        }
    },
    "Path": "/entrypoint.sh",
    "ProcessLabel": "",
    "ResolvConfPath": "/var/lib/docker/containers/958cfdfeba97cea71dfd161a8f6ea1f3c0d61cbac0889bdb6c842864a9401c32/resolv.conf",
    "State": {
        "Error": "",
        "ExitCode": 0,
        "FinishedAt": "0001-01-01T00:00:00Z",
        "OOMKilled": false,
        "Paused": false,
        "Pid": 64677,
        "Restarting": false,
        "Running": true,
        "StartedAt": "2015-03-12T19:52:38.035473391Z"
    },
    "Volumes": {
        "/etc/ceph": "/var/lib/ceph-photos/etc"
    },
    "VolumesRW": {
        "/etc/ceph": true
    }
}
]

It should be equal to -v /var/lib/ceph-photos/etc:/etc/ceph:rw -e RGW_NAME=something

[bobrik]

Image is up to date, btw.

[leseb]

Hum @Ulexus did we also upgrade the firefly image?

[Ulexus]

@leseb No, I didn't backport the civetweb changes to firefly. Is civetweb ready to use for firefly? I'm happy to backport it, if so.

[leseb]

Hum actually I had issues configuring civetweb on Firefly, so I guess we should stick with this one.
I can try later.

[Ulexus]

I seem to recall civetweb in Firefly was a barebones preview, while in giant it is soft-release status

[Ulexus]

@bobrik Do you have the stderr output from the container? I'm guessing radosgw itself is failing to start (or blocking, rather, since the container should exit if either daemon dies).

[bobrik]

It does't exit.

# docker logs ceph.radosgw0
2015-03-12 19:52:38.261737 7f46b12f57c0  0 ceph version 0.80.9 (b5a67f0e1d15385bc0d60a6da6e7fc810bde6047), process radosgw, pid 65
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.18. Set the 'ServerName' directive globally to suppress this message
2015-03-12 19:52:46.138416 7f46b12f57c0  0 framework: fastcgi
2015-03-12 19:52:46.138421 7f46b12f57c0  0 starting handler: fastcgi

# docker exec ceph.radosgw0 ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0  17976  1576 ?        Ss   19:52   0:00 /bin/bash /entrypoint.sh
root        63  0.0  0.0   4660  1180 ?        Sl   19:52   0:00 /startRadosgw
root        65  0.1  0.0 5234328 12008 ?       Sl   19:52   0:01 /usr/bin/radosgw -d -c /etc/ceph/ceph.conf -n client.radosgw.gateway -k /var/lib/ceph/radosgw/something/keyring
root        67  0.0  0.0  75592  3468 ?        S    19:52   0:00 /usr/sbin/apache2 -DFOREGROUND
www-data    83  0.0  0.0  75332  1672 ?        S    19:52   0:00 /usr/sbin/fcgi-pm -DFOREGROUND
www-data    84  0.0  0.0 430412  2868 ?        Sl   19:52   0:00 /usr/sbin/apache2 -DFOREGROUND
www-data    86  0.0  0.0 430412  2872 ?        Sl   19:52   0:00 /usr/sbin/apache2 -DFOREGROUND
root       402  0.0  0.0 5217680 7220 ?        Ssl  19:59   0:01 /usr/bin/radosgw -c /etc/ceph/ceph.conf -n client.radosgw.gateway
root       535  0.0  0.0 5217680 7196 ?        Ssl  19:59   0:01 /usr/bin/radosgw -c /etc/ceph/ceph.conf -n client.radosgw.gateway
root       662  0.0  0.0 5217680 7452 ?        Ssl  20:00   0:01 /usr/bin/radosgw -c /etc/ceph/ceph.conf -n client.radosgw.gateway
root       787  0.0  0.0  15564  1140 ?        R    20:20   0:00 ps aux

[Ulexus]

Oh, right. You're using a difference fcgi socket path from the container's (hard-coded) default, so radosgw will use your path, but the fcgi service is trying to connect to the container-default socket.

The giant version allows your to specify a socket path, but the firefly version has it hard-coded to /tmp/radosgw.sock.

If you update your ceph.conf to have rgw socket path = /tmp/radosgw.sock, I believe that will fix it.

[Ulexus]

Anyway, you're right; that should be documented.

[Ulexus]

Updated docs

[Ulexus]

There's no particular reason this has to be hard-coded, so if you would like to submit a PR against firefly for it, I'll be happy to merge it in (if I can figure out how to merge a PR to a different branch, anyway).

[bobrik]

You can choose branch where PR goes when you create one.

Can you also add info about concurrency and expected throughput of ceph/radosgw? The last time I checked (ages ago) the number of concurrent connections for apache was limited by number of apache processes. Each process needs some memory, so this gets escalated quickly.

This isssue could be closed, I'll probably stick with bobrik/radosgw at least before upgrading for giant.