yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1635458] [NEW] Swift CLI is OK, but Swift Horizon shows: "Error: Unable to create container"

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Launchpad Bug Tracker <1635458@xxxxxxxxxxxxxxxxxx>
Date: Mon, 24 Oct 2016 20:55:17 -0000
Reply-to: Bug 1635458 <1635458@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
You have been subscribed to a public bug:

I am deploying multinode openstack:


ENVIRONMENT:

 - 1 controller node
 - 1 network node
 - 1 compute node
 - 1 cinder node
 - 1 swift node
 - 1 swift proxy node

Distributor ID:	Ubuntu
Description:	Ubuntu 14.04.5 LTS
Release:	14.04
Codename:	trusty


Linux controller 4.4.0-31-generic #50~14.04.1-Ubuntu SMP Wed Jul 13 01:07:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Python 2.7.6
neutron 2.3.4
swift 2.0.3
glance 0.12.0
swift 2.0.3
nova 2.17.0
cinder 1.0.8
keystone 0.7.1

----------------
1) admin CLI + Horizon works as expected:
2) demo CLI client works as expected:

root@controller:/home/ajn\# `swift stat`

>        Account: AUTH_cdbcbc3eeb8a472c8abb2b67e749774c
>     Containers: 2
>        Objects: 4
>          Bytes: 55534
>  Accept-Ranges: bytes
>  X-Timestamp: 1476882488.69520
>  X-Trans-Id: txb644cc0c76504910bd30f-005808f556
>  Content-Type: text/plain; charset=utf-8

root@controller:/home/ajn\# `swift upload demo-container-test1 text-
demo1.txt `

> text-demo1.txt

root@controller:/home/ajn\# `swift stat demo-container-test1`

>        Account: AUTH_cdbcbc3eeb8a472c8abb2b67e749774c
>      Container: demo-container-test1
>        Objects: 1
>          Bytes: 10
>       Read ACL:
>      Write ACL:
>        Sync To:
>       Sync Key:
>  Accept-Ranges: bytes
>  X-Timestamp: 1476982183.68498
>  X-Trans-Id: tx268ae3529da9422cae649-005808f5e6
>  Content-Type: text/plain; charset=utf-8

root@controller:/home/ajn\# `rm text-demo1.txt`
root@controller:/home/ajn\# `swift download demo-container-test1`

> text-demo1.txt [auth 0.107s, headers 0.162s, total 0.163s, 0.000 MB/s]

3) Horizon object storage under demo user shows "Error: Enable to create
container"

![selection_493](https://cloud.githubusercontent.com/assets/1716020/19568645/a8fffd74-96f2-11e6
-81ab-c2a939420f5a.png)

--------

Apache2 log are very iniformative:

root@controller:/home/ajn\#` tail /var/log/apache2/error.log`

> [Thu Oct 20 23:44:49.756116 2016] [:error] [pid 8646:tid 139712725325568] RESP HEADERS: [('date', 'Thu, 20 Oct 2016 23:44:49 GMT'), ('content-length', '73'), ('content-type', 'text/html; charset=UTF-8'), ('x-trans-id', 'tx975635b8d06c40d7a6626-00580956f1')]
> [Thu Oct 20 23:44:49.756185 2016] [:error] [pid 8646:tid 139712725325568] RESP BODY: \<html>\<h1>Forbidden\</h1>\<p>Access was denied to this resource.\</p>\</html>
> [Thu Oct 20 23:44:49.756441 2016] [:error] [pid 8646:tid 139712725325568] Container PUT failed: http://swiftProxy:**8080:8080**/v1/AUTH_62c353aacad140488aac55552b659334/container-test2 403 Forbidden  [first 60 chars of response] \<html>\<h1>Forbidden\</h1>\<p>Access was denied to this resourc
> [Thu Oct 20 23:44:49.756455 2016] [:error] [pid 8646:tid 139712725325568] Traceback (most recent call last):
> [Thu Oct 20 23:44:49.756459 2016] [:error] [pid 8646:tid 139712725325568]   File "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line 1189, in _retry
> [Thu Oct 20 23:44:49.756463 2016] [:error] [pid 8646:tid 139712725325568]     rv = func(self.url, self.token, *args, **kwargs)
> [Thu Oct 20 23:44:49.756467 2016] [:error] [pid 8646:tid 139712725325568]   File "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line 674, in put_container
> [Thu Oct 20 23:44:49.756486 2016] [:error] [pid 8646:tid 139712725325568]     http_response_content=body)
> [Thu Oct 20 23:44:49.756491 2016] [:error] [pid 8646:tid 139712725325568] ClientException: Container PUT failed: http://swiftProxy:**8080:8080**/v1/AUTH_62c353aacad140488aac55552b659334/container-test2 403 Forbidden  [first 60 chars of response] \<html>\<h1>Forbidden\</h1>\<p>Access was denied to this resourc
> [Thu Oct 20 23:44:49.756718 2016] [:error] [pid 8646:tid 139712725325568] Recoverable error: Container PUT failed: http://swiftProxy:**8080:8080**/v1/AUTH_62c353aacad140488aac55552b659334/container-test2 403 Forbidden  [first 60 chars of response] \<html>\<h1>Forbidden\</h1>\<p>Access was denied to this resourc
>

Looks like Horizon forms a wrong query URL, it adds superfluous port
number:

> http://swiftProxy:8080:8080

Comparing to the the command line, a debug swift command shows that the
cli client queries the correct url:

oot@controller:/home/ajn\# `swift --debug stat`

> DEBUG:keystoneclient.session:REQ: curl -i -X POST http://controller:5000/v2.0/tokens -H "Content-Type: application/json" -H "User-Agent: python-keystoneclient" -d '{"auth": {"passwordCredentials": {"username": "admin", "password": "linuxacademy123"}, "tenantId": "cdbcbc3eeb8a472c8abb2b67e749774c"}}'
> INFO:urllib3.connectionpool:Starting new HTTP connection (1): controller
> DEBUG:urllib3.connectionpool:Setting read timeout to None
> DEBUG:urllib3.connectionpool:"POST /v2.0/tokens HTTP/1.1" 200 7125
> DEBUG:keystoneclient.session:RESP: [200] CaseInsensitiveDict({'date': 'Fri, 21 Oct 2016 00:21:11 GMT', 'vary': 'X-Auth-Token', 'content-length': '7125', 'content-type': 'application/json', 'x-distribution': 'Ubuntu'})
> RESP BODY: {"access": {"token": {"issued_at": "2016-10-21T00:21:11.915583", "expires": "2016-10-21T01:21:11Z", "id": "MIIMsQYJKoZIhvcNAQcCoIIMojCCDJ4CAQExDTALBglghkgBZQMEAgEwggr-
>
> \#\#\#\# TRUNKATED  \#\#\#\#
>
> PmTrffWUVJU-CChlhSElfwgp7XW0igpLv18A7ThnOaSXQMQIMFV0O46v+1W2tc725XtM6vgz62-sfu6ZSG66VbH4zjf7lkAVtPyxRbGNYSUk1eQPLo-FHc1cbule4gnk2CAL10Hy4zRtaFBmPZ62qt50ZLp-MFAt6HVzGR8abTWqvpSFlc3arWSk2vSmSPInWoZy6g==", "tenant": {"description": "Admin tenant", "enabled": true, "id": "cdbcbc3eeb8a472c8abb2b67e749774c", "name": "admin"}}, "serviceCatalog": [{"endpoints": [{"adminURL": "http://controller:8774/v2/cdbcbc3eeb8a472c8abb2b67e749774c";, "region": "regionOne", "internalURL": "http://controller:8774/v2/cdbcbc3eeb8a472c8abb2b67e749774c";, "id": "23c8d788422c466cb6c97dcb92c2b0b2", "publicURL": "http://controller:8774/v2/cdbcbc3eeb8a472c8abb2b67e749774c"}], "endpoints_links": [], "type": "compute", "name": "nova"}, {"endpoints": [{"adminURL": "http://controller:9696";, "region": "regionOne", "internalURL": "http://controller:9696";, "id": "534fefe0f0414b8ebd3d54b9f4be7d93", "publicURL": "http://controller:9696"}], "endpoints_links": [], "type": "network", "name": "neutron"}, {"endpoints": [{"adminURL": "http://controller:8776/v2/cdbcbc3eeb8a472c8abb2b67e749774c";, "region": "regionOne", "internalURL": "http://controller:8776/v2/cdbcbc3eeb8a472c8abb2b67e749774c";, "id": "5700f74a49ad447d88e9d7189b4532ed", "publicURL": "http://controller:8776/v2/cdbcbc3eeb8a472c8abb2b67e749774c"}], "endpoints_links": [], "type": "volumev2", "name": "cinderv2"}, {"endpoints": [{"adminURL": "http://controller:9292";, "region": "regionOne", "internalURL": "http://controller:9292";, "id": "1edb475e3de84f7abf92b01701404aee", "publicURL": "http://controller:9292"}], "endpoints_links": [], "type": "image", "name": "glance"}, {"endpoints": [{"adminURL": "http://controller:8776/v1/cdbcbc3eeb8a472c8abb2b67e749774c";, "region": "regionOne", "internalURL": "http://controller:8776/v1/cdbcbc3eeb8a472c8abb2b67e749774c";, "id": "60bf9cad87df41f3b58badf3dd81fc1a", "publicURL": "http://controller:8776/v1/cdbcbc3eeb8a472c8abb2b67e749774c"}], "endpoints_links": [], "type": "volume", "name": "cinder"}, {"endpoints": [{"adminURL": "http://swiftProxy:8080";, "region": "regionOne", "internalURL": "**http://swiftProxy:8080**/v1/AUTH_cdbcbc3eeb8a472c8abb2b67e749774c";, "id": "34b5dcc64c354e2d80f4d94e5fbab2ff", "publicURL": "http://swiftProxy:8080/v1/AUTH_cdbcbc3eeb8a472c8abb2b67e749774c"}], "endpoints_links": [], "type": "object-store", "name": "swift"}, {"endpoints": [{"adminURL": "http://controller:35357/v2.0";, "region": "regionOne", "internalURL": "http://controller:5000/v2.0";, "id": "8f7bce4e0629429e9204efb9d1399ee1", "publicURL": "http://controller:5000/v2.0"}], "endpoints_links": [], "type": "identity", "name": "keystone"}], "user": {"username": "admin", "roles_links": [], "id": "4ac4bb45dd504f53a027b79ca3e9f2e8", "roles": [{"name": "admin"}, {"name": "_member_"}], "name": "admin"}, "metadata": {"is_admin": 0, "roles": ["5910c3c96ca54403b189d00f9a70b8be", "9fe2ff9ee4384b1894a90878d3e92bab"]}}}
>
> \#\#\#\# TRUNKATED  \#\#\#\#
>
> INFO:urllib3.connectionpool:Starting new HTTP connection (1): swiftProxy
> DEBUG:urllib3.connectionpool:Setting read timeout to <object object at 0x7fd5a645b090>
> DEBUG:urllib3.connectionpool:"HEAD /v1/AUTH_cdbcbc3eeb8a472c8abb2b67e749774c HTTP/1.1" 204 0
> DEBUG:swiftclient:REQ: curl -i **http://swiftProxy:8080**/v1/AUTH_cdbcbc3eeb8a472c8abb2b67e749774c -I -H "X-Auth-Token:
> \#\#\#\# TRUNKATED  \#\#\#\#
> DEBUG:swiftclient:RESP STATUS: 204 No Content
> DEBUG:swiftclient:RESP HEADERS: [('content-length', '0'), ('accept-ranges', 'bytes'), ('x-timestamp', '1476882488.69520'), ('x-trans-id', 'tx2f3a80acaf3149d78ca21-0058095f77'), ('date', 'Fri, 21 Oct 2016 00:21:11 GMT'), ('x-account-bytes-used', '55544'), ('x-account-container-count', '3'), ('content-type', 'text/plain; charset=utf-8'), ('x-account-object-count', '5')]
>        Account: AUTH_cdbcbc3eeb8a472c8abb2b67e749774c
>     Containers: 3
>        Objects: 5
>          Bytes: 55544
>  Accept-Ranges: bytes
>    X-Timestamp: 1476882488.69520
>     X-Trans-Id: tx2f3a80acaf3149d78ca21-0058095f77
>   Content-Type: text/plain; charset=utf-8
>

4) Same erroneous url is used by CLI + Horizon of any newly created
user:

root@controller:/home/ajn# . ajn-openrc.sh
Please enter your OpenStack Password:
root@controller:/home/ajn#
root@controller:/home/ajn# swift stat
Account HEAD failed: http://swiftProxy:8080:8080/v1/AUTH_513a3eb3b3b742d09e89fffb64a53a6d 403 Forbidden

** Affects: horizon
     Importance: Undecided
         Status: New


** Tags: horizon swift
-- 
Swift CLI is OK, but Swift Horizon shows: "Error: Unable to create container"
https://bugs.launchpad.net/bugs/1635458
You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon).