cf-charmers team mailing list archive
-
cf-charmers team
-
Mailing list archive
-
Message #00156
Re: Ben: status on Acceptance Tests CI integration? Any assistance?
Could you guys please find some time to pair on this? It will be much faster to fix.
There are ‘known’ failing tests related to the lack of some components (Health Manager) as Alexander mentioned in some emails but the majority of the tests should be passing.
On May 15, 2014, at 3:52 AM, Benjamin Saller <benjamin.saller@xxxxxxxxxxxxx> wrote:
> I did source it, the env vars are as expected, I source the following and then run setup_yeti
>
> #!/bin/bash
>
> HOSTNAME=`juju status router/0 |grep public-address|cut -f 2 -d : `
> IP=`dig +short $HOSTNAME`
>
>
> export VCAP_BVT_API_ENDPOINT="http://api.${IP}.xip.io"
> export VCAP_BVT_UAA_CC_SECRET="habr"
>
> export VCAP_BVT_ADMIN_USER="admin"
> export VCAP_BVT_ADMIN_USER_PASSWD="admin"
> export VCAP_BVT_USER="test"
> export VCAP_BVT_USER_PASSWD="password"
>
>
>
> function setup_yeti() {
> cf api $VCAP_BVT_API_ENDPOINT
> cf login -u $VCAP_BVT_ADMIN_USER -p $VCAP_BVT_ADMIN_USER_PASSWD
> cf create-space my-space
> cf target -s my-space
> cf create-user $VCAP_BVT_USER $VCAP_BVT_USER_PASSWD
> cf set-org-role $VCAP_BVT_USER my-org OrgManager
> cf set-space-role $VCAP_BVT_USER my-org my-space SpaceManager
> cf set-space-role $VCAP_BVT_USER my-org my-space SpaceDeveloper
> }
>
> and then bundle exec rspec. This results in a slew of failures, it seems like some of this might be that even though loggregator is in a started start I see
>
> Failure/Error: Unable to find matching line from backtrace
>
> There is noting I've seen in the loggregator logs indicating the nature of the failure
>
> -Ben
>
>
>
>
>
>
> On Wed, May 14, 2014 at 11:22 PM, Prismakov Alexander <alexander.prismakov@xxxxxxxxxxx> wrote:
> Hi Ben,
>
> You need “source” the script not just run. Could you please check environment variables before running test?
> I’m ready to pair with you to solve the issues. Just say when and how.
> We may discuss it via Skype/IRC etc.
> My Skype:prismakov
>
> - Alex Prismakov
>
> On May 15, 2014, at 9:14, Benjamin Saller <benjamin.saller@xxxxxxxxxxxxx> wrote:
>
>> I had tried settings those, its implied in the docs that if you rake prepare it will create additional accounts (which works) and use the first of those automatically. The tests are almost all still failing even after sourcing that script. There was some breakage in the bundle as you all noted but given that the deployment I was using could push a working app I would expect this to work as well.
>>
>> I'll continue to investigate, I'd very much like to have (even a working subset of) the YETI tests running before we can switch to CATs. If you have any additional insights they are welcome.
>>
>>
>> Thanks,
>> Ben
>>
>>
>> On Wed, May 14, 2014 at 3:22 AM, Alexander Lomov <lomov.as@xxxxxxxxx> wrote:
>> It seems that you'll need VCAP_BVT_USER
>> and VCAP_BVT_USER_PASSWD variables additionally. You can try this script to prepare your CF installation for the tests: https://gist.github.com/allomov/bd0d77703df42fe29de9 (many thanks to Alex P. for it).
>>
>> Best wishes,
>> Alex L.
>>
>>
>> On 14 May 2014 02:31, Benjamin Saller <benjamin.saller@xxxxxxxxxxxxx> wrote:
>> I've been unable to replicate these test results on the single aws node deployment. I am hoping its getting the UAA_CC_SECRET wrong, are your settings different?
>>
>> VCAP_BTV_ORG_NAMESPACE=jenkins
>> VCAP_BVT_ADMIN_USER_PASSWD=admin
>> VCAP_BVT_ADMIN_USER=admin
>> VCAP_BVT_API_ENDPOINT=http://api.<proper ip>.xip.io
>> VCAP_BVT_UAA_CC_SECRET=admin
>>
>> from the proper hashed version of YETI mentioned in the README
>>
>> bundle
>> bundle prepare
>> bundle exec rspec
>>
>> This does run the tests though almost everything fails. With the deployment I'm using I've been able to push and validate that the fib-cpu app mentioned in the README works, so the basic setup seemed fine. My being unable to get the tests to pass or have clear insight into what is going wrong has kept me from putting the CI code in place.
>>
>> http://api.54.198.98.13.xip.io
>>
>> is the current endpoint. I'll leave this up tonight and maybe you can see if the tests run for you and what I'm missing in the current setup?
>>
>> Thanks,
>> Ben
>>
>>
>> On Mon, May 12, 2014 at 5:56 AM, Manuel Garcia <manuel.garcia@xxxxxxxxxxx> wrote:
>> Hi Ben,
>>
>> Will you need any assistance from us?
>>
>> - Manuel Garcia
>>
>>
>> On May 7, 2014, at 1:49 PM, Prismakov Alexander <alexander.prismakov@xxxxxxxxxxx> wrote:
>>
>>> Hi Ben,
>>>
>>> I spend some time comparing results of integration tests on both cloudfoundry deployments - one deployed with BOSH Lite and second one deployed with JUJU.
>>> To integrate it to our CI we need disable some tests. The link (https://docs.google.com/spreadsheets/d/17CAkRv_-vOfjBw_jgmNU6PAvIeRtWQDnD_hesdcdpIU/edit?usp=sharing) contains the list. Fill free to add your results/comments.
>>> Also sometimes some tests fail for me when I run them as a bunch but then they pass if I rerun them individually. You need to keep it in mind for CI. May be it make sense to run all tests from CI individually.
>>>
>>> -Alex Prismakov
>>
>>
>>
>> --
>> Mailing list: https://launchpad.net/~cf-charmers
>> Post to : cf-charmers@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~cf-charmers
>> More help : https://help.launchpad.net/ListHelp
>>
>>
>>
>> --
>> Mailing list: https://launchpad.net/~cf-charmers
>> Post to : cf-charmers@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~cf-charmers
>> More help : https://help.launchpad.net/ListHelp
>
>
> --
> Mailing list: https://launchpad.net/~cf-charmers
> Post to : cf-charmers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~cf-charmers
> More help : https://help.launchpad.net/ListHelp
References