openstack team mailing list archive

["Mark Washenberger" <mark.washenberger@xxxxxxxxxxxxx>]

I'm tending to agree with Sandy's comments.

I think we all agree that we have a mess with the database stubbing that is going on. And I'm confident that the db fake would make that mess more manageable.

But the way I see the mess, it comes from having a giant flat db interface and really large testcases. The giant flat db.api class means it can be daunting to create a nicely packaged collection of test double functions.

Also, by having such large test cases, the generality & complexity of setUp() as well as its distance from any given test method increase tremendously. All of this makes it harder to understand the impact of any one line in setUp and nearly impossible to tell which parts of setUp are relevant to a given test.

But, I have to admit that I'm not backing up the way I think we could make the best improvements with any code at the moment--and Soren is. So I'm not sure I really want to stand in the way.

My only detraction from a big fake is that it in general has to be a pretty faithful implementation of the real interface. However, if I use a bunch of small special-purpose fakes, I can usually get the same coverage without working as hard.

"Sandy Walsh" <sandy.walsh@xxxxxxxxxxxxx> said:

> I suppose there is a matter of preference here. I prefer to look in the setup()
> and teardown() methods of my test suite to find out how everything hangs together.
> Otherwise I have to check nova.TestCase when things break. The closer my test can
> stay to my expectations from unittest.TestCase the happier I am.
> 
> I can't comment on your fake db implementation, but my fear is this scenario:
> 
> Test1 assumes db.create_foo() will return 123 and Test2 assumes it will return
> "abc". How do they both comfortably co-exist? And whatever the mechanism, why is
> it better than just stubs.Set("db.create_foo", _my_create_foo)?
> 
> It's local and it makes sense in the context of that file.
> 
> -S
> 
> ________________________________________
> From: Soren Hansen [soren@xxxxxxxxxxx]
> 
> 2011/11/22 Sandy Walsh <sandy.walsh@xxxxxxxxxxxxx>:
>> I'm not a big fan of faking a database, not only for the reasons outlined
>> already, but because it makes the tests harder to understand.
> 
> Can you give me an example? I find the opposite to be true, so I'd
> love to see counterexamples. Most of the time, the data store is not
> relevant for the tests. I just need to stick an instance into the db,
> do some stuff, and verify that I get the correct (direct and indirect)
> output. I don't see how having a mocked db.instance_get is any more
> readable than a db.instance_create() (or a parameterised
> create_instance utility method for testing purposes or whatever).
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>