openstack team mailing list archive

[corvus@xxxxxxxxxxxx (James E. Blair)]

Justin Santa Barbara <justin@xxxxxxxxxxxx> writes:

> This is a plugin for OpenStack; it won't support any other clouds.  This
> means there will be no baggage from having to support EC2, or any of the
> other clouds that JClouds supports.  We can support all the OpenStack
> goodness directly.

I second Andrew's commendation on your work on Java bindings for
OpenStack, but I also share his concern that effort would be well spent
helping with established projects.

What you wrote above is pretty vague, and in my mind, not a compelling
reason to start a new project rather than contributing to an existing
one actively developed by a diverse and engaged group of people.

> JClouds is great, but it's an abstraction layer that doesn't give OpenStack
> users anything.

On the contrary, there are a lot of people with apps running in clouds
that are not OpenStack right now who would probably like to use
OpenStack soon.  Being able to run in multiple clouds simultaneously
while migrating, not to mention being able to migrate without changing
code, is a great reason to have an abstraction layer.

I'm not so worried about OpenStack's performance that I think
source-level vendor-lock-in is the way to keep users.

> I hope the OpenStack CI team will join me in working on something that
> advances OpenStack, rather than spending any more time propping up the
> dying clouds :-)

I think providing a migration path for users to OpenStack is an
excellent way to advance it.

Since the Essex design summit, the OpenStack CI team has learned a lot
about running tests on cloud nodes.  In order to run the kinds of tests
we need to run, we need features that aren't in any of the Jenkins cloud
plugins currently.  We need disposable machines that are only used for
one test, which means spinning up and terminating hundreds of machines
per day.  We need to use machines from multiple providers simultaneously
so that we're resilient against errors with one provider.  We need to
pull nodes from a pool of machines that have been spun up ahead of time
for speed.  We need to be able to select from different kinds of images
for certain tests.  Machines need to be checked for basic functionality
before being added to the pool (we frequently get nodes without a
functioning network).  They need to be started from snapshots with
cached data on them to avoid false negatives from network problems.  We
need to keep them around after failures to diagnose problems, and we
need to delete those after a certain amount of time.  These are just
some of the things we've learned.

None of those tasks are OpenStack specific, but they are all useful to
someone who wants to use cloud computing nodes with Jenkins.  In other
words, an abstraction layer is the perfect place to put that kind of
work.  A layer that knows about things like images, pools of machines,
how to retrieve them from a provider (any provider), and then another
layer that hands that information to Jenkins.

The devstack-gate[1] project is the system we use to do that right now.
It's a set of python scripts that manages all of what I've described
above, external to Jenkins.  We consider it a prototype.  That logic
belongs in something accessible to Jenkins directly so that Jenkins can
manage the full life-cycle of the node.  That's why we're working with
the jclouds developers on incorporating everything we've learned into
jclouds (and yes, the pooling logic, etc, are going in jclouds, not the
jclouds plugin -- Jenkins isn't the only system that might use that
tactic), so that the jclouds Jenkins plugin can eventually support
everything we're doing now, and more.

I wish you well as I would any software that supports OpenStack.  But I
think the jclouds project is going to provide a great benefit to
OpenStack as I outlined above, and I'm going to be thrilled when we are
able to express the cloud testing requirements of the OpenStack CI
project directly in Jenkins.

As Monty has for quite some time, I would encourage anyone interested in
working in this area to contribute to the jclouds project.  And of
course anyone interested in CI or developer tooling is welcome to
contribute to the OpenStack CI system.  We run the project
infrastructure just like any other OpenStack project -- all of our
code[2] and puppet scripts[3] are in gerrit and our bugs (we have a huge
task list) are in launchpad[4].  We meet weekly on IRC[5].

[1] https://github.com/openstack-ci/devstack-gate
[2] https://github.com/openstack/openstack-ci
[3] https://github.com/openstack/openstack-ci-puppet
[4] https://bugs.launchpad.net/openstack-ci/
[5] http://wiki.openstack.org/Meetings

-Jim