← Back to team overview

nova team mailing list archive

Re: Database is locked after running nova-manage

 

Hi Manuel,

Here's some more food for thought...there is the ongoing question of
making all openstack projects consistent in the way they process CLI
options *and* configuration file options.  Currently, there is quite a
difference in how this is handled for Nova and Swift.  I've outlined a
proposal (not quite done yet, but 95% done) here:

http://wiki.openstack.org/CommonOptionsProcessing

I'll be proposing this shortly to the Nova/OpenStack mailing lists,
but work will likely not go forward until the Bexar release (after Oct
21st).

So, in short, yep, I understand your concerns and yep, we're trying to
come up with a solution that makes sense for all the projects and
would love to get your input!  Please do feel free to pudate the wiki
page above with any ideas you may have. :)

Cheers!

jay

On Fri, Sep 17, 2010 at 7:54 AM, Joshua McKenty <jmckenty@xxxxxxxxx> wrote:
> You're welcome to submit a patch for review :)
>
> On Fri, Sep 17, 2010 at 1:47 PM, Manuel Amador (Rudd-O) <rudd-o@xxxxxxxxxx>
> wrote:
>>
>> It's really unexpected.  No UNIX utility behaves like that.  The closest
>> you
>> can see in actual usage is ssh, where you specify configuration options
>> with
>> the -o 'ConfigName ConfigValue' command line option, but actual options
>> like
>> --some-option have no equivalent in the configuration.
>>
>> El Friday, September 17, 2010, Joshua McKenty escribió:
>> > There's a blueprint on this topic (optparser) already, and a rather
>> > lengthy
>> > debate went on early in nova development about whether config and CLI
>> > should be overlapping. As I pointed out in IRC the other day, we went
>> > with
>> > the gflags approach (e.g., doing it the way google does), simply because
>> > that was what termie was most familiar with, and none of the rest of us
>> > had a strong enough preference to argue (much).
>> >
>> > As far as it being totally unexpected, it's not really. I had thought
>> > that
>> > the debian package at least generated some flagfiles - I'm sure that the
>> > chef cookbook does.
>> >
>> > Joshua
>> >
>> > On Fri, Sep 17, 2010 at 2:55 AM, Manuel Amador (Rudd-O)
>> >
>> > <rudd-o@xxxxxxxxxx>wrote:
>> > > Seriously -- it is totally unexpected that the command-line options
>> > > double as
>> > > the run-time configuration of the programs, or that a specified config
>> > > file option invalid for a particular program does not make it exit
>> > > with
>> > > $? = 64.
>> > >
>> > > If the complexity of the stack is such that you are in practice going
>> > > to
>> > > require people to create config files, then I highly suggest not
>> > > abusing
>> > > the
>> > > concept of command-line options for that; rather, just ship example
>> > > configuration files with sane defaults, and document the alternatives
>> > > right in
>> > > the config file.
>> > >
>> > > This makes it *WAY* easier for the beginner user to get around to set
>> > > up
>> > > the
>> > > openstack, and it *does not* break formal expectations of command line
>> > > options.
>> > >
>> > > El Thursday, September 16, 2010, Vishvananda Ishaya escribió:
>> > > > Yes we modified python-gflags to ignore unknown options.  This
>> > > > allows
>> > > > us
>> > >
>> > > to
>> > >
>> > > > specify options in dynamically imported files and to use the same
>> > >
>> > > flagfile
>> > >
>> > > > for all of the binaries if necessary.
>> > > >
>> > > > --help should show the majority of the available flags, but if there
>> > > > are flags in dynamically imported files they will not show up.  For
>> > > > example flags in volume/manager.py do not show up in --help, nor do
>> > > > the flags in volume/driver.py, but they are used in nova-volume
>> > > >
>> > > > we should probably use flags.DECLARE to show the possible flags in
>> > >
>> > > --help,
>> > >
>> > > > but it is a little challenging with pluggable backends. Perhaps the
>> > > > best solution would be to add a flags.DEFINE_plugin which works like
>> > > > DEFINE_string but will import the specified file and add any other
>> > > > flags that are defined in it.
>> > > >
>> > > > As far as validating the flags, it is a tradeoff.  Not validating
>> > > > means
>> > >
>> > > we
>> > >
>> > > > can throw a bunch of flags into a file and not care if there are
>> > > > extras
>> > > > specified, but validating saves you from mistyped flags.  If we had
>> > > > the
>> > > > above DEFINE support we could switch to validating flags again.
>> > > >
>> > > > Vish
>> > > >
>> > > > On Sep 16, 2010, at 5:16 PM, Manuel Amador (Rudd-O) wrote:
>> > > > > So when you say flags, what you mean is command-line options?
>> > > > >
>> > > > > Then I have to report that I can pass any number of crazy nonsense
>> > > > > options to the nova programs, and the programs do not validate
>> > > > > them.
>> > > > >
>> > > > > I did not use pip.  I do not like to rely on non-package manager
>> > >
>> > > programs
>> > >
>> > > > > modifying my /usr.
>> > > > >
>> > > > > OK, so I can ignore the errors.
>> > > > >
>> > > > > I can see the project admin was created.
>> > > > >
>> > > > > THANKS!
>> > > > >
>> > > > > El Thursday, September 16, 2010, Vishvananda Ishaya escribió:
>> > > > >> Those errors are normal.  SQLite isn't very good with multiple
>> > >
>> > > threads,
>> > >
>> > > > >> but it doesn't actually cause any real problems.
>> > > > >>
>> > > > >> Don't know why redis wasn't installed, it should have gotten it
>> > > > >> from
>> > >
>> > > the
>> > >
>> > > > >> pip requires.
>> > > > >>
>> > > > >> You can ignore the scheduler error, it is in the scheduler
>> > > > >> branch,
>> > >
>> > > which
>> > >
>> > > > >> hasn't been merged into trunk yet.
>> > > > >>
>> > > > >> nova uses python-gflags.  To set flags you can either use
>> > > > >> --FLAGNAME=FLAGVALUE on the commandline when you run each of the
>> > > > >> bins
>> > >
>> > > or
>> > >
>> > > > >> you can stick a bunch of them into a file with one line per flag,
>> > > > >> like so --s3_url=127.0.0.1
>> > > > >> --auth_driver=nova.auth.ldapdriver.LdapDriver
>> > > > >> and specify the file on the commandline:
>> > > > >> ./nova-compute --flagfile=/path/to/flagfile
>> > > > >>
>> > > > >> If you have further questions, you might want to just jump in
>> > >
>> > > #openstack
>> > >
>> > > > >> and irc.freenode.net and we can discuss there.
>> > > > >>
>> > > > >> On Sep 16, 2010, at 2:20 PM, Manuel Amador (Rudd-O) wrote:
>> > > > >>> When I ran
>> > > > >>>
>> > > > >>> bin/nova-manage project create admin admin
>> > > > >>>
>> > > > >>> Nova-volume said this:
>> > > > >>>
>> > > > >>> -------------------------
>> > > > >>> ERROR:root:model server went away
>> > > > >>>
>> > > > >>> Traceback (most recent call last):
>> > > > >>> File "/home/rudd-o/openstack/nova/nova/service.py", line 139, in
>> > > > >>>
>> > > > >>> report_state
>> > > > >>>
>> > > > >>>   service_ref = db.service_get(context, self.service_id)
>> > > > >>>
>> > > > >>> File "/home/rudd-o/openstack/nova/nova/db/api.py", line 56, in
>> > > > >>> service_get
>> > > > >>>
>> > > > >>>   return IMPL.service_get(context, service_id)
>> > > > >>>
>> > > > >>> File "/home/rudd-o/openstack/nova/nova/db/sqlalchemy/api.py",
>> > > > >>> line
>> > >
>> > > 50,
>> > >
>> > > > >>> in
>> > > > >>>
>> > > > >>> service_get
>> > > > >>>
>> > > > >>>   return models.Service.find(service_id)
>> > > > >>>
>> > > > >>> File "/home/rudd-o/openstack/nova/nova/db/sqlalchemy/models.py",
>> > > > >>> line 79, in
>> > > > >>>
>> > > > >>> find
>> > > > >>>
>> > > > >>>   ).filter_by(deleted=deleted
>> > > > >>>
>> > > > >>> File "/usr/lib/python2.6/dist-packages/sqlalchemy/orm/query.py",
>> > > > >>> line 1522,
>> > > > >>>
>> > > > >>> in one
>> > > > >>>
>> > > > >>>   ret = list(self)
>> > > > >>>
>> > > > >>> File "/usr/lib/python2.6/dist-packages/sqlalchemy/orm/query.py",
>> > > > >>> line 1565,
>> > > > >>>
>> > > > >>> in __iter__
>> > > > >>>
>> > > > >>>   return self._execute_and_instances(context)
>> > > > >>>
>> > > > >>> File "/usr/lib/python2.6/dist-packages/sqlalchemy/orm/query.py",
>> > > > >>> line 1570,
>> > > > >>>
>> > > > >>> in _execute_and_instances
>> > > > >>>
>> > > > >>>   mapper=self._mapper_zero_or_none())
>> > > > >>>
>> > > > >>> File
>> > > > >>> "/usr/lib/python2.6/dist-packages/sqlalchemy/orm/session.py",
>> > >
>> > > line
>> > >
>> > > > >>> 735,
>> > > > >>>
>> > > > >>> in execute
>> > > > >>>
>> > > > >>>   clause, params or {})
>> > > > >>>
>> > > > >>> File
>> > > > >>> "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py",
>> > >
>> > > line
>> > >
>> > > > >>> 1157, in execute
>> > > > >>>
>> > > > >>>   params)
>> > > > >>>
>> > > > >>> File
>> > > > >>> "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py",
>> > >
>> > > line
>> > >
>> > > > >>> 1237, in _execute_clauseelement
>> > > > >>>
>> > > > >>>   return self.__execute_context(context)
>> > > > >>>
>> > > > >>> File
>> > > > >>> "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py",
>> > >
>> > > line
>> > >
>> > > > >>> 1268, in __execute_context
>> > > > >>>
>> > > > >>>   context.parameters[0], context=context)
>> > > > >>>
>> > > > >>> File
>> > > > >>> "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py",
>> > >
>> > > line
>> > >
>> > > > >>> 1367, in _cursor_execute
>> > > > >>>
>> > > > >>>   context)
>> > > > >>>
>> > > > >>> File
>> > > > >>> "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py",
>> > >
>> > > line
>> > >
>> > > > >>> 1360, in _cursor_execute
>> > > > >>>
>> > > > >>>   context)
>> > > > >>>
>> > > > >>> File
>> > > > >>> "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/default.py",
>> > > > >>> line
>> > > > >>>
>> > > > >>> 288, in do_execute
>> > > > >>>
>> > > > >>>   cursor.execute(statement, parameters)
>> > > > >>>
>> > > > >>> OperationalError: (OperationalError) database is locked u'SELECT
>> > > > >>> services.created_at AS services_created_at, services.updated_at
>> > > > >>> AS
>> > > > >>> services_updated_at, services.deleted_at AS services_deleted_at,
>> > > > >>> services.deleted AS services_deleted, services.id AS
>> > > > >>> services_id,
>> > > > >>> services.host AS services_host, services.binary AS
>> > > > >>> services_binary,
>> > > > >>> services.topic AS services_topic, services.report_count AS
>> > > > >>> services_report_count \nFROM services \nWHERE services.id = ?
>> > > > >>> AND
>> > > > >>> services.deleted = ?' (2, False)
>> > > > >>> 2010-09-16 14:19:17-0700 [-] (root): ERROR model server went
>> > > > >>> away
>> > > > >>> 2010-09-16 14:19:17-0700 [-] Traceback (most recent call last):
>> > > > >>> 2010-09-16 14:19:17-0700 [-]   File "/home/rudd-
>> > > > >>> o/openstack/nova/nova/service.py", line 139, in report_state
>> > > > >>> 2010-09-16 14:19:17-0700 [-]     service_ref =
>> > >
>> > > db.service_get(context,
>> > >
>> > > > >>> self.service_id)
>> > > > >>> 2010-09-16 14:19:17-0700 [-]   File "/home/rudd-
>> > > > >>> o/openstack/nova/nova/db/api.py", line 56, in service_get
>> > > > >>> 2010-09-16 14:19:17-0700 [-]     return
>> > > > >>> IMPL.service_get(context,
>> > > > >>> service_id) 2010-09-16 14:19:17-0700 [-]   File "/home/rudd-
>> > > > >>> o/openstack/nova/nova/db/sqlalchemy/api.py", line 50, in
>> > > > >>> service_get 2010-09-16 14:19:17-0700 [-]     return
>> > >
>> > > models.Service.find(service_id)
>> > >
>> > > > >>> 2010-09-16 14:19:17-0700 [-]   File "/home/rudd-
>> > > > >>> o/openstack/nova/nova/db/sqlalchemy/models.py", line 79, in find
>> > > > >>> 2010-09-16 14:19:17-0700 [-]     ).filter_by(deleted=deleted
>> > > > >>> 2010-09-16 14:19:17-0700 [-]   File "/usr/lib/python2.6/dist-
>> > > > >>> packages/sqlalchemy/orm/query.py", line 1522, in one
>> > > > >>> 2010-09-16 14:19:17-0700 [-]     ret = list(self)
>> > > > >>> 2010-09-16 14:19:17-0700 [-]   File "/usr/lib/python2.6/dist-
>> > > > >>> packages/sqlalchemy/orm/query.py", line 1565, in __iter__
>> > > > >>> 2010-09-16 14:19:17-0700 [-]     return
>> > > > >>> self._execute_and_instances(context) 2010-09-16 14:19:17-0700
>> > > > >>> [-]
>> > > > >>> File "/usr/lib/python2.6/dist-
>> > > > >>> packages/sqlalchemy/orm/query.py", line 1570, in
>> > >
>> > > _execute_and_instances
>> > >
>> > > > >>> 2010-09-16 14:19:17-0700 [-]
>> > > > >>> mapper=self._mapper_zero_or_none()) 2010-09-16 14:19:17-0700 [-]
>> > > > >>> File "/usr/lib/python2.6/dist-
>> > > > >>> packages/sqlalchemy/orm/session.py", line 735, in execute
>> > > > >>> 2010-09-16 14:19:17-0700 [-]     clause, params or {})
>> > > > >>> 2010-09-16 14:19:17-0700 [-]   File "/usr/lib/python2.6/dist-
>> > > > >>> packages/sqlalchemy/engine/base.py", line 1157, in execute
>> > > > >>> 2010-09-16 14:19:17-0700 [-]     params)
>> > > > >>> 2010-09-16 14:19:17-0700 [-]   File "/usr/lib/python2.6/dist-
>> > > > >>> packages/sqlalchemy/engine/base.py", line 1237, in
>> > > > >>> _execute_clauseelement 2010-09-16 14:19:17-0700 [-]     return
>> > > > >>> self.__execute_context(context) 2010-09-16 14:19:17-0700 [-]
>> > > > >>> File
>> > > > >>> "/usr/lib/python2.6/dist-
>> > > > >>> packages/sqlalchemy/engine/base.py", line 1268, in
>> > > > >>> __execute_context 2010-09-16 14:19:17-0700 [-]
>> > > > >>> context.parameters[0],
>> > > > >>> context=context) 2010-09-16 14:19:17-0700 [-]   File
>> > > > >>> "/usr/lib/python2.6/dist-
>> > > > >>> packages/sqlalchemy/engine/base.py", line 1367, in
>> > > > >>> _cursor_execute
>> > > > >>> 2010-09-16 14:19:17-0700 [-]     context)
>> > > > >>> 2010-09-16 14:19:17-0700 [-]   File "/usr/lib/python2.6/dist-
>> > > > >>> packages/sqlalchemy/engine/base.py", line 1360, in
>> > > > >>> _cursor_execute
>> > > > >>> 2010-09-16 14:19:17-0700 [-]     context)
>> > > > >>> 2010-09-16 14:19:17-0700 [-]   File "/usr/lib/python2.6/dist-
>> > > > >>> packages/sqlalchemy/engine/default.py", line 288, in do_execute
>> > > > >>> 2010-09-16 14:19:17-0700 [-]     cursor.execute(statement,
>> > >
>> > > parameters)
>> > >
>> > > > >>> 2010-09-16 14:19:17-0700 [-] OperationalError:
>> > > > >>> (OperationalError)
>> > > > >>> database is locked u'SELECT services.created_at AS
>> > >
>> > > services_created_at,
>> > >
>> > > > >>> services.updated_at AS services_updated_at, services.deleted_at
>> > > > >>> AS
>> > > > >>> services_deleted_at, services.deleted AS services_deleted,
>> > >
>> > > services.id
>> > >
>> > > > >>> AS services_id, services.host AS services_host, services.binary
>> > > > >>> AS
>> > > > >>> services_binary, services.topic AS services_topic,
>> > > > >>> services.report_count AS services_report_count \nFROM services
>> > >
>> > > \nWHERE
>> > >
>> > > > >>> services.id = ? AND services.deleted = ?' (2, False)
>> > > > >>> ERROR:root:Recovered model server connection!
>> > > > >>> 2010-09-16 14:19:52-0700 [-] (root): ERROR Recovered model
>> > > > >>> server
>> > > > >>> connection ---------------------------
>> > > > >>>
>> > > > >>> Identical tracebacks in nova-network and nova-compute.
>> > > > >>>
>> > > > >>> I am stuck here :-)
>> > > > >>>
>> > > > >>> _______________________________________________
>> > > > >>> Mailing list: https://launchpad.net/~nova
>> > > > >>> Post to     : nova@xxxxxxxxxxxxxxxxxxx
>> > > > >>> Unsubscribe : https://launchpad.net/~nova
>> > > > >>> More help   : https://help.launchpad.net/ListHelp
>> > >
>> > > _______________________________________________
>> > > Mailing list: https://launchpad.net/~nova
>> > > Post to     : nova@xxxxxxxxxxxxxxxxxxx
>> > > Unsubscribe : https://launchpad.net/~nova
>> > > More help   : https://help.launchpad.net/ListHelp
>>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~nova
> Post to     : nova@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~nova
> More help   : https://help.launchpad.net/ListHelp
>
>



References