openstack team mailing list archive
-
openstack team
-
Mailing list archive
-
Message #08056
Re: Howto Nova setup with HA?
Hi Jay,
I am using a Gelera master/master setup with Pacemaker providing an floating IP.
Best regards,
Tristan van Bokkem
Datacenter Operations
Contact:
E-mail Personal: tristanvanbokkem@xxxxxxx
E-mail Support: info@xxxxxxx
E-mail NOC: noc@xxxxxxx
Website: http://www.i3d.net Office:
Interactive 3D B.V.
Meent 93b
3011 JG Rotterdam
The Netherlands
Visit www.smartdc.net – SmartDC is our in-house 36,000 sq. ft. datacenter in Rotterdam, The Netherlands. High density hosting – multiple fiber carriers in-house – Level3 PoP.
Interactive 3D (i3D.net) is a company registered in The Netherlands at Meent 93b, Rotterdam. Registration #: 14074337 - VAT # NL 8202.63.886.B01. Interactive 3D (i3D.net) is CDSA certified on content protection and security. We are ranked in the Deloitte Technology Fast 50 as one of the fastest growing technology companies.
_____
From: Jay Pipes [mailto:jaypipes@xxxxxxxxx]
To: openstack@xxxxxxxxxxxxxxxxxxx
Sent: Mon, 27 Feb 2012 21:40:21 +0100
Subject: Re: [Openstack] Howto Nova setup with HA?
On 02/27/2012 08:06 AM, i3D.net - Tristan van Bokkem wrote:
> MySQL DB
> * multi-master configuration with pacemaker providing an floating IP. <-
> seems to work.
The term "multi-master" in the MySQL world typically means something
very specific: you are running multiple master nodes, all in an active
setup using something like MMM, MySQL Cluster or Galera Cluster. I don't
think this is what you are referring to above. Use of pacemaker is
typically an active/passive setup, where Pacemaker can be used to
redirect requests from a failed active master to the passive master node
that is in standby mode (basically, DRBD is used to replicate the InnoDB
and binary logs from the active master to the passive node).
So, I would avoid using the term "multi-master" unless you are
specifically talking about an active-active setup, but maybe that's just
me being picky ;)
> Glance
> * Run on multiple servers <-- how do they get synced?
For the Glance registry, you can set up a passive glance-registry node
that has connection information to your main HA-setup MySQL database
instance.
You can do a number of things for the image storage side of things (the
Glance API servers, not the Glance registry). You could set Glance to
use a distributed storage backend (like Swift or RADOS) and let that
distributed backend provide high availability.
Alternately, for smaller setups, you could do like we are doing in the
TryStack project and use the filesystem storage backend in Glance and
have Pacemaker monitor a passive Glance API node with a simple rsync
script that runs on a periodic interval that syncs the image data.
> Compute servers
> * Each run their own copy of nova-api (only instances running on the
> node use this)
No. Compute nodes run nova-compute, not nova-api.
> * nova-network (multi-host configuration) with private network <-- Needs
> its own discussion I guess.
>
> Keystone
> * How about keystone, same as glance I guess?
Similar to glance-registry.
Cheers,
-jay
> Best regards,
>
> Tristan van Bokkem
> Datacenter Operations
>
> Contact:
> E-mail Personal: tristanvanbokkem@xxxxxxx <mailto:tristanvanbokkem@xxxxxxx>
> E-mail Support: info@xxxxxxx <mailto:info@xxxxxxx>
> E-mail NOC: noc@xxxxxxx <mailto:noc@xxxxxxx>
> Website: http://www.i3d.net Office:
> Interactive 3D B.V.
> Meent 93b
> 3011 JG Rotterdam
> The Netherlands
>
> Visit www.smartdc.net – SmartDC is our in-house 36,000 sq. ft.
> datacenter in Rotterdam, The Netherlands. High density hosting –
> multiple fiber carriers in-house – Level3 PoP.
>
> Interactive 3D (i3D.net) is a company registered in The Netherlands at
> Meent 93b, Rotterdam. Registration #: 14074337 - VAT # NL
> 8202.63.886.B01. Interactive 3D (i3D.net) is CDSA certified on content
> protection and security. We are ranked in the Deloitte Technology Fast
> 50 as one of the fastest growing technology companies.
>
> ------------------------------------------------------------------------
> *From:* Dan Wendlandt [mailto:dan@xxxxxxxxxx]
> *To:* Ghe Rivero [mailto:ghe.rivero@xxxxxxxxxxxx]
> *Cc:* netstack@xxxxxxxxxxxxxxxxxxx, darren.birkett@xxxxxxxxx,
> openstack@xxxxxxxxxxxxxxxxxxx
> *Sent:* Tue, 21 Feb 2012 18:13:55 +0100
> *Subject:* Re: [Openstack] Howto Nova setup with HA?
>
>
>
> 2012/2/21 Ghe Rivero <ghe.rivero@xxxxxxxxxxxx
> <mailto:ghe.rivero@xxxxxxxxxxxx>>
>
> BigSwitch and Nicira have created different blueprints in
> quantum to add their plugins for essex-4 about ten days ago,
> Cisco already have it included almost from the beginning. No
> news of them for the moment, but i really hope that both of them
> make it in time!
> Ghe Rivero
>
> https://blueprints.launchpad.net/quantum/+spec/quantum-nvp-plugin
> https://blueprints.launchpad.net/quantum/+spec/quantum-bigswitch-plugin
>
>
> To be clear, there are several more existing and planned plugins for
> Quantum, even beyond the three you mention. Each plugin will make
> its own trade-offs in terms of HA for control plane and data plane
> components. I'm planning on organizing a talk at the Folsom
> summit/conference where people have a chance to talk about their
> plugins. No need for a flame war here :)
>
> One other point of confusion that I'd like to address is that there
> is NO requirement that a plugin be merged into the main Quantum repo
> for it to be a "valid" plugin. I know of several plugins that are
> used despite not being merged in (in fact, at the summit, we're
> going to discuss if ANY plugins should remain in the main repo...
> this is likely a larger decision that should be made consistently
> across all openstack projects).
>
> Dan
>
>
>
> 2012/2/21 Diego Parrilla Santamaría
> <diego.parrilla.santamaria@xxxxxxxxx
> <mailto:diego.parrilla.santamaria@xxxxxxxxx>>
>
> This is the kind of flame war I love... ;-)
>
> if you think the HA challenges in Openstack Nova are at
> MySQL and Rabbit level, you are not putting the focus on the
> right components. The biggest HA (and scalability)
> challenges in Nova are in the networking side. There are a
> lot of companies tackling the networking challenges of Nova
> (BigSwitch, Nicira, Cisco, etc...), and I would like to see
> how they evolve in the future and how they will address this
> problem. Right now, only promises.
>
> Fight!
> --
> Diego Parrilla
> <http://www.stackops.com/>*CEO*
> **www.stackops.com* <http://www.stackops.com/> |
> *diego.parrilla@xxxxxxxxxxxx
> <mailto:diego.parrilla@xxxxxxxxxxxx>****| +34 649 94 43 29
> <tel:%2B34%20649%2094%2043%2029> | skype:diegoparrilla*
> * <http://www.stackops.com/>
> **
>
>
>
> On Tue, Feb 21, 2012 at 1:04 PM, i3D.net - Tristan van
> Bokkem <tristanvanbokkem@xxxxxx
> <mailto:tristanvanbokkem@xxxxxx>> wrote:
>
> __
> Hi Darren,
>
> No, I am not looking at a performance increasment at
> this moment. Just trying to make the different
> components Nova is build on redundant. Of course a
> master/master setup of mysql / rabbitmq with a floating
> IP will probably work but it bugs me that Nova is not
> really build with flexibility of it components in mind.
>
> I understand it is impossible to cover every scenario
> and you can make things redundant into the infinitive
> but why doesn't have Nova any of this build in? Why do I
> need to setup a Virtual IP/floating IP where Nova simple
> could accept multiple locations and failover if the
> first location fails. Or even round-robin to distribute
> the load. All I am doing now is yet add another
> component that can possibly fail as well.
>
>
> Best regards,
>
> Tristan van Bokkem
> Datacenter Operations
>
> Contact:
> E-mail Personal: tristanvanbokkem@xxxxxxx
> <mailto:tristanvanbokkem@xxxxxxx>
> E-mail Support: info@xxxxxxx <mailto:info@xxxxxxx>
> E-mail NOC: noc@xxxxxxx <mailto:noc@xxxxxxx>
> Website: http://www.i3d.net Office:
> Interactive 3D B.V.
> Meent 93b
> 3011 JG Rotterdam
> The Netherlands
>
> Visit www.smartdc.net <http://www.smartdc.net> – SmartDC
> is our in-house 36,000 sq. ft. datacenter in Rotterdam,
> The Netherlands. High density hosting – multiple fiber
> carriers in-house – Level3 PoP.
>
> Interactive 3D (i3D.net) is a company registered in The
> Netherlands at Meent 93b, Rotterdam. Registration #:
> 14074337 - VAT # NL 8202.63.886 <tel:8202.63.886>.B01.
> Interactive 3D (i3D.net) is CDSA certified on content
> protection and security. We are ranked in the Deloitte
> Technology Fast 50 as one of the fastest growing
> technology companies.
>
> ------------------------------------------------------------------------
> *From:* Darren Birkett
> [mailto:darren.birkett@xxxxxxxxx
> <mailto:darren.birkett@xxxxxxxxx>]
> *To:* i3D.net - Tristan van Bokkem
> [mailto:tristanvanbokkem@xxxxxx
> <mailto:tristanvanbokkem@xxxxxx>]
> *Cc:* Florian Haas [mailto:florian@xxxxxxxxxxx
> <mailto:florian@xxxxxxxxxxx>],
> openstack@xxxxxxxxxxxxxxxxxxx
> <mailto:openstack@xxxxxxxxxxxxxxxxxxx>
> *Sent:* Tue, 21 Feb 2012 12:51:11 +0100
>
> *Subject:* Re: [Openstack] Howto Nova setup with HA?
>
> Surely it depends what you want out of it? If just
> pure high
> availability (ie ability to continue functioning
> when primary node
> goes down), then doesn't a master/master with
> floating IP fit the
> bill?
>
> It sounds like what you're after is better
> performance/throughput
> rather than HA - the ability to write to multiple
> masters at the same
> time. Even here if nova *was* able to hit multiple
> databases, a
> master/master setup isn't going to suit, as each
> write to one master
> still needs to be replicated to the other so
> throughput is going to be
> the same. You'd then need to look to something like
> a native mysql
> cluster to get actual increased throughput.
>
> Darren
>
> On 21 February 2012 11:34, i3D.net - Tristan van Bokkem
>
> <tristanvanbokkem@xxxxxx
> <mailto:tristanvanbokkem@xxxxxx>> wrote:
> > Ho Florian,
> >
> > I respect your opinion of course and I am not
> here to start a fight. Point
> > is, how I understand it Nova really isn't very
> flexible on High Availability
> > terms. I have now a master/master mysql setup
> with a Virtual IP working just
> > to work around the shortcomings Nova imo
> introduces. This same story goes
> > for RabbitMQ, I need to setup RabbitMQ in HA
> (which is fine) but I need to
> > setup a Virtual IP for it as well just so Nova
> can correctly connect to it.
> > Without the VIP the hole point of setting it up
> in a cluster is useless as
> > you still will point nova to one single host.
> >
> > It shouldn't be that hard to make Nova listen to
> multiple mysql addresses
> > and rabbitmq hosts in a cluster?
> >
> >
> > Best regards,
> >
> > Tristan van Bokkem
> > Datacenter Operations
> >
> > Contact:
> > E-mail Personal: tristanvanbokkem@xxxxxxx
> <mailto:tristanvanbokkem@xxxxxxx>
> > E-mail Support: info@xxxxxxx <mailto:info@xxxxxxx>
> > E-mail NOC: noc@xxxxxxx <mailto:noc@xxxxxxx>
> > Website: http://www.i3d.net Office:
> > Interactive 3D B.V.
> > Meent 93b
> > 3011 JG Rotterdam
> > The Netherlands
> >
> > Visit www.smartdc.net <http://www.smartdc.net> –
> SmartDC is our in-house 36,000 sq. ft. datacenter in
> > Rotterdam, The Netherlands. High density hosting
> – multiple fiber carriers
> > in-house – Level3 PoP.
> >
> > Interactive 3D (i3D.net) is a company registered
> in The Netherlands at Meent
> > 93b, Rotterdam. Registration #: 14074337 - VAT #
> NL 8202.63.886 <tel:8202.63.886>.B01.
> > Interactive 3D (i3D.net) is CDSA certified on
> content protection and
> > security. We are ranked in the Deloitte
> Technology Fast 50 as one of the
> > fastest growing technology companies.
> >
> > ________________________________
> > From: Florian Haas [mailto:florian@xxxxxxxxxxx
> <mailto:florian@xxxxxxxxxxx>]
> > To: openstack@xxxxxxxxxxxxxxxxxxx
> <mailto:openstack@xxxxxxxxxxxxxxxxxxx>
> > Sent: Fri, 17 Feb 2012 13:32:51 +0100
> >
> > Subject: Re: [Openstack] Howto Nova setup with HA?
> >
> > On Thu, Feb 16, 2012 at 3:28 PM, i3D.net -
> Tristan van Bokkem
> > <tristanvanbokkem@xxxxxx
> <mailto:tristanvanbokkem@xxxxxx>> wrote:
> >> Hi Tom,
> >>
> >> Thank you very much for sharing your knowledge,
> this will help not only
> >> me!
> >> As I am still working on the MySQL part, you
> mentioned multi-master which
> >> I
> >> have running indeed, but I suppose there still
> will need to be some kind
> >> of
> >> entering point within the cluster. How did you
> managed this?
> >
> > Virtual cluster IP addresses and highly available
> MySQL (either DRBD
> > or MySQL replication based). Pacemaker is very
> well capable of
> > managing this.
> >
> > Of course, since you also quoted
> >
> http://openlife.cc/blogs/2011/july/ultimate-mysql-high-availability-solution
> > parts of which I wholeheartedly disagree with (as
> with all of Henrik's
> > Pacemaker bashing, but he's of course entitled to
> his own opinion),
> > then you might run away screaming from anything
> that barely mentions
> > Pacemaker in passing. But in reality[1],
> Pacemaker is an extremely
> > capable HA stack that would be well suited for this.
> >
> > Cheers,
> > Florian
> >
> > [1] To be polite, I should say "in my humble
> opinion," but my opinion
> > on this really isn't humble at all, so I prefer
> being honest over
> > being polite. :)
> >
> > --
> > Need help with High Availability?
> > http://www.hastexo.com/now
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~openstack
> <https://launchpad.net/%7Eopenstack>
> > Post to : openstack@xxxxxxxxxxxxxxxxxxx
> <mailto:openstack@xxxxxxxxxxxxxxxxxxx>
> > Unsubscribe : https://launchpad.net/~openstack
> <https://launchpad.net/%7Eopenstack>
> > More help : https://help.launchpad.net/ListHelp
> >
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~openstack
> <https://launchpad.net/%7Eopenstack>
> > Post to : openstack@xxxxxxxxxxxxxxxxxxx
> <mailto:openstack@xxxxxxxxxxxxxxxxxxx>
> > Unsubscribe : https://launchpad.net/~openstack
> <https://launchpad.net/%7Eopenstack>
> > More help : https://help.launchpad.net/ListHelp
> >
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> <https://launchpad.net/%7Eopenstack>
> Post to : openstack@xxxxxxxxxxxxxxxxxxx
> <mailto:openstack@xxxxxxxxxxxxxxxxxxx>
> Unsubscribe : https://launchpad.net/~openstack
> <https://launchpad.net/%7Eopenstack>
> More help : https://help.launchpad.net/ListHelp
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> <https://launchpad.net/%7Eopenstack>
> Post to : openstack@xxxxxxxxxxxxxxxxxxx
> <mailto:openstack@xxxxxxxxxxxxxxxxxxx>
> Unsubscribe : https://launchpad.net/~openstack
> <https://launchpad.net/%7Eopenstack>
> More help : https://help.launchpad.net/ListHelp
>
>
>
>
> --
> Ghe Rivero
> *OpenStack & Distribution Engineer
> ***www.stackops.com* <http://www.stackops.com/> |
> *ghe.rivero@xxxxxxxxxxxx
> <mailto:diego.parrilla@xxxxxxxxxxxx>****| +34 625 63 45 23 |
> skype:ghe.rivero*
> * <http://www.stackops.com/>
> *
>
> ****
>
> *
>
> ******************** ADVERTENCIA LEGAL ********************
> Le informamos, como destinatario de este mensaje, que el correo
> electrónico y las comunicaciones por medio de Internet no
> permiten asegurar ni garantizar la confidencialidad de los
> mensajes transmitidos, así como tampoco su integridad o su
> correcta recepción, por lo que STACKOPS TECHNOLOGIES S.L. no
> asume responsabilidad alguna por tales circunstancias. Si no
> consintiese en la utilización del correo electrónico o de las
> comunicaciones vía Internet le rogamos nos lo comunique y ponga
> en nuestro conocimiento de manera inmediata. Este mensaje va
> dirigido, de manera exclusiva, a su destinatario y contiene
> información confidencial y sujeta al secreto profesional, cuya
> divulgación no está permitida por la ley. En caso de haber
> recibido este mensaje por error, le rogamos que, de forma
> inmediata, nos lo comunique mediante correo electrónico remitido
> a nuestra atención y proceda a su eliminación, así como a la de
> cualquier documento adjunto al mismo. Asimismo, le comunicamos
> que la distribución, copia o utilización de este mensaje, o de
> cualquier documento adjunto al mismo, cualquiera que fuera su
> finalidad, están prohibidas por la ley.
>
> ***************** PRIVILEGED AND CONFIDENTIAL ****************
> We hereby inform you, as addressee of this message, that e-mail
> and Internet do not guarantee the confidentiality, nor the
> completeness or proper reception of the messages sent and, thus,
> STACKOPS TECHNOLOGIES S.L. does not assume any liability for
> those circumstances. Should you not agree to the use of e-mail
> or to communications via Internet, you are kindly requested to
> notify us immediately. This message is intended exclusively for
> the person to whom it is addressed and contains privileged and
> confidential information protected from disclosure by law. If
> you are not the addressee indicated in this message, you should
> immediately delete it and any attachments and notify the sender
> by reply e-mail. In such case, you are hereby notified that any
> dissemination, distribution, copying or use of this message or
> any attachments, for any purpose, is strictly prohibited by law.
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> <https://launchpad.net/%7Eopenstack>
> Post to : openstack@xxxxxxxxxxxxxxxxxxx
> <mailto:openstack@xxxxxxxxxxxxxxxxxxx>
> Unsubscribe : https://launchpad.net/~openstack
> <https://launchpad.net/%7Eopenstack>
> More help : https://help.launchpad.net/ListHelp
>
>
>
>
> --
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Dan Wendlandt
> Nicira Networks: www.nicira.com <http://www.nicira.com>
> twitter: danwendlandt
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Follow ups
References
