maria-developers team mailing list archive

[Kristian Nielsen <knielsen@xxxxxxxxxxxxxxx>]

Colin Charles <colin@xxxxxxxxxxx> writes:

> I was speaking to Otto (CEO, MariaDB Foundation) and discussing that
> we should move the lists from Launchpad to something else. We

The most important is that people can subscribe to the list without having
to create an account. This is the problem with Launchpad.

However, the problem with the mailing list is not where it is hosted. It is
that the majority of development happens off-list!

The developer mailing list is the core of a healthy open-source project that
treats developers equally. It is where everyone, newbie or core developer,
can first learn how the project runs, and later follow what is going on.

But not so in MariaDB. So many people from SkySQL casually divide the world
into first-class SkySQL employees, who are included in discussions. And
second-class others, who are casually kept out, as a matter of course.

Just look at the below random example from a closed mailing list open only
to employees. Stuff like this happens all the time, on closed mailing lists,
closed IRC channels, and latest I think a company-internal chat tool called
"Slack".

Basically, MariaDB is turning into a proprietary project to build a business
for a struggling venture-funded startup. It just happens to be bound by the
original GPL license to release source of the main product, though even that
is desperately attempted being circumvented by stuff like "MariaDB
enterprise".

So Colin, and others: Why bother working to move the mailing list elsewhere?
Spend first your efforts to coerce everyone to actually use the mailing
list, and to remove non-public communication channels. Start with yourself.
Cc all mails you send to the public mailing list. Stop using non-puclic IRC
or other channels, and ask people to use the public channels if they want to
contact you.

Thanks,

 - Kristian.

=======================================================================

From: Rasmus Johansson <rasmus@xxxxxxxxxxx>
Subject: [All] Current and last 10.1.8 sprint
To: "all@xxxxxxxxxxx" <all@xxxxxxxxxxx>
Date: Wed, 30 Sep 2015 11:17:21 +0300 (2 weeks, 6 days, 21 hours ago)

Hi all,

I've now populated sprint 10.1.8-4 with the issues that we agreed upon
yesterday on the maria call or over IRC after that. The sprint can be
found here:

https://mariadb.atlassian.net/secure/RapidBoard.jspa?rapidView=27

There are a few things missing still which I hope we can add today:
- Bar will add some issues from the list of debian patches
- Holyfoot might have one issue missing. I had documented that
MDEV-7817 should be added to Holyfoot, but I think I got that number
wrong. HF, could you give me the correct number?

PLEASE NOTICE that this now is the last sprint of 10.1.8 after which
the plan is to release it as the first stable GA release of 10.1 as
discussed on the call. The release will happen after next week's Maria
call. Please scream and shout if you think there are some issues in
addition to the ones on the sprint that you think still must be fixed
before going GA.

All, please check that the sprint now includes what we discussed
yesterday and that it does look realistic for yourself.

BR,
Rasmus

=======================================================================

From: Elena Stepanova <elenst@xxxxxxxxxxxxxxxx>
Subject: Re: [All] Current and last 10.1.8 sprint
To: all@xxxxxxxxxxxxxxxxxx
Date: Thu, 1 Oct 2015 18:43:52 +0300 (2 weeks, 5 days, 13 hours ago)

Hi Rasmus, Serg, all,

On 30.09.2015 11:17, Rasmus Johansson wrote:
> Hi all,
>
> I've now populated sprint 10.1.8-4 with the issues that we agreed upon
> yesterday on the maria call or over IRC after that. The sprint can be
> found here:
>
> https://mariadb.atlassian.net/secure/RapidBoard.jspa?rapidView=27
>
> There are a few things missing still which I hope we can add today:
> - Bar will add some issues from the list of debian patches
> - Holyfoot might have one issue missing. I had documented that
> MDEV-7817 should be added to Holyfoot, but I think I got that number
> wrong. HF, could you give me the correct number?
>
> PLEASE NOTICE that this now is the last sprint of 10.1.8 after which
> the plan is to release it as the first stable GA release of 10.1 as
> discussed on the call. The release will happen after next week's Maria
> call. Please scream and shout if you think there are some issues in
> addition to the ones on the sprint that you think still must be fixed
> before going GA.


/me screaming #1:
https://mariadb.atlassian.net/browse/MDEV-8813

I don't see how we can go GA with binlog encryption (even disabled by
default), but without any ability to use the logs for
backup/SST/investigation/what-not.
And it's a new feature, which we technically cannot add after GA.

Regards,
/E

(to be continued)

=======================================================================

From: Colin Charles <colin@xxxxxxxxxxx>
Subject: Re: [All] Current and last 10.1.8 sprint
To: Elena Stepanova <elenst@xxxxxxxxxxxxxxxx>
Cc: all@xxxxxxxxxxxxxxxxxx
Date: Fri, 2 Oct 2015 00:00:15 +0800 (2 weeks, 5 days, 13 hours ago)

Hi Elena, all,

> On 1 Oct 2015, at 23:43, Elena Stepanova <elenst@xxxxxxxxxxxxxxxx> wrote:
> 
>> PLEASE NOTICE that this now is the last sprint of 10.1.8 after which
>> the plan is to release it as the first stable GA release of 10.1 as
>> discussed on the call. The release will happen after next week's Maria
>> call. Please scream and shout if you think there are some issues in
>> addition to the ones on the sprint that you think still must be fixed
>> before going GA.
> 
> 
> /me screaming #1:
> https://mariadb.atlassian.net/browse/MDEV-8813
> 
> I don't see how we can go GA with binlog encryption (even disabled
> by default), but without any ability to use the logs for
> backup/SST/investigation/what-not.
> And it's a new feature, which we technically cannot add after GA.

mysqlbinlog is a client utility. It already has “differences”, i.e. no
streaming backup, needing GTID support
(https://mariadb.atlassian.net/browse/MDEV-4989), etc. I don’t see how
this can hold back our GA, so that we can include this tool to work
with things post-GA (maybe even for 10.2)

But of course, I agree we should aim for everything to be “usable”. I
don’t have a good solution to this problem, and I don’t see a time
estimate for adding support to this for MDEV-8813. I also know that we
definitely have to release 10.1 as GA *before* 5.7 is GA, so
presumably all this happens before our company/dev meeting.

Maybe we can just add this as a warning in the release notes and also
when someone is turning on encryption?

cheers,
-colin

=======================================================================

From: Elena Stepanova <elenst@xxxxxxxxxxxxxxxx>
Subject: Re: [All] Current and last 10.1.8 sprint
To: Colin Charles <colin@xxxxxxxxxxx>
Cc: all@xxxxxxxxxxxxxxxxxx
Date: Thu, 1 Oct 2015 20:01:14 +0300 (2 weeks, 5 days, 12 hours ago)

Hi Colin, all,

> mysqlbinlog is a client utility. It already has “differences”,
> i.e. no streaming backup, needing GTID support
> (https://mariadb.atlassian.net/browse/MDEV-4989), etc. I don’t see
> how this can hold back our GA, so that we can include this tool to
> work with things post-GA (maybe even for 10.2)
>
> But of course, I agree we should aim for everything to be
> “usable”. I don’t have a good solution to this problem, and I don’t
> see a time estimate for adding support to this for MDEV-8813. I also
> know that we definitely have to release 10.1 as GA *before* 5.7 is
> GA, so presumably all this happens before our company/dev meeting.
>
> Maybe we can just add this as a warning in the release notes and
> also when someone is turning on encryption?


Okay, I understand -- at the end, business needs are above everything
else (no sarcasm). Usability and quality are first to give way.

But then, this call for "issues that must be fixed" is a pointless
exercise -- there is nothing we can realistically add to the sprint
without postponing the release, so lets just agree right now that 10.1
goes GA next week no matter what, and add a warning "if you use this
server, you will get problems" (sarcasm... kind of).

And please actually *remember* this decision, so that there is no "how
come nobody said there were problems, we would have of course fixed
everything before releasing" complaints later. I clearly remember them
after our previous business-need-driven releases, and it was not fun.


In fact, given that Daniel probably leaves for the meeting next
Thursday (?), and knowing our buildbot and other infrastructural
instability, we should aim to release the tree which we will have on
Tuesday, the very latest. Which basically means it should be frozen on
Saturday, to let it build fully, and leave Monday for fixing whatever
gets broken by Friday pushes.

Among other things, it most likely means *no* to systemd -- unless we
are unbelievably lucky and the first systemd-enabled builds in
buildbot (which will hopefully happen tonight) show great success,
there will be no time to fix anything.


Regards,
/E

=======================================================================

From: Colin Charles <colin@xxxxxxxxxxx>
Subject: Re: [All] Current and last 10.1.8 sprint
To: Elena Stepanova <elenst@xxxxxxxxxxxxxxxx>
Cc: all@xxxxxxxxxxxxxxxxxx
Date: Fri, 2 Oct 2015 01:32:32 +0800 (2 weeks, 5 days, 11 hours ago)

Hi Elena, all,

> Okay, I understand -- at the end, business needs are above
> everything else (no sarcasm). Usability and quality are first to
> give way.
> 
> But then, this call for "issues that must be fixed" is a pointless
> exercise -- there is nothing we can realistically add to the sprint
> without postponing the release, so lets just agree right now that
> 10.1 goes GA next week no matter what, and add a warning "if you use
> this server, you will get problems" (sarcasm... kind of).
> 

Overall, very good arguments and I cannot really claim to deviate from
them. Rasmus might want to chime in, based on pressures from the
products team?

> And please actually *remember* this decision, so that there is no
> "how come nobody said there were problems, we would have of course
> fixed everything before releasing" complaints later. I clearly
> remember them after our previous business-need-driven releases, and
> it was not fun.
> 

I wonder, what we do in situations like this. I mean there’s also the
MariaDB Foundation whom are supposed to decide on things like this,
but in theory we’re all captains… Do we query support @ MariaDB
Corporation to figure out if this is something acceptable for them?

> In fact, given that Daniel probably leaves for the meeting next
> Thursday (?), and knowing our buildbot and other infrastructural
> instability, we should aim to release the tree which we will have on
> Tuesday, the very latest. Which basically means it should be frozen
> on Saturday, to let it build fully, and leave Monday for fixing
> whatever gets broken by Friday pushes.
> 
> Among other things, it most likely means *no* to systemd -- unless
> we are unbelievably lucky and the first systemd-enabled builds in
> buildbot (which will hopefully happen tonight) show great success,
> there will be no time to fix anything.

Again more good points that I can’t argue with.

Original email says, "The release will happen after next week's Maria
call. Please scream and shout if you think there are some issues in
addition to the ones on the sprint that you think still must be fixed
before going GA.”

You bring up a good point that nothing realistically gets fixed from
now till then. So while I’m not Rasmus, maybe we do have to “delay”
the release till we’re at the company meeting. Again, I have no idea
what can get done from now till… whenever we freeze

Frankly, as long as we release before 19 Oct, I think we’ll be before
MySQL 5.7. However I think we need to also plan this as MariaDB
Corporation will want to do marketing around this (and we now have new
marketing lead(s)).

I think the GA release is 5.7.9, seeing
http://dev.mysql.com/doc/relnotes/mysql/5.7/en/news-5-7-9.html, but
also remember
http://dev.mysql.com/doc/relnotes/mysql/5.7/en/news-5-7-10.html is
out, which means they’ve pretty much baked GA and are doing “internal
smoke testing”, while ensuring that bug fixes go into 5.7.10. So to
me, their GA is “Ready”, and I can only imagine when they’ll drop it
to the world

=======================================================================

From: Rasmus Johansson <rasmus@xxxxxxxxxxx>
Subject: Re: [All] Current and last 10.1.8 sprint
To: Colin Charles <colin@xxxxxxxxxxx>
Cc: all@xxxxxxxxxxxxxxxxxx
Date: Thu, 1 Oct 2015 23:31:31 +0300 (2 weeks, 5 days, 9 hours ago)

1.  (*) text/plain          ( ) text/html           

Hi,

I suggest we have a conversation about the critical items like systemd,
encrypted binlogs and the timeline of the release tomorrow. I'll initiate a
discussion on IRC and set up a call if needed.

=======================================================================

From: Elena Stepanova <elenst@xxxxxxxxxxxxxxxx>
Subject: Re: [All] Current and last 10.1.8 sprint
To: Colin Charles <colin@xxxxxxxxxxx>
Cc: all@xxxxxxxxxxxxxxxxxx
Date: Fri, 2 Oct 2015 03:30:07 +0300 (2 weeks, 5 days, 5 hours ago)

Hi Colin, all,

On 01.10.2015 20:32, Colin Charles wrote:
>
> <skip>
>
> Overall, very good arguments and I cannot really claim to deviate
> from them. Rasmus might want to chime in, based on pressures from
> the products team?
>
>> And please actually *remember* this decision, so that there is no
>> "how come nobody said there were problems, we would have of course
>> fixed everything before releasing" complaints later. I clearly
>> remember them after our previous business-need-driven releases, and
>> it was not fun.
>>
>
> I wonder, what we do in situations like this. I mean there’s also
> the MariaDB Foundation whom are supposed to decide on things like
> this, but in theory we’re all captains… Do we query support @
> MariaDB Corporation to figure out if this is something acceptable
> for them?

It's easy to predict the feedback from support, as they are the ones
who have to deal with customers' problems in GA. I suppose that's why
they were kept out of the loop on previous occasions, at least they
what they claimed.

To make sure that at least members of this list are well-informed, I
collected some quick statistics off the top of our JIRA. It's not very
accurate, but it gives a good enough approximation.

*5.5* (bugs said to affect 5.5 and possibly higher versions)

~100 bugs of Major Priority or higher;
out of them, ~27 crashes.

*10.0* (bugs said to affect 10.0 and possibly 10.1)

~ 178 bugs of Major Priority or higher;
out of them, ~36 crashes.

*10.1* (bugs said to affect 10.1 only)

~50 bugs of Major Priority or higher;
out of them, ~14 crashes.

Don't get excited yet.
Most of 5.5 bugs affect 10.0 and 10.1.
Probably nearly all 10.0 bugs affect 10.1.

Remember, all of this is open statistics, everyone who wants can do
the same search in JIRA. It does not evaluate the actual quality of
the release, but it can tell people what *we know* about the quality
of the release.

Like, it makes it clear that we are about to release GA with ~70
crashing bugs, give or take.

"Wrong result" bugs, which are actually even worse for many real-life
scenarios, are more difficult to count, but there are plenty of them
as well.

I can admit right away that the release is by no means sufficiently
tested, not even close. The problem is, even if it was tested more,
the quality would not have become better -- as you can easily see from
the numbers above, we don't have enough manpower for bugfixing.

>
>> In fact, given that Daniel probably leaves for the meeting next
>> Thursday (?), and knowing our buildbot and other infrastructural
>> instability, we should aim to release the tree which we will have
>> on Tuesday, the very latest. Which basically means it should be
>> frozen on Saturday, to let it build fully, and leave Monday for
>> fixing whatever gets broken by Friday pushes.
>>
>> Among other things, it most likely means *no* to systemd -- unless
>> we are unbelievably lucky and the first systemd-enabled builds in
>> buildbot (which will hopefully happen tonight) show great success,
>> there will be no time to fix anything.
>
> Again more good points that I can’t argue with.
>
> Original email says, "The release will happen after next week's Maria
> call. Please scream and shout if you think there are some issues in
> addition to the ones on the sprint that you think still must be fixed
> before going GA.”
>
> You bring up a good point that nothing realistically gets fixed from
> now till then. So while I’m not Rasmus, maybe we do have to “delay”
> the release till we’re at the company meeting. Again, I have no idea
> what can get done from now till… whenever we freeze
>
> Frankly, as long as we release before 19 Oct, I think we’ll be
> before MySQL 5.7. However I think we need to also plan this as
> MariaDB Corporation will want to do marketing around this (and we
> now have new marketing lead(s)).

If we really must release before 19 Oct, I would not count on
releasing during the meeting, it's too optimistic, we don't even know
how well the connection will work.
And bugfixing during the meeting should also be minimized, previous
experience shows it does not do much good, probably due to all the
distractions.

Regards,
/E

=======================================================================

From: Sergei Golubchik <serg@xxxxxxxxxxx>
Subject: Re: [All] Current and last 10.1.8 sprint
To: Elena Stepanova <elenst@xxxxxxxxxxxxxxxx>
Cc: all@xxxxxxxxxxxxxxxxxx, Colin Charles <colin@xxxxxxxxxxx>
Date: Fri, 2 Oct 2015 10:00:45 +0200 (2 weeks, 4 days, 21 hours ago)

Hi, Elena!

On Oct 02, Elena Stepanova wrote:

> To make sure that at least members of this list are well-informed, I 
> collected some quick statistics off the top of our JIRA. It's not very 
> accurate, but it gives a good enough approximation.
> 
> *5.5* (bugs said to affect 5.5 and possibly higher versions)

I've also looked at bugs that we plan to fix in 5.5,
that's about 2/3rd of affected.

> ~100 bugs of Major Priority or higher;

~67 bugs here, including, say, "MDEV-8557 [PATCH] Spelling mistakes".

> out of them, ~27 crashes.
>
> *10.0* (bugs said to affect 10.0 and possibly 10.1)
> 
> ~ 178 bugs of Major Priority or higher;
> out of them, ~36 crashes.
> 
> *10.1* (bugs said to affect 10.1 only)
> 
> ~50 bugs of Major Priority or higher;
> out of them, ~14 crashes.
> 
> Remember, all of this is open statistics, everyone who wants can do the 
> same search in JIRA. It does not evaluate the actual quality of the 
> release, but it can tell people what *we know* about the quality of the 
> release.

Yes. In average (over last two years) we're getting ~20 major (or
higher) bugs for 5.5 in a month. And fixing about as much.

> If we really must release before 19 Oct, I would not count on
> releasing during the meeting, it's too optimistic, we don't even know
> how well the connection will work.
> And bugfixing during the meeting should also be minimized, previous 
> experience shows it does not do much good, probably due to all the 
> distractions.

It's not difficult to find good internet connection in Amsterdam.
I woulnd't suggest to do bugfixing at the meeting, but there's still a
week before it. And releasing on the meeting is possible, we've done it
before.

Regards,
Sergei

=======================================================================

From: Elena Stepanova <elenst@xxxxxxxxxxxxxxxx>
Subject: Re: [All] Current and last 10.1.8 sprint
To: Sergei Golubchik <serg@xxxxxxxxxxx>
Cc: all@xxxxxxxxxxxxxxxxxx, Colin Charles <colin@xxxxxxxxxxx>
Date: Fri, 2 Oct 2015 13:37:44 +0300 (2 weeks, 4 days, 18 hours ago)

Hi Sergei,


On 02.10.2015 11:00, Sergei Golubchik wrote:
> Hi, Elena!
>
> On Oct 02, Elena Stepanova wrote:
>
>> To make sure that at least members of this list are well-informed, I
>> collected some quick statistics off the top of our JIRA. It's not very
>> accurate, but it gives a good enough approximation.
>>
>> *5.5* (bugs said to affect 5.5 and possibly higher versions)
>
> I've also looked at bugs that we plan to fix in 5.5,
> that's about 2/3rd of affected.
>
>> ~100 bugs of Major Priority or higher;
>
> ~67 bugs here, including, say, "MDEV-8557 [PATCH] Spelling mistakes".

Right.

>From the external perspective, I find 'Affects version' more useful
than 'Fix version', because the former shows something real: version
X.Y[.Z] is, or once was, affected by this bug. The meaning of our 'Fix
version' field is vague, it's something like "We would like to fix it
in version X.Y[.Z] and now we think there is a chance we will do it,
but there is no guarantee when it will happen, or whether it will
happen at all". Normally, users should not pay attention to it.

My point was to show what an interested party (or an unfriendly
blogger) can gather about any of our GA releases by a simple JIRA
search.

In fact, I'm surprised nobody out there blogged about our releases in
this manner yet, they could make a good fun of us by matching these
results with our release criteria published in the KB. Sooner or later
it will happen, and I'd rather remind about it now than later listen
to complaints from marketing or sales "It's bad for business, how
could engineering allow it to happen?!".


> It's not difficult to find good internet connection in Amsterdam.
> I woulnd't suggest to do bugfixing at the meeting, but there's still a
> week before it. And releasing on the meeting is possible, we've done it
> before.

Of course it's possible, it's just not reliable (even less reliable
than doing it from home). What I meant was that if we really, really
*must* release by 19th, I would not rely on doing it on the meeting.

But even if we decide to do it from Amsterdam -- the current sprint
ends on Tuesday, developers leave on Friday, so there is only
Wednesday and Thursday to fix last-moment failures in buildbot, thus
there is still no much room for fixing anything apart from the current
sprint.


Regards,
/E

=======================================================================