trafodion-development team mailing list archive

["Birdsall, Dave" <dave.birdsall@xxxxxx>]

Good topic… forwarding to the development community…

Dave

From: Trafodion-technical-committee [mailto:trafodion-technical-committee-bounces+dave.birdsall=hp.com@xxxxxxxxxxxxxxxxxxx] On Behalf Of Cooper, Joanie
Sent: Wednesday, July 09, 2014 6:44 PM
To: trafodion-technical-committee@xxxxxxxxxxxxxxxxxxx
Cc: Sandstrom, Susan; TrafodionDepAndInt; Capirala, Tharaknath; Sharma, Anoop
Subject: [Trafodion-technical-committee] FW: Change in trafodion/infra[master]: Separate install packaging from product packaging

Hi,

Steve V. had asked that the TDI team take another look at
the idea of separating out the install repo into its own
packaging from the core/dcs packaging.

There had been a lot of previous discussion about how
we package Trafodion and how we handle version control
for the various components.

The TDI team is very interested in having an installation
package separate from the core/dcs packaging.

We have found that as we are all still learning about
the various distros and how customers may install Trafodion,
that the installation process has needed to change and
be made available faster than the core/dcs components.

We would like to propose that we have two separate packages.

e.g.  installer-0.8.5_v1.tar.gz and trafodion-0.8.5.tar.gz.

The proposal is to name the installer by what version of Trafodion it supports (e.g., 0.8.5) and then have a version number of the installer.  If that proposal goes forward, we’d have to be clear on what the format of the installer version number is (initial proposal was single number), and how we name the files to differentiate the two different numbers.

The installer version number would be reset back to 1 with each trafodion release.

I’ve included the discussion below that the TDI team had around this proposal.

We have reached a team consensus and would like to send our proposal
on to the technical committee for your review.

Please let us know your thoughts and if we can answer any questions for you.

Thanks,
   Joanie
_____________________________________________
From: Cooper, Joanie
Sent: Wednesday, July 09, 2014 17:31
To: Varnau, Steve (Trafodion)
Subject: RE: Change in trafodion/infra[master]: Separate install packaging from product packaging


Hi Steve,

It looks like we may have closure on this one from the TDI team.

Do you have what you need or would you like an email
sent from the TDI team to the technical committee?

Thanks,
   Joanie

_____________________________________________
From: Berman, Vladimir
Sent: Wednesday, July 09, 2014 5:29 PM
To: Varnau, Steve (Trafodion); Cooper, Joanie; Bhalgami, Chirag; Anderson, Marvin; TrafodionDepAndInt
Subject: RE: Change in trafodion/infra[master]: Separate install packaging from product packaging


I see your point … makes sense. I am okay with separate packaging for installer and Trafodion core.
Thanks,
Vlad


_____________________________________________
From: Varnau, Steve (Trafodion)
Sent: Wednesday, July 09, 2014 4:50 PM
To: Berman, Vladimir; Cooper, Joanie; Bhalgami, Chirag; Anderson, Marvin; TrafodionDepAndInt
Subject: RE: Change in trafodion/infra[master]: Separate install packaging from product packaging


Regarding suggestion to change all the component version numbers. As a user, I would be even more confused if the version numbers changed when the content had not.  I don’t think we should do that.

Currently the packaging process does rebuild each component, but we trust that process to work repeatably. So if the release candidate version worked and passed testing, then we tag the same version of code with the release tag, it gets built and published.  Likewise, if tag 0.8.2 of core at the same version that 0.8.1 is pointing to, then the assumption is that we don’t need to re-test core.

That is why I was able to turn around the RC and release builds pretty fast. The only testing done was the smoke testing by Marvin.

Actually changing a release number would change the content of the build, so that would make me more nervous about not re-testing

-Steve


_____________________________________________
From: Berman, Vladimir
Sent: Wednesday, July 09, 2014 16:01
To: Varnau, Steve (Trafodion); Cooper, Joanie; Bhalgami, Chirag; Anderson, Marvin; TrafodionDepAndInt
Subject: RE: Change in trafodion/infra[master]: Separate install packaging from product packaging


Hi Steve,

Regarding “can we release a change to the installer package much faster than to the entire Trafodion package?” … My thinking was not about the build and upload process, but more about the required testing. In other words, from the time an installer bug is found and fixed, how long will it take to do the required testing before the new build can be uploaded. Is there a difference in the testing we would do for an entire build vs. installer-only build. For example, if the difference is 1 day, then obviously the installer-only packaging wins. But if the difference is about 1 hour, then probably it doesn’t matter.

Your point about versioning is the other factor here. Yes, it’s confusing when the versions of different components are out of sync. I think a possible solution is to have an automated script that changes all versions as part of the build process.

Thanks,
Vlad


_____________________________________________
From: Varnau, Steve (Trafodion)
Sent: Wednesday, July 09, 2014 3:52 PM
To: Berman, Vladimir; Cooper, Joanie; Bhalgami, Chirag; Anderson, Marvin; TrafodionDepAndInt
Subject: RE: Change in trafodion/infra[master]: Separate install packaging from product packaging


Vlad,

From the time I apply the tags, it takes roughly half an hour to build all the components, package, and post them on the download site.  So, in my mind that is not the driving factor.  Though if we did an installer-only package, that time would probably take about 1 minute, or less.

My concern is customer confusion of what the releases contain.  Currently our download site has the windows driver and 3 trafodion tarballs (0.8.0, 0.8.1, 0.8.2). In a week or so, we’ll add another one called 0.8.3.  The first three differ only in installer, the next release will be everything. But that will not be obvious.  If I was an early user, I might have seen 0.8.2 come out and think, oh I’ll pick up the latest fixes. I download the huge tarball only to find core and dcs are still 0.8.0 versions.

Sure documentation can improve that, but my thinking is that our packaging should make things more obvious.  But maybe that’s not a real problem.  I’m fine with keeping the packaging as is, if that’s what you guys want.  But that is the downside from my point of view.

-Steve


_____________________________________________
From: Berman, Vladimir
Sent: Wednesday, July 09, 2014 13:59
To: Cooper, Joanie; Varnau, Steve (Trafodion); Bhalgami, Chirag; Anderson, Marvin; TrafodionDepAndInt
Subject: RE: Change in trafodion/infra[master]: Separate install packaging from product packaging


I still think it’s kind of awkward to have separate packages for the installer and Trafodion. Are we sure that we can release a change to the installer package much faster than to the entire Trafodion package? I think that’s the deciding factor here.
Thanks,
Vlad


_____________________________________________
From: Cooper, Joanie
Sent: Wednesday, July 09, 2014 9:21 AM
To: Varnau, Steve (Trafodion); Bhalgami, Chirag; Anderson, Marvin; Berman, Vladimir; TrafodionDepAndInt
Subject: RE: Change in trafodion/infra[master]: Separate install packaging from product packaging


Hi,

I’d suggest we reset the installer version back to 1 for a new dcs/core release.

Just as a somewhat unrelated question, what are the dcs/core changes that have us move to “0.9.0”?
Is this the upcoming July release or some other criteria?

Are we in agreement to the separate installer and dcs/core packaging?
Are we ready to pass our recommendations on to the technical committee?

Thanks,
   Joanie

_____________________________________________
From: Varnau, Steve (Trafodion)
Sent: Wednesday, July 09, 2014 9:10 AM
To: Bhalgami, Chirag; Cooper, Joanie; Anderson, Marvin; Berman, Vladimir; TrafodionDepAndInt
Subject: RE: Change in trafodion/infra[master]: Separate install packaging from product packaging


Yes, we need to clarify the naming here.  The proposal is to name the installer by what version of Trafodion it supports (e.g., 0.8.5) and then have a version number of the installer.  If that proposal goes forward, we’d have to be clear on what the format of the installer version number is (initial proposal was single number), and how we name the files to differentiate the two different numbers.

Perhaps:  installer-0.8.5_v1.tar.gz

And then do the installer version number reset back to 1 with each trafodion release, or just keep incrementing?  If we keep incrementing, then perhaps more than one number is needed.

-Steve


_____________________________________________
From: Bhalgami, Chirag
Sent: Tuesday, July 08, 2014 16:17
To: Cooper, Joanie; Varnau, Steve (Trafodion); Anderson, Marvin; Berman, Vladimir; TrafodionDepAndInt
Subject: RE: Change in trafodion/infra[master]: Separate install packaging from product packaging


Correction…

Deleted & Replaced

Thanks,
- Chirag

From: Bhalgami, Chirag
Sent: Tuesday, July 08, 2014 3:40 PM
To: Cooper, Joanie; Varnau, Steve (Trafodion); Anderson, Marvin; Berman, Vladimir; TrafodionDepAndInt
Subject: RE: Change in trafodion/infra[master]: Separate install packaging from product packaging

Just my 2 cents…

SemVer (Semantic Versioning) is the best way to go for open source products and I suggest that is what we should follow too. The standard format should be major.minor.patch as it exist for Trafodion. We have few sub-components, so it may not make much difference between having trafodion-0.8.1 & installer-0.8.1-1 Vs. trafodion-0.8 & installer-0.8.1 – it all depends on what’s the max value we want to keep for minor. If plan is to keep 99 as max # for minor (e.g. v0.78.3) then we can have v1.0.0 for the next trafodion release (and keep it stable until v2.0.0 release– after a year?) and keep incrementing minor and patch numbers for each Trafodion and Installer/other releases respectively. Like – v1.2.0 for Trafodion release in Dec-14 and 1.2.6 for any Installer release in Feb-15 and we can still have other Trafodion Installer releases in Oct-14 with v1.1.4) . If plan is to keep 9 as max # for minor then we can go with what Steve has suggested – add one more number and make it look like major.minor.patch.build in order to avoid having Trafodion v28.0.0 in Dec-15. ☺

Adding too many numbers in a build is only recommended when versioning of all components are automated and “nothing” is manual (which eventually keeps adding build# at the end) but we don’t have it at this moment. So decision depends on max number we want to keep for minor and patch along with when we are planning to automate versioning of all sub-components – all we have to keep in mind is how we will handle versions when versioning will be automated.

Thanks,
- Chirag

P.S.: We can also have 3-digit (or even 4-digit) number for patch/build.



-----Original Message-----
From: Cooper, Joanie
Sent: Tuesday, July 08, 2014 3:31 PM
To: Varnau, Steve (Trafodion); Anderson, Marvin; Berman, Vladimir; TrafodionDepAndInt
Subject: RE: Change in trafodion/infra[master]: Separate install packaging from product packaging

Hi,

I think that's a very interesting proposal.

It would solve the combined packaging version issues and yet allow us to just release any emergency fixes for the installer.

Thoughts?

Thanks,
   Joanie

-----Original Message-----
From: Varnau, Steve (Trafodion)
Sent: Tuesday, July 08, 2014 2:00 PM
To: Cooper, Joanie; Anderson, Marvin; Berman, Vladimir; TrafodionDepAndInt
Subject: RE: Change in trafodion/infra[master]: Separate install packaging from product packaging

What if we package them separately, but the installer version number is based on the trafodion version?

So, we'd have trafodion-0.8.1.tar.gz and installer-0.8.1-1.tar.gz.
Next installer release would be installer-0.8.1-2.tar.gz.  So, the trafodion version it supports is part of the name, but there is an additional number indicating the installer version?

This does mean that every time core/dcs changes we'd also need to release a matching installer, but not the other way around.  Does that help at all?

-Steve

> -----Original Message-----
> From: Cooper, Joanie
> Sent: Tuesday, July 08, 2014 13:03
> To: Anderson, Marvin; Berman, Vladimir; TrafodionDepAndInt
> Cc: Varnau, Steve (Trafodion)
> Subject: RE: Change in trafodion/infra[master]: Separate install
> packaging from product packaging
>
> Hi,
>
> I too am okay with the install not being a separate package.
> If we always create a matching package/bundle that the user downloads,
> they can assume all components should work together successfully.
>
> There is then no need to manage a matrix or test various components in
> varied combinations.
>
> But, we do need to ensure we provide the ability to turn around a new
> package, extremely quickly, with just installation changes.
>
> It’s a bit of the chicken and the egg problem.
>
> If they can't install, they can't run with Trafodion.
> But, if we require a full release with just an install fix, we have to
> be able to do this quickly.
>
> Does this assume the only version number incrementing in the new
> release will be to the installation component?
>
> One advantage I can see with a separate installation package, is the
> user only needs to download one component and continue with a fix to
> the installation.
>
> Otherwise, they need to download a whole bundle again (does download
> speed matter?).
>
> I would argue that the installation component is unique to other
> product components.
>
> How do other open source vendors handle this particular problem?
> It might be nice to know if there is a convention out there that would
> be good for us to follow.
>
> Thanks,
>    Joanie
>
> -----Original Message-----
> From: Anderson, Marvin
> Sent: Tuesday, July 08, 2014 12:49 PM
> To: Berman, Vladimir; Cooper, Joanie; TrafodionDepAndInt
> Cc: Varnau, Steve (Trafodion)
> Subject: RE: Change in trafodion/infra[master]: Separate install
> packaging from product packaging
>
> Hi,
> I'm ok with it not being a separate package, if (and this is the one
> of the things that started this) the install component doesn't have to
> adhere to the gating policies near the end of a release cycle.  We've
> shot ourselves in the foot twice on the install component because of
> that and I'd like to not continue to do that.  We make bug fixes to
> the installer that are totally independent of Trafodion as in there is
> no corresponding Trafodion core code to match it as we are making
> adjustments to deal with various user environments as we discover them.
>
> The way the install component would like to work is someone downloads
> and tries to install, runs into a problem because their environment is
> slightly different than what we have tried to anticipate since we
> cannot anticipate the 100s of possible environments out there.
> Install makes a fix so that environment can now work and we want to
> put that new version of install back out for immediate download before
> we get someone else hitting the same issue.  It’s a little different
> with install as in if the user can't install they can't do anything as
> opposed to other bugs like "can't INSERT a 5000 byte row".  And there
> is no way we can test the 100s of possible various environments.
> Separating install packaging from the product seemed the best way to
> make this scenario occur.  If we can release install fast enough, I don't really care how it's packaged.
>
> --Marvin
>
> -----Original Message-----
> From: Berman, Vladimir
> Sent: Tuesday, July 08, 2014 2:56 PM
> To: Cooper, Joanie; TrafodionDepAndInt
> Cc: Varnau, Steve (Trafodion)
> Subject: RE: Change in trafodion/infra[master]: Separate install
> packaging from product packaging
>
> I agree with Hans.
>
> Thanks,
> Vlad
>
> -----Original Message-----
> From: Cooper, Joanie
> Sent: Tuesday, July 08, 2014 11:40 AM
> To: TrafodionDepAndInt
> Cc: Varnau, Steve (Trafodion)
> Subject: FW: Change in trafodion/infra[master]: Separate install
> packaging from product packaging
>
> Hi,
>
> Hans made a few comments below about the idea of packaging/bundling
> and compatible versions.
>
> Does this sound good to you?
>
> If so, then we can pass this on to Steve and help influence the
> technical committee decisions.
>
> Thanks,
>   Joanie
>
> -----Original Message-----
> From: Varnau, Steve (Trafodion)
> Sent: Tuesday, July 08, 2014 11:27 AM
> To: Cooper, Joanie
> Subject: FW: Change in trafodion/infra[master]: Separate install
> packaging from product packaging
>
> fyi...
>
> -Steve
>
> -----Original Message-----
> From: Hans Zeller (Code Review) [mailto:review@xxxxxxxxxxxxxxxxxxxx]
> Sent: Tuesday, July 08, 2014 11:25
> To: Varnau, Steve (Trafodion)
> Cc: Anderson, Marvin
> Subject: Change in trafodion/infra[master]: Separate install packaging
> from product packaging
>
> Hans Zeller has posted comments on this change.
>
> Change subject: Separate install packaging from product packaging
> ......................................................................
>
>
> Patch Set 2: Code-Review-1
>
> I would prefer to have one package with compatible versions. An
> installer without the product to install is useless and the concept of
> "upgrading the installer" without upgrading the already installed
> product makes no sense. If we publish separate components then we need
> to maintain a compatibility matrix and automated checks for
> compatibility - which may be much worse than just packaging compatible
> pieces together. If we have a few more dot releases where the only
> change is the installer, that's fine with me.
>
> --
> To view, visit https://review.trafodion.org/21 To unsubscribe, visit
> https://review.trafodion.org/settings
>
> Gerrit-MessageType: comment
> Gerrit-Change-Id: I64b77d3d052a4badac033cec4f7c0605ed42f930
> Gerrit-PatchSet: 2
> Gerrit-Project: trafodion/infra
> Gerrit-Branch: master
> Gerrit-Owner: Steve Varnau <steve.varnau@xxxxxx<mailto:steve.varnau@xxxxxx>>
> Gerrit-Reviewer: Hans Zeller <hans.zeller@xxxxxx<mailto:hans.zeller@xxxxxx>>
> Gerrit-Reviewer: Marvin Anderson <marvin.anderson@xxxxxx<mailto:marvin.anderson@xxxxxx>>
> Gerrit-Reviewer: Trafodion Jenkins
> Gerrit-HasComments: No