Airflow voting on release artifacts

classic Classic list List threaded Threaded
26 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Airflow voting on release artifacts

Chris Riccomini
Hey all,

We've had a question arise in the Airflow project. We're currently cutting
release candidates (RCs), and voting on those. These RCs contain an
artifact with the suffix -rc0, -rc1, etc. The problem with this is that the
final RC that passes the vote still has an -rcX in its version number. To
get around this, the release manager has just been rebuilding the .asc,
.sha, etc, and publishing this new artifact as the final release.

Hitesh recently raised the issue that the artifact that passes the vote
MUST be the one that we actually release. The problem with this is that it
would force us to publish multiple RCs with the exact same version number.
This is a terrible experience from a developer and user point of view.
Airflow is a Python project, and many users pip install various versions of
Airflow to test things out. Having multiple RCs with the same version
number is going to make it really difficult to keep things straight, both
during the VOTE, and afterwards.

Based on a cursory look, it seems that other projects handle this problem
in one of three ways:

* Double vote (the final RC gets voted AGAIN, this time, with proper naming)
* Rename/rebuild after final vote (This is what Airflow is doing, and Beam
does this, I believe)
* All RCs have the exact same version number

Most Java projects are just publishing multiple RCs with the exact same
version number. This seems bad from a general software engineering
perspective, since it's going to cause build cache issues (e.g. .mvn
already contains version 1.2.3, even though a new RC with the same version
should be installed). Beam does seem to be using -SNAPSHOT releases in Java
for RCs, which is CLOSER to what we've been doing in Airflow (-rc0, -rc1,
etc), and they seem to be renaming after the release, just like we were.

What's the guidance here? We seem to have two requirements that are at odds
with eachtother:

1) The RC artifact that we vote on is the one that we release
2) We have real version numbers for RCs which are independent of one
another so build systems, dependency managements systems, and package
management systems can properly handle them.

Any help or thoughts would be appreciated.

Cheers,
Chris
Reply | Threaded
Open this post in threaded view
|

Re: Airflow voting on release artifacts

John D. Ament-2
Do you need to cut a vote with something named rc?  Why can't you just use
the version #?

On Mon, Apr 24, 2017 at 4:18 PM Chris Riccomini <[hidden email]>
wrote:

> Hey all,
>
> We've had a question arise in the Airflow project. We're currently cutting
> release candidates (RCs), and voting on those. These RCs contain an
> artifact with the suffix -rc0, -rc1, etc. The problem with this is that the
> final RC that passes the vote still has an -rcX in its version number. To
> get around this, the release manager has just been rebuilding the .asc,
> .sha, etc, and publishing this new artifact as the final release.
>
> Hitesh recently raised the issue that the artifact that passes the vote
> MUST be the one that we actually release. The problem with this is that it
> would force us to publish multiple RCs with the exact same version number.
> This is a terrible experience from a developer and user point of view.
> Airflow is a Python project, and many users pip install various versions of
> Airflow to test things out. Having multiple RCs with the same version
> number is going to make it really difficult to keep things straight, both
> during the VOTE, and afterwards.
>
> Based on a cursory look, it seems that other projects handle this problem
> in one of three ways:
>
> * Double vote (the final RC gets voted AGAIN, this time, with proper
> naming)
> * Rename/rebuild after final vote (This is what Airflow is doing, and Beam
> does this, I believe)
> * All RCs have the exact same version number
>
> Most Java projects are just publishing multiple RCs with the exact same
> version number. This seems bad from a general software engineering
> perspective, since it's going to cause build cache issues (e.g. .mvn
> already contains version 1.2.3, even though a new RC with the same version
> should be installed). Beam does seem to be using -SNAPSHOT releases in Java
> for RCs, which is CLOSER to what we've been doing in Airflow (-rc0, -rc1,
> etc), and they seem to be renaming after the release, just like we were.
>
> What's the guidance here? We seem to have two requirements that are at odds
> with eachtother:
>
> 1) The RC artifact that we vote on is the one that we release
> 2) We have real version numbers for RCs which are independent of one
> another so build systems, dependency managements systems, and package
> management systems can properly handle them.
>
> Any help or thoughts would be appreciated.
>
> Cheers,
> Chris
>
Reply | Threaded
Open this post in threaded view
|

Re: Airflow voting on release artifacts

Bolke de Bruin
Hi John,

Typically one handles a “release candidate” equal to a release, except for the fact that it hasn’t been fully tested and reviewed yet. In case everyone (all entities) is satisfied with the RC it is then just rebranded to Release. Unfortunately, in the python ecosystem the version information is contained within the (source) package and as metadata by means of the filename. These two need to be in sync, otherwise package installers will get confused.

So imagine, if we started voting on “apache-airflow-1.8.1+incubating”, but it does *not* pass the vote due to a blocker bug or a license issue. We would need to restart the vote with an updated package but with the same name. Semantically, this is already confusing as you would need to examine the signatures to see if the version has changed. It is also risky for the users/testers and therefore for the developers: the new version will not trigger an upgrade with package installers and a new installation would need to be forced (contrary to how maven handles things for example). This means, we could receive bug reports on “1.8.1” which actually are “1.8.1-RC1” or “1.8.1-RC2” etc.

In Apache we vote on artefacts. A rebrand delivers us two different artefacts: RC + Release. For us, as a podling, that would mean 4 votes: 2 at the PMC and 2 at the IPMC. This is very time consuming for a very minor change.

Anecdotical evidence from two other projects shows that Apache Beam voted on 0.6.0-RC2 as a release and then rebranded and updated packages after the vote (please correct me if I am wrong, just did a mailinglist scan and a comparison on the trees). Apache Libcloud (a pure python project) votes twice, but they seem to handle RCs differently, as there are large time gaps between the votes.

So we are seeking advice how to make efficient use of the Apache process, given the limitations of the ecosystem. I prefer what Apache Beam seems to be doing. I can also imagine to combine two artefacts in one vote: pointing everyone to the RC including it as a tar ball, but also including the tar ball and signatures for the *potential* rebranded version to release. Somehow we would need to hide the *potential* release as much from plain sight as possible, as we would have no way to distinguish between the actual release and the potential.

Bolke

> On 25 Apr 2017, at 02:11, John D. Ament <[hidden email]> wrote:
>
> Do you need to cut a vote with something named rc?  Why can't you just use
> the version #?
>
> On Mon, Apr 24, 2017 at 4:18 PM Chris Riccomini <[hidden email]>
> wrote:
>
>> Hey all,
>>
>> We've had a question arise in the Airflow project. We're currently cutting
>> release candidates (RCs), and voting on those. These RCs contain an
>> artifact with the suffix -rc0, -rc1, etc. The problem with this is that the
>> final RC that passes the vote still has an -rcX in its version number. To
>> get around this, the release manager has just been rebuilding the .asc,
>> .sha, etc, and publishing this new artifact as the final release.
>>
>> Hitesh recently raised the issue that the artifact that passes the vote
>> MUST be the one that we actually release. The problem with this is that it
>> would force us to publish multiple RCs with the exact same version number.
>> This is a terrible experience from a developer and user point of view.
>> Airflow is a Python project, and many users pip install various versions of
>> Airflow to test things out. Having multiple RCs with the same version
>> number is going to make it really difficult to keep things straight, both
>> during the VOTE, and afterwards.
>>
>> Based on a cursory look, it seems that other projects handle this problem
>> in one of three ways:
>>
>> * Double vote (the final RC gets voted AGAIN, this time, with proper
>> naming)
>> * Rename/rebuild after final vote (This is what Airflow is doing, and Beam
>> does this, I believe)
>> * All RCs have the exact same version number
>>
>> Most Java projects are just publishing multiple RCs with the exact same
>> version number. This seems bad from a general software engineering
>> perspective, since it's going to cause build cache issues (e.g. .mvn
>> already contains version 1.2.3, even though a new RC with the same version
>> should be installed). Beam does seem to be using -SNAPSHOT releases in Java
>> for RCs, which is CLOSER to what we've been doing in Airflow (-rc0, -rc1,
>> etc), and they seem to be renaming after the release, just like we were.
>>
>> What's the guidance here? We seem to have two requirements that are at odds
>> with eachtother:
>>
>> 1) The RC artifact that we vote on is the one that we release
>> 2) We have real version numbers for RCs which are independent of one
>> another so build systems, dependency managements systems, and package
>> management systems can properly handle them.
>>
>> Any help or thoughts would be appreciated.
>>
>> Cheers,
>> Chris
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Airflow voting on release artifacts

Bertrand Delacretaz
In reply to this post by Chris Riccomini
Hi,

On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini <[hidden email]> wrote:
> ...Hitesh recently raised the issue that the artifact that passes the vote
> MUST be the one that we actually release...

Yes in terms of having the same binary digests and signatures, but
renaming the files is fine IMO, especially for removing an -rc suffix
which makes total sense. I would just add that step to your release
process documentation to make it clear.

> ...Rename/rebuild after final vote (This is what Airflow is doing, and Beam
> does this, I believe)...

I'd say rename yes but rebuild no, in order to keep the same digests
and signatures.

-Bertrand

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Airflow voting on release artifacts

Bolke de Bruin
Hi Bertrand,

> On 25 Apr 2017, at 09:04, Bertrand Delacretaz <[hidden email]> wrote:
>
> Hi,
>
> On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini <[hidden email]> wrote:
>> ...Hitesh recently raised the issue that the artifact that passes the vote
>> MUST be the one that we actually release...
>
> Yes in terms of having the same binary digests and signatures, but
> renaming the files is fine IMO, especially for removing an -rc suffix
> which makes total sense. I would just add that step to your release
> process documentation to make it clear.
>
>> ...Rename/rebuild after final vote (This is what Airflow is doing, and Beam
>> does this, I believe)...
>
> I'd say rename yes but rebuild no, in order to keep the same digests
> and signatures.
>

As mentioned earlier, that seems not to be possible. The metadata (filename) and version information inside the package need to be in sync. This how the python build tools and python ecosystem works.

- Bolke.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Airflow voting on release artifacts

Niclas Hedhman
We have a similar issue in Polygene, but the internal version is simply the
expected version, say 1.2.3 and the RC has the different file name. No
packagers will ever get the -RC named artifact and no confusion is among
community members as they are aware of this. IF the RC passes, then the
rename can happen. IF the RC doesn't happen, you can rebuild with new
content and same internal version.

Cheers
Niclas

On Tue, Apr 25, 2017 at 4:43 PM, Bolke de Bruin <[hidden email]> wrote:

> Hi Bertrand,
>
> > On 25 Apr 2017, at 09:04, Bertrand Delacretaz <
> [hidden email]> wrote:
> >
> > Hi,
> >
> > On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini <[hidden email]>
> wrote:
> >> ...Hitesh recently raised the issue that the artifact that passes the
> vote
> >> MUST be the one that we actually release...
> >
> > Yes in terms of having the same binary digests and signatures, but
> > renaming the files is fine IMO, especially for removing an -rc suffix
> > which makes total sense. I would just add that step to your release
> > process documentation to make it clear.
> >
> >> ...Rename/rebuild after final vote (This is what Airflow is doing, and
> Beam
> >> does this, I believe)...
> >
> > I'd say rename yes but rebuild no, in order to keep the same digests
> > and signatures.
> >
>
> As mentioned earlier, that seems not to be possible. The metadata
> (filename) and version information inside the package need to be in sync.
> This how the python build tools and python ecosystem works.
>
> - Bolke.
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>


--
Niclas Hedhman, Software Developer
http://polygene.apache.org - New Energy for Java
Reply | Threaded
Open this post in threaded view
|

Re: Airflow voting on release artifacts

Bolke de Bruin
Hi Niclas,

Is this Java or Python? I can only find Java for Polygene.

Furthermore, how do you manage this you repository? Do you have the release already set in one of your files, e.g. something like this:
https://github.com/apache/incubator-airflow/blob/v1-8-test/airflow/version.py <https://github.com/apache/incubator-airflow/blob/v1-8-test/airflow/version.py>

The build system generates the metadata from there, which is used by the package installers (e.g. pip).

Cheers
Bolke


> On 25 Apr 2017, at 10:47, Niclas Hedhman <[hidden email]> wrote:
>
> We have a similar issue in Polygene, but the internal version is simply the
> expected version, say 1.2.3 and the RC has the different file name. No
> packagers will ever get the -RC named artifact and no confusion is among
> community members as they are aware of this. IF the RC passes, then the
> rename can happen. IF the RC doesn't happen, you can rebuild with new
> content and same internal version.
>
> Cheers
> Niclas
>
> On Tue, Apr 25, 2017 at 4:43 PM, Bolke de Bruin <[hidden email]> wrote:
>
>> Hi Bertrand,
>>
>>> On 25 Apr 2017, at 09:04, Bertrand Delacretaz <
>> [hidden email]> wrote:
>>>
>>> Hi,
>>>
>>> On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini <[hidden email]>
>> wrote:
>>>> ...Hitesh recently raised the issue that the artifact that passes the
>> vote
>>>> MUST be the one that we actually release...
>>>
>>> Yes in terms of having the same binary digests and signatures, but
>>> renaming the files is fine IMO, especially for removing an -rc suffix
>>> which makes total sense. I would just add that step to your release
>>> process documentation to make it clear.
>>>
>>>> ...Rename/rebuild after final vote (This is what Airflow is doing, and
>> Beam
>>>> does this, I believe)...
>>>
>>> I'd say rename yes but rebuild no, in order to keep the same digests
>>> and signatures.
>>>
>>
>> As mentioned earlier, that seems not to be possible. The metadata
>> (filename) and version information inside the package need to be in sync.
>> This how the python build tools and python ecosystem works.
>>
>> - Bolke.
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>
>
> --
> Niclas Hedhman, Software Developer
> http://polygene.apache.org - New Energy for Java

Reply | Threaded
Open this post in threaded view
|

Re: Airflow voting on release artifacts

John D. Ament-2
I'll point out that this is a great conversation and set of questions to
have with your mentors first, the IPMC later if they didn't respond/give
you a clear path forward.  Did you engage with them?

I personally hate seeing rc's being put forward to the incubator.  Most of
us use git these days, branches are cheap.  Even in my $dayjob when rolling
a release we always create a release branch, cut the release from there.
If something broke delete it from our internal artifact repository and drop
the branch - effectively starting over.

Would following an approach like that work for you?

John

On Tue, Apr 25, 2017 at 5:18 AM Bolke de Bruin <[hidden email]> wrote:

> Hi Niclas,
>
> Is this Java or Python? I can only find Java for Polygene.
>
> Furthermore, how do you manage this you repository? Do you have the
> release already set in one of your files, e.g. something like this:
>
> https://github.com/apache/incubator-airflow/blob/v1-8-test/airflow/version.py
> <
> https://github.com/apache/incubator-airflow/blob/v1-8-test/airflow/version.py
> >
>
> The build system generates the metadata from there, which is used by the
> package installers (e.g. pip).
>
> Cheers
> Bolke
>
>
> > On 25 Apr 2017, at 10:47, Niclas Hedhman <[hidden email]> wrote:
> >
> > We have a similar issue in Polygene, but the internal version is simply
> the
> > expected version, say 1.2.3 and the RC has the different file name. No
> > packagers will ever get the -RC named artifact and no confusion is among
> > community members as they are aware of this. IF the RC passes, then the
> > rename can happen. IF the RC doesn't happen, you can rebuild with new
> > content and same internal version.
> >
> > Cheers
> > Niclas
> >
> > On Tue, Apr 25, 2017 at 4:43 PM, Bolke de Bruin <[hidden email]>
> wrote:
> >
> >> Hi Bertrand,
> >>
> >>> On 25 Apr 2017, at 09:04, Bertrand Delacretaz <
> >> [hidden email]> wrote:
> >>>
> >>> Hi,
> >>>
> >>> On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini <
> [hidden email]>
> >> wrote:
> >>>> ...Hitesh recently raised the issue that the artifact that passes the
> >> vote
> >>>> MUST be the one that we actually release...
> >>>
> >>> Yes in terms of having the same binary digests and signatures, but
> >>> renaming the files is fine IMO, especially for removing an -rc suffix
> >>> which makes total sense. I would just add that step to your release
> >>> process documentation to make it clear.
> >>>
> >>>> ...Rename/rebuild after final vote (This is what Airflow is doing, and
> >> Beam
> >>>> does this, I believe)...
> >>>
> >>> I'd say rename yes but rebuild no, in order to keep the same digests
> >>> and signatures.
> >>>
> >>
> >> As mentioned earlier, that seems not to be possible. The metadata
> >> (filename) and version information inside the package need to be in
> sync.
> >> This how the python build tools and python ecosystem works.
> >>
> >> - Bolke.
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [hidden email]
> >> For additional commands, e-mail: [hidden email]
> >>
> >>
> >
> >
> > --
> > Niclas Hedhman, Software Developer
> > http://polygene.apache.org - New Energy for Java
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Airflow voting on release artifacts

Niclas Hedhman
In reply to this post by Bolke de Bruin
It is Java, but we have version references internally in all kinds of
places.
So, what happens is that the build creates the "expected final release",
say "1.2.3" and sets all internal references to that. But the tarball will
be -RCx, which is voted upon. And then, as Bertrand suggested, a rename of
that tarball if the vote passes, is not a problem.

And as John points out, the release build tags "1.2.3" in git and we can
(re)move that later if the vote fails.


On Tue, Apr 25, 2017 at 5:18 PM, Bolke de Bruin <[hidden email]> wrote:

> Hi Niclas,
>
> Is this Java or Python? I can only find Java for Polygene.
>
> Furthermore, how do you manage this you repository? Do you have the
> release already set in one of your files, e.g. something like this:
> https://github.com/apache/incubator-airflow/blob/v1-8-
> test/airflow/version.py <https://github.com/apache/
> incubator-airflow/blob/v1-8-test/airflow/version.py>
>
> The build system generates the metadata from there, which is used by the
> package installers (e.g. pip).
>
> Cheers
> Bolke
>
>
> > On 25 Apr 2017, at 10:47, Niclas Hedhman <[hidden email]> wrote:
> >
> > We have a similar issue in Polygene, but the internal version is simply
> the
> > expected version, say 1.2.3 and the RC has the different file name. No
> > packagers will ever get the -RC named artifact and no confusion is among
> > community members as they are aware of this. IF the RC passes, then the
> > rename can happen. IF the RC doesn't happen, you can rebuild with new
> > content and same internal version.
> >
> > Cheers
> > Niclas
> >
> > On Tue, Apr 25, 2017 at 4:43 PM, Bolke de Bruin <[hidden email]>
> wrote:
> >
> >> Hi Bertrand,
> >>
> >>> On 25 Apr 2017, at 09:04, Bertrand Delacretaz <
> >> [hidden email]> wrote:
> >>>
> >>> Hi,
> >>>
> >>> On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini <
> [hidden email]>
> >> wrote:
> >>>> ...Hitesh recently raised the issue that the artifact that passes the
> >> vote
> >>>> MUST be the one that we actually release...
> >>>
> >>> Yes in terms of having the same binary digests and signatures, but
> >>> renaming the files is fine IMO, especially for removing an -rc suffix
> >>> which makes total sense. I would just add that step to your release
> >>> process documentation to make it clear.
> >>>
> >>>> ...Rename/rebuild after final vote (This is what Airflow is doing, and
> >> Beam
> >>>> does this, I believe)...
> >>>
> >>> I'd say rename yes but rebuild no, in order to keep the same digests
> >>> and signatures.
> >>>
> >>
> >> As mentioned earlier, that seems not to be possible. The metadata
> >> (filename) and version information inside the package need to be in
> sync.
> >> This how the python build tools and python ecosystem works.
> >>
> >> - Bolke.
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [hidden email]
> >> For additional commands, e-mail: [hidden email]
> >>
> >>
> >
> >
> > --
> > Niclas Hedhman, Software Developer
> > http://polygene.apache.org - New Energy for Java
>
>


--
Niclas Hedhman, Software Developer
http://polygene.apache.org - New Energy for Java
Reply | Threaded
Open this post in threaded view
|

Re: Airflow voting on release artifacts

Alex Harui
In reply to this post by Bolke de Bruin


On 4/25/17, 1:43 AM, "Bolke de Bruin" <[hidden email]> wrote:

>Hi Bertrand,
>
>> On 25 Apr 2017, at 09:04, Bertrand Delacretaz
>><[hidden email]> wrote:
>>
>> Hi,
>>
>> On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini
>><[hidden email]> wrote:
>>> ...Hitesh recently raised the issue that the artifact that passes the
>>>vote
>>> MUST be the one that we actually release...
>>
>> Yes in terms of having the same binary digests and signatures, but
>> renaming the files is fine IMO, especially for removing an -rc suffix
>> which makes total sense. I would just add that step to your release
>> process documentation to make it clear.
>>
>>> ...Rename/rebuild after final vote (This is what Airflow is doing, and
>>>Beam
>>> does this, I believe)...
>>
>> I'd say rename yes but rebuild no, in order to keep the same digests
>> and signatures.
>>
>
>As mentioned earlier, that seems not to be possible. The metadata
>(filename) and version information inside the package need to be in sync.
>This how the python build tools and python ecosystem works.

I'm not familiar with Python, but is it possible to have a command line
option that adds the "-rc" suffix in the right places?

IMO, there are two "audiences".  One is the voters, and one is your
customers.  The customers should not be using the RC until after it is
approved unless they are volunteering to be a voter.  Voters are primarily
supposed to examine a source artifact, but if you also release a
"convenience binary" artifact, there are some examinations of that
"binary" artifact required.  For many projects this "convenience" artifact
is the one that the vast majority of customers consume.  AIUI, Python
doesn't have binaries, but IMO, there is no requirement that this
"convenience" artifact actually contains binaries.  A "convenience"
artifact is just supposed to be the source artifact processed by a build
script since many of your customer may not have, or may not want to
configure their machines to run whatever build script engine you choose
for your project.  Further, there is no requirement that I know of that
voters must test the "convenience" artifact by actually running it.  It
just makes sense to do so in most cases.  But if Python is interpreted
source, you may be able to use this to your advantage.

So, your source package should consist of the source and build script
required to build this "customer"/"convenience" package, and the build
script should allow a command line option to add the "-rc" suffix.  Then
voters would be instructed to download the source package, and to test it,
build a "customer" package with the "-rc" suffix and test the results.
And voters would also be instructed to download the "customer" package
that doesn't have the -rc suffix.  But in order to test it, they only have
to diff that package against the "customer" package they built from the
source package.  It should be the same except for the metadata.

We'll see if others can find a problem with this plan, but IMO, that would
be good enough for me as an PMC voter.

HTH,
-Alex


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Airflow voting on release artifacts

Bolke de Bruin
Thank you thinking with us, that’s really appreciated. Unfortunately, many of the assumptions from the Java world do not apply to the Python world:

Version information inside the artefact needs to be in sync with the external filename.

Examples:

1. Plain rename of the tar ball

bolke$ pip install airflow-1.8.0rc11+apache.incubating.tar.gz
Processing ./airflow-1.8.0rc11+apache.incubating.tar.gz
  Requirement already satisfied (use --upgrade to upgrade): airflow==1.8.0rc5+apache.incubating from file:///Users/bolke/Documents/dev/airflow/dist/airflow-1.8.0rc11%2Bapache.incubating.tar.gz in /Users/bolke/Documents/dev/airflow_env/lib/python2.7/site-packages

Note that although RC11 is mentioned RC5 is contained in the package and is not upgraded.

2. Rename of the package inside the tar ball
bolke$ mv airflow-1.8.0rc5+apache.incubating airflow-1.8.0rc11+apache.incubating
bolke$ tar cfz ../airflow-1.8.0rc11+apache.incubating.tar.gz airflow-1.8.0rc11+apache.incubating/

bolke$ pip install airflow-1.8.0rc11+apache.incubating.tar.gz
Processing ./airflow-1.8.0rc11+apache.incubating.tar.gz
  Requirement already satisfied (use --upgrade to upgrade): airflow==1.8.0rc5+apache.incubating from file:///Users/bolke/Documents/dev/airflow/dist/airflow-1.8.0rc11%2Bapache.incubating.tar.gz in /Users/bolke/Documents/dev/airflow_env/lib/python2.7/site-packages

3. There is no “separate” build script. Pip will just install a binary (“wheel”) or uses the source package (as shown above). Both are used interchangeable by users. We only distribute source packages at the moment.

@Alex: I have to think a little bit more about what you wrote, but it is currently confusing the hell out of me :). Furthermore, I am not sure if it applies considering the above #3.

At the moment we seem stuck between a rock and a hard place. And as we would like to release a new version pretty soon, we probably want to vote twice for now, and including the IPMC, four times. Hopefully we can have a vote in rapid succession :-).

Bolke


> On 25 Apr 2017, at 17:33, Alex Harui <[hidden email]> wrote:
>
>
>
> On 4/25/17, 1:43 AM, "Bolke de Bruin" <[hidden email] <mailto:[hidden email]>> wrote:
>
>> Hi Bertrand,
>>
>>> On 25 Apr 2017, at 09:04, Bertrand Delacretaz
>>> <[hidden email]> wrote:
>>>
>>> Hi,
>>>
>>> On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini
>>> <[hidden email]> wrote:
>>>> ...Hitesh recently raised the issue that the artifact that passes the
>>>> vote
>>>> MUST be the one that we actually release...
>>>
>>> Yes in terms of having the same binary digests and signatures, but
>>> renaming the files is fine IMO, especially for removing an -rc suffix
>>> which makes total sense. I would just add that step to your release
>>> process documentation to make it clear.
>>>
>>>> ...Rename/rebuild after final vote (This is what Airflow is doing, and
>>>> Beam
>>>> does this, I believe)...
>>>
>>> I'd say rename yes but rebuild no, in order to keep the same digests
>>> and signatures.
>>>
>>
>> As mentioned earlier, that seems not to be possible. The metadata
>> (filename) and version information inside the package need to be in sync.
>> This how the python build tools and python ecosystem works.
>
> I'm not familiar with Python, but is it possible to have a command line
> option that adds the "-rc" suffix in the right places?
>
> IMO, there are two "audiences".  One is the voters, and one is your
> customers.  The customers should not be using the RC until after it is
> approved unless they are volunteering to be a voter.  Voters are primarily
> supposed to examine a source artifact, but if you also release a
> "convenience binary" artifact, there are some examinations of that
> "binary" artifact required.  For many projects this "convenience" artifact
> is the one that the vast majority of customers consume.  AIUI, Python
> doesn't have binaries, but IMO, there is no requirement that this
> "convenience" artifact actually contains binaries.  A "convenience"
> artifact is just supposed to be the source artifact processed by a build
> script since many of your customer may not have, or may not want to
> configure their machines to run whatever build script engine you choose
> for your project.  Further, there is no requirement that I know of that
> voters must test the "convenience" artifact by actually running it.  It
> just makes sense to do so in most cases.  But if Python is interpreted
> source, you may be able to use this to your advantage.
>
> So, your source package should consist of the source and build script
> required to build this "customer"/"convenience" package, and the build
> script should allow a command line option to add the "-rc" suffix.  Then
> voters would be instructed to download the source package, and to test it,
> build a "customer" package with the "-rc" suffix and test the results.
> And voters would also be instructed to download the "customer" package
> that doesn't have the -rc suffix.  But in order to test it, they only have
> to diff that package against the "customer" package they built from the
> source package.  It should be the same except for the metadata.
>
> We'll see if others can find a problem with this plan, but IMO, that would
> be good enough for me as an PMC voter.
>
> HTH,
> -Alex
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Airflow voting on release artifacts

Jim Apple
Would virtualenv help by avoiding the "Requirement already satisfied"
complaints?

On Tue, Apr 25, 2017 at 2:05 PM, Bolke de Bruin <[hidden email]> wrote:

> Thank you thinking with us, that’s really appreciated. Unfortunately, many
> of the assumptions from the Java world do not apply to the Python world:
>
> Version information inside the artefact needs to be in sync with the
> external filename.
>
> Examples:
>
> 1. Plain rename of the tar ball
>
> bolke$ pip install airflow-1.8.0rc11+apache.incubating.tar.gz
> Processing ./airflow-1.8.0rc11+apache.incubating.tar.gz
>   Requirement already satisfied (use --upgrade to upgrade):
> airflow==1.8.0rc5+apache.incubating from file:///Users/bolke/Documents/
> dev/airflow/dist/airflow-1.8.0rc11%2Bapache.incubating.tar.gz in
> /Users/bolke/Documents/dev/airflow_env/lib/python2.7/site-packages
>
> Note that although RC11 is mentioned RC5 is contained in the package and
> is not upgraded.
>
> 2. Rename of the package inside the tar ball
> bolke$ mv airflow-1.8.0rc5+apache.incubating airflow-1.8.0rc11+apache.
> incubating
> bolke$ tar cfz ../airflow-1.8.0rc11+apache.incubating.tar.gz
> airflow-1.8.0rc11+apache.incubating/
>
> bolke$ pip install airflow-1.8.0rc11+apache.incubating.tar.gz
> Processing ./airflow-1.8.0rc11+apache.incubating.tar.gz
>   Requirement already satisfied (use --upgrade to upgrade):
> airflow==1.8.0rc5+apache.incubating from file:///Users/bolke/Documents/
> dev/airflow/dist/airflow-1.8.0rc11%2Bapache.incubating.tar.gz in
> /Users/bolke/Documents/dev/airflow_env/lib/python2.7/site-packages
>
> 3. There is no “separate” build script. Pip will just install a binary
> (“wheel”) or uses the source package (as shown above). Both are used
> interchangeable by users. We only distribute source packages at the moment.
>
> @Alex: I have to think a little bit more about what you wrote, but it is
> currently confusing the hell out of me :). Furthermore, I am not sure if it
> applies considering the above #3.
>
> At the moment we seem stuck between a rock and a hard place. And as we
> would like to release a new version pretty soon, we probably want to vote
> twice for now, and including the IPMC, four times. Hopefully we can have a
> vote in rapid succession :-).
>
> Bolke
>
>
> > On 25 Apr 2017, at 17:33, Alex Harui <[hidden email]> wrote:
> >
> >
> >
> > On 4/25/17, 1:43 AM, "Bolke de Bruin" <[hidden email] <mailto:
> [hidden email]>> wrote:
> >
> >> Hi Bertrand,
> >>
> >>> On 25 Apr 2017, at 09:04, Bertrand Delacretaz
> >>> <[hidden email]> wrote:
> >>>
> >>> Hi,
> >>>
> >>> On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini
> >>> <[hidden email]> wrote:
> >>>> ...Hitesh recently raised the issue that the artifact that passes the
> >>>> vote
> >>>> MUST be the one that we actually release...
> >>>
> >>> Yes in terms of having the same binary digests and signatures, but
> >>> renaming the files is fine IMO, especially for removing an -rc suffix
> >>> which makes total sense. I would just add that step to your release
> >>> process documentation to make it clear.
> >>>
> >>>> ...Rename/rebuild after final vote (This is what Airflow is doing, and
> >>>> Beam
> >>>> does this, I believe)...
> >>>
> >>> I'd say rename yes but rebuild no, in order to keep the same digests
> >>> and signatures.
> >>>
> >>
> >> As mentioned earlier, that seems not to be possible. The metadata
> >> (filename) and version information inside the package need to be in
> sync.
> >> This how the python build tools and python ecosystem works.
> >
> > I'm not familiar with Python, but is it possible to have a command line
> > option that adds the "-rc" suffix in the right places?
> >
> > IMO, there are two "audiences".  One is the voters, and one is your
> > customers.  The customers should not be using the RC until after it is
> > approved unless they are volunteering to be a voter.  Voters are
> primarily
> > supposed to examine a source artifact, but if you also release a
> > "convenience binary" artifact, there are some examinations of that
> > "binary" artifact required.  For many projects this "convenience"
> artifact
> > is the one that the vast majority of customers consume.  AIUI, Python
> > doesn't have binaries, but IMO, there is no requirement that this
> > "convenience" artifact actually contains binaries.  A "convenience"
> > artifact is just supposed to be the source artifact processed by a build
> > script since many of your customer may not have, or may not want to
> > configure their machines to run whatever build script engine you choose
> > for your project.  Further, there is no requirement that I know of that
> > voters must test the "convenience" artifact by actually running it.  It
> > just makes sense to do so in most cases.  But if Python is interpreted
> > source, you may be able to use this to your advantage.
> >
> > So, your source package should consist of the source and build script
> > required to build this "customer"/"convenience" package, and the build
> > script should allow a command line option to add the "-rc" suffix.  Then
> > voters would be instructed to download the source package, and to test
> it,
> > build a "customer" package with the "-rc" suffix and test the results.
> > And voters would also be instructed to download the "customer" package
> > that doesn't have the -rc suffix.  But in order to test it, they only
> have
> > to diff that package against the "customer" package they built from the
> > source package.  It should be the same except for the metadata.
> >
> > We'll see if others can find a problem with this plan, but IMO, that
> would
> > be good enough for me as an PMC voter.
> >
> > HTH,
> > -Alex
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> <mailto:[hidden email]>
> > For additional commands, e-mail: [hidden email]
> <mailto:[hidden email]>
>
Reply | Threaded
Open this post in threaded view
|

Re: Airflow voting on release artifacts

Alex Harui
In reply to this post by Bolke de Bruin


>
>3. There is no “separate” build script. Pip will just install a binary
>(“wheel”) or uses the source package (as shown above). Both are used
>interchangeable by users. We only distribute source packages at the
>moment.
>
>@Alex: I have to think a little bit more about what you wrote, but it is
>currently confusing the hell out of me :). Furthermore, I am not sure if
>it applies considering the above #3.

It could still apply.  You would just have to add a build script that
renames the package and metadata.

Let's say I wanted to release a single file that reported the version
number.  Forgive me that I don't know Python so I just grabbed what I
think it would look like from the internet.

---MyRelease.py---
print("I am version 1.2.3.")

Let's assume this is what your "customers" want to use.

I am proposing that the Apache Source Package also contain the following
file:

---BuildScript.sh---
# creates Customer Package in out folder.
mkdir out
sed s/1.2.3/1.2.3$1/ < MyRelease.py > out/MyRelease$1.py

Voters would run:

   BuildScript.sh RC1

That would result in:

---out/MyReleaseRC1.py---
print("I am version 1.2.3RC1.")

And this version would be tested by the voters.  The source package being
voted on contains the original MyRelease.py and BuildScript.sh.  The
release manager would also run:


    BuildScript.sh

That would result in:

---out/MyRelease.py---
print("I am version 1.2.3.")

In our project, the RM posts the source package in the RC folder and
creates a folder called "binaries" for the compiled source.  You could
call the folder something else, but let's keep the names for now.  The RM
would copy a zip of MyRelease.py and BuildScript.sh (and LICENSE, NOTICE,
README) into the RC folder and out/MyRelease.py to the "binaries" folder.
Along with signatures and checksum files.


Voters would download the zip, expand it, run "BuildScript.sh RC1" and
test with their out/MyReleaseRC1.py.  They would examine the zip to make
sure it is compliant with Apache release policy.  This is what all other
voters on all other projects generally do.  But they would perform one
different step, which is, instead of testing the MyRelease.py in the
"binaries" folder, they would simply diff their MyReleaseRC1.py against
the MyRelease.py in the "binaries" folder.  If the only diffs are the
version, they should feel satisfied that the resulting "customer" packages
is ok for release.

Of course, I could be wrong...

HTH,
-Alex


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Airflow voting on release artifacts

Hitesh Shah
Hi folks,

Given that the source bits are the official release, would it be okay if
the community as a whole decided on say the following approach:

1) Bundle source with version set to 1.8.1
2) Bundle binary convenience artifacts built using version set to 1.8.1rc1
3) Publish source and binary bits to dist.a.o/dev for vote
4) If vote passes, publish the source tarball that was voted upon and
"modified" binary convenience artifacts built with version set to 1.8.1

The implication here is that the release manager is being trusted by the
PMC to release the modified convenience artifacts from the voted-upon
source without a new vote.

If it helps, there are a couple of variations of the above which could be
applied to reduce the no. of total votes:

2a) Create 2 binary sets for the vote - one with version 1.8.1rc1 and other
with version 1.8.1 ( with only 1.8.1 being published on a successful vote)

OR

4a) After PPMC vote passes, use the original source and modified binaries
for the IPMC vote and therefore the IPMC vote is on the final bits being
published.

Any comments?

thanks
-- Hitesh


On Tue, Apr 25, 2017 at 9:02 PM, Alex Harui <[hidden email]> wrote:

>
>
> >
> >3. There is no “separate” build script. Pip will just install a binary
> >(“wheel”) or uses the source package (as shown above). Both are used
> >interchangeable by users. We only distribute source packages at the
> >moment.
> >
> >@Alex: I have to think a little bit more about what you wrote, but it is
> >currently confusing the hell out of me :). Furthermore, I am not sure if
> >it applies considering the above #3.
>
> It could still apply.  You would just have to add a build script that
> renames the package and metadata.
>
> Let's say I wanted to release a single file that reported the version
> number.  Forgive me that I don't know Python so I just grabbed what I
> think it would look like from the internet.
>
> ---MyRelease.py---
> print("I am version 1.2.3.")
>
> Let's assume this is what your "customers" want to use.
>
> I am proposing that the Apache Source Package also contain the following
> file:
>
> ---BuildScript.sh---
> # creates Customer Package in out folder.
> mkdir out
> sed s/1.2.3/1.2.3$1/ < MyRelease.py > out/MyRelease$1.py
>
> Voters would run:
>
>    BuildScript.sh RC1
>
> That would result in:
>
> ---out/MyReleaseRC1.py---
> print("I am version 1.2.3RC1.")
>
> And this version would be tested by the voters.  The source package being
> voted on contains the original MyRelease.py and BuildScript.sh.  The
> release manager would also run:
>
>
>     BuildScript.sh
>
> That would result in:
>
> ---out/MyRelease.py---
> print("I am version 1.2.3.")
>
> In our project, the RM posts the source package in the RC folder and
> creates a folder called "binaries" for the compiled source.  You could
> call the folder something else, but let's keep the names for now.  The RM
> would copy a zip of MyRelease.py and BuildScript.sh (and LICENSE, NOTICE,
> README) into the RC folder and out/MyRelease.py to the "binaries" folder.
> Along with signatures and checksum files.
>
>
> Voters would download the zip, expand it, run "BuildScript.sh RC1" and
> test with their out/MyReleaseRC1.py.  They would examine the zip to make
> sure it is compliant with Apache release policy.  This is what all other
> voters on all other projects generally do.  But they would perform one
> different step, which is, instead of testing the MyRelease.py in the
> "binaries" folder, they would simply diff their MyReleaseRC1.py against
> the MyRelease.py in the "binaries" folder.  If the only diffs are the
> version, they should feel satisfied that the resulting "customer" packages
> is ok for release.
>
> Of course, I could be wrong...
>
> HTH,
> -Alex
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Airflow voting on release artifacts

Alex Harui
I'm not on the IPMC, so I don't have an official vote, but for me, it
would be ok if you include step 2a and the voters have a way to validate
that the 1.8.1rc1 binary they tested is "the same"=="identical except for
version numbers" as the 1.8.1 binaries you want to distribute to customers.

Also, I believe the source package build script must be able to reproduce
the 1.8.1 binary you want to distribute to customers.

So if you can do that I would hope the IPMC would approve.

-Alex

On 4/27/17, 3:57 PM, "Hitesh Shah" <[hidden email]> wrote:

>Hi folks,
>
>Given that the source bits are the official release, would it be okay if
>the community as a whole decided on say the following approach:
>
>1) Bundle source with version set to 1.8.1
>2) Bundle binary convenience artifacts built using version set to 1.8.1rc1
>3) Publish source and binary bits to dist.a.o/dev for vote
>4) If vote passes, publish the source tarball that was voted upon and
>"modified" binary convenience artifacts built with version set to 1.8.1
>
>The implication here is that the release manager is being trusted by the
>PMC to release the modified convenience artifacts from the voted-upon
>source without a new vote.
>
>If it helps, there are a couple of variations of the above which could be
>applied to reduce the no. of total votes:
>
>2a) Create 2 binary sets for the vote - one with version 1.8.1rc1 and
>other
>with version 1.8.1 ( with only 1.8.1 being published on a successful vote)
>
>OR
>
>4a) After PPMC vote passes, use the original source and modified binaries
>for the IPMC vote and therefore the IPMC vote is on the final bits being
>published.
>
>Any comments?
>
>thanks
>-- Hitesh
>
>
>On Tue, Apr 25, 2017 at 9:02 PM, Alex Harui <[hidden email]> wrote:
>
>>
>>
>> >
>> >3. There is no “separate” build script. Pip will just install a binary
>> >(“wheel”) or uses the source package (as shown above). Both are used
>> >interchangeable by users. We only distribute source packages at the
>> >moment.
>> >
>> >@Alex: I have to think a little bit more about what you wrote, but it
>>is
>> >currently confusing the hell out of me :). Furthermore, I am not sure
>>if
>> >it applies considering the above #3.
>>
>> It could still apply.  You would just have to add a build script that
>> renames the package and metadata.
>>
>> Let's say I wanted to release a single file that reported the version
>> number.  Forgive me that I don't know Python so I just grabbed what I
>> think it would look like from the internet.
>>
>> ---MyRelease.py---
>> print("I am version 1.2.3.")
>>
>> Let's assume this is what your "customers" want to use.
>>
>> I am proposing that the Apache Source Package also contain the following
>> file:
>>
>> ---BuildScript.sh---
>> # creates Customer Package in out folder.
>> mkdir out
>> sed s/1.2.3/1.2.3$1/ < MyRelease.py > out/MyRelease$1.py
>>
>> Voters would run:
>>
>>    BuildScript.sh RC1
>>
>> That would result in:
>>
>> ---out/MyReleaseRC1.py---
>> print("I am version 1.2.3RC1.")
>>
>> And this version would be tested by the voters.  The source package
>>being
>> voted on contains the original MyRelease.py and BuildScript.sh.  The
>> release manager would also run:
>>
>>
>>     BuildScript.sh
>>
>> That would result in:
>>
>> ---out/MyRelease.py---
>> print("I am version 1.2.3.")
>>
>> In our project, the RM posts the source package in the RC folder and
>> creates a folder called "binaries" for the compiled source.  You could
>> call the folder something else, but let's keep the names for now.  The
>>RM
>> would copy a zip of MyRelease.py and BuildScript.sh (and LICENSE,
>>NOTICE,
>> README) into the RC folder and out/MyRelease.py to the "binaries"
>>folder.
>> Along with signatures and checksum files.
>>
>>
>> Voters would download the zip, expand it, run "BuildScript.sh RC1" and
>> test with their out/MyReleaseRC1.py.  They would examine the zip to make
>> sure it is compliant with Apache release policy.  This is what all other
>> voters on all other projects generally do.  But they would perform one
>> different step, which is, instead of testing the MyRelease.py in the
>> "binaries" folder, they would simply diff their MyReleaseRC1.py against
>> the MyRelease.py in the "binaries" folder.  If the only diffs are the
>> version, they should feel satisfied that the resulting "customer"
>>packages
>> is ok for release.
>>
>> Of course, I could be wrong...
>>
>> HTH,
>> -Alex
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Airflow voting on release artifacts

Justin Mclean
In reply to this post by Hitesh Shah
Hi,

> The implication here is that the release manager is being trusted by the
> PMC to release the modified convenience artifacts from the voted-upon
> source without a new vote.

How are the artefacts modified after the vote?

IMO As long as the hashes and signature are the same there is no issue. If any of the contents change, rather than just files names, then it would no longer be a valid release.

If it is just a rename, a safer way to rename the file this would be to do so when you do the svn move from dev staging area to the release area.

Thanks,
Justin
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Airflow voting on release artifacts

Hitesh Shah
Hi Justin,

Currently, the podling has been modifying the contents and hence this
discussion.

thanks
-- Hitesh

On Thu, Apr 27, 2017 at 8:48 PM, Justin Mclean <[hidden email]>
wrote:

> Hi,
>
> > The implication here is that the release manager is being trusted by the
> > PMC to release the modified convenience artifacts from the voted-upon
> > source without a new vote.
>
> How are the artefacts modified after the vote?
>
> IMO As long as the hashes and signature are the same there is no issue. If
> any of the contents change, rather than just files names, then it would no
> longer be a valid release.
>
> If it is just a rename, a safer way to rename the file this would be to do
> so when you do the svn move from dev staging area to the release area.
>
> Thanks,
> Justin
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Airflow voting on release artifacts

Alex Harui


On 5/1/17, 7:44 AM, "Hitesh Shah" <[hidden email]> wrote:

>Hi Justin,
>
>Currently, the podling has been modifying the contents and hence this
>discussion.

I agree with Justin and others that modification after the vote is not a
good thing.  So my assumption was that if you add your 2a step and modify
the binary before the vote, it will be acceptable.  IMO, all you need is a
way to verify that the binary the voters test is essentially the same as
the binary you want to actually release.

-Alex


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Airflow voting on release artifacts

Bolke de Bruin

> On 1 May 2017, at 17:36, Alex Harui <[hidden email]> wrote:
>
>
>
> On 5/1/17, 7:44 AM, "Hitesh Shah" <[hidden email]> wrote:
>
>> Hi Justin,
>>
>> Currently, the podling has been modifying the contents and hence this
>> discussion.
>
> I agree with Justin and others that modification after the vote is not a
> good thing.  So my assumption was that if you add your 2a step and modify
> the binary before the vote, it will be acceptable.  IMO, all you need is a
> way to verify that the binary the voters test is essentially the same as
> the binary you want to actually release.
>
> -Alex
>
>

Hi Alex,

As mentioned earlier this is not possible in a clean way. Version information is contained within the source package and it is required by specification to be. Installation happens from this source package. There are no “binaries”.

We understand the need to vote on the artefacts, however the way it is required to work put us between a rock and a hard place: either our users can end up with an outdated pre-release while reporting they have the release installed or we need to vote 2+2 times (PMC+IPMC).

We are looking to optimize this process either technically or procedurally, but until so far haven’t been able to distill anything that really helps.

Cheers
Bolke



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Airflow voting on release artifacts

Alex Harui


On 5/1/17, 11:44 AM, "Bolke de Bruin" <[hidden email]> wrote:

>
>> On 1 May 2017, at 17:36, Alex Harui <[hidden email]> wrote:
>>
>>
>>
>> On 5/1/17, 7:44 AM, "Hitesh Shah" <[hidden email]> wrote:
>>
>>> Hi Justin,
>>>
>>> Currently, the podling has been modifying the contents and hence this
>>> discussion.
>>
>> I agree with Justin and others that modification after the vote is not a
>> good thing.  So my assumption was that if you add your 2a step and
>>modify
>> the binary before the vote, it will be acceptable.  IMO, all you need
>>is a
>> way to verify that the binary the voters test is essentially the same as
>> the binary you want to actually release.
>>
>> -Alex
>>
>>
>
>Hi Alex,
>
>As mentioned earlier this is not possible in a clean way. Version
>information is contained within the source package and it is required by
>specification to be. Installation happens from this source package. There
>are no “binaries”.
>
>We understand the need to vote on the artefacts, however the way it is
>required to work put us between a rock and a hard place: either our users
>can end up with an outdated pre-release while reporting they have the
>release installed or we need to vote 2+2 times (PMC+IPMC).
>
>We are looking to optimize this process either technically or
>procedurally, but until so far haven’t been able to distill anything that
>really helps.

Well, I'm quite confused now.  Hitesh seems to say there are binaries.
And I have proposed a couple of ideas where you create different artifacts
for voters vs. customers that I think get around all of these issues.
AFAIK, nobody on this list has objected to those proposals.

Maybe there is something about Python I don't understand, but if I had to
ship a set of Javascript files with an embedded version number in one of
those files, I would use what I proposed.  AFAICT, there is no obligation
to make your customers (not your voters) consume the source package, it
just has to be possible to generate what the customers use from the source
package.

HTH,
-Alex


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
12