openstack-doc-core team mailing list archive

[Lana Brindley <openstack@xxxxxxxxxxxxxxxx>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 25/06/15 18:05, Tom Fifield wrote:
> On 25/06/15 10:03, Lana Brindley wrote:
>> Hi core team,
>>
>> One of the things that came up as a pain point at Summit was the DocImpact automation. I�ve been doing some thinking about this, and this is where I�m at right now:
>>
>> Workflow: Devs create a patch, and at commit time they think "oh yeah, probably there should be some docs about that" and whack in a 'DocImpact' flag. In other words, there's not a lot of thought going in here, it's just an easy thing to do, they get a warm fuzzy, and there�s an assumption that the documentation fairy comes along and takes care of things. In reality, what then happens is some docs triager spends an hour looking at the patch, searching through the docs, then deciding it has nothing to do with openstack-manuals. Often, the actual doc impact (if there even is one) is against docs in the dev repo, not openstack-manuals.
>>
>> In short, devs recognise they need docs, and DocImpact is an easy way to feel like they're doing something productive to make that happen. What really happens is that those bugs are largely irrelevant for openstack-manuals, which puts pressure on our core team to triage them effectively. To try and get some perspective on the size of the problem, here are some numbers:
>>
>> Of the 508 current openstack-manuals bugs, 214 are the result of the DocImpact script. 51 of these are yet to be triaged.
>> Right now, there are 170 patches in-flight with the DocImpact tag. To state the obvious, if all them land, that�s 170 new bugs in our queue.
>>
>> So, solutions:
>>
>> 1: We have a crack team of docs people (potentially with automation assistance) going through docimpact bugs, cc'ing the original patch authors, and moving them into dev repos where necessarily, and marking the rest invalid. We could team this with a an education campaign on the dev mailing list.
>>
>> 2: We ditch docimpact, and force devs to create their own docs bugs (and patches). This would mean fewer devs get on with the job of documentation, but at least what we do get would be well-considered, rather than hastily added to their commit message.
>>
>> 3: We adjust docimpact to raise a bug in their own dev repo. So, patch against swift with docimpact raises a bug in the swift bug queue, not the openstack-manuals doc queue. Maintenance overhead then remains with dev, and I bet you any money they would self-police that better than we ever could.
>>
>> 4: Some other thing I haven�t thought of yet � ?
>>
>> I have a feeling 3 is where the money's at. I took the liberty of quickly checking with an Infra core, and this change should be relatively easy to implement, too, for the record.
>>
>> Thoughts?
> 
> 
> Combo of modified #1 & modified #3, with a bit of #4 (communication back
> on their patch), with a bit of #5 (modified docimpact parameters).

Can you elaborate?

> 
> 
> While I fully agree that DocImpact is in am imperfect state, and
> developers are falling into the "magic fairy" trap, I want to first go
> back to the reason DocImpact exists and understand the large benefit it
> does give us, despite misgivings.
> 
> Prior to enlisting developer assistance to identify patches that
> potentially resulted in the need for some documentation, our team had to
> put in an enormous amount of effort to try and work out what was going
> on dev-wise. We kinda had an inkling that there were some new features
> going in, and there were probably some pretty important bugs being
> fixed. To work that out, it involved reading almost every single
> blueprint for features, and then trying to read a few hundred random
> patches that looked interesting enough that they might result in docs.
> 
> What the introduction of DocImpact did was, for the most part, solve
> this tracking problem. We successfully leveraged the collective
> developer effort to let us know what the interesting things were.

For the record, I don't think DocImpact is a bad thing, which is why I
probably wouldn't implement solution #2.

If nothing else, it got devs thinking about docs with every patch, and
that's a valuable thing.

> 
> Our developers aren't perfect in doing this, but in general, we get bugs
> lodged so we can track major features, ensure our scripts for config
> changes are working, and find out rather important updates.
> 
> In addition, we are able to fairly apply a priority to the patch that
> comes in. Vendor driver? Don't really care: low.
> Deprecation/Introduction of a major feature? Important: High. Bug Fix
> that seriously changes behaviour: High. See also
> https://wiki.openstack.org/wiki/Documentation/HowTo#Doc_Bug_Triaging_Guidelines

I suspect we're not actually very good at doing this, though.

> 
> Though flawed and in need of a serious kick in the pants, currently
> DocImpact provides the only place this tracking occurs.
> 
> So, whatever solution we adopt, wherever/however it ends up, we need to
> make sure this tracking ability continues - leveraging the effort of
> many to filter the firehose a little.

+1

L

- -- 
Lana Brindley
Technical Writer
Rackspace Cloud Builders Australia
http://lanabrindley.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJVjJZgAAoJELppzVb4+KUyx/gH/jeQI1zprjpMyfvaV6f3cbcf
xkifT6GBz7+ElOAquPzQRJZT63kl80n6GKoMBq8Pu38ab6L3GDf3dI3kzqzvXk+m
orYTuh0qKF3Pcq8ikDIQGINLIK95wds/S1vPNOqy/zEbteRbLU+dIzlg+0b7XYW9
k0zeKEa4sGGk7PdQTebcWt42c/mml2GVsb+WsgxP/Aagq/r9PO5DA5IX/BbQqa5i
fwgaqGVWzqqiWreRBCnp5/g7w4fa0oO2UvVU3JoOK9qpvkTTkxMTzqWMHjsHpWmX
OybNvtYIWIRIIBnzqXE3LBZ6kyAXqEi0xv48HDpl8Gq94LWzmihcCo23pLDixeY=
=njpD
-----END PGP SIGNATURE-----