Quality Assurance
February 28th, 2006In other news, the dev mailing list had a nice little flamefest over what is Quality Assurance and what exactly the QA team’s job is and how they are to do it. There was a lot of the extremes where QA would have no power to change anything other then report bugs and hope the developer who maintains the application (if one actually does) will fix it. The other extreme that seemed to come from the same people, appeared to make it appear that QA wanted to go in and basically take over another persons ebuild. Its been a long standing thing with gentoo that people take personal space with the packages they maintain. To a extent this is warranted, however as we do have cvs…we can see exactly what a person did to your package. Sometimes they screw up the delicate balance or sometimes they just go in to fix added white spaces or a improperly committed manifest. In the latter case we can safely assume that minor changes will not kill anything and shouldn’t be any real issue. Fixing a white space problem shouldn’t have to go to the council for approval as was suggested by another developer. This would just jam up the council with a lot of pointless business that is like taking a squabble over the last cup of coffee to the CEO of the company…its just not something you do.
As always the answer lies somewhere in the middle. Now I by no means have the answer, and have stayed out of the topic because I am not impartial, since I am a member of the QA team. The x86 team has I hope shown that QA does work with this project on a smaller scale. I believe that overall x86 has become stable, instead of being called “stable” by user’s and developers alike. Now we have a full fledged group of people who want to focus on QA with the specific goal of improving the entire tree instead of just individual archs.
Without flames, and discussion about other people’s comments. I’d like to hear what everyone thinks QA actually is and the responsibilities that go along with that. Yes I know I sound like a teacher in that last line. I will however snap a ruler on anyones fingers who doesn’t follow the rules of my question. You’ve been warned.
February 28th, 2006 at 8:05 pm
I’m taking a bit of a stab in the dark here as I am somewhat new to the development side of things; I’ve been mentored and I’m waiting for recruiters to look at my dev bug. I also stopped reading the thread about 100 posts in. I had to go to dinner, and when I got back it was up to 150 posts
Anyway, below are my thoughts on what QA is and what the QA team could do.
The root cause of most problems is portage, duh! While portage is nice and flexible and I’d never want to see it replaced, it allows people to do stupid things in ebuilds. Most of the time the package maintainer does it without noticing. The job of the QA people is to notice these subtle mistakes and point them out to the package maintainer.
As part of letting the maintainer know about the problem, a bug should be filed so that repeat offenders can be noticed, and so common errors can be highlighted in the developer documentation.
Another part of the QA team’s job should be documentation. They should maintain documentation about all of the problems they’ve come across and how to fix them. They should do this so that a developer can learn from the past mistakes of others.
Minor QA issues that don’t affect the installed package (example: description typos, HOMEPAGE changes, etc) should never be fixed by QA, the package maintainer should fix these issues. Major issues that causes loss of data, breakages, etc should be dealt with by the QA team only if the package maintainer is not around on IRC and no one in the herd is available to make the change. For everything in between QA should always attempt to contact the package maintainer, but if he is unresponsive or away they should make the change.
If there is a dispute about a request from the QA team, then there should be some sort of “vote” by the QA team to determine who is in the right. Even though maintainership gives some sense of “ownership” to a particular package, if a certain number of QA members (say 5) agree that a change should be made, then the package maintainer must make the change. He then can bring his case to the council if he is still upset with the decision.