To Software Software - To Overview Projects Board - Overview Projects Board - To Software-Assessment - Software/Assessment - To Software-Assessment Current Tests - Software-Assessment Current Test

Software-Assessment Documentation

Practice On Software-Assessment - The Guide to Software-Assessment

How the process should work: Description of Software Development Update Cycle (Proposal)
- (wiki:Software/Assessment/Documentation/UpdateCycle)
This document defines the Process Workflow of a complete Software-Assessment Cycle with responsibilities, communication channels, working steps, documentation and reporting requirements by each step
This document describes the steps that has been deployed into practice (actual)
Description of Software Development Update Cycle (Proposal) describes how it should work (nominal)

1. Software Developers

Software Development Workflow (wiki:Software/DevelopmentWorkflow) (git commands, git repositories list)
Software Developers can help writing patches or write code for feature requests by eiter way
- Have a look into bugs.cacert.org with not yet fixed bugs. Search for ...
  - Needs work
- Found a bug in the production system by themselves
- Hear about new requirements from Policy Group
- Following Arbitration cases and pickup Arbitrators ruling recommendations
Current Webdb code you'll get from
- Webdb website: About CAcert.Org - Sourcecode
- git central repository: git central repositories
Developers can send in their patches by either way:
- Create your own Developers branch in git repository (git://git-cacert.it-sls.de/<reponame>)
- Send your patch by email to the Software Assessors (-> Software-Assessment Team)
- Attach the patch to the bug# on bugs.cacert.org
- Send your patch thru cacert-devel mailing list
Switch state under bugs.cacert.org to ...
- Fix available

2. Software-Assessors (part I)

Software-Assessors review the bugs list and selects the bugs to be tested next
- This decision can be discussed with other Software-Assessors, Testteam, Developers, Board, or alone
- review of open bugs list on bugs.cacert.org and/or received bugs by email and/or cacert-devel mailing list and/or git developers branch
If there is no bug# created yet under bugs.cacert.org, Software-Assessor or Developer or Testteam t/l add a new bug#
- go to bugs.cacert.org - Add new bug
Software-Assessors adds the patch to the cacert-devel repository
- "git commit" to local repository, then push to centralized repository with "git push"
Software-Assessor triggers the transfer of cacert-devel repository to the testserver
- How ?
  - trigger a script ?
  - start a git command ?
  - from command line ? Markus: currently the update is done manually from the command line.
  - from a website ?
  - BernhardFröhlich: My current procedure to bring the patch onto the testserver:
    - Merge the bug's branch into one of the test branches
    - Note that test branches usually have multiple bugs merged in, so they can be tested simultaneously. If the patch is big, risky or has complications with other patches a new test branch can be created. If necessary the bug's branch may even be merged into multiple test branches...
    - Add a line to pages/index/feed.rss and commit it to the test branch so testers know that a new bug has been merged in.
    - Install the test branch on the testserver using git checkout and/or git pull
    - Switch between the different test branches as necessary...
Software-Assessor informs Testteam t/l about the new added bugs by either email, phone, irc or other communication channels
Switch bugs.cacert.org state to ...
- Needs Review & Testing

3. Testteam t/l (part I)

Testteam t/l is responsible
- to maintain the Testers info page
- and also the finished Tests wiki pages
- to inform the testers
- to signal the end of test round to Software-Assessment team
- to maintain the testteam (recruitment of new testers)
Testteam t/l cleans up the Main Entry Info Page for Software Testers from previous test round by creating a new wiki page Software/CurrentTest/testfinishedYYYYMMDD and copies the informations from the CurrentTest page onto the finished Test page and adds a link onto the main test info page
- Template for testfinishedYYYYMMDD
  - testfinishedYYYYMMDD
    
    template for new testfinished page
  - type testfinishedYYYYMMDD :
Testteam t/l prepares the Main Entry Info Page for Software Testers with the infos about the added bugs
Testteam t/l mails out to the testteam the starting of a new test round

4. Testers

Testers visits the Main Entry Info Page for Software Testers wiki page, to get informed about the new patches.
Testers selects one of the listed bugs and tests the system
Testers reports to the bug# under bugs.cacert.org about their tests by adding a note
- if the bug is fixed
- if there are more problems with the bug
- if there are side effects
- if there are other problems
- at least Testers add a Checked note
- You can add as many notes you want
Tester is free in what he is testing, the only requirement -> reporting
If there are several bugs in one test round, Tester can choose one, two, three, ... bugs to test. It only depends on the time the Tester spends testing
If you want to help in Software testing, please contact Testteam t/l

5. Testteam t/l (part II) (optional)

Testteam t/l contacts Software-Assessors team, that the testround has been finished by either email or other communications channel
Switch state under bugs.cacert.org from .. to ...
- Needs Review & Testing
  
  Needs Review
  
  Needs Testing
  
  Needs Review

6. Software-Assessors (part II)

Software-Assessor reviews the current test round and signals the end of the testround to Testteam t/l (optional)
1st Software-Assessor reviews the patch(es) and adds a Checked mark to the bug# under bugs.cacert.org
- New Mantis State for Check 1 is under deployment "review-feedback" or "need-review"
1st Software-Assessor requests a review to one of the other Software-Assessors by either email or other communications channel
2nd Software-Assessor reviews the patch(es) and adds a Checked mark to the bug# under bugs.cacert.org
- New Mantis State for Check 2 is under deployment "reviewed-done"
One of the involved Software-Assessors triggers the transfer of the new Software-Revision to the Critical Team by sending an e-mail with at least the following information:
- Reason for change
- Reference to Bug number in the database on bugs.cacert.org
  Markus: I usually put "patch request: <mantis ticket url> into the subject, then add a small abstract about the contents / intention of the patch. Add list of to be deleted files (though I'm not sure if the unified diff does handle deletions automatically), add output of "git diff oldhash..newhash >patchrequest_bugXXX.diff", do not copy&paste the output from shell to the mail, as I found that this causes problems with the patch command sometimes. Be careful with the hashes, ideally the patch is introduced in one single commit, so the two hashes are adjacent to each other. If thats not the case, maybe its better to make a new clone of cacert (live webdb) to an branch of cacert-devel, apply the patched files from cacert-devel master, commit and diff from there.
- Patch in unified diff format for each file to be changed (as attachment) "git diff oldhash..newhash >patchrequest_bugXXX.diff"
- Full source of any new file(s) to be added (as attachment)

Markus: its ok for the Critical Admin Team if they get the unified diff, no need to add the plain text files as well.

Name(s) of file(s) to be removed
If necessary: specific non-standard installation instructions
Switching bugs.cacert.org state to ...
- Ready to deploy

7. Critical Team

Critical team receives the request by e-mail to critical-admin@cacert.org
A member of the critical team will lookup the information for the specified Bug number in the bugs.cacert.org database to verify that the requested change is fully "signed-off"
Patch(es) and any new file(s) are uploaded to critical system
Patch(es) are applied to live system, new files are added to and obsoleted files are removed from the live system
All changes to the critical system are committed in the CVS repository on the critical system
A new tarball is created, containing full new version of the application as it is deployed on the live system
Information from the CVS commit is mailed to cacert-systemlog@lists.cacert.org including reference to the Bug number
New tarball is mirrored to the test system (cacert1.it-sls.de:/home/cacert/www/tarballs)
- u60: who imports the mirror into cacert.git repository ?
Switch state under bugs.cacert.org to ...
- Solved?

8. Software-Assessors (part III)

How does the Software-Assessors becomes informed, that the patches had been applied ?
- Software-Assessors have actively to monitor the cacert-systemlog@lists.cacert.org mailing list for notification of updates from step before
- Has the Critical Sysadmin send an email to the Software-Assessor who started the Update request ?
  - wytze: No, the Software-Assessor will monitor the cacert-systemlog@lists.cacert.org mailing list for such a notification
What are the next steps?
Markus: As Software Assessors can not change an Ticket status from resolved to closed, the SA need to inform people with enough karma to close the tickets (i.e. Andreas Bäß, Ulrich Schröter, ...). Maybe permissions for SA need to be adapted.
Inform Testteam t/l that the testround has been finished and new testround begins?
Markus: Update the Newsfeed text to push the current status out to the testers. Maybe an mailing list for test staff would be great?
Is there maintenance on repositories needed?
Markus: After an successful patch, it would be wise to start over with the latest copy from the live repository. I.e. remove all files from cacert-devel and copy fresh code from cacert OR make a new branch based on cacert.
Switch bugs.cacert.org state to ...
- Closed

Software/Assessment/Documentation (last edited 2021-04-11 14:54:35 by BernhardFröhlich)

Needs Review & Testing	Needs Review
Needs Testing	Needs Review