Handling the comments received during a public review

Note: A short list for the steps in handling comments is available in OpenDocument and Office formats. A basic template for logging comments is available in OpenDocument and Office formats.

Soliciting, receiving and handling comments on the work produced by a Technical Committee is a key step in the OASIS Process. All TC Work Products must be submitted to public review and a record of all comments received and their resolutions presented to TC Administration before a Work Product can be approved as an OASIS Final Deliverable (Committee Specification or OASIS Standard).

In addition, it is good practice. Soliciting public feedback helps ensure that you have thought through every eventuality and potential usage scenario. Comments can give you suggestions for future directions and, by fostering a dialog with users outside your TC, help promote interest in and adoption and implementation of your specification.

The OASIS TC Process defines a limited number of policies regarding when TCs must seek feedback and what they must do with the comments they receive. In keeping with the OASIS principle of simplicity, these policies impose a minimal set of obligations. TCs are free to go beyond these minimums should they wish. In particular, TCs that plan to submit their work to other standards bodies such as the ISO or ANSI may need to be more proactive in handling comments in order to satisfy their requirements.

A TC's obligations for handling comments

The obligations placed on TCs for handling comments are specificed in six sections of the TC Process:

  1. While the TC is first forming, section 2.2, TC Formation requires that the proposers of a new TC post a pointer to an account of each of the comments / issues raised during that review, along with its resolution.
  2. Section 2.8, TC Visibility requires that comments received through the TC's comment emailing list shall be forwarded to one or more Members of the TC including the TC Chair. It also states that comments can only be received through the comment emailing list.
  3. Section 3.2, Public Review of a Committee Draft requires that comments from non-TC Members must be collected via the TC's archived public comment facility and that the TC must acknowledge the receipt of each comment, track the comments received, and post to its primary e-mail list its disposition of each comment at the end of the review period.
  4. Section 3.4.1, Submission of a Candidate OASIS Standard requires that when the TC resolves to submit a Committee Specification to the membership as a Candidate OASIS Standard, the TC Chair must submit to TC Administration (g) … a pointer to an account of each of the comments/issues raised during the public review period(s), along with its resolution as well as (j) A pointer to the publicly visible comments archive for the originating TC.
  5. Section 3.4.2, Public Review of a Candidate OASIS Standard requires that comments be received from non-members via the comment emailing list and that the TC acknowledge the receipt of each comment, track the comments received, and post to its primary e-mail list the disposition of each comment at the end of the review period.
  6. Section 3.4.3, Balloting for OASIS Standard approval requires that if the TC wishes to submit an amended specification to the membership for a vote, it must provide any disposition of comments received in previous votes.

In summary, a TC's obligations for handling comments are:

  • The TC Chair and optionally other TC members must subscribe to the TC comment mailing list and monitor it for comments.
    • Each OASIS TC is provided with a mailing list for receiving comments. It has the same short-name as the TC mailing list along with "-comment" (e.g. amqp-comment@lists.oasis-open.org)
  • The TC must only accept comments provided via the comment mailing list. Any other source must be ignored.
  • Someone from the TC must reply to the senders of comments to acknowledge their receipt.
  • The TC must maintain a cumulative running list of the comments (a.k.a. a "comment resolution log") received along with the TC's decisions on how to handle each of them.
  • At designated steps in the TC Process, the TC must post a pointer to the comment resolution log to the TC's primary email list. That pointer must be a publicly-accessible form that provides non-TC members and members of the general public with access to the log.

Note that the comment mailing list is always open and a TC may receive comments at any time, in or out of a public review cycle. While the TC Process does not explicitly require that the TC follow the above requirements for comments received outside of public review periods, it is good practice for TCs to do so. In particular, if the TC submits its OASIS Standard to other SDOs, that body's requirements for receiving comments may be more stringent.

Steps for handling comments

For the purposes of these steps, a comment is a statement received (a) in an email to a TC's -comment@ mailing list or (b) in the case of a TC member, in an email to the TC's primary mailing list during an announced public review period that refers to the work product under review. Verbal comments or comments on something other than the work product being reviewed, while important, are not governed by this process.

To meet the requirements for handling comments, take these steps:

  1. Receive the comment: The member responsible for monitoring the comment list should regularly check the TC's comment mailing list and, when a comment is sent it, should report the comment to the TC following whatever procedure the TC has agreed to follow. For example, if the TC is using a JIRA project to track issues, the designated person may enter a JIRA ticket with the relevant information and report it to the TC at a regular meeting.

  2. Acknowledge the comment: The member responsible for acknowledging comments should send a reply email to the comment provider acknowledging its receipt and, optionally, providing feedback on what the TC will do with it. The reply need not be extensive or detailed but can be as simple as a thank you. For example, an email like the below:

    [commenter-name],

    We have received your comments on [draft in review] sent by you on [date]. On behalf of the members of the TC, I thank you for your feedback.

    The TC maintains a log of all comments received on its work and periodically posts a log of all comments along with the TC’s decisions on how to address them to its primary emailing list. The archives of this list are publicly visible at [tc mailing list url].

    Again, thank you for your comment and please feel free to send along additional observations.

    is sufficient to satisfy the requirement.

    Note that for some standards bodies such as ANSI acknowledging comments and reporting their disposition to the commenters is a requirement for standards submitted to them for adoption.

  3. Track the comment and record how it will be handled: The TC must create a record of each comment received and, in due course, of the TCs decision on how to dispose of it. The record must, at a minimum, record:

    • The date the comment was received
    • A link to the email in which a comment was received
    • The name of the person or entity providing the comment
    • A brief summary of the comment
    • A statement of the TC’s decision on how to handle the comment, in as much detail as the TC wishes to provide

    The TC is free to include any other information it finds useful in the record but must include the above as a minimum.

    The TC is free to use any template it chooses so long as it meets the criteria above. A simple template is available from TC Administration in OpenDocument and Office formats. If the TC uses JIRA or a Wiki to track comments, the TC must export or otherwise create a stand-alone document from those tools in order to produce an immutable, persistent record.

    Note that the TC is not required to act on any comment received. The TC is free to decide that a comment is not relevant and take no further action on it. Also, the TC is not required to provide a detailed explanation of its reasoning or any actions taken. All the TC is required to do is provide a statement of its decision and wording as brief as TC decided to take no action on the comment or Specification will be edited in line with the suggestion is perfectly sufficient.

  4. Prepare and store the comment resolution log: The TC may produce its comment resolution log as a PDF file, a spreadsheet or a word processing file in OpenDocument or Office format but the comment resolution log must be a stand-alone document. If the TC uses JIRA or a Wiki to track comments, then a stand-alone record must be extracted from this archive. A stand-alone document is required to ensure that the comment resolution log is persistent and immutable.

    After preparing the comment resolution log, store it in the TCs OASIS-provided document repository. When you load the document to the repository, Kavi will generate an email with a pointer to the document. This email will satisfy the TC Process requirement the TC post a pointer to the document.

    Note that the TC Process requires the pointer be posted to the TC’s mailing list “at the end of the review period.” Clearly, reviewing comments and deciding how to address them takes time beyond the formal end of a public review. Therefore, for the purposes of this procedure, the "end of the review period" is deemed to be the point at time in which the TC requests its next formal action on the draft (e.g. next public review, Special Majority Vote to approve as a Committee Specification). The TC can, of course, post the document earlier or post iterations of the document. The availability of the final comment disposition log will be required when the TC is ready to request the next step in the document's development.

  5. Provide the link to the comment resolution log to TC Admin: When requesting TC Admin action on the draft after the public review, please provide the link to the log. TC Admin will ensure that record of comments and their resolutions can readily be found by anyone interested in the TCs work.

    • A link to the comment resolution document will be included in the announcements of subsequent public reviews, document approvals, Special Majority ballots and other communications where it will provide additional, useful information.
    • The comment resolution log will be stored in the OASIS Library along with the public review draft to which it applies. This will help to ensure transparency and traceability between comments received during a public review and their resolution.

    Note that TC Administration will not proceed with a requested action until the comment resolution log meeting these requirements is provided.