[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: RE: [tag] requirements = test assertion
Forwarding this to the TC as it is clearly yet another
prior art of interest.
My comments:
- I like the "class" approach, in section 2.5.2. that
identifies classes of implementations, with the purpose of targeting a set of
conformance statements to each class. By knowing the relationship between
classes, you can infer which requirements apply, e.g. if a class A inherits from
a class B, the requirements targeting A also include requirements targeting
B.
To me, that says that it helps to think of targets (IUTs)
in tems of classes of items.
Specific cases we could reuse in a guideline
doc:
-Requirement that is written is such a precise and testable
way that nothing more need be added to a related TA
(3...)The system SHALL achieve a Perfect Ballot Index of at least 2.33 as measured by the VPP. - Requirement that would require a more precise
expression when writing a TA, as in itself it does not provide any
measurable or testable behavior. A TA woudl for example narrow the notion of
"help" (or define the forms help can take) in a way that can be
evaluated.
(3...) The voting system
SHALL provide a means for the voter
to get help directly from the system at any time during the voting session.
- Requirement
that poses the interesting question of "composed" targets. In the following
case: how should a TA for assessing the fulfillment of "Voting systems"
to this requirement below, relate to the subtargets that "EMS" and
"vote-capture device" are? Shouldn't we say that a voting system that
contains exactly one EMS instance which is not conforming to the EMS
requirements, is itself NOT conforming? (since a non-conforming EMS actually
might not be considered to be an EMS?) And if it contains one conforming
EMS and one non-conforming EMS, shouldn't the voting system be considered
conforming to the requirement below?
(6.1) Voting systems SHALL contain at least one EMS and at least one vote-capture
device.
leading to similar to the notion
Jacques
From: Lynne Rosenthal [mailto:lynne.rosenthal@nist.gov] Sent: Thursday, December 06, 2007 4:07 PM To: Durand, Jacques R. Subject: RE: [tag] requirements = test assertion This is the standard I
spoke about today, where we wrote the requirements as testable statements – so
basically, the requirement = a test assertion. Look at chapters
3-7. chapter 2 explains conformance which is in terms of conforming to a
‘class of voting system’. lynne From: Durand,
Jacques R. [mailto:JDurand@us.fujitsu.com] Quick summary of current status of
discussions on Anatomy of a TA: (this is not exhaustive of what has been said
this afternoon!) - The TA "model" (what are the
elements of a TA, whether optional or
mandatory, and what they mean) is to be
distinguished from the representation structure(s)
of a TA. In other words, saying that
the "target" is a mandatory element does not
mean that it must be defined explicitly
in every TA representation: as long as a TA representation provides means to
access this element (URL, inheritance, context
info...) then it is still a representation
compliant with our model. - The assertion target identifies
actually a class of items to which the TA applies. There is a rough agreement
that the "target" is more than just a tag or classification means. It helps
identify the [part of an] implementation
that will be under test, for a test case
derived from this TA. - As a normative specification
statement may concern different targets
depending on the user perspective, it is OK to
arbitrarily use any of these targets, or even
several. - The target of a TA may be (and in
general is) a finer grained item than
the target of a conformance clause,
which usually has an intuitive user-level business function. The latter
corresponds to current definition of
"implementation" (product / service /
document...) - The ID of a TA must be globally
unique, and should reflect spec ID and version
#. - Testability is the prime quality a
test assertion writer should be concerned with. If the addressed normative statement
is unambiguously testable, it is sufficient to precisely refer to it, in order
to have a valid TA (no need to write a separate "assertion expression" or
so.) - Optionality in a specification
(SHOULD, MAY...) is NOT an obstacle to
testability, as the optional quality does not
affect the testability of the feature described, if described precisely
enough. The test assertion may however need
to be worded as [for derived test cases] to verify that "WHEN <the feature is
used > THEN <it MUST be used accordingly to
spec>". Normally, any optional feature can
be interpreted that way and *in the case it is used* is subject to clear
normative statements of mandatory character, in a well written specification. The
WHEN part could be defined in a qualifier / precondition part of the TA. To be
more specific: when a spec says: "the widget X MAY be present in the
package. The widget X MUST satisfy A,B and C." The first sentence in itself is not
material for a TA. The two sentences together are. Should the "package" be chosen
as target, the presence of X is a precondition or qualifier of the
TA. - "pre-requisites" of a TA, as
references to other TAs that a target is
supposed to satisfy for this TA to apply, are
just some particular kind of qualifier, i.e. are part of the "qualifier" (or
precondition) for this TA. |
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]