軟件測試與評估：02-Intro to Testing

PrinciplesofSoftwareTestingforTesters

Module2

CoreConceptsofSoftwareTestingObjectivesIntroducefoundationtopicsoffunctionaltestingProvidestakeholder-centricvisionsofqualityanddefectExplaintestideasIntroducetestmatricesModule2ContentOutlineDefinitionsDefiningfunctionaltestingDefinitionsofqualityApragmaticdefinitionofdefectDimensionsofqualityTestideasTestideacatalogsTestmatricesFunctionalTestingInthiscourse,weadoptacommon,broadcurrentmeaningforfunctionaltesting.ItisBlackboxInterestedinanyexternallyvisibleormeasurableattributesofthesoftwareotherthanperformance.Infunctionaltesting,wethinkoftheprogramasacollectionoffunctionsWetestitintermsofitsinputsandoutputs.DiscussionExercise2.1:DefineQualityFormpairsDefinequalityWritethedefinitiondownIsthisyourcompany’sdefinition?Isthisyourpartner’scompany’sdefinition?Doesyourpartneragreewithhis/hercorporatedefinitionofquality?HowSomeExpertsHaveDefinedQualityDefinequality

Fitnessforuse(Dr.JosephM.Juran)Thetotalityoffeaturesandcharacteristicsofaproductthatbearonitsabilitytosatisfyagivenneed(AmericanSocietyforQuality)Conformancewithrequirements(PhilipCrosby)Thetotalcompositeproductandservicecharacteristicsofmarketing,engineering,manufacturingandmaintenancethroughwhichtheproductandserviceinusewillmeetexpectationsofthecustomer(ArmandV.Feigenbaum)Noteabsenceof“conformstospecifications.”QualityAsSatisfiersandDissatisfiersJosephJurandistinguishesbetweenCustomerSatisfiersandDissatisfiersaskeydimensionsofquality:CustomerSatisfierstherightfeaturesadequateinstructionCustomerDissatisfiersunreliablehardtousetooslowincompatiblewiththecustomer’sequipmentQualityInvolvesManyStakeholdersInaprojectteammeeting,eachpersonintheroomhasadifferentvisionofwhata“quality”productwouldbe.Fixingdefectsisjustoneissue.ProjectManagerProgrammingUserInterfaceDesignMarketingGlassBoxTestingBlackBoxTestingCustomerServiceWritingManufacturingMultimediaProductionContentDevelopmentExercise2.2:QualityHasManyStakeholders(1/2)Aprogramminggroupagreesonvariablenamingconventionsandfollowsthem.Thentheyhireanewseniorprogrammer,whowon’tfollowtheconventions.Programmingthewayhedidinthe1970’s,hegivesvariablesnameslikeMabelandAl.Someoftheotherprogrammersareunhappyaboutthisandsotheywanttoenterabugreportintothebugtrackingsystem,sayingthat“Mabelisnotapropervariablename.”Question:Doesthisreportbelonginthechangerequestsystem?Whyorwhynot?Exercise2.2:QualityHasManyStakeholders(2/2)AcompanyisdevelopingaproductthattheywillreleaseinEnglishfirst.ThecompanyexpectstocreateotherlanguageversionsbychangingstringsandotherresourcevariablesfromtheEnglishversion.Theyplantolocalizeversionswithoutrecompilingorrelinkingthecode.However,theEnglishcodehasahyphenationerrorthatdoesn’taffectEnglishwordsbutwillmishandlesomeGermanwords.ThelocalizationmanagerwantsthisfiledasabugagainsttheEnglishbasecode.FixingtheproblemintheGermanversionwouldrequireacodechangeandrecompilation.TheAmericanprojectmanagersaysthatitisnotabugbecausetheEnglishversionworksOK.Whoisright?ShouldthechangerequestgowiththeEnglishproduct,theGermanproduct,orboth?AWorkingDefinitionofQualityQualityisvaluetosomeperson. ----GeraldM.WeinbergChangeRequestsandQualityA“defect”–intheeyesofaprojectstakeholder–canincludeanythingabouttheprogramthatcausestheprogramtohavelowervalue.It’sappropriatetoreportanyaspectofthesoftwarethat,inyouropinion(orintheopinionofastakeholderwhoseinterestsyouadvocate)causestheprogramtohavelowervalue.DimensionsofQuality:FURPSReliabilitye.g.,Testtheapplicationbehavesconsistentlyandpredictably.Performancee.g.,TestonlineresponseunderaverageandpeakloadingFunctionalitye.g.,TesttheaccurateworkingsofeachusagescenarioUsabilitye.g.,Testapplicationfromtheperspectiveofconveniencetoend-user.Supportabilitye.g.,Testtheabilitytomaintainandsupportapplicationunderproductionuse1.Functionaltesting

verifiesthatasystemexecutestherequireduse-casescenariosasintended.Functionaltestsmayincludethetestingoffeatures,usagescenariosandsecurity.ThetraditionalviewofFURPSThetraditionalviewofFURPS2.Usabilitytesting

evaluatestheapplicationfromtheuser’sperspective.Usabilitytestsfocusonhumanfactors,aesthetics,consistencyintheuserinterface,

onlineandcontext-sensitivehelp,wizardsandagents,userdocumentation,and

trainingmaterials.ThetraditionalviewofFURPS3.Reliabilitytesting

verifiesthattheapplicationperformsreliablyandisnotpronetofailuresduringexecution(crashes,hangs,memoryleaks).Effectivereliabilitytestingrequiresspecializedtools.Reliabilitytestsincludeintegrity,structure,stress,contentionandvolumetests.ThetraditionalviewofFURPS4.Performancetesting

checksthatthetargetsystemworksfunctionallyandreliablyunderproductionload.Performancetestsincludebenchmarktests,loadtests,andperformanceprofiletests.ThetraditionalviewofFURPS5.Supportabilitytesting

verifiesthattheapplicationcanbedeployedasintended.Supportabilitytestsincludeinstallationandconfigurationtests.ABroaderDefinitionofDimensionsofQualityAccessibilityCapabilityCompatibility ConcurrencyConformancetostandardsEfficiencyInstallabilityanduninstallability

LocalizabilityMaintainabilityPerformancePortability ReliabilityScalabilitySecuritySupportabilityTestabilityUsabilityCollectively,theseareoftencalledQualitiesofService,NonfunctionalRequirements,Attributes,orsimplythe-ilitiesModule2ContentOutlineDefinitionsDefiningfunctionaltestingDefinitionsofqualityApragmaticdefinitionofdefectDimensionsofqualityTestideasTestideacatalogsTestmatricesTestIdeasAtestideaisabriefstatementthatidentifiesatestthatmightbeuseful.Atestideadiffersfromatestcase,inthatthetestideacontainsnospecificationofthetestworkings,onlytheessenceoftheideabehindthetest.Testideasaregeneratorsfortestcases:potentialtestcasesarederivedfromatestideaslist.Akeyquestionforthetesterortestanalystiswhichonesaretheonesworthtrying.Exercise2.3:BrainstormTestIdeas(1/2)GroundRulesforBrainstormingThegoalistogetlotsofideas.Youbrainstormtogethertodiscovercategoriesofpossibletests—goodideasthatyoucanrefinelater.Therearemoregreatideasouttherethanyouthink.Don’tcriticizeothers’contributions.JokesareOK,andareoftenvaluable.Worklater,aloneorinamuchsmallergroup,toeliminateredundancy,cutbadideas,andrefineandoptimizethespecifictests.Often,thesemeetingshaveafacilitator(whorunsthemeeting)andarecorder(whowritesgoodstuffontoflipcharts).Thesetwokeeptheiropinionstothemselves.Exercise2.3:BrainstormTestIdeas(2/2)Afieldcanacceptintegervaluesbetween20and50.Whattestsshouldyoutry?ATestIdeasListforInteger-InputTestsCommonanswerstotheexercisewouldinclude:TestWhyit’sinterestingExpectedresult20SmallestvalidvalueAcceptsit19Smallest-1Reject,errormsg00isalwaysinterestingReject,errormsgBlankEmptyfield,what’sitdo?Reject?Ignore?49ValidvalueAcceptsit50LargestvalidvalueAcceptsit51Largest+1Reject,errormsg-1NegativenumberReject,errormsg42949672962^32,overflowinteger?Reject,errormsgDiscussion2.4:WhereDoTestIdeasComeFrom?WherewouldyouderiveTestIdeasLists?ModelsSpecificationsCustomercomplaintsBrainstormsessionsamongcolleaguesHowdoyoucreatetheminyourcompany? Module2ContentOutlineDefinitionsDefiningfunctionaltestingDefinitionsofqualityApragmaticdefinitionofdefectDimensionsofqualityTestideasTestideacatalogsTestmatricesIdentifyaGenericListofTestIdeasThinkaboutthecategoriesofvaluesyou’dconsidergenerallyapplicableinanintegerinputfieldwithLowerBound(LB)andUpperBound(UB).TestWhyit’sinterestingExpectedresultACatalogofTestIdeasforInteger-InputtestsNothingValidvalueAtLBofvalue AtUBofvalueAtLBofvalue-1 AtUBofvalue+1 OutsideofLBofvalue OutsideofUBofvalue 0 Negative AtLBnumberofdigitsorcharsAtUBnumberofdigitsorcharsEmptyfield(clearthedefaultvalue)OutsideofUBnumberofdigitsorchars Non-digits Wrongdatatype(e.g.decimalintointeger)ExpressionsSpaceNon-printingchar(e.g.,Ctrl+char)DOSfilenamereservedchars(e.g.,"\*.:")UpperASCII(128-254)UppercasecharsLowercasecharsModifiers(e.g.,Ctrl,Alt,Shift-Ctrl,etc.)Functionkey(F2,F3,F4,etc.)TheTest-Ideas

CatalogAtest-ideascatalogisalistofrelatedtestideast