This site is from a past semester! The current version will be here when the new semester starts.
CS2113/T 2020 Jan-Apr
  • Full Timeline
  • Week 1 [from Mon Jan 13]
  • Week 2 [from Wed Jan 15 noon]
  • Week 3 [from Wed Jan 22 noon]
  • Week 4 [from Wed Jan 29 noon]
  • Week 5 [from Wed Feb 5 noon]
  • Week 6 [from Wed Feb 12 noon]
  • Week 7 [from Wed Feb 19 noon]
  • Week 8 [from Wed Mar 4 noon]
  • Week 9 [from Wed Mar 11 noon]
  • Week 10 [from Wed Mar 18 noon]
  • Week 11 [from Wed Mar 25 noon]
  • Week 12 [from Wed Apr 1 noon]
  • Week 13 [from Wed Apr 8 noon]
  • Textbook
  • Admin Info
  • Report Bugs
  • Forum
  • Instructors
  • Announcements
  • File Submissions
  • Tutorial Schedule
  • repl.it link
  • Java Coding Standard
  • Forum Activities Dashboard
  • Participation Dashboard

  •  Individual Project (iP):
  • Individual Project Info
  • Duke Upstream Repo
  • iP Code Dashboard
  • iP Progress Dashboard

  •  Team Project (tP):
  • Team Project Info
  • Team List
  • tP Code Dashboard
  • tP Progress Dashboard
  • tP: mid-v2.1 [week 12] tP: Deliverables


    tP: v2.1 [week 13]

    1. Submit deliverables by Saturday 11th Apr 23.59
    2. Submit demo video by Monday 13th Apr 23.59
    3. Prepare for the practical exam before the lecture
    4. Attend the practical exam during the lecture

    1 Submit deliverables by Saturday 11th Apr 23.59

    • Deadline for all v2.1 submissions is Week 12 Saturday 23.59 unless stated otherwise.
    • Submit to LumiNUS folder we have set up, not to your project space.
      cs2113T students: documents should be submitted to both modules. It's not enough to submit to CS2101 side only.
    • Penalty for late submission:
      -1 mark for missing the deadline (up to 2 hour of delay).
      -2 for an extended delay (up to 24 hours late).
      Penalty for delays beyond 24 hours is determined on a case by case basis.
      • Even a one-second delay is considered late, irrespective of the reason.
      • For submissions done via LumiNUS, the submission time is the timestamp shown by LumiNUS.
      • When determining the late submission penalty, we take the latest submission even if the same exact file was submitted earlier. Do not submit the same file multiple times if you want to avoid unnecessary late submission penalties.
      • The whole team is penalized for problems in team submissions. Only the respective student is penalized for problems in individual submissions.
    • Please follow submission instructions closely. Any non-compliance will be penalized. e.g. wrong file name/format.
      • For pdf submissions, ensure the file is usable and hyperlinks in the file are correct. Problems in documents are considered bugs too e.g. broken links, outdated diagrams/instructions etc..
    • Do not update the code during the 14 days after the deadline. Get our permission first if you need to update the code in the repo during that freeze period.
      • You can update issues/milestones/PRs even during the freeze period.
      • You can update the code during the freeze period if the change is related to a late submission approved by us.
      • You can continue to evolve your repo after the freeze period.

    Submissions:

    • To convert the UG/DG/PPP into PDF format, go to the generated page in your project's github.io site and use this technique to save as a pdf file. Using other techniques can result in poor quality resolution (will be considered a bug) and unnecessarily large files. You may un-tick the background graphics option as well if you wish.

    Saving as PDF files

    1. Use Chrome to load the page you want to save as pdf.

    2. Click on the Print option in Chrome’s menu.

    3. Set the destination to Save as PDF, then click Save to save a copy of the file in PDF format. For best results, use the settings indicated in the screenshot below.

    • Ensure hyperlinks in the pdf files work. Your UG/DG/PPP will be evaluated using PDF files during the PE. Broken/non-working hyperlinks in the PDF files will be reported as bugs and will count against your project score. Again, use the conversion technique given above to ensure links in the PDF files work.
    • Product:
      • Do a release on GitHub, tagged as v2.1.
      • Upload the jar file to LumiNUS (in addition making a release on GitHub).
        File name: [team][product name].jar e.g. [CS2113T-W09-1][ContactsPlus].jar
    • Should be an executable jar file.
    • Should be i.e., it can be used by end-usersreleasable. While some features may be scheduled for later versions, the features in v2.1 should be good enough to make it usable by at least some of the target users.
    • Also note the following constraint:

    Constraint-File-Size

    The file sizes of the deliverables should not exceed the limits given below.
    Reason: It is hard to download big files during the practical exam due to limited WiFi bandwidth at the venue:

    • JAR file: 100Mb (Some third-party software -- e.g., Stanford NLP library, certain graphics libraries -- can cause you to exceed this limit)
    • PDF files: 15Mb/file (Not following the recommended method of converting to PDF can cause big PDF files. Another cause is using unnecessarily high resolution images for screenshots).

    • Source Code: Push the code to GitHub and tag with the version number. Source code (please ensure the code reported by RepoSense as yours is correct; any updates to RepoSense config files or @@author annotations after the deadline will be considered a later submission). Note that the quality of the code attributed to you accounts for a significant component of your final score, graded individually.
    • Should match v2.1 deliverables i.e., executable, docs, website, etc.
    • To be delivered as a Git repo. Ensure your GitHub team repo is updated to match the executable.

    • User Guide: Convert to pdf and upload to LumiNUS.
      Upload to LumiNUS.
      File name: [TEAM_ID][product Name]UG.pdf e.g.[CS2113T-W09-1][Contacts Plus]UG.pdf

    In UG/DG, using hierarchical section numbering and figure numbering is optional (reason: it's not easy to do in MarkDown), but make sure it does not inconvenience the reader (e.g., use section/figure title and/or hyperlinks to point to the section/figure being referred to). Examples:

    In the section Implementation given above ...

    • Follow the AddressBook-Level3 (AB3) UG structure.
    • Should cover all v2.1 features.
      Ensure those descriptions match the product precisely, as it will be used by peer testers (inaccuracies will be considered bugs).
    • Optionally, can also cover future features. Mark those as Coming in v3.0.
    • Also note the following constraint:

    Constraint-File-Size

    The file sizes of the deliverables should not exceed the limits given below.
    Reason: It is hard to download big files during the practical exam due to limited WiFi bandwidth at the venue:

    • JAR file: 100Mb (Some third-party software -- e.g., Stanford NLP library, certain graphics libraries -- can cause you to exceed this limit)
    • PDF files: 15Mb/file (Not following the recommended method of converting to PDF can cause big PDF files. Another cause is using unnecessarily high resolution images for screenshots).

    • Developer Guide: submit similar to UG
      Upload to LumiNUS.
      File name: [TEAM_ID][product Name]DG.pdf e.g. [CS2113T-W09-1][Contacts Plus]DG.pdf
    • Should match the v2.1 implementation. Optionally, can include proposed implementations of future features.

    • Follow the AddressBook-Level3 (AB3) DG structure. Sections to include in your DG:

      • Design: similar to AB3 DG except,
        • you may omit the Architecture section (no penalty)
        • if you have not organized the code into clearly divided components (no penalty if you didn't), you can use a single class diagram (if it is not too complicated) or use several class diagrams each describing a different area of the system.
      • Implementation: similar to AB3 DG
      • Appendix A: Product Scope
      • Appendix B: User Stories
      • Appendix C: Non Functional Requirements
      • Appendix D: Glossary
      • Appendix E: Instructions for Manual Testing (more details below)
    • Include an appendix named Instructions for Manual Testing, to give some guidance to the tester to chart a path through the features, and provide some important test inputs the tester can copy-paste into the app.

      • Cover all user-testable features.
      • No need to give a long list of test cases including all possible variations. It is upto the tester to come up with those variations.
      • Inaccurate instructions will be considered bugs.
    DG Tips
    • Aim to showcase your documentation skills. The stated objective of the DG is to explain the implementation to a future developer, but a secondary objective is to serve as evidence of your ability to document deeply-technical content using prose, examples, diagrams, code snippets, etc. appropriately. To that end, you may also describe features that you plan to implement in the future, even beyond v2.1 (hypothetically).
      For an example, see the description of the undo/redo feature implementation in the AddressBook-Level3 (AB3) developer guide.
    • Use multiple UML diagram types. Following from the point above, try to include UML diagrams of multiple types to showcase your ability to use different UML diagrams.
    • Keep diagrams simple. The aim is to make diagrams comprehensible, not necessarily comprehensive.
      Ways to simplify diagrams:
      • Omit less important details. Examples:
        • a class diagram can omit minor utility classes, private/unimportant members; some less-important associations can be shown as attributes instead.
        • a sequence diagram can omit less important interactions, self-calls.
      • Omit repetitive details e.g., a class diagram can show only a few representative ones in place of many similar classes (note how the AB3 Logic class diagram shows concrete *Command classes).
      • Limit the scope of a diagram. Decide the purpose of the diagram (i.e., what does it help to explain?) and omit details not relevant to that purpose.
      • Break diagrams into smaller fragments when possible.
        • If a component has a lot of classes, consider further dividing into sub-components (e.g., a Parser sub-component inside the Logic component). After that, sub-components can be shown as black-boxes in the main diagram and their details can be shown as separate diagrams.
        • You can use ref frames to break sequence diagrams to multiple diagrams.
    • Integrate diagrams into the description. Place the diagram close to where it is being described.
    • Use code snippets sparingly. The more you use code snippets in the DG, and longer the code snippet, the higher the risk of it getting outdated quickly. Instead, use code snippets only when necessary and cite only the strictly relevant parts only.
    • Resize diagrams so that the text size in the diagram matches the the text size of the main text of the diagram. See example.

    These class diagrams seem to have lot of member details, which can get outdated pretty quickly:


    This class diagram seems to have too many classes:
    These sequence diagrams are bordering on 'too complicated':

    In this negative example, the text size in the diagram is much bigger than the text size used by the document:

    It will look more 'polished' if the two text sizes match.

    • Project Portfolio Page (PPP):
      • PDF version: upload to LumiNUS
        File name: [TEAM_ID][Your full Name as Given in LumiNUS]PPP.pdf e.g.[CS2113T-W09-1][Leow Wai Kit, John]PPP.pdf
        Use - in place of / if your name has it e.g., Ravi s/o VeeganRavi s-o Veegan (reason: Windows does not allow / in file names)
      • HTML version: make available on github.io

    At the end of the project each student is required to submit a Project Portfolio Page.

    PPP Objectives

    • For you to use (e.g. in your resume) as a well-documented data point of your SE experience
    • For evaluators to use as a data point to evaluate your project contributions

    PPP Sections to include

    • Overview: A short overview of your product to provide some context to the reader. The opening 1-2 sentences may be reused by all team members. If your product overview extends beyond 1-2 sentences, the remainder should be written by yourself.
    • Summary of Contributions --Suggested items to include:
      • Code contributed: Give a link to your code on tP Code Dashboard. The link is available in the Project List Page -- linked to the icon under your photo.
      • Enhancements implemented: A summary of the enhancements you implemented.
      • Contributions to documentation: Which sections did you contribute to the UG?
      • Contributions to the DG: Which sections did you contribute to the DG? Which UML diagrams did you add/updated?
      • Contributions to team-based tasks :
      • Review/mentoring contributions: Links to PRs reviewed, instances of helping team members in other ways
      • Contributions beyond the project team:
        • Evidence of helping others e.g. responses you posted in our forum, bugs you reported in other team's products,
        • Evidence of technical leadership e.g. sharing useful information in the forum

    Team-tasks are the tasks that someone in the team has to do. Marks allocated to team-tasks will be divided among team members based on how much each member contributed to those tasks.

    Here is a non-exhaustive list of team-tasks:

    1. Necessary general code enhancements
    2. Setting up tools e.g., GitHub, Gradle
    3. Maintaining the issue tracker
    4. Release management
    5. Updating user/developer docs that are not specific to a feature e.g. documenting the target user profile
    6. Incorporating more useful tools/libraries/frameworks into the product or the project workflow (e.g. automate more aspects of the project workflow using a GitHub plugin)

    Keep in mind that evaluators will use the PPP to estimate your project effort. We recommend that you mention things that will earn you a fair score e.g., explain how deep the enhancement is, why it is complete, how hard it was to implement etc..

    • [Optional] Contributions to the User Guide (Extracts): Reproduce the parts in the User Guide that you wrote. This can include features you implemented as well as features you propose to implement.
      The purpose of allowing you to include proposed features is to provide you more flexibility to show your documentation skills. e.g. you can bring in a proposed feature just to give you an opportunity to use a UML diagram type not used by the actual features.

    • [Optional] Contributions to the Developer Guide (Extracts): Reproduce the parts in the Developer Guide that you wrote. Ensure there is enough content to evaluate your technical documentation skills and UML modelling skills. You can include descriptions of your design/implementations, possible alternatives, pros and cons of alternatives, etc.

    • [Optional] If you plan to use the PPP in your Resume, you can also include your SE work outside of the module (will not be graded).

    PPP Format

    • To convert the UG/DG/PPP into PDF format, go to the generated page in your project's github.io site and use this technique to save as a pdf file. Using other techniques can result in poor quality resolution (will be considered a bug) and unnecessarily large files. You may un-tick the background graphics option as well if you wish.

    Saving as PDF files

    1. Use Chrome to load the page you want to save as pdf.

    2. Click on the Print option in Chrome’s menu.

    3. Set the destination to Save as PDF, then click Save to save a copy of the file in PDF format. For best results, use the settings indicated in the screenshot below.

    • Ensure hyperlinks in the pdf files work. Your UG/DG/PPP will be evaluated using PDF files during the PE. Broken/non-working hyperlinks in the PDF files will be reported as bugs and will count against your project score. Again, use the conversion technique given above to ensure links in the PDF files work.

    PPP Page Limit

    Content Recommended Hard Limit
    Overview + Summary of contributions 0.5-1 2
    [Optional] Contributions to the User Guide 1-3
    [Optional] Contributions to the Developer Guide 3-6
    • The page limits given above are after converting to PDF format. The actual amount of content you require is actually less than what these numbers suggest because the HTML → PDF conversion adds a lot of spacing around content.

    • Product Website: Ensure the website is updated and auto-published.
    • Use the GitHub Pages feature to publish your tP repo as a website.

    Website [optional] README

    This deliverable is optional.

    • Ensure the Ui.png matches the current product

    Some common sense tips for a good product screenshot

    Ui.png represents your product in its full glory.

    • Before taking the screenshot, populate the product with data that makes the product look good. For example,
    • If the product is supposed to show photos, use real photos instead of dummy placeholders.
    • If the product doesn't have nice line wrapping for long inputs/outputs, don't use such inputs/outputs for the screenshot.
    • It should show a state in which the product is well-populated i.e., don't leave data panels largely blank
    • Choose a state that showcase the main features of the product i.e., the login screen is not usually a good choice
    • Take a clean screenshot with a decent resolution. Some screenshot tools can capture a specified window only. If your tool cannot do that, make sure you crop away the extraneous parts captured by the screenshot.
    • Avoid annotations (arrows, callouts, explanatory text etc.); it should look like the product is in use for real.
     

    Reason: Distracting annotations.

     

    Reason: Not enough data.

     

    Reason: screenshot not cropped cleanly (contains extra background details)

     

    Website [optional] AboutUs Page

    This deliverable is optional.

    • Use a suitable profile photo.

    The purpose of the profile photo is for the teaching team to identify you. Therefore, choose a recent individual photo showing your face clearly (i.e., not too small) -- somewhat similar to a passport photo. Some examples can be seen in the 'Teaching team' page. Given below are some examples of good and bad profile photos.

    If you are uncomfortable posting your photo due to security reasons, you can post a lower resolution image so that it is hard for someone to misuse that image for fraudulent purposes. If you are concerned about privacy, you may use a placeholder image in place of the photo in module-related documents that are publicly visible.

    • Include a link to each person's PPP page.
    • Team member names match full names used by LumiNUS.

    Website UG (Web Page)

    • Should match the submitted PDF file.

    Website DG (Web Page)

    • Should match the submitted PDF file.

    2 Submit demo video by Monday 13th Apr 23.59

    • Submit the demo video to LumiNUS.

    Due to the COVID-19 situation, this deliverable has been changed as follows:

    • Screen-record a demo of all the product features, in a reasonable order.
    • Including an audio explanation is strongly encouraged (not compulsory) -- alternatively, you can switch between slides and the app to give additional explanations. Annotations and other enhancements to the video are optional (those will not earn any extra marks).
    • All members taking part in the demo video is encouraged but not compulsory.
    • File name: [TEAM_ID][product Name].mp4 e.g.[CS2113T-W09-1][Contacts Plus].mp4 (other video formats are acceptable but use a format that works on all major OS'es).
    • File size: Recommended to keep below 200Mb. You can use a low resolution as long as the video is in usable quality.
    • Submission: Submit to LumiNUS (different folder).
    • Deadline: Monday 13th April 23.59 (i.e., 2 days after the main deadline)
    • Also refer to info given below.

    Demo Duration

    • Strictly 18 minutes for a 5-person team, 15 minutes for a 4-person team, 21 minutes for a 6-person team. Exceeding this limit will be penalized.

    Demo Target audience

    • Assume you are giving a demo to a higher-level manager of your company, to brief him/her on the current capabilities of the product. This is the first time they are seeing the new product you developed. The actual audience are the evaluators (the team supervisor and another tutor).

    Demo Scope

    • Start by giving an overview of the product so that the evaluators get a sense of the full picture early. Include the following:
      • What is it? e.g., FooBar is a product to ensure the user takes frequent standing-breaks while working.
      • Who is it for? e.g., It is for someone who works at a PC, prefers typing, and wants to avoid prolonged periods of sitting.
      • How does it help? Give an overview of how the product's features help to solve the target problem for the target user

    Here is an example:

    Hi, welcome to the demo of our product FooBar. It is a product to ensure the user takes
    frequent standing-breaks while working.
    It is for someone who works at a PC, prefers typing, and wants to avoid prolonged periods
    of sitting.
    The user first sets the parameters such as frequency and targets, and then enters a
    command to record the start of the sitting time, ... The app shows the length of the
    sitting periods, and alerts the user if ...
    ...
    • There is no need to introduce team members or explain who did what. Reason: to save time.
    • Present the features in a reasonable order: Organize the demo to present a cohesive picture of the product as a whole, presented in a logical order.
    • No need to cover design/implementation details as the manager is not interested in those details.

    Demo Structure

    • Demo the product using the same executable you submitted
    • Use a sufficient amount of Mr aaa is not a realistic person namerealistic demo data. e.g at least 20 data items. Trying to demo a product using just 1-2 sample data creates a bad impression.

    Demo Tips

    • Plan the demo to be in sync with the impression you want to create. For example, if you are trying to convince that the product is easy to use, show the easiest way to perform a task before you show the full command with all the bells and whistles.

    3 Prepare for the practical exam before the lecture

    Objectives:

    • The primary objective of the PE is to increase the rigor of project grading. Assessing most aspects of the project involves an element subjectivity. As the project counts for a large percentage of the final grade, it is not prudent to rely on evaluations of tutors alone as there can be significant variations between how different tutors assess projects. That is why we collect more data points via the PE so as to minimize the chance of your project being affected by evaluator-bias.
    • PE is also used to evaluate your manual testing skills, product evaluation skills, effort estimation skills etc.
    • Note that significant project components are not graded solely based on peer ratings. Rather, PE data are mostly used to cross-validate tutors' grades and identify cases that need further investigation. When peer inputs are used for grading, they are usually combined with tutor evaluations with appropriate weight for each. In some cases ratings from team members are given a higher weight compared to ratings from other peers, if that is appropriate.
    • PE is not a means of pitting you against each other. Developers and testers play for the same side; they need to push each other to improve the quality of their work -- not bring down each other.

    Grading:

    • Your performance in the practical exam will affect your final grade and your peers', as explained in Admin: Project Grading section.
    • As such, we have put in measures to identify and penalize insincere/random evaluations.
    • Also see:
    Grading bugs found in the PE
    • Of Developer Testing component, based on the bugs found in your code3A and System/Acceptance Testing component, based on the bugs found in others' code3B above, the one you do better will be given a 70% weight and the other a 30% weight so that your total score is driven by your strengths rather than weaknesses.
    • Bugs rejected by the dev team, if the rejection is approved by the teaching team, will not affect marks of the tester or the developer.
    • The penalty/credit for a bug varies based on,
      • The severity of the bug: severity.High > severity.Medium > severity.Low > severity.VeryLow
      • The type of the bug: type.FunctionalityBug > type.DocumentationBug > type.FeatureFlaw
    • The penalty for a bug is divided equally among assignees.
    • Developers are not penalized for duplicate bug reports they received but the testers earn credit for duplicate bug reports they submitted as long as the duplicates are not submitted by the same tester.
    • i.e., the same bug reported by many testersObvious bugs earn less credit for the tester and slightly more penalty for the developer.
    • If the team you tested has a low bug count i.e., total bugs found by all testers is low, we will fall back on other means (e.g., performance in PE dry run) to calculate your marks for system/acceptance testing.
    • Your marks for developer testing depends on the bug density rather than total bug count. Here's an example:
      • n bugs found in your feature; it is a difficult feature consisting of lot of code → 4/5 marks
      • n bugs found in your feature; it is a small feature with a small amount of code → 1/5 marks
    • You don't need to find all bugs in the product to get full marks. For example, finding half of the bugs of that product or 4 bugs, whichever the lower, could earn you full marks.
    • Excessive incorrect downgrading/rejecting/marking as duplicatesduplicate-flagging, if deemed an attempt to game the system, will be penalized.

    • Ensure that you have accepted the invitation to join the GitHub org used by the module. Go to https://github.com/nus-cs2113-AY1920S2 to accept the invitation.

    • Ensure you have access to a computer that is able to run module projects e.g. has the right Java version.

    • Download the latest CATcher and ensure you can run it on your computer.

    Issues created for PE-D and PE need to be in a precise format for our grading scripts to work. Incorrectly-formatted responses will have to discarded. Therefore, you are not allowed to use the GitHub interface for PE-D and PE activities, unless you have obtained our permission first.

    • Create a public repo in your GitHub account with the following name:
      • PE Dry Run: ped
      • PE: pe
    • Enable its issue tracker and add the following labels to it (the label names should be precisely as given).

    Bug Severity labels:

    • severity.VeryLow : A flaw that is purely cosmetic and does not affect usage e.g., a typo/spacing/layout/color/font issues in the docs or the UI that doesn't affect usage.
    • severity.Low : A flaw that is unlikely to affect normal operations of the product. Appears only in very rare situations and causes a minor inconvenience only.
    • severity.Medium : A flaw that causes occasional inconvenience to some users but they can continue to use the product.
    • severity.High : A flaw that affects most users and causes major problems for users. i.e., makes the product almost unusable for most users.

    Type labels:

    • type.FunctionalityBug: A functionality does not work as specified/expected.
    • type.FeatureFlaw: Some functionality missing from a feature delivered in v2.1 in a way that the feature becomes less useful to the intended target user for normal usage. i.e., the feature is not 'complete'. In other words, an acceptance testing bug that falls within the scope of v2.1 features. These issues are counted against the 'depth and completeness' of the feature.
    • type.DocumentationBug: A flaw in the documentation that can potentially affect the user e.g., a missing step, a wrong instruction, typos that affect users

    • Have a good screen grab tool with annotation features so that you can quickly take a screenshot of a bug, annotate it, and post in the issue tracker.

      • You can use Ctrl+V to paste a picture from the clipboard into a text box in bug report.
    • Download the product to be tested.

    • After you have been notified which team to test (likely to be in the morning of PE-D day), download the jar file from the team's releases page.
    • After you have been notified of the download location, download the zip file that bears your name.

    • Charge your computer before coming to the session. The testing venue might not have enough charging points.

    4 Attend the practical exam during the lecture

    • Attend the practical test, to be done during the lecture.

    PE Overview

    Objectives:

    • The primary objective of the PE is to increase the rigor of project grading. Assessing most aspects of the project involves an element subjectivity. As the project counts for a large percentage of the final grade, it is not prudent to rely on evaluations of tutors alone as there can be significant variations between how different tutors assess projects. That is why we collect more data points via the PE so as to minimize the chance of your project being affected by evaluator-bias.
    • PE is also used to evaluate your manual testing skills, product evaluation skills, effort estimation skills etc.
    • Note that significant project components are not graded solely based on peer ratings. Rather, PE data are mostly used to cross-validate tutors' grades and identify cases that need further investigation. When peer inputs are used for grading, they are usually combined with tutor evaluations with appropriate weight for each. In some cases ratings from team members are given a higher weight compared to ratings from other peers, if that is appropriate.
    • PE is not a means of pitting you against each other. Developers and testers play for the same side; they need to push each other to improve the quality of their work -- not bring down each other.

    Grading:

    • Your performance in the practical exam will affect your final grade and your peers', as explained in Admin: Project Grading section.
    • As such, we have put in measures to identify and penalize insincere/random evaluations.
    • Also see:
    Grading bugs found in the PE
    • Of Developer Testing component, based on the bugs found in your code3A and System/Acceptance Testing component, based on the bugs found in others' code3B above, the one you do better will be given a 70% weight and the other a 30% weight so that your total score is driven by your strengths rather than weaknesses.
    • Bugs rejected by the dev team, if the rejection is approved by the teaching team, will not affect marks of the tester or the developer.
    • The penalty/credit for a bug varies based on,
      • The severity of the bug: severity.High > severity.Medium > severity.Low > severity.VeryLow
      • The type of the bug: type.FunctionalityBug > type.DocumentationBug > type.FeatureFlaw
    • The penalty for a bug is divided equally among assignees.
    • Developers are not penalized for duplicate bug reports they received but the testers earn credit for duplicate bug reports they submitted as long as the duplicates are not submitted by the same tester.
    • i.e., the same bug reported by many testersObvious bugs earn less credit for the tester and slightly more penalty for the developer.
    • If the team you tested has a low bug count i.e., total bugs found by all testers is low, we will fall back on other means (e.g., performance in PE dry run) to calculate your marks for system/acceptance testing.
    • Your marks for developer testing depends on the bug density rather than total bug count. Here's an example:
      • n bugs found in your feature; it is a difficult feature consisting of lot of code → 4/5 marks
      • n bugs found in your feature; it is a small feature with a small amount of code → 1/5 marks
    • You don't need to find all bugs in the product to get full marks. For example, finding half of the bugs of that product or 4 bugs, whichever the lower, could earn you full marks.
    • Excessive incorrect downgrading/rejecting/marking as duplicatesduplicate-flagging, if deemed an attempt to game the system, will be penalized.

    PE Preparation

    • It's similar to,
    • Ensure that you have accepted the invitation to join the GitHub org used by the module. Go to https://github.com/nus-cs2113-AY1920S2 to accept the invitation.

    • Ensure you have access to a computer that is able to run module projects e.g. has the right Java version.

    • Download the latest CATcher and ensure you can run it on your computer.

    Issues created for PE-D and PE need to be in a precise format for our grading scripts to work. Incorrectly-formatted responses will have to discarded. Therefore, you are not allowed to use the GitHub interface for PE-D and PE activities, unless you have obtained our permission first.

    • Create a public repo in your GitHub account with the following name:
      • PE Dry Run: ped
      • PE: pe
    • Enable its issue tracker and add the following labels to it (the label names should be precisely as given).

    Bug Severity labels:

    • severity.VeryLow : A flaw that is purely cosmetic and does not affect usage e.g., a typo/spacing/layout/color/font issues in the docs or the UI that doesn't affect usage.
    • severity.Low : A flaw that is unlikely to affect normal operations of the product. Appears only in very rare situations and causes a minor inconvenience only.
    • severity.Medium : A flaw that causes occasional inconvenience to some users but they can continue to use the product.
    • severity.High : A flaw that affects most users and causes major problems for users. i.e., makes the product almost unusable for most users.

    Type labels:

    • type.FunctionalityBug: A functionality does not work as specified/expected.
    • type.FeatureFlaw: Some functionality missing from a feature delivered in v2.1 in a way that the feature becomes less useful to the intended target user for normal usage. i.e., the feature is not 'complete'. In other words, an acceptance testing bug that falls within the scope of v2.1 features. These issues are counted against the 'depth and completeness' of the feature.
    • type.DocumentationBug: A flaw in the documentation that can potentially affect the user e.g., a missing step, a wrong instruction, typos that affect users

    • Have a good screen grab tool with annotation features so that you can quickly take a screenshot of a bug, annotate it, and post in the issue tracker.

      • You can use Ctrl+V to paste a picture from the clipboard into a text box in bug report.
    • Download the product to be tested.

    • After you have been notified which team to test (likely to be in the morning of PE-D day), download the jar file from the team's releases page.
    • After you have been notified of the download location, download the zip file that bears your name.

    • Charge your computer before coming to the session. The testing venue might not have enough charging points.

    PE Phase 1: Bug Reporting

    PE Phase 1 is an 'exam', to be done under exam conditions:

    • You are not allowed to communicate with others (except the invigilators).
    • You are not allowed to share your work with others.
    • Violators will be reported to the university for disciplinary action.

    Online channels used for the PE:

    • Gitter:
      • Prof will be online in Gitter.
      • Direct all your queries to prof as private messages to @damithc. Do not use the public chat channel . If you use the main channel, you might accidentally reveal which team you are testing.
    • Zoom:
      • There will be a Zoom meeting (details will be announced later) to provide you with audio announcements.
      • Join it and keep the speaker on during the PE session so that you can hear our announcements.
      • You can use it to chat with the prof (but Gitter is preferred for chatting as it's harder to post screenshots in Zoom chats).
      • You will not be able to share your screen, share your cam, or talk to us.
    • When, where: Week 13 lecture slot
    PE Phase 1 - Part I Product Testing [60 minutes]
    • Test the product and report bugs as described below. You may report both product bugs and documentation bugs during this period.
    Testing instructions for PE and PE-D
    a) Launching the JAR file
    • Get the jar file to be tested:
    • Download the jar file from the team's releases page, if you haven't done this already.
    • Download the zip file from the given location, if you haven't done that already.
    • Unzip the downloaded zip file with the password (to be given to you at the start of the PE, via LumiNUS gradebook). This will give you another zip file with the name suffix _inner.zip.
    • Unzip the inner zip file. This will give you the jar file and other PDF files needed for the PE. Warning: do not run the jar file while it is still inside the zip file.

    • Put the downloaded jar file in an empty folder.
    • Open a command window. Run the java -version command to ensure you are using Java 11.
    • Launch the jar file using the java -jar command (do not use double-clicking).
    • If the product doesn't work at all: If the product fails catastrophically e.g., cannot even launch, or even the basic commands crash the app, contact the invigilator (via Gitter or Zoom chat) to receive a fallback team to test.
    b) What to test
    • Test the product based on the User Guide available from their GitHub website https://{team-id}.github.io/tp/UserGuide.html.
    • Do system testing first i.e., does the product work as specified by the documentation?. If there is time left, you can do acceptance testing as well i.e., does the product solve the problem it claims to solve?.
    • Test based on the Developer Guide (Appendix named Instructions for Manual Testing) and the User Guide. The testing instructions in the Developer Guide can provide you some guidance but if you follow those instructions strictly, you are unlikely to find many bugs. You can deviate from the instructions to probe areas that are more likely to have bugs.
    • As before, do both system testing and acceptance testing but give priority to system testing as system-testing bugs can earn you more credit.

    c) What bugs to report?
    • You may report functionality bugs, UG bugs, and feature flaws.

    These are considered functionality bugs:
    Behavior differs from the User Guide
    A legitimate user behavior is not handled e.g. incorrect commands, extra parameters
    Behavior is not specified and differs from normal expectations e.g. error message does not match the error

    These are considered feature flaws:
    The feature does not solve the stated problem of the intended user i.e., the feature is 'incomplete'
    Hard-to-test features
    Features that don't fit well with the product
    Features that are not optimized enough for fast-typists or target users

    These are considered UG bugs (if they hinder the reader):

    Use of visuals

    • Not enough visuals e.g., screenshots/diagrams
    • The visuals are not well integrated to the explanation
    • The visuals are unnecessarily repetitive e.g., same visual repeated with minor changes

    Use of examples:

    • Not enough or too many examples e.g., sample inputs/outputs

    Explanations:

    • The explanation is too brief or unnecessarily long.
    • The information is hard to understand for the target audience. e.g., using terms the reader might not know

    Neatness/Correctness:

    • looks messy
    • not well-formatted
    • broken links, other inaccuracies, typos, etc.

    • You can also post suggestions on how to improve the product.
      Be diplomatic when reporting bugs or suggesting improvements. For example, instead of criticising the current behavior, simply suggest alternatives to consider.
    • Report functionality bugs:

    These are considered functionality bugs:
    Behavior differs from the User Guide
    A legitimate user behavior is not handled e.g. incorrect commands, extra parameters
    Behavior is not specified and differs from normal expectations e.g. error message does not match the error

    • Do not post suggestions but if a feature is missing a critical functionality that makes the feature less useful to the intended user, it can be reported as a bug of type Type.FeatureFlaw.

    These are considered feature flaws:
    The feature does not solve the stated problem of the intended user i.e., the feature is 'incomplete'
    Hard-to-test features
    Features that don't fit well with the product
    Features that are not optimized enough for fast-typists or target users

    • You may also report UG bugs.

    These are considered UG bugs (if they hinder the reader):

    Use of visuals

    • Not enough visuals e.g., screenshots/diagrams
    • The visuals are not well integrated to the explanation
    • The visuals are unnecessarily repetitive e.g., same visual repeated with minor changes

    Use of examples:

    • Not enough or too many examples e.g., sample inputs/outputs

    Explanations:

    • The explanation is too brief or unnecessarily long.
    • The information is hard to understand for the target audience. e.g., using terms the reader might not know

    Neatness/Correctness:

    • looks messy
    • not well-formatted
    • broken links, other inaccuracies, typos, etc.

    d) How to report bugs
    • Post bugs as you find them (i.e., do not wait to post all bugs at the end) because bug reports created/modified after the allocated time will not count.
    • Launch CATcher, and login to the correct profile:
      • PE Dry Run: CS2113/T PE Dry run
      • PE: CS2113/T PE
    • Post bugs using CATcher.
    • Post bug reports in the following repo you created earlier:
      • PE Dry Run: ped
      • PE: pe
    • The whole description of the bug should be in the issue description i.e., do not add comments to the issue.
    e) Bug report format
    • Each bug should be a separate issue.
    • Write good quality bug reports; poor quality or incorrect bug reports will not earn credit.
    • Use a descriptive title.
    • Give a good description of the bug with steps to reproduce and screenshots. If the receiving team cannot reproduce the bug, you will not be able to get credit for it.
    • Assign exactly one severity.* label to the bug report. Bug report without a severity label are considered severity.Low (lower severity bugs earn lower credit)

    Bug Severity labels:

    • severity.VeryLow : A flaw that is purely cosmetic and does not affect usage e.g., a typo/spacing/layout/color/font issues in the docs or the UI that doesn't affect usage.
    • severity.Low : A flaw that is unlikely to affect normal operations of the product. Appears only in very rare situations and causes a minor inconvenience only.
    • severity.Medium : A flaw that causes occasional inconvenience to some users but they can continue to use the product.
    • severity.High : A flaw that affects most users and causes major problems for users. i.e., makes the product almost unusable for most users.
    • Assign exactly one type.* label to the issue.

    Type labels:

    • type.FunctionalityBug: A functionality does not work as specified/expected.
    • type.FeatureFlaw: Some functionality missing from a feature delivered in v2.1 in a way that the feature becomes less useful to the intended target user for normal usage. i.e., the feature is not 'complete'. In other words, an acceptance testing bug that falls within the scope of v2.1 features. These issues are counted against the 'depth and completeness' of the feature.
    • type.DocumentationBug: A flaw in the documentation that can potentially affect the user e.g., a missing step, a wrong instruction, typos that affect users
    PE Phase 1 - Part II Evaluating Documents [30 minutes]
    • This slot is for reporting documentation bugs only. You may report bugs related to the UG and the DG.
    • For each bug reported, cite evidence and justify. For example, if you think the explanation of a feature is too brief, explain what information is missing and why the omission hinders the reader.

    These are considered UG bugs (if they hinder the reader):

    Use of visuals

    • Not enough visuals e.g., screenshots/diagrams
    • The visuals are not well integrated to the explanation
    • The visuals are unnecessarily repetitive e.g., same visual repeated with minor changes

    Use of examples:

    • Not enough or too many examples e.g., sample inputs/outputs

    Explanations:

    • The explanation is too brief or unnecessarily long.
    • The information is hard to understand for the target audience. e.g., using terms the reader might not know

    Neatness/Correctness:

    • looks messy
    • not well-formatted
    • broken links, other inaccuracies, typos, etc.

    These are considered DG bugs (if they hinder the reader):

    These are considered UG bugs (if they hinder the reader):

    Use of visuals

    • Not enough visuals e.g., screenshots/diagrams
    • The visuals are not well integrated to the explanation
    • The visuals are unnecessarily repetitive e.g., same visual repeated with minor changes

    Use of examples:

    • Not enough or too many examples e.g., sample inputs/outputs

    Explanations:

    • The explanation is too brief or unnecessarily long.
    • The information is hard to understand for the target audience. e.g., using terms the reader might not know

    Neatness/Correctness:

    • looks messy
    • not well-formatted
    • broken links, other inaccuracies, typos, etc.

    UML diagrams:

    • Notation incorrect or not compliant with the notation covered in the module.
    • Some other type of diagram used when a UML diagram would have worked just as well.
    • The diagram used is not suitable for the purpose it is used.
    • The diagram is too complicated.

    Code snippets:

    • Excessive use of code e.g., a large chunk of code is cited when a smaller extract of would have sufficed.

    Problems in User Stories. Examples:

    • Incorrect format
    • All three parts are not present
    • Benefit does not match the function
    • Important user stories missing

    Problems in NFRs. Examples:

    • Not really a Non-Functional Requirement
    • Not well-defined (i.e., hard to decide when it has been met)
    • Not reasonably achievable
    • Highly relevant NFRs missing

    Problems in Glossary. Examples:

    • Unnecessary terms included
    • Important terms missing

    PE Phase 1 - Part III Overall Evaluation [15 minutes]
    • To be submitted via TEAMMATES. You are recommended to complete this during the PE session itself, but you have until the end of the day to submit (or revise) your submissions.
    Important questions included in the evaluation:

    Q Quality of the product design,
    Evaluate based on the User Guide and the actual product behavior.

    Criterion Unable to judge Low Medium High
    target user not specified clearly specified and narrowed down appropriately
    value proposition not specified The value to target user is low. App is not worth using Some small group of target users might find the app worth using Most of the target users are likely to find the app worth using
    optimized for target user Not enough focus for CLI users Mostly CLI-based, but cumbersome to use most of the time feels like a fast typist can be more productive with the app, compared to an equivalent GUI app without a CLI

    Q Compared to AddressBoook-Level3 (AB3), the overall quality of the UG you evaluated is,
    Evaluate based on fit-for-purpose, from the perspective of a target user. For reference, the AB3 UG is here.

    Q Compared to AB3, the overall quality of the DG you evaluated is,
    Evaluate based on fit-for-purpose from the perspective of a new team member trying to understand the product's internal design by reading the DG. For reference, the AB3 DG is here.

    Q [For each member] The functional code contributed by the person is,
    Consider implementation work only (i.e., exclude testing, documentation, project management etc.)
    The typical iP refers to an iP where all the requirements are met at the minimal expectations given.
    Use the person's PPP and RepoSense page to evaluate the effort.

    Q [Optional] Concerns or any noteworthy observations about the product you evaluated

    PE Phase 2: Developer Response

    This phase is for you to respond to the bug reports you received.

    Duration: The review period will start around 1 day after the PE (exact time to be announced) and will last until the following Saturday midnight (i.e., within 3 days). However, you are recommended to finish this task ASAP, to minimize cutting into your exam preparation work.

    Bug reviewing is recommended to be done as a team as some of the decisions need team consensus.

    Instructions for Reviewing Bug Reports

    • Don't freak out if there are lot of bug reports. Many can be duplicates and some can be false positives. In any case, we anticipate that all of these products will have some bugs and our penalty for bugs is not harsh. Furthermore, it depends on the severity of the bug. Some bug may not even be penalized.
    • CATcher does not come with a UG, but the UI is fairly intuitive (there are tool tips too). Do post in the forum if you need any guidance with its usage.
    • Also note that CATcher hasn't been battle-tested for this phase, in particular, w.r.t. multiple team members editing the same issue concurrently. It is ideal if the team members get together and work through the issues together. If you think others might be editing the same issues at the same time, use the Sync button at the top to force-sync your view with the latest data from GitHub.
  • Launch CATcher, and login to the profile CS2113/T PE. It will show all the bugs assigned to your team, divided into three sections:
    1. Issues Pending Responses - Issues that your team has not processed yet.
    2. Issues Responded - Your job is to get all issues to the second category.
    3. Faulty Issues - e.g., Bugs marked as duplicates of each other, or causing circular duplicate relationships. Fix the problem given so that no issues remain in this category.
  • Respond to the bug reports shown.
  • You must use CATcher. You are strictly prohibited from editing PE bug reports using the GitHub Web interface as it can can render bug reports unprocessable by CATcher, sometimes in an irreversible ways, and can affect the entire class. Please contact the prof if you are unable to use CATcher for some reason.

    • If a bug seems to be for a different product (i.e. wrongly assigned to your team), let us know ASAP.
    • If the bug is reported multiple times,
      • Mark all copies EXCEPT one as duplicates of the one left out (let's call that one the original) using the A Duplicate of tick box..
      • If the duplicates have different severity levels, you should keep the one with the highest severity as the original. But you can downgrade the severity of the original or the duplicates.
      • For each group of duplicates, all duplicates should point to one original i.e., no multiple levels of duplicates, and no cyclical duplication relationships.
      • If the duplication status is eventually accepted, all duplicates will be assumed to have inherited the type.* and severity.* from the original.

    • Apply one of these labels (if missing, we assign: response.Accepted)

    Response Labels:

    • response.Accepted: You accept it as a bug.
    • response.NotInScope: It is a valid issue but not something the team should be penalized for e.g., it was not related to features delivered in v2.1.
    • response.Rejected: What tester treated as a bug is in fact the expected behavior. You can reject bugs that you inherited from AB3.
    • response.CannotReproduce: You are unable to reproduce the behavior reported in the bug after multiple tries.
    • response.IssueUnclear: The issue description is not clear. Don't post comments asking the tester to give more info. The tester will not be able to see those comments because the bug reports are anonymous.
    • Apply one of these labels (if missing, we assign: type.FunctionalityBug)

    Type labels:

    • type.FunctionalityBug: A functionality does not work as specified/expected.
    • type.FeatureFlaw: Some functionality missing from a feature delivered in v2.1 in a way that the feature becomes less useful to the intended target user for normal usage. i.e., the feature is not 'complete'. In other words, an acceptance testing bug that falls within the scope of v2.1 features. These issues are counted against the 'depth and completeness' of the feature.
    • type.DocumentationBug: A flaw in the documentation that can potentially affect the user e.g., a missing step, a wrong instruction, typos that affect users
    • If you disagree with the original severity assigned to the bug, you may change it to the correct level.

    Bug Severity labels:

    • severity.VeryLow : A flaw that is purely cosmetic and does not affect usage e.g., a typo/spacing/layout/color/font issues in the docs or the UI that doesn't affect usage.
    • severity.Low : A flaw that is unlikely to affect normal operations of the product. Appears only in very rare situations and causes a minor inconvenience only.
    • severity.Medium : A flaw that causes occasional inconvenience to some users but they can continue to use the product.
    • severity.High : A flaw that affects most users and causes major problems for users. i.e., makes the product almost unusable for most users.
    • If you need teaching team's inputs when deciding on a bug e.g., if you are not sure if the UML notation is correct, post in the forum. Remember to quote the issue number shown in CATcher (it appears at the end of the issue title).
    • Broken links in UG/DG: Severity can be low or medium depending on how many such cases and how much inconvenience they cause to the reader.
    • UML notation variations caused by the diagramming tool: Can be rejected if not contradicting the standard notation (as given by the textbook) i.e., extra decorations that are not misleading.
      If the same problem is reported for multiple diagrams, can be flagged as duplicates.
      Omit optional notations is not a bug as long it doesn't hinder understanding.
    • Minor typos and grammar errors: These are still considered as severity.VeryLow type.DocumentationBug bugs (even if it is in the actual UI) which carry a very tiny penalty.
    • How to prove that something is 'not in scope': The following (at least one) can be used to prove that a feature was left out deliberately because it was not in scope:
      • The UG specifies it as not supported or coming in a future version.
      • The user cannot attempt to use the missing feature or when the user does so, the software fails gracefully, possibly with a suitable error message i.e., the software should not crash.
    • If a missing feature is essential for the app to be reasonably useful, its omission can be considered a feature flaw even if it can be proven as not in scope as given in the previous point.
    • Decide who should fix the bug. Use the Assignees field to assign the issue to that person(s). There is no need to actually fix the bug though. It's simply an indication/acceptance of responsibility. If there is no assignee, we will distribute the penalty for that bug (if any) among all team members.
      • If it is not easy to decide the assignee(s), we recommend (but not enforce) that the feature owner should be assigned bugs related to the feature, Reason: The feature owner should have defended the feature against bugs using automated tests and defensive coding techniques.
      • It is expected that PR reviewers will take part of the responsibilities if they were given a reasonable time to review the PR (even if they never reviewed it), provided the bug should have been reasonably apparent to a reviewer. For example, if someone put up a PR for review but merged it un-reviewed after two days because no one bothered to review it, it's only fair that the whole team (or whoever was assigned to review the PR) take a share of the blame.

    • As far as possible, choose the correct type.*, severity.*, response.*, assignees, and duplicate status even for bugs you are not accepting or marking as duplicates. Reason: your non-acceptance or duplication status may be rejected in a later phase, in which case we need to grade it as an accepted/non-duplicate bug.

    • Justify your response. For all of the following cases, you must add a comment justifying your stance. Testers will get to respond to all those cases and will be double-checked by the teaching team in later phases. Indiscriminate/unreasonable dev/tester responses, if deemed as a case of trying to game the system, will be penalized.

      • downgrading severity
      • non-acceptance of a bug
      • changing the bug type
      • non-obvious duplicate

    PE Phase 3: Tester Response

    • In this phase you will get to state whether you agree or disagree with the dev team's response to the bugs you reported. If a bug reported has been subjected to any of the below by the receiving dev team, you can record your objections and the reason for the objection.
      • not accepted
      • severity downgraded
      • bug type changed
    • As before, consider carefully before you object to a dev team's response. If many of your objections were overruled by the teaching team later, you will lose marks for not being able to evaluate a bug report properly.
    • You can also refer to the below guidelines:
    • Broken links in UG/DG: Severity can be low or medium depending on how many such cases and how much inconvenience they cause to the reader.
    • UML notation variations caused by the diagramming tool: Can be rejected if not contradicting the standard notation (as given by the textbook) i.e., extra decorations that are not misleading.
      If the same problem is reported for multiple diagrams, can be flagged as duplicates.
      Omit optional notations is not a bug as long it doesn't hinder understanding.
    • Minor typos and grammar errors: These are still considered as severity.VeryLow type.DocumentationBug bugs (even if it is in the actual UI) which carry a very tiny penalty.
    • How to prove that something is 'not in scope': The following (at least one) can be used to prove that a feature was left out deliberately because it was not in scope:
      • The UG specifies it as not supported or coming in a future version.
      • The user cannot attempt to use the missing feature or when the user does so, the software fails gracefully, possibly with a suitable error message i.e., the software should not crash.
    • If a missing feature is essential for the app to be reasonably useful, its omission can be considered a feature flaw even if it can be proven as not in scope as given in the previous point.

    • This phase is optional. If you do not respond to a dev response, we'll assume that you agree with it.
    • Deadline: Reading week Tuesday 2359 (i.e., within 3 days)
    • Procedure:
    • When the phase has been announced as open, login to CATcher as usual (profile: CS2113/T PE).
    • For each issues listed in the Issues Pending Responses section:,
      • Go to the details, and read the dev team's response.
      • If you disagree with any of the items listed, tick on the I disagree tick box and enter your justification for the disagreement, and click Save.
      • If you are fine with the team's changes, click Save without any other changes upon which the issue will move to the Issue Responded section.
    • Note that only bugs that require your response will be shown by CATcher. Bugs already accepted by the team will not appear in CATcher as there is nothing for you to do about them.

    You must use CATcher. You are strictly prohibited from editing PE bug reports using the GitHub Web interface as it can can render bug reports unprocessable by CATcher, sometimes in an irreversible ways, and can affect the entire class. Please contact the prof if you are unable to use CATcher for some reason.

    PE Phase 4: Tutor Moderation

    • In this phase tutors will look through all dev responses you objected to in the previous phase and decide on a final outcome.
    • In the unlikely we need your inputs, the tutor will contact you.


    tP: mid-v2.1 [week 12] tP: Deliverables