kubuntu-council team mailing list archive

[Engineering and Systems Software Research Group - UFAL <mp+466403@xxxxxxxxxxxxxxxxxx>]

Engineering and Systems Software Research Group - UFAL has proposed merging lp:~easy-ufal/ubuntu-manual-tests/improve-1549 into lp:ubuntu-manual-tests.

Commit message:
Improve test case 1549_Transmission

Requested reviews:
  Ubuntu Testcase Admins (ubuntu-testcase)

For more details, see:
https://code.launchpad.net/~easy-ufal/ubuntu-manual-tests/improve-1549/+merge/466403

The goal of this merge request is to improve test cases by identifying and removing test smells. The term test smell is often used in automated tests but current research has demonstrated that they also exist in natural language test cases. Potential problems they can cause are non-deterministic behavior,  incomplete verification, poor maintainability, and loss of productivity.

In this merge request, we address the Ambiguous Test smell, which indicates an action in a test case not specific enough, leaving room for different interpretations by the testers.

In the third action, the test states “select a torrent file”. This action should be more specific, otherwise it could lead to:

1. Loss of productivity, as the tester would have to search for a random and suitable .torrent file.
2. Non-deterministic behavior:
    - A tester could select a .torrent file that is too small, small enough for the download to end so quickly that the following action that needs to be verified no longer makes sense.
    - A tester could select a .torrent that is too big, which could be a problem if the test is performed in a VM.
    - A tester could select a .torrent file that has no seed, so the download never starts.

To improve this test and avoid the mentioned problems we propose adding a precondition statement (enclosed in a `<em></em>` tag) that indicates to the tester that a file is needed to perform the test and also specifies a link to easily download the file needed. Regarding the file, we have chosen an Ubuntu ISO .torrent file, as we believe it is a stable choice.

References:

Manoel Aranda, et Al. "A Catalog of Transformations to Remove Test Smells in Natural Language Tests." 28th International Conference on Evaluation and Assessment in Software Engineering (EASE 2024).

Elvys Soares, et Al. "Manual Tests Do Smell! Cataloging and Identifying Natural Language Test Smells." 17th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM 2023).

-- 
Your team Ubuntu Testcase Admins is requested to review the proposed merge of lp:~easy-ufal/ubuntu-manual-tests/improve-1549 into lp:ubuntu-manual-tests.

=== modified file 'testcases/packages/1549_Transmission'
--- testcases/packages/1549_Transmission	2014-02-28 10:59:15 +0000
+++ testcases/packages/1549_Transmission	2024-05-25 11:53:32 +0000
@@ -1,10 +1,16 @@
 This test will check that transmission manage torrents correctly
+<em>
+    To perform this test download the following .torrent file to the "~/Downloads" folder:
+    <ul>
+        <li> https://releases.ubuntu.com/22.04/ubuntu-22.04.4-desktop-amd64.iso.torrent </li>
+    </ul>
+</em>
 <dl>
     <dt> Open transmission</dt>
         <dd>Transmission opens correctly</dd>
     <dt> Click on "Open a Torrent" button</dt>
         <dd>A new window appears</dd>
-    <dt>Select a torrent file</dt>
+    <dt>Select the file "ubuntu-22.04.4-desktop-amd64.iso.torrent" (located at "~/Downloads")</dt>
         <dd>The torrent starts to be downloaded</dd>
     <dt>Click on "Pause Torrent" button</dt>
         <dd>The torrent is paused</dd>