kubuntu-council team mailing list archive

Thread
Date
Re: [Merge] ubuntu-manual-tests:fix_missing_tests into ubuntu-manual-tests:main

To: mp+451378@xxxxxxxxxxxxxxxxxx
From: Brian Murray <mp+451378@xxxxxxxxxxxxxxxxxx>
Date: Fri, 15 Sep 2023 00:31:57 -0000
In-reply-to: <169470885341.3332505.2836169077275124427.launchpad@juju-98d295-prod-launchpad-17>
Reply-to: mp+451378@xxxxxxxxxxxxxxxxxx
Sender: noreply@xxxxxxxxxxxxx
While reviewing this MP I noticed a couple of discrepancies between the test cases and the actual installer screens for the 'daily-live' image. Please do a test run of the new installer and update the test cases accordingly as I may not have identified all the discrepancies in my in-line comments.

Diff comments:

> diff --git a/definitions/canary_desktop_cases.xml b/definitions/canary_desktop_cases.xml
> index 1c3ea11..bb0470e 100644
> --- a/definitions/canary_desktop_cases.xml
> +++ b/definitions/canary_desktop_cases.xml
> @@ -272,6 +272,79 @@
>      <dt>Login, ensure that the /home partition is still in use and the data from the previous installation have been preserved</dt>
>    </ut:test>
>  
> +  <ut:test id="entire-disk-zfs">
> +    <dt>Note the state of the 'Erase disk and install FAMILY' radio button</dt>
> +        <dd>The 'Erase disk and install FAMILY' radio button is selected and the 'Advanced features' button is active</dd>
> +        <dt>Click on the 'Advanced features...' button</dt>
> +        <dd>The 'Advanced Features' dialog is displayed</dd>
> +    <dt>Select 'Erase disk and use ZFS'</dt>
> +        <dd>'Erase disk and use ZFS' is selected</dd>
> +    <dt>Click on the 'OK' button</dt>
> +        <dd>The dialog closes and 'ZFS selected' is displayed next to the 'Advanced features...' button</dd>
> +    <dt>Click on the 'Install Now' button</dt>

This is a 'Next' button in the new installer.

> +        <dd>'Write the changes to disks' dialogue appears</dd>
> +    <dt>Click Continue</dt>
> +    <dt>If there is only one hard disk, skip to step 18 (On the 'Where are you?' screen...). Otherwise, on the 'Installation type' screen verify that the drive selected on the Select drive list corresponds to the drive on the chart (e.g /dev/sda)</dt>
> +        <dd>Selected drive is displayed on the chart</dd>
> +    <dt>Verify that the full drive space is allocated</dt>
> +        <dd>Full drive space is allocated for installation</dd>
> +    <dt>Click on the Install Now button</dt>
> +        <dd>The 'Where are you?' screen is displayed</dd>
> +  </ut:test>
> +
> +  <ut:test id="reboot-zfs">
> +    <dt>Allow the machine to reboot</dt>
> +        <dd>The system boots properly</dd>
> +        <dd>The system loads into FAMILY showing username selected</dd>
> +        <dd>Upon login, open a terminal, run the following commands and verify it matches the output</dd>
> +        <dd><code>$ zfs mount | sort
> +            bpool/BOOT/ubuntu_UUID        /boot
> +            rpool/ROOT/ubuntu_UUID        /
> +            rpool/ROOT/ubuntu_UUID/srv    /srv
> +            rpool/ROOT/ubuntu_UUID/usr/local  /usr/local
> +            rpool/ROOT/ubuntu_UUID/var/games  /var/games
> +            rpool/ROOT/ubuntu_UUID/var/lib/AccountsService  /var/lib/AccountsService
> +            rpool/ROOT/ubuntu_UUID/var/lib/apt  /var/lib/apt
> +            rpool/ROOT/ubuntu_UUID/var/lib/dpkg  /var/lib/dpkg
> +            rpool/ROOT/ubuntu_UUID/var/lib/NetworkManager  /var/lib/NetworkManager
> +            rpool/ROOT/ubuntu_UUID/var/lib  /var/lib
> +            rpool/ROOT/ubuntu_UUID/var/log  /var/log
> +            rpool/ROOT/ubuntu_UUID/var/mail  /var/mail
> +            rpool/ROOT/ubuntu_UUID/var/snap  /var/snap
> +            rpool/ROOT/ubuntu_UUID/var/spool  /var/spool
> +            rpool/ROOT/ubuntu_UUID/var/www  /var/www
> +            rpool/USERDATA/root_0y7dio      /root
> +            rpool/USERDATA/u_0y7dio         /home/u
> +        </code></dd>
> +  </ut:test>
> +
> +  <ut:test id="entire-disk-lvm">
> +    <dt>Note the state of the 'Erase disk and install FAMILY' radio button</dt>
> +        <dd>The 'Erase disk and install FAMILY' radio button is selected and the 'Advanced features' button is active</dd>
> +        <dt>Click on the 'Advanced features...' button</dt>
> +        <dd>The 'Advanced Features' dialog is displayed</dd>
> +    <dt>Select 'Use LVM with the new FAMILY installation' and check 'Encrypt the new FAMILY installation for security'</dt>
> +        <dd>'Encrypt the new FAMILY Installation for security' and 'Use LVM with the new FAMILY Installation' is checked</dd>
> +    <dt>Click on the 'OK' button</dt>
> +        <dd>The dialog closes and 'LVM and encryption selected' is displayed next to the 'Advanced features...' button</dd>
> +    <dt>Click on the 'Install Now' button</dt>

It's actually a 'Next' button.

> +        <dd>The 'Choose a security key' screen is displayed with the 'Install Now' button greyed out</dd>
> +    <dt>Enter a security key and type a different key into the 'confirm the security key' input box</dt>
> +        <dd>The installer displays 'Passwords do not match'</dd>
> +    <dt>Enter a security key and type the same key into the 'Confirm the security key' input box</dt>
> +        <dd>The passwords are accepted and the 'Install Now' button can be clicked</dd>
> +    <dt>Check 'Overwrite empty disk space'</dt>
> +    <dt>Click on the Continue button (if there is only one hard disk in the system, the button should read 'Install Now')</dt>
> +        <dd>'Write the changes to disks' dialogue appears</dd>
> +    <dt>Click Continue</dt>
> +    <dt>If there is only one hard disk, skip to step 18 (On the 'Where are you?' screen...). Otherwise, on the 'Installation type' screen verify that the drive selected on the Select drive list corresponds to the drive on the chart (e.g /dev/sda)</dt>

I suspicious of "step 18" being correct. Maybe removing the number would be easiest?

> +        <dd>Selected drive is displayed on the chart</dd>
> +    <dt>Verify that the full drive space is allocated</dt>
> +        <dd>Full drive space is allocated for installation</dd>
> +    <dt>Click on the Install Now button</dt>
> +        <dd>The 'Where are you?' screen is displayed</dd>
> +  </ut:test>
> +
>    <ut:case id="1761_Canary Install (manual partitioning)">
>      <ut:include ref="grub" />
>      <ut:include ref="try-or-install" />
> @@ -498,4 +571,77 @@
>      <ut:include ref="reboot-ubuntu" />
>    </ut:case>
>  
> +  <ut:case id="1783_Canary Install Test GNOME Suite">
> +    <ut:include ref="grub" />
> +    <ut:include ref="try-or-install" />
> +    <ut:include ref="start" />
> +    <ut:include ref="select-installation-type" />
> +    <ut:include ref="entire-disk" />
> +    <ut:include ref="setup" />
> +    <ut:include ref="reboot-ubuntu" />
> +    <dt>After rebooting, follow the two test cases here to test the GNOME suite of applications:</dt>
> +        <dd>https://wiki.ubuntu.com/DesktopTeam/TestPlans/gjs</dd>
> +    <dt>This test case is primarily useful when there have been updates to gjs, but is still worthy of testing on every release.</dt>
> +  </ut:case>
> +
> +  <ut:case id="1784_Install (entire disk with ZFS)">
> +    <ut:include ref="start" />
> +    <ut:include ref="select-installation-type" />
> +    <ut:include ref="entire-disk-zfs" />
> +    <ut:include ref="setup" />
> +    <ut:include ref="reboot-zfs" />
> +  </ut:case>
> +
> +  <ut:case id="1785_Install (entire disk with lvm and encryption)">
> +    <ut:include ref="start" />
> +    <ut:include ref="select-installation-type" />
> +    <ut:include ref="entire-disk-lvm" />
> +    <ut:include ref="setup" />
> +    <ut:include ref="reboot-ubuntu" />
> +  </ut:case>
> +
> +  <ut:case id="1786_Canary Non-English Installation Online">
> +    <p>The goal of this test case is to check that localization support is functional during the installation, that language packs are downloaded and installed correctly from the Internet and that the input methods for either Chinese, Japanese or Korean are working.</p>
> +    <ut:include ref="grub" />
> +    <ut:include ref="try-or-install" />
> +    <ut:include ref="start" />
> +    <ut:include ref="select-installation-type" />
> +    <ut:include ref="entire-disk" />
> +    <ut:include ref="setup" />
> +    <ut:include ref="reboot-ubuntu" />
> +    <dt>Verify that your system is localized</dt>
> +    <dt>Verify that the input method works correctly by entering text in the editor. The input method of choice should be available from the panel and typing in the editor should properly display the ibus chars selector.</dt>
> +  </ut:case>
> +
> +  <ut:case id="1787_Non-English Live Session and Installation; No Network">
> +    <dt>Before doing this test, make sure when booting into the live session you've selected a non-english language. You must also make sure the machine you're installing on has no internet access.</dt>
> +    <ut:include ref="session-start" />
> +    <dt>Verify that the language you've selected is being used in the live session.</dt>
> +    <dt>Verify applications and basic functionality work in the live session, then click the install icon.</dt>
> +    <ut:include ref="try-or-install" />
> +    <ut:include ref="start" />
> +    <ut:include ref="select-installation-type" />
> +    <ut:include ref="entire-disk" />
> +    <ut:include ref="setup" />
> +    <ut:include ref="reboot-ubuntu" />
> +  </ut:case>
> +
> +  <ut:case id="1788_Non-English Live Session and Installation; Full Network Support">
> +    <dt>Before doing this test, make sure when booting into the live session you've selected a non-english language.</dt>
> +    <ut:include ref="session-start" />
> +    <dt>Verify that the language you've selected is being used in the live session.</dt>
> +    <dt>Verify applications and basic functionality work in the live session, then click the install icon.</dt>
> +    <ut:include ref="try-or-install" />
> +    <ut:include ref="start" />
> +    <ut:include ref="select-installation-type" />
> +    <ut:include ref="entire-disk" />
> +    <ut:include ref="setup" />
> +    <ut:include ref="reboot-ubuntu" />
> +  </ut:case>
> +
> +  <ut:case id="1789_Install Screen Reader">
> +    <dt>For this test, you must boot the iso and after hearing the "online" sound, press super+alt+s to activate the screen reader.</dt>

It's actually the Ubuntu start up sound.

> +    <dt>After you've activated the screen reader, you must navigate through the install process doing a "Install (Entire Disk)" install, reboot and verify everthing works as intended.</dt>
> +  </ut:case>
> +
>  </ut:configuration>
> diff --git a/testcases/image/1784_Install (entire disk with ZFS) b/testcases/image/1784_Install (entire disk with ZFS)
> new file mode 100644
> index 0000000..f0da7e8
> --- /dev/null
> +++ b/testcases/image/1784_Install (entire disk with ZFS)
> @@ -0,0 +1,106 @@
> +<!-- Please do not edit this file directly; it was generated with the
> +     tools/test_case_gen script using the following configuration as input:
> +     definitions/canary_desktop_cases.xml
> +-->
> +
> +    <p><em>Proceed in your native language if you wish. Instructions will remain in English.</em></p>
> +    <dl>
> +      
> +    
> +    <dt>Click on 'Continue'</dt>
> +      <dd>The 'Try or Install' screen is displayed with 'try FAMILY' and 'Install FAMILY' buttons</dd>
> +    <dt>Click on the release notes hyperlink to confirm that a browser launches and you are taken to the release notes discourse page.</dt>
> +    <dt>Click on the 'Install FAMILY' icon to select the option and click on the Continue button</dt>
> +      <dd>The 'Keyboard layout' screen appears</dd>
> +      <dd>The proposed keyboard corresponds with your keyboard</dd>
> +    <dt>Select your keyboard layout and click on Continue</dt>
> +      <dd>The 'Connect to internet' screen is displayed</dd>
> +      <dd>The screen should reflect the current status and display those elements</dd>
> +      <dd>
> +         <li>Wired connection</li>
> +         <li>Connect to a Wi-Fi network followed by a scrollable list of available APs, displaying an active one colored with a leading checkmark</li>
> +         <li>Connect to a hidden Wi-Fi network</li>
> +         <li>I don't want to connect to internet for now</li>
> +      </dd>
> +    <dt>Click on 'Continue'</dt>
> +      <dd>The 'Updates and other software' screen is displayed, listing normal and minimal installation.</dd>
> +  

This is also missing the "Other options" section which includes checkboxes for installing thrid-party software and the additional media formats stuff.

> +    
> +    <dt>Click on 'Continue'</dt>
> +      <dd>The 'Installation type' screen is displayed</dd>
> +  
> +    
> +    <dt>Note the state of the 'Erase disk and install FAMILY' radio button</dt>
> +        <dd>The 'Erase disk and install FAMILY' radio button is selected and the 'Advanced features' button is active</dd>
> +        <dt>Click on the 'Advanced features...' button</dt>
> +        <dd>The 'Advanced Features' dialog is displayed</dd>
> +    <dt>Select 'Erase disk and use ZFS'</dt>
> +        <dd>'Erase disk and use ZFS' is selected</dd>
> +    <dt>Click on the 'OK' button</dt>
> +        <dd>The dialog closes and 'ZFS selected' is displayed next to the 'Advanced features...' button</dd>
> +    <dt>Click on the 'Install Now' button</dt>
> +        <dd>'Write the changes to disks' dialogue appears</dd>
> +    <dt>Click Continue</dt>
> +    <dt>If there is only one hard disk, skip to step 18 (On the 'Where are you?' screen...). Otherwise, on the 'Installation type' screen verify that the drive selected on the Select drive list corresponds to the drive on the chart (e.g /dev/sda)</dt>
> +        <dd>Selected drive is displayed on the chart</dd>
> +    <dt>Verify that the full drive space is allocated</dt>
> +        <dd>Full drive space is allocated for installation</dd>
> +    <dt>Click on the Install Now button</dt>
> +        <dd>The 'Where are you?' screen is displayed</dd>
> +  
> +    
> +      <dd>The 'Where are you?' screen is displayed</dd>
> +    <dt>If your system is connected to the network, note the preselected timezone corresponds with your timezone and the city indicated in the text box</dt>
> +      <dd>The timezone and city displayed match your timezone and a major city from your area</dd>
> +    <dt>Select your timezone, and click on the Continue button</dt>
> +      <dd>The 'Who are you?' screen appears</dd>
> +    <dt>Input your initial user details and password <em>admin can not be used - it is a dedicated Linux User</em></dt>
> +
> +      <dd>Name, username and password are accepted.</dd>
> +      <dd>Continue button becomes available</dd>
> +    <dt>Click on 'Continue'</dt>
> +      <dd>The 'Choose your look' screen is displayed showing light and dark options</dd>
> +    <dt>Click on 'Continue'</dt>
> +      <dd>The 'Welcome to FAMILY' slide is displayed</dd>
> +      <dd>The slideshow is entirely in your language</dd>
> +    <dt>Wait for the installer to finish</dt>
> +      <dd>An 'Installation Complete' dialog appears</dd>
> +    <dt>Click the 'Restart into FAMILY XX.XX' button</dt>
> +      <dd>GUI is shut down, a prompt to remove media and press Enter appears</dd>
> +    <dt>Remove the disc and press enter</dt>
> +      <dd>The machine is rebooted</dd>
> +  
> +    
> +    <dt>Allow the machine to reboot</dt>
> +        <dd>The system boots properly</dd>
> +        <dd>The system loads into FAMILY showing username selected</dd>
> +        <dd>Upon login, open a terminal, run the following commands and verify it matches the output</dd>
> +        <dd><code>$ zfs mount | sort
> +            bpool/BOOT/ubuntu_UUID        /boot
> +            rpool/ROOT/ubuntu_UUID        /
> +            rpool/ROOT/ubuntu_UUID/srv    /srv
> +            rpool/ROOT/ubuntu_UUID/usr/local  /usr/local
> +            rpool/ROOT/ubuntu_UUID/var/games  /var/games
> +            rpool/ROOT/ubuntu_UUID/var/lib/AccountsService  /var/lib/AccountsService
> +            rpool/ROOT/ubuntu_UUID/var/lib/apt  /var/lib/apt
> +            rpool/ROOT/ubuntu_UUID/var/lib/dpkg  /var/lib/dpkg
> +            rpool/ROOT/ubuntu_UUID/var/lib/NetworkManager  /var/lib/NetworkManager
> +            rpool/ROOT/ubuntu_UUID/var/lib  /var/lib
> +            rpool/ROOT/ubuntu_UUID/var/log  /var/log
> +            rpool/ROOT/ubuntu_UUID/var/mail  /var/mail
> +            rpool/ROOT/ubuntu_UUID/var/snap  /var/snap
> +            rpool/ROOT/ubuntu_UUID/var/spool  /var/spool
> +            rpool/ROOT/ubuntu_UUID/var/www  /var/www
> +            rpool/USERDATA/root_0y7dio      /root
> +            rpool/USERDATA/u_0y7dio         /home/u
> +        </code></dd>
> +  
> +  
> +    </dl>
> +    <p>If <strong>all</strong> actions produce the expected results described,
> +      please <a href="results#add_result">submit</a> a 'passed' result.</p>
> +    <p>If <strong>any</strong> action fails, or produces an unexpected result,
> +      please <a href="results#add_result">submit</a> a 'failed' result and <a href="../../buginstructions">file a bug</a>. Please be sure to include
> +      the bug number when you <a href="results#add_result">submit</a> your
> +      result.</p>
> +  
> \ No newline at end of file


-- 
https://code.launchpad.net/~ubuntu-testcase/ubuntu-manual-tests/+git/ubuntu-manual-tests/+merge/451378
Your team Ubuntu Testcase Admins is requested to review the proposed merge of ubuntu-manual-tests:fix_missing_tests into ubuntu-manual-tests:main.
References

[Merge] ubuntu-manual-tests:fix_missing_tests into ubuntu-manual-tests:main
From: Tim Andersson, 2023-09-14