launchpad-reviewers team mailing list archive
-
launchpad-reviewers team
-
Mailing list archive
-
Message #26309
[Merge] ~cjwatson/launchpad:py3-flattened-emails into launchpad:master
Colin Watson has proposed merging ~cjwatson/launchpad:py3-flattened-emails into launchpad:master.
Commit message:
Reduce dependencies on flattened email message representation
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
For more details, see:
https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/398029
There are various minor differences in the flattened email message representation produced by `message.as_string()` and similar methods between Python 2 and 3: for instance, quoted-printable encoding of whitespace at the end of a line is handled slightly differently. In most cases we only really care that the message decodes to the expected content, so adjust various tests to compare the decoded output.
--
Your team Launchpad code reviewers is requested to review the proposed merge of ~cjwatson/launchpad:py3-flattened-emails into launchpad:master.
diff --git a/lib/lp/archiveuploader/tests/nascentupload-announcements.txt b/lib/lp/archiveuploader/tests/nascentupload-announcements.txt
index 9e908d5..4557415 100644
--- a/lib/lp/archiveuploader/tests/nascentupload-announcements.txt
+++ b/lib/lp/archiveuploader/tests/nascentupload-announcements.txt
@@ -176,17 +176,16 @@ header however.
This is the body of the rejection email.
>>> body = notification.get_payload()[0]
- >>> print body.as_string() # doctest: -NORMALIZE_WHITESPACE
- Content-Type: text/plain; charset="utf-8"
- MIME-Version: 1.0
- Content-Transfer-Encoding: quoted-printable
- <BLANKLINE>
+ >>> print(body.get('Content-Disposition'))
+ None
+ >>> print(body.get('Content-Type'))
+ text/plain; charset="utf-8"
+ >>> print(body.get_payload(decode=True).decode('UTF-8'))
+ ... # doctest: -NORMALIZE_WHITESPACE
Rejected:
- The source bar - 1.0-1 is already accepted in ubuntu/hoary and you cannot u=
- pload the same version within the same distribution. You have to modify the=
- source version and re-upload.
+ The source bar - 1.0-1 is already accepted in ubuntu/hoary and you cannot upload the same version within the same distribution. You have to modify the source version and re-upload.
<BLANKLINE>
- bar (1.0-1) breezy; urgency=3Dlow
+ bar (1.0-1) breezy; urgency=low
<BLANKLINE>
* Initial version
<BLANKLINE>
@@ -194,14 +193,13 @@ This is the body of the rejection email.
Changed-By: Daniel Silverstone <daniel.silverstone@xxxxxxxxxxxxx>
Maintainer: Launchpad team <launchpad@xxxxxxxxxxxxxxxxxxx>
<BLANKLINE>
- =3D=3D=3D
+ ===
<BLANKLINE>
If you don't understand why your files were rejected, or if the
override file requires editing, please go to:
http://answers.launchpad.net/soyuz
<BLANKLINE>
- -- =
- <BLANKLINE>
+ --
You are receiving this email because you are the most recent person
listed in this package's changelog.
<BLANKLINE>
@@ -210,12 +208,12 @@ In order to facilitate automated processing of announcement emails, the
changes file is enclosed as an attachment.
>>> attachment = notification.get_payload()[1]
- >>> print attachment.as_string() # doctest: -NORMALIZE_WHITESPACE
- Content-Disposition: attachment; filename="changesfile"
- MIME-Version: 1.0
- Content-Type: text/plain; charset="utf-8"
- Content-Transfer-Encoding: quoted-printable
- <BLANKLINE>
+ >>> print(attachment.get('Content-Disposition'))
+ attachment; filename="changesfile"
+ >>> print(attachment.get('Content-Type'))
+ text/plain; charset="utf-8"
+ >>> print(attachment.get_payload(decode=True).decode('UTF-8'))
+ ... # doctest: -NORMALIZE_WHITESPACE
Format: 1.7
Date: Thu, 16 Feb 2006 15:34:09 +0000
Source: bar
@@ -226,16 +224,13 @@ changes file is enclosed as an attachment.
Urgency: low
Maintainer: Launchpad team <launchpad@xxxxxxxxxxxxxxxxxxx>
Changed-By: Daniel Silverstone <daniel.silverstone@xxxxxxxxxxxxx>
- Description: =
- <BLANKLINE>
+ Description:
bar - Stuff for testing
- Changes: =
- <BLANKLINE>
- bar (1.0-1) breezy; urgency=3Dlow
+ Changes:
+ bar (1.0-1) breezy; urgency=low
.
* Initial version
- Files: =
- <BLANKLINE>
+ Files:
5d533778b698edc1a122098a98c8490e 512 devel optional bar_1.0-1.dsc
fc1464e5985b962a042d5354452f361d 164 devel optional bar_1.0.orig.tar.gz
1e35b810764f140af9616de8274e6e73 537 devel optional bar_1.0-1.diff.gz
@@ -666,27 +661,26 @@ changes file is mentioned toward the end of the email.
True
>>> body = announcement_email.get_payload()[0]
- >>> print body.as_string() # doctest: -NORMALIZE_WHITESPACE
- Content-Type: text/plain; charset="utf-8"
- MIME-Version: 1.0
- Content-Transfer-Encoding: quoted-printable
- <BLANKLINE>
- bar (1.0-10) breezy; urgency=3Dlow
+ >>> print(body.get('Content-Disposition'))
+ None
+ >>> print(body.get('Content-Type'))
+ text/plain; charset="utf-8"
+ >>> print(body.get_payload(decode=True).decode('UTF-8'))
+ ... # doctest: -NORMALIZE_WHITESPACE
+ bar (1.0-10) breezy; urgency=low
<BLANKLINE>
* Changes file that contains UTF-8
<BLANKLINE>
- * Non-ascii text: =C4=8Ciha=C5=99
+ * Non-ascii text: Čihař
<BLANKLINE>
<BLANKLINE>
Date: Thu, 30 Mar 2006 01:36:14 +0100
- Changed-By: Non-ascii changed-by =C4=8Ciha=C5=99 <daniel.silverstone@canoni=
- cal.com>
- Maintainer: Non-ascii maintainer =C4=8Ciha=C5=99 <launchpad@lists.canonical=
- .com>
+ Changed-By: Non-ascii changed-by Čihař <daniel.silverstone@xxxxxxxxxxxxx>
+ Maintainer: Non-ascii maintainer Čihař <launchpad@xxxxxxxxxxxxxxxxxxx>
Signed-By: Foo Bar <foo.bar@xxxxxxxxxxxxx>
http://launchpad.test/ubuntu/+source/bar/1.0-10
<BLANKLINE>
- =3D=3D
+ ==
<BLANKLINE>
OK: bar_1.0.orig.tar.gz
OK: bar_1.0-10.diff.gz
@@ -697,8 +691,7 @@ changes file is mentioned toward the end of the email.
<BLANKLINE>
Thank you for your contribution to Ubuntu.
<BLANKLINE>
- -- =
- <BLANKLINE>
+ --
You are receiving this email because you are the most recent person
listed in this package's changelog.
<BLANKLINE>
@@ -715,12 +708,12 @@ Here's the attachment metadata.
And what follows is the content of the attachment.
- >>> print attachment.as_string() # doctest: -NORMALIZE_WHITESPACE
- Content-Disposition: attachment; filename="changesfile"
- MIME-Version: 1.0
- Content-Type: text/plain; charset="utf-8"
- Content-Transfer-Encoding: quoted-printable
- <BLANKLINE>
+ >>> print(attachment.get('Content-Disposition'))
+ attachment; filename="changesfile"
+ >>> print(attachment.get('Content-Type'))
+ text/plain; charset="utf-8"
+ >>> print(attachment.get_payload(decode=True).decode('UTF-8'))
+ ... # doctest: -NORMALIZE_WHITESPACE
Format: 1.7
Date: Thu, 30 Mar 2006 01:36:14 +0100
Source: bar
@@ -729,23 +722,18 @@ And what follows is the content of the attachment.
Version: 1.0-10
Distribution: breezy
Urgency: low
- Maintainer: Non-ascii maintainer =C4=8Ciha=C5=99 <launchpad@lists.canonical=
- .com>
- Changed-By: Non-ascii changed-by =C4=8Ciha=C5=99 <daniel.silverstone@canoni=
- cal.com>
- Description: =
- <BLANKLINE>
+ Maintainer: Non-ascii maintainer Čihař <launchpad@xxxxxxxxxxxxxxxxxxx>
+ Changed-By: Non-ascii changed-by Čihař <daniel.silverstone@xxxxxxxxxxxxx>
+ Description:
bar - Stuff for testing
- Changes: =
- <BLANKLINE>
- bar (1.0-10) breezy; urgency=3Dlow
+ Changes:
+ bar (1.0-10) breezy; urgency=low
.
* Changes file that contains UTF-8
.
- * Non-ascii text: =C4=8Ciha=C5=99
+ * Non-ascii text: Čihař
.
- Files: =
- <BLANKLINE>
+ Files:
a4932aa84fdb62819b49f3dda163fc0d 514 devel optional bar_1.0-10.dsc
ac6b4efe44e31f47ec9f0d0fac6935f4 622 devel optional bar_1.0-10.diff.gz
<BLANKLINE>
diff --git a/lib/lp/bugs/doc/externalbugtracker-debbugs.txt b/lib/lp/bugs/doc/externalbugtracker-debbugs.txt
index 741dcd8..f81a0d2 100644
--- a/lib/lp/bugs/doc/externalbugtracker-debbugs.txt
+++ b/lib/lp/bugs/doc/externalbugtracker-debbugs.txt
@@ -607,15 +607,15 @@ We can look for the mail that would have been sent.
>>> print(msg['X-Envelope-To'])
1234@xxxxxxxxxxx
- >>> print(str(msg))
- From ...
- Content-Type...
- Message-Id: <123456@xxxxxxxxxxxxx>
- To: 1234@xxxxxxxxxxx
- From: debbugs@xxxxxxxxxxxxxxxxxx
- Subject: Re: Example bug 1234
- Date...
- <BLANKLINE>
+ >>> print(msg['Message-Id'])
+ <123456@xxxxxxxxxxxxx>
+ >>> print(msg['To'])
+ 1234@xxxxxxxxxxx
+ >>> print(msg['From'])
+ debbugs@xxxxxxxxxxxxxxxxxx
+ >>> print(msg['Subject'])
+ Re: Example bug 1234
+ >>> print(msg.get_payload(decode=True).decode('UTF-8'))
A little fermented curd will do the trick!
diff --git a/lib/lp/services/mail/doc/emailauthentication.txt b/lib/lp/services/mail/doc/emailauthentication.txt
index 48427bb..ac58668 100644
--- a/lib/lp/services/mail/doc/emailauthentication.txt
+++ b/lib/lp/services/mail/doc/emailauthentication.txt
@@ -160,9 +160,8 @@ hack can be removed.
boundary="--------------------EuxKj2iCbKjpUGkD"
...
- >>> print(msg.get_payload(i=0).as_string()) #doctest: -NORMALIZE_WHITESPACE
- Content-Type: multipart/mixed; boundary="--------------------EuxKj2iCbKjpUGkD"
- ...
+ >>> print(msg.get_payload(i=0)['Content-Type'])
+ multipart/mixed; boundary="--------------------EuxKj2iCbKjpUGkD"
>>> principal = authenticateEmail(msg, accept_any_timestamp)
>>> print(IPerson(principal).displayname)
diff --git a/lib/lp/services/mail/tests/incomingmail.txt b/lib/lp/services/mail/tests/incomingmail.txt
index c42ef12..57556e6 100644
--- a/lib/lp/services/mail/tests/incomingmail.txt
+++ b/lib/lp/services/mail/tests/incomingmail.txt
@@ -122,11 +122,17 @@ wasn't a handler registered for that domain, an OOPS was recorded with
a link to the original message.
>>> from lp.testing.mail_helpers import pop_notifications
- >>> print(str(pop_notifications()[-1]))
- From ...
- Content-Type: multipart/mixed...
- ...
- To: Sample Person <test@xxxxxxxxxxxxx>
+ >>> notification = pop_notifications()[-1]
+ >>> print(notification.get_content_type())
+ multipart/mixed
+ >>> print(notification['To'])
+ Sample Person <test@xxxxxxxxxxxxx>
+ >>> error_message, original_message = notification.get_payload()
+ >>> print(error_message.get_content_type())
+ text/plain
+ >>> print(error_message.get_payload(decode=True).decode())
+ An error occurred while processing a mail you sent to Launchpad's email
+ interface.
...
Sorry, something went wrong when Launchpad tried processing your mail.
We've recorded what happened, and we'll fix it as soon as possible.
@@ -135,11 +141,14 @@ a link to the original message.
If this is blocking your work, please file a question at
https://answers.launchpad.net/launchpad/+addquestion
and include the error ID OOPS-... in the descr...
- ...
- From: Sample Person <test@xxxxxxxxxxxxx>
- To: 123@xxxxxxx
- Subject: Signed Email
- ...
+ >>> print(original_message.get_content_type())
+ message/rfc822
+ >>> print(original_message.get_payload(0)['From'])
+ Sample Person <test@xxxxxxxxxxxxx>
+ >>> print(original_message.get_payload(0)['To'])
+ 123@xxxxxxx
+ >>> print(original_message.get_payload(0)['Subject'])
+ Signed Email
---------------------------------------------
Mail from Persons not registered in Launchpad
@@ -259,11 +268,17 @@ And submit an email to the handler.
An exception is raised, an OOPS is recorded, and an email is sent back
to the user, citing the OOPS ID, with the original message attached.
- >>> print(str(pop_notifications()[-1]))
- From ...
- Content-Type: multipart/mixed...
- ...
- To: Foo Bar <foo.bar@xxxxxxxxxxxxx>
+ >>> notification = pop_notifications()[-1]
+ >>> print(notification.get_content_type())
+ multipart/mixed
+ >>> print(notification['To'])
+ Foo Bar <foo.bar@xxxxxxxxxxxxx>
+ >>> error_message, original_message = notification.get_payload()
+ >>> print(error_message.get_content_type())
+ text/plain
+ >>> print(error_message.get_payload(decode=True).decode())
+ An error occurred while processing a mail you sent to Launchpad's email
+ interface.
...
Sorry, something went wrong when Launchpad tried processing your mail.
We've recorded what happened, and we'll fix it as soon as possible.
@@ -271,13 +286,17 @@ to the user, citing the OOPS ID, with the original message attached.
<BLANKLINE>
If this is blocking your work, please file a question at
https://answers.launchpad.net/launchpad/+addquestion
- and include the error ID OOPS-...... in the descr...
- ...
- From: Foo Bar <foo.bar@xxxxxxxxxxxxx>
- To: launchpad@xxxxxxxx
- X-Launchpad-Original-To: launchpad@xxxxxxxx
- Subject: doesn't matter
- ...
+ and include the error ID OOPS-... in the descr...
+ >>> print(original_message.get_content_type())
+ message/rfc822
+ >>> print(original_message.get_payload(0)['From'])
+ Foo Bar <foo.bar@xxxxxxxxxxxxx>
+ >>> print(original_message.get_payload(0)['To'])
+ launchpad@xxxxxxxx
+ >>> print(original_message.get_payload(0)['X-Launchpad-Original-To'])
+ launchpad@xxxxxxxx
+ >>> print(original_message.get_payload(0)['Subject'])
+ doesn't matter
Unauthorized exceptions, which are ignored for the purpose of OOPS
reporting in the web interface, are not ignored in the email interface.
@@ -303,11 +322,17 @@ reporting in the web interface, are not ignored in the email interface.
...
twisted.cred.error.Unauthorized
- >>> print(str(pop_notifications()[-1]))
- From ...
- Content-Type: multipart/mixed...
- ...
- To: Foo Bar <foo.bar@xxxxxxxxxxxxx>
+ >>> notification = pop_notifications()[-1]
+ >>> print(notification.get_content_type())
+ multipart/mixed
+ >>> print(notification['To'])
+ Foo Bar <foo.bar@xxxxxxxxxxxxx>
+ >>> error_message, original_message = notification.get_payload()
+ >>> print(error_message.get_content_type())
+ text/plain
+ >>> print(error_message.get_payload(decode=True).decode())
+ An error occurred while processing a mail you sent to Launchpad's email
+ interface.
...
Sorry, something went wrong when Launchpad tried processing your mail.
We've recorded what happened, and we'll fix it as soon as possible.
@@ -315,13 +340,17 @@ reporting in the web interface, are not ignored in the email interface.
<BLANKLINE>
If this is blocking your work, please file a question at
https://answers.launchpad.net/launchpad/+addquestion
- and include the error ID OOPS-...... in the descr...
- ...
- From: Foo Bar <foo.bar@xxxxxxxxxxxxx>
- To: launchpad@xxxxxxxxxxxxxxxx
- X-Launchpad-Original-To: launchpad@xxxxxxxxxxxxxxxx
- Subject: doesn't matter
- ...
+ and include the error ID OOPS-... in the descr...
+ >>> print(original_message.get_content_type())
+ message/rfc822
+ >>> print(original_message.get_payload(0)['From'])
+ Foo Bar <foo.bar@xxxxxxxxxxxxx>
+ >>> print(original_message.get_payload(0)['To'])
+ launchpad@xxxxxxxxxxxxxxxx
+ >>> print(original_message.get_payload(0)['X-Launchpad-Original-To'])
+ launchpad@xxxxxxxxxxxxxxxx
+ >>> print(original_message.get_payload(0)['Subject'])
+ doesn't matter
-------------
DB exceptions
diff --git a/lib/lp/soyuz/doc/distroseriesqueue-notify.txt b/lib/lp/soyuz/doc/distroseriesqueue-notify.txt
index 2176890..42cfdbc 100644
--- a/lib/lp/soyuz/doc/distroseriesqueue-notify.txt
+++ b/lib/lp/soyuz/doc/distroseriesqueue-notify.txt
@@ -84,9 +84,9 @@ uploader's email address.
The mail body contains a list of files that were accepted:
- >>> print(notification.get_payload(0)) # doctest: -NORMALIZE_WHITESPACE
- From nobody ...
- ...
+ >>> print(notification.get_payload(0).get_payload(
+ ... decode=True).decode('UTF-8'))
+ ... # doctest: -NORMALIZE_WHITESPACE
NEW: netapplet_1.0-1.dsc
NEW: netapplet_1.0.orig.tar.gz
NEW: netapplet_1.0-1.diff.gz
@@ -95,8 +95,7 @@ The mail body contains a list of files that were accepted:
You may have gotten the distroseries wrong. If so, you may get warnings
above if files already exist in other distroseries.
<BLANKLINE>
- -- =
- <BLANKLINE>
+ --
You are receiving this email because you are the most recent person
listed in this package's changelog.
<BLANKLINE>
@@ -184,17 +183,17 @@ the uploader and the changer) and the changer.
The mail body contains the same list of files again:
- >>> print(msgs[0].get_payload(0)) # doctest: -NORMALIZE_WHITESPACE
- From nobody ...
- ...
+ >>> print(msgs[0].get_payload(0).get_payload(decode=True).decode('UTF-8'))
+ ... # doctest: -NORMALIZE_WHITESPACE
+ netapplet (1.0-1) ...
+ <BLANKLINE>
OK: netapplet_1.0-1.dsc
-> Component: main Section: web
OK: netapplet_1.0.orig.tar.gz
OK: netapplet_1.0-1.diff.gz
<BLANKLINE>
...
- -- =
- <BLANKLINE>
+ --
You are receiving this email because you are the most recent person
listed in this package's changelog.
<BLANKLINE>
@@ -267,10 +266,9 @@ The mail headers are the same as before:
The mail body contains the same list of files again:
- >>> print(changer_notification.get_payload(0))
+ >>> print(changer_notification.get_payload(0).get_payload(
+ ... decode=True).decode('UTF-8'))
... # doctest: -NORMALIZE_WHITESPACE
- From nobody ...
- ...
NEW: netapplet_1.0-1.dsc
NEW: netapplet_1.0.orig.tar.gz
NEW: netapplet_1.0-1.diff.gz
@@ -279,15 +277,13 @@ The mail body contains the same list of files again:
You may have gotten the distroseries wrong. If so, you may get warnings
above if files already exist in other distroseries.
<BLANKLINE>
- -- =
- <BLANKLINE>
+ --
You are receiving this email because you are the most recent person
listed in this package's changelog.
<BLANKLINE>
- >>> print(signer_notification.get_payload(0))
+ >>> print(signer_notification.get_payload(0).get_payload(
+ ... decode=True).decode('UTF-8'))
... # doctest: -NORMALIZE_WHITESPACE
- From nobody ...
- ...
NEW: netapplet_1.0-1.dsc
NEW: netapplet_1.0.orig.tar.gz
NEW: netapplet_1.0-1.diff.gz
@@ -296,8 +292,7 @@ The mail body contains the same list of files again:
You may have gotten the distroseries wrong. If so, you may get warnings
above if files already exist in other distroseries.
<BLANKLINE>
- -- =
- <BLANKLINE>
+ --
You are receiving this email because you made this upload.
<BLANKLINE>
diff --git a/lib/lp/soyuz/doc/soyuz-set-of-uploads.txt b/lib/lp/soyuz/doc/soyuz-set-of-uploads.txt
index 563cbc9..61f08cc 100644
--- a/lib/lp/soyuz/doc/soyuz-set-of-uploads.txt
+++ b/lib/lp/soyuz/doc/soyuz-set-of-uploads.txt
@@ -284,7 +284,11 @@ succeed. A helper function, simulate_upload does that with all the checking.
... for message in pop_notifications(commit=False):
... print("To:", sort_addresses(message['to']))
... print("Subject:", message['subject'])
- ... print(message.get_payload()[0].as_string())
+ ... print("Content-Type:",
+ ... message.get_payload()[0]['content-type'])
+ ... print()
+ ... print(message.get_payload()[0].get_payload(
+ ... decode=True).decode('UTF-8'))
... print()
The 'bar' package' is an arch-all package. We have four stages to the
@@ -334,10 +338,8 @@ Check the email recipient for displayname containing special chars,
To: "Foo B. Bar" <foo.bar@xxxxxxxxxxxxx>
Subject: [ubuntutest/breezy] bar 1.0-4 (Accepted)
Content-Type: text/plain; charset="utf-8"
- MIME-Version: 1.0
- Content-Transfer-Encoding: quoted-printable
<BLANKLINE>
- bar (1.0-4) breezy; urgency=3Dlow
+ bar (1.0-4) breezy; urgency=low
<BLANKLINE>
* Changer using non-preferred email
<BLANKLINE>
@@ -347,7 +349,7 @@ Check the email recipient for displayname containing special chars,
Signed-By: foo.bar@xxxxxxxxxxxxx (Foo B. Bar)
http://launchpad.test/ubuntutest/+source/bar/1.0-4
<BLANKLINE>
- =3D=3D
+ ==
<BLANKLINE>
OK: bar_1.0.orig.tar.gz
OK: bar_1.0-4.diff.gz
@@ -358,8 +360,7 @@ Check the email recipient for displayname containing special chars,
<BLANKLINE>
Thank you for your contribution to ubuntutest.
<BLANKLINE>
- -- =
- <BLANKLINE>
+ --
You are receiving this email because you made this upload.
<BLANKLINE>
<BLANKLINE>
