-
-
Notifications
You must be signed in to change notification settings - Fork 609
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
16.0 - Migration mail_embed_image #1402
Open
imlopes
wants to merge
18
commits into
OCA:16.0
Choose a base branch
from
imlopes:16.0-mig-mail_embed_image
base: 16.0
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
18 commits
Select commit
Hold shift + click to select a range
583310e
[ADD] mail_embed_image module
a5497fc
[FIX] solves SMTP line limit error [maximum allowed line length is 998
gfcapalbo 32725e0
[REF] refactor and improvements
gfcapalbo e290c4e
[FIX]
gfcapalbo 45e109f
[FIX]
gfcapalbo 4ae2560
[add] debug feedback
gfcapalbo e11fedc
f
gfcapalbo 857d108
[FIX] exacly reproduce super's default values
hbrunn f05a2ab
[IMP] depend on base
hbrunn 08a7e5c
[FIX] run tests at_install
hbrunn 4178906
[FIX] improve error handling
hbrunn bc55edf
[FIX] make tests more resilient against different testing envs
hbrunn bc3323f
[UPD] Update mail_embed_image.pot
oca-travis 14ea417
[UPD] README.rst
OCA-git-bot 8b930f2
Apply dotfiles
TDu d0f19c2
mail_embed_image 10.0.1.0.1
OCA-git-bot fe62a97
[UPD] README.rst
OCA-git-bot 8fcc1b9
[16.0][MIG] migrating to 16.0
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -17,13 +17,13 @@ Mail Embed Image | |
:target: http://www.gnu.org/licenses/agpl-3.0-standalone.html | ||
:alt: License: AGPL-3 | ||
.. |badge3| image:: https://img.shields.io/badge/github-OCA%2Fsocial-lightgray.png?logo=github | ||
:target: https://github.com/OCA/social/tree/10.0/mail_embed_image | ||
:target: https://github.com/OCA/social/tree/16.0/mail_embed_image | ||
:alt: OCA/social | ||
.. |badge4| image:: https://img.shields.io/badge/weblate-Translate%20me-F47D42.png | ||
:target: https://translation.odoo-community.org/projects/social-10-0/social-10-0-mail_embed_image | ||
:target: https://translation.odoo-community.org/projects/social-16-0/social-16-0-mail_embed_image | ||
:alt: Translate me on Weblate | ||
.. |badge5| image:: https://img.shields.io/badge/runboat-Try%20me-875A7B.png | ||
:target: https://runboat.odoo-community.org/builds?repo=OCA/social&target_branch=10.0 | ||
:target: https://runboat.odoo-community.org/builds?repo=OCA/social&target_branch=16.0 | ||
:alt: Try me on Runboat | ||
|
||
|badge1| |badge2| |badge3| |badge4| |badge5| | ||
|
@@ -42,7 +42,7 @@ Bug Tracker | |
Bugs are tracked on `GitHub Issues <https://github.com/OCA/social/issues>`_. | ||
In case of trouble, please check there if your issue has already been reported. | ||
If you spotted it first, help us to smash it by providing a detailed and welcomed | ||
`feedback <https://github.com/OCA/social/issues/new?body=module:%20mail_embed_image%0Aversion:%2010.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**>`_. | ||
`feedback <https://github.com/OCA/social/issues/new?body=module:%20mail_embed_image%0Aversion:%2016.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**>`_. | ||
|
||
Do not contact contributors directly about support or help with technical issues. | ||
|
||
|
@@ -59,6 +59,7 @@ Contributors | |
|
||
* George Daramouskas <[email protected]> | ||
* Giovanni Francesco Capalbo <[email protected]> | ||
* Italo LOPES <[email protected]> | ||
|
||
Maintainers | ||
~~~~~~~~~~~ | ||
|
@@ -73,6 +74,6 @@ OCA, or the Odoo Community Association, is a nonprofit organization whose | |
mission is to support the collaborative development of Odoo features and | ||
promote its widespread use. | ||
|
||
This module is part of the `OCA/social <https://github.com/OCA/social/tree/10.0/mail_embed_image>`_ project on GitHub. | ||
This module is part of the `OCA/social <https://github.com/OCA/social/tree/16.0/mail_embed_image>`_ project on GitHub. | ||
|
||
You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,3 @@ | ||
# -*- coding: utf-8 -*- | ||
# Copyright 2019 Therp BV <https://therp.nl> | ||
# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl.html). | ||
from . import models |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,3 @@ | ||
# -*- coding: utf-8 -*- | ||
# Copyright 2019 Therp BV <https://therp.nl> | ||
# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl.html). | ||
from . import ir_mail_server |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,71 +1,40 @@ | ||
# -*- coding: utf-8 -*- | ||
# Copyright 2019 Therp BV <https://therp.nl> | ||
# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl.html). | ||
import uuid | ||
import logging | ||
from contextlib import contextmanager | ||
from odoo import models, http | ||
from odoo.addons.base.ir.ir_mail_server import encode_header_param | ||
from werkzeug.test import EnvironBuilder | ||
from werkzeug.wrappers import Request as WerkzeugRequest | ||
from lxml.html.soupparser import fromstring | ||
from lxml.etree import tostring | ||
from base64 import encodestring | ||
import threading | ||
from odoo.http import root as root_wsgi | ||
import uuid | ||
from base64 import b64encode | ||
from email.mime.image import MIMEImage | ||
|
||
import requests | ||
from lxml.html import fromstring, tostring | ||
|
||
logger = logging.getLogger(__name__) | ||
from odoo import models | ||
|
||
_logger = logging.getLogger(__name__) | ||
|
||
class IrMailServer(models.Model): | ||
_inherit = 'ir.mail_server' | ||
|
||
@contextmanager | ||
def _fetch_image(self, path): | ||
public_user = self.env.ref('base.public_user') | ||
session_store = root_wsgi.session_store | ||
session = session_store.new() | ||
session.update({ | ||
'db': threading.current_thread().dbname, | ||
'login': public_user.login, | ||
'uid': public_user.id, | ||
'context': self.env.context, | ||
}) | ||
werkzeug_env = EnvironBuilder(path).get_environ() | ||
werkzeug_request = WerkzeugRequest(werkzeug_env) | ||
werkzeug_request.session = session | ||
# construct an odoo request with this werkzeug request. | ||
request = http.HttpRequest(werkzeug_request) | ||
with request: | ||
request._env = self.env(user=public_user) | ||
endpoint, arguments = http.routing_map( | ||
self.env.registry._init_modules, | ||
False, | ||
self.env['ir.http']._get_converters() | ||
).bind_to_environ( | ||
werkzeug_env).match(return_rule=False,) | ||
yield endpoint, arguments | ||
class IrMailServer(models.Model): | ||
_inherit = "ir.mail_server" | ||
|
||
def build_email( | ||
self, | ||
email_from, | ||
email_to, | ||
subject, | ||
body, | ||
email_cc=None, | ||
email_bcc=None, | ||
reply_to=False, | ||
attachments=None, | ||
message_id=None, | ||
references=None, | ||
object_id=False, | ||
subtype='plain', | ||
headers=None, | ||
body_alternative=None, | ||
subtype_alternative='plain', | ||
self, | ||
email_from, | ||
email_to, | ||
subject, | ||
body, | ||
email_cc=None, | ||
email_bcc=None, | ||
reply_to=False, | ||
attachments=None, | ||
message_id=None, | ||
references=None, | ||
object_id=False, | ||
subtype="plain", | ||
headers=None, | ||
body_alternative=None, | ||
subtype_alternative="plain", | ||
): | ||
fileparts = None | ||
if subtype == "html": | ||
body, fileparts = self._build_email_replace_img_src(body) | ||
result = super(IrMailServer, self).build_email( | ||
email_from=email_from, | ||
email_to=email_to, | ||
|
@@ -83,48 +52,41 @@ | |
body_alternative=body_alternative, | ||
subtype_alternative=subtype_alternative, | ||
) | ||
return self._build_email_replace_img_src(result) | ||
if fileparts: | ||
for fpart in fileparts: | ||
result.attach(fpart) | ||
return result | ||
|
||
def _build_email_replace_img_src(self, email): | ||
""" Given a message, find it's img tags and if they | ||
are URLs, replace them with cids. | ||
""" | ||
for part in email.walk(): | ||
if part.get_content_type() == 'text/html': | ||
body = part.get_payload(decode=True) | ||
if not body or body == '\n': | ||
continue | ||
root = self._build_email_process_img_body( | ||
fromstring(body), email) | ||
# encodestring will put a newline every 74 char | ||
part.set_payload(encodestring(tostring(root))) | ||
return email | ||
def _build_email_replace_img_src(self, html_body): | ||
"""Replace img src with base64 encoded image.""" | ||
if not html_body: | ||
return html_body | ||
|
||
def _build_email_process_img_body(self, root, email): | ||
base_url = self.env['ir.config_parameter'].get_param( | ||
'web.base.url') | ||
for img in root.xpath( | ||
".//img[starts-with(@src, '%s/web/image')]" | ||
"| .//img[starts-with(@src, '/web/image')]" % (base_url)): | ||
image_path = img.get('src').replace(base_url, '') | ||
with self._fetch_image(image_path) as (endpoint, arguments): | ||
# now go ahead and call the endpoint and fetch the data | ||
response = endpoint.method(**arguments) | ||
if not response or response.status_code != 200: | ||
logger.warning('Could not get %s', img.get('src')) | ||
continue | ||
cid = uuid.uuid4().hex | ||
filename_rfc2047 = encode_header_param(cid) | ||
filepart = MIMEImage(response.data) | ||
# TODO check if filepart exists (do not attach twice) | ||
filepart.set_param('name', filename_rfc2047) | ||
filepart.add_header( | ||
'Content-Disposition', | ||
'inline', | ||
cid=cid, | ||
filename=filename_rfc2047, | ||
) | ||
# attach the image into the email as attachment | ||
email.attach(filepart) | ||
img.set('src', 'cid:%s' % (str(cid))) | ||
return root | ||
root = fromstring(html_body) | ||
images = root.xpath("//img") | ||
fileparts = [] | ||
for img in images: | ||
src = img.get("src") | ||
if src and not src.startswith("data:") and not src.startswith("base64:"): | ||
try: | ||
response = requests.get(src, timeout=10) | ||
_logger.debug("Fetching image from %s", src) | ||
if response.status_code == 200: | ||
cid = uuid.uuid4().hex | ||
# convert cid to rfc2047 encoding | ||
filename_encoded = "=?utf-8?b?%s?=" % b64encode( | ||
cid.encode("utf-8") | ||
).decode("utf-8") | ||
image_content = response.content | ||
filepart = MIMEImage(image_content) | ||
filepart.add_header("Content-ID", f"<{cid}>") | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ask : shall not be the cid.encode("utf-8") ? |
||
filepart.add_header( | ||
"Content-Disposition", | ||
"inline", | ||
filename=filename_encoded, | ||
) | ||
img.set("src", f"cid:{cid}") | ||
fileparts.append(filepart) | ||
except Exception as e: | ||
_logger.warning("Could not get %s: %s", img.get("src"), str(e)) | ||
return tostring(root, encoding="unicode"), fileparts |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,3 @@ | ||
* George Daramouskas <[email protected]> | ||
* Giovanni Francesco Capalbo <[email protected]> | ||
* Italo LOPES <[email protected]> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,3 @@ | ||
# -*- coding: utf-8 -*- | ||
# Copyright 2019 Therp BV <https://therp.nl> | ||
# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl.html). | ||
from . import test_mail_embed_image |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the reason the original did the fetching in a somewhat roundabout way is that what you do here allows malicious users to craft emails to themselves and have Odoo fetch arbitrary resources from the internal network. Can be harmless, can be catastrophic depending on what's accessible from there
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
May be harmless or catastrophic...
Yes, probably.
Is functionally really expected, yes too.
Thanks @imlopes for this code.