[13.0][MIG][IMP] adyen statement import #384

NL66278 · 2021-03-30T10:38:44Z

Migration of account_bank_statement_import_adyen, but extended with possibility to import csv files, and with a module to automatically import them form the Adyen website.

This PR also contains some changes to other modules, that were made necessary by the unfortunate logic of the import modules to try the import in turns and then call super when failing. Not all modules where handling this in a proper way.

It would be better if, like the original modules in 7.0, an import was always linked to a certain format, so this dubious try..catch..pass on logic could be avoided.

StefanRijnhart · 2021-04-16T16:24:37Z

Looks great, you've solved all the technical debt from the 12.0 version and greatly enhanced the functionality! Are you suggesting we keep all the individual commits or can some of them be squashed?
Just curious: if you don't enable the clearing account mechanism for your adyen imports, what does your accounting setup looks like? In our setup, I think the amounts are transferred by batch so each statement represents one actual bank transaction, corresponding to the outstanding amount on the clearing account. In any case, please let me know what your considerations are not to use the clearing account mechanism.

NL66278 · 2021-04-20T15:59:27Z

@StefanRijnhart with the customer that needed the migration, it is not yet clear how they want to handle the downloaded statements. The clearing account is still in the migration. I just removed the dependency from the adyen import. But there is no obstacle to use the modules in combination.

I will squash the commits where logical.

StefanRijnhart · 2021-04-20T16:09:37Z

Mmm... In our case, we implemented the import module first, then when testing it turned out that the financial logic required this clearing account mechanism. So I'm not sure removing the dependency is a good idea.

NL66278 · 2021-04-21T07:32:56Z

@StefanRijnhart Zou je kunnen leven met een dringend advies in de README om ook de clearing_account module te installeren ipv een technische dependency?

StefanRijnhart · 2021-04-21T08:03:28Z

I'm curious if the Adyen module is useful without the clearing account mechanism, and if we don't know, then what is the reason that you decoupled those?

NL66278 · 2021-04-21T09:19:19Z

@StefanRijnhart I discussed it and will put back the clearing_account dependency. I will also go through all the commits once more to minimize them even more and rectify a few oversights.

lfreeke · 2021-12-13T17:56:57Z

@NL66278 I think this should be an OCA icon -> account_bank_statement_import_online_adyen/static/description/icon.png

NL66278 · 2021-12-14T06:30:49Z

@lfreeke I did not change the icon, do you expect problems with this?

lfreeke · 2021-12-15T16:56:13Z

@NL66278 It has the Ponto icon now.

NL66278 · 2021-12-16T17:28:32Z

@lfreeke I replaced the Ponto logo with the Adyen logo

NL66278 · 2022-03-02T11:55:44Z

Runboat sunk?

bash /runboat/runboat-clone-and-install.sh
rm -fr /mnt/data/initialized
rm -fr /mnt/data/odoo-addons-dir
mkdir -p /mnt/data/odoo-addons-dir
cd /mnt/data/odoo-addons-dir
curl -sSL https://github.com/oca/bank-statement-import/tarball/5885781ce65e877ce51e8bb83f095d73b517abd8
tar zxf - --strip-components=1
INSTALL_METHOD=oca_install_addons
[[ oca_install_addons == \o\c\a_\i\n\s\t\a\l\l_\a\d\d\o\n\s ]]
oca_install_addons
setuptools-odoo-get-requirements --include-addons --addons-dir /mnt/data/odoo-addons-dir
/root/.local/pipx/shared/lib/python3.8/site-packages/_distutils_hack/init.py:17: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the distutils module in sys.modules. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
warnings.warn(
/root/.local/pipx/shared/lib/python3.8/site-packages/_distutils_hack/init.py:30: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
cat test-requirements.txt
csv
odoo13-addon-multi_step_wizard
odoo13-addon-web_widget_dropdown_dynamic
odoo>=13.0a,<13.1dev
xlrd
pip install -r test-requirements.txt
Looking in indexes: https://wheelhouse.odoo-community.org/oca-simple-and-pypi
ERROR: Could not find a version that satisfies the requirement csv (from versions: none)
ERROR: No matching distribution found for csv

github-actions · 2022-07-31T12:22:41Z

There hasn't been any activity on this pull request in the past 4 months, so it has been marked as stale and it will be closed automatically if no further activity occurs in the next 30 days.
If you want this PR to never become stale, please ask a PSC member to apply the "no stale" label.

lfreeke

👍 Functional test, works well for our customer.

NL66278 · 2023-09-15T12:24:34Z

@StefanRijnhart Could you re-open, and possibly merge this one? Working very well for our customer for a long time now. And I will be migrating it to 16.0.

StefanRijnhart

Great work, thanks!

StefanRijnhart · 2023-09-25T07:14:53Z

/ocabot merge nobump

OCA-git-bot · 2023-09-25T07:14:59Z

Hey, thanks for contributing! Proceeding to merge this for you.
Prepared branch 13.0-ocabot-merge-pr-384-by-StefanRijnhart-bump-nobump, awaiting test results.

…t_clearing_account

If passed any csv file, paypal module would try to use it, even if clearly not a paypal file.

… encode bytes

Prevent other statement import modules from processing the file. Without this for instance the Enterprise account_bank_statement_import_csv module will process a retrieved cv file. In addition this is more efficient as it saves a decoding step.

csv is part of the standard library. Defining this as an external dependency crashes runboat

Put more information in the generated statement line. Also put the for users more recognizable merchant ref in the name (Label) field, and put the psp reference in ref. With a small refactoring the information retrieved can also more easily be customized.

Move parsing code to separate class; Break up large and unwieldy parsing method in separate methods.

NL66278 · 2023-09-25T08:04:48Z

@StefanRijnhart Thanks for your actions and comments. I did the rebase and the linting problems should be fixed now.

StefanRijnhart · 2023-09-25T08:22:15Z

Alright!

/ocabot merge nobump

OCA-git-bot · 2023-09-25T08:22:21Z

What a great day to merge this nice PR. Let's do it!
Prepared branch 13.0-ocabot-merge-pr-384-by-StefanRijnhart-bump-nobump, awaiting test results.

OCA-git-bot · 2023-09-25T08:29:04Z

Congratulations, your PR was merged at f8c9430. Thanks a lot for contributing to OCA. ❤️

NL66278 force-pushed the 13.0-adyen_statement_import branch from 1b28ba1 to 78cd582 Compare April 20, 2021 16:22

NL66278 force-pushed the 13.0-adyen_statement_import branch 2 times, most recently from ff35e7e to 28be6f9 Compare April 21, 2021 10:55

NL66278 force-pushed the 13.0-adyen_statement_import branch 2 times, most recently from d183517 to caafca9 Compare July 28, 2021 14:30

NL66278 force-pushed the 13.0-adyen_statement_import branch from fe4cf65 to 8342b59 Compare December 17, 2021 12:36

NL66278 force-pushed the 13.0-adyen_statement_import branch from 423c568 to 5885781 Compare March 1, 2022 13:31

NL66278 force-pushed the 13.0-adyen_statement_import branch from d4c9715 to 1ef50a7 Compare March 18, 2022 16:24

NL66278 force-pushed the 13.0-adyen_statement_import branch from e44ec2e to 943d783 Compare March 29, 2022 07:51

github-actions bot added the stale PR/Issue without recent activity, it'll be soon closed automatically. label Jul 31, 2022

github-actions bot closed this Sep 4, 2022

lfreeke reviewed Aug 17, 2023

View reviewed changes

StefanRijnhart reopened this Sep 25, 2023

StefanRijnhart approved these changes Sep 25, 2023

View reviewed changes

OCA-git-bot added the bot is merging ⏳ label Sep 25, 2023

MPishpecki and others added 21 commits September 25, 2023 09:56

[MIG] 12.0 account_bank_statement_import_adyen, account_bank_statemen…

10d9639

…t_clearing_account

[IMP] : black, isort, prettier

fc8b6db

[MIG] account_bank_statement_clearing_account Migrate to 13.0

9e2a82c

[MIG] account_bank_statement_import_adyen Migrate to 13.0

302e0d8

[ADD] account_bank_statement_import_online_adyen

f637f44

[FIX] absi- _paypal Module did not properly check input.

2552e1e

If passed any csv file, paypal module would try to use it, even if clearly not a paypal file.

[FIX] absi- _transfer_move. Correct test and duplicate class name

3afe152

[IMP] absi- _adyen. Fully support csv files and reordered columns

dee4db1

[FIX] *_online_adyen: correct icon

d179f15

[FIX] *_online_adyen: make sure downloaded file contains valid base64…

20d37ac

… encode bytes

[IMP] *_online_adyen: more debugging on retrieved statement

f780815

[IMP] *_online_adyen base64 encode retrieved file

318c671

[FIX] *adyen*: support xlsx as well as csv

cab6718

[IMP] *import_adyen: log rows read and transaction count

2edfc66

[IMP] *_import_adyen: Support statements without payout

0f54cfb

[FIX] *adyen*: Test file currency should be same as on journal

c96f2e9

[FIX] *_paypal: remove external dependency csv

71ac840

csv is part of the standard library. Defining this as an external dependency crashes runboat

[IMP] *_adyen: Check merchant account as early as possible

a3f5847

[RFR] *_import_adyen: refactor

7a4da01

Move parsing code to separate class; Break up large and unwieldy parsing method in separate methods.

NL66278 force-pushed the 13.0-adyen_statement_import branch from 943d783 to 7a4da01 Compare September 25, 2023 07:59

OCA-git-bot added the bot is merging ⏳ label Sep 25, 2023

OCA-git-bot merged commit 204a0e4 into OCA:13.0 Sep 25, 2023
6 checks passed

OCA-git-bot added merged 🎉 and removed bot is merging ⏳ labels Sep 25, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[13.0][MIG][IMP] adyen statement import #384

[13.0][MIG][IMP] adyen statement import #384

NL66278 commented Mar 30, 2021

StefanRijnhart commented Apr 16, 2021

NL66278 commented Apr 20, 2021

StefanRijnhart commented Apr 20, 2021

NL66278 commented Apr 21, 2021

StefanRijnhart commented Apr 21, 2021

NL66278 commented Apr 21, 2021

lfreeke commented Dec 13, 2021

NL66278 commented Dec 14, 2021

lfreeke commented Dec 15, 2021 •

edited

Loading

NL66278 commented Dec 16, 2021

NL66278 commented Mar 2, 2022

github-actions bot commented Jul 31, 2022

lfreeke left a comment

NL66278 commented Sep 15, 2023

StefanRijnhart left a comment

StefanRijnhart commented Sep 25, 2023

OCA-git-bot commented Sep 25, 2023

NL66278 commented Sep 25, 2023

StefanRijnhart commented Sep 25, 2023

OCA-git-bot commented Sep 25, 2023

OCA-git-bot commented Sep 25, 2023

[13.0][MIG][IMP] adyen statement import #384

[13.0][MIG][IMP] adyen statement import #384

Conversation

NL66278 commented Mar 30, 2021

StefanRijnhart commented Apr 16, 2021

NL66278 commented Apr 20, 2021

StefanRijnhart commented Apr 20, 2021

NL66278 commented Apr 21, 2021

StefanRijnhart commented Apr 21, 2021

NL66278 commented Apr 21, 2021

lfreeke commented Dec 13, 2021

NL66278 commented Dec 14, 2021

lfreeke commented Dec 15, 2021 • edited Loading

NL66278 commented Dec 16, 2021

NL66278 commented Mar 2, 2022

github-actions bot commented Jul 31, 2022

lfreeke left a comment

Choose a reason for hiding this comment

NL66278 commented Sep 15, 2023

StefanRijnhart left a comment

Choose a reason for hiding this comment

StefanRijnhart commented Sep 25, 2023

OCA-git-bot commented Sep 25, 2023

NL66278 commented Sep 25, 2023

StefanRijnhart commented Sep 25, 2023

OCA-git-bot commented Sep 25, 2023

OCA-git-bot commented Sep 25, 2023

lfreeke commented Dec 15, 2021 •

edited

Loading