Skip to content

Commit

Permalink
TP2000 1119 create measure command class (#1127)
Browse files Browse the repository at this point in the history
* Form mixin and adapt view.

* adding command class CreateMeasures; migration for same

* adding infra for new celery task

* bulk_handling celery task

* bulk_handling celery task

* TP2000-1142 Synchronous only operation errors (#1114)

* Updated NIG5 to correctly handle only goods create and update and NIG5_origin to only handle origin deletions (#1116)

Also added NIG11 skip delete

* TP2000-1160 Fix publish_to_api task envelope ID error (#1118)

* Use seed envelope ID env variable

* Remove int type conversion from seed envelope ID setting value

* Update to remove validate_taric_xml_record_order as a requirement for export / publishing of TARIC data (#1119)

* Quota origins and exclusions geo area description query optimisation (#1117)

* Optimise geo area description queries

* Add tests

* Tp2000 819 importer refactor (#1015)

* initial new parser commit

* add new parsers and process for getting info from xml to dictionary for message

* updated to using subclasses to find all parsers, then populate the object from the data provided

* updated / added new parsers for all objects

* updated parsers and linked to handlers (for dependency data, may refactor)

* added validation + working on reporting issues

* TP2000-844  Add suffix and indent columns to find commodities view (#885)

* Add suffix and indent column

* Update test

* Show indent as at today

* Update test

* Refactor

* Use optional type hint

* TP2000-738 Display rule check progress (#887)

* Display rule check progress in UI

* Add test

* Add test docstring

* TP2000-849  Add active state filter to quota search (#886)

* Add active state filter

* Add test

* Test invalid active state filter option

* Rename fixture variable

* TP2000-741 ME32 & ME16 rules checker failing (#874)

* TP2000-477 Prepopulate geo exclusions (#889)

* Pre-populate geo exclusions formset

* Add conditions initial data, update tests

* Fix tests

* Fix tests

* Add test for formset_add_or_delete

* Add erga omnes prefix

* Use regex

* Tp2000 762  virus checker (#888)

* adding docker network

* virus check validator

* skip virus check localy

* removing variable

* validator migration

* PR comments and handler update

* updating to make it obvious creds aren't real

* removing dev file upoad handler

* testing defaults

* updated local config

* TP2000-806 get envelope history (#890)

* TP2000-860 Sqlite dump and upload failing (#892)

* Skip migration during sqlite database creation.

* Run Sqlite export process during out of hours.

* updated exclusion mapping list as country has none (#895)

* TP2000-861 Fix footnote delete button on measure edit view (#894)

* Fix delete button

* Add trailing slash to auto-generated URLs

* Order indents to ensure access to latest version. (#897)

* updating link data (WIP)

* updating link data (WIP)

* updating link data (WIP)

* set parent import model (WIP)

* set parent import model (WIP)

* WIP - adding tests with example envelopes for the new importer

* wip

* wip

* initial new parser commit

* add new parsers and process for getting info from xml to dictionary for message

* updated to using subclasses to find all parsers, then populate the object from the data provided

* updated / added new parsers for all objects

* updated parsers and linked to handlers (for dependency data, may refactor)

* added validation + working on reporting issues

* updating link data (WIP)

* updating link data (WIP)

* updating link data (WIP)

* set parent import model (WIP)

* set parent import model (WIP)

* WIP - adding tests with example envelopes for the new importer

* wip

* wip

* test updates

* test updates

* test updates

* test updates

* test updates

* test updates

* test updates

* test updates

* test updates

* test updates

* test updates

* test updates

* test updates

* test updates

* building out some UI for the new importer

* building out some UI for the new importer

* building out some UI for the new importer

* building out some UI for the new importer

* adding identity fields to all parsers

* fix flaky tests

* testing updates

* testing updates

* testing updates - finished update tests for all TARIC objects we use currently

* testing updates - added add codes delete tests

* testing updates - added add codes delete tests

* testing updates - added certificate, commodity, footnote, geoarea, and measure delete tests

* moved new parser / importer to its own app

* finished adding delete tests and removed unused methods

* added documentation

* added documentation + small fixes

* added documentation + small fixes

* added documentation + small fixes

* update comm code import objects

* added documentation + small fixes

* added documentation + small fixes

* added documentation + small fixes

* minor updates based on PR comments

* minor updates based on PR comments

* minor updates based on PR comments

* minor updates based on PR comments

* minor updates based on PR comments

* minor updates based on PR comments

* test fix for py 3.8

* test fix for py 3.8

* tidy up chunker a little and add tests

* duplicated a bunch of files referenced in the importer app - anything thats not a model now also lives in the taric_parsers app.

* duplicated a bunch of files referenced in the importer app - anything that's not a model now also lives in the taric_parsers app.

* minor changes from PR review

* updated issue_type for import error creation to a text choice

* added tests to UI elements

* altered ot absolute imports

* altered exception raised on TaricXmlSourceBase

* import tidy up - absolute imports

* minor updates

* minor updates

* minor updates

* minor updates

* minor updates

* minor updates

* minor updates

* minor updates

* Updating documentation + PR feedback

* Updating documentation + PR feedback

* updates based on PR feedback

* updates based on PR feedback

* updates based on PR feedback

* updates based on PR feedback

* updates based on PR feedback

* updates based on PR feedback

* updates based on PR feedback

* updates based on PR feedback

* updates based on PR feedback

* updates based on PR feedback

* updates based on PR feedback

* updates based on PR feedback

* updates based on PR feedback

* update pre-commit-hook format, was showing as invalid yaml - no functional change - just formatting

* updates to readme, adding pre-commit info

* updates to readme, adding pre-commit info

* updates to readme, adding pre-commit info

* updates to readme, adding pre-commit info

* updates to readme, adding pre-commit info

* rename to remove New prefix

* rename to remove New prefix

* PR updates

* PR updates

* PR updates

* PR updates - adding in importerv2 to main comm code UI

* PR updates : integrating UI - mid progress

* minor UI update

* fix unit tests mid way

* interim commit

* interim commit

* interim commit

* interim commit

* fixed tests

* fixed tests

* fixed tests

* add in missing migration

* update NIG5 to not trigger when goods nomenclature also being deleted in the same transaction

---------

Co-authored-by: Dale Cannon <[email protected]>
Co-authored-by: Paul Pepper <[email protected]>
Co-authored-by: Edie Pearce <[email protected]>
Co-authored-by: Anthoni Gleeson <[email protected]>

* feat: Improvement tweaks for creating quota definition periods (#1120)

* feat: Allow table reports to be exported as a CSV (#1122)

* feat: Quotas Definitions' That Are Expiring Soon (#1121)

* Bump jinja2 from 3.1.2 to 3.1.3 (#1126)

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.2 to 3.1.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.2...3.1.3)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* empty

* empty

* empty

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Paul Pepper <[email protected]>
Co-authored-by: Paul Pepper <[email protected]>
Co-authored-by: Doug Mills <[email protected]>
Co-authored-by: Dale Cannon <[email protected]>
Co-authored-by: Edie Pearce <[email protected]>
Co-authored-by: Anthoni Gleeson <[email protected]>
Co-authored-by: Tash Boyse <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
  • Loading branch information
9 people authored Jan 12, 2024
1 parent 194fafe commit afb2280
Show file tree
Hide file tree
Showing 3 changed files with 76 additions and 2 deletions.
28 changes: 28 additions & 0 deletions measures/migrations/0016_createmeasures.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Generated by Django 3.2.23 on 2023-12-13 15:56

from django.db import migrations
from django.db import models


class Migration(migrations.Migration):
dependencies = [
("measures", "0015_alter_measure_dead_order_number"),
]

operations = [
migrations.CreateModel(
name="CreateMeasures",
fields=[
(
"id",
models.AutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
("cleaned_data", models.JSONField()),
],
),
]
18 changes: 18 additions & 0 deletions measures/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -967,3 +967,21 @@ class FootnoteAssociationMeasure(TrackedModel):
business_rules.ME73,
UpdateValidity,
)


class CreateMeasures(models.Model):
cleaned_data = models.JSONField()
# Each sub-form in the CreateMeasuresWizard should have a function called
# create_from_serialized_cleaned_data()
# Loop through these sub-forms and call this function, passing in self.cleaned_data.

def create_measures(self):
pass
# NOTE: import must be here to avoid circular import errors
# from measures import forms

# for form in [
# forms.MeasureDetailsForm,
# ]:
# form = form.create_from_serialized_cleaned_data(self.cleaned_data)
# TODO: ensure each form returns correct form object
32 changes: 30 additions & 2 deletions measures/views.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
from django.http import HttpResponseRedirect
from django.http import JsonResponse
from django.shortcuts import redirect
from django.shortcuts import render
from django.utils.decorators import method_decorator
from django.views import View
from django.views.generic.base import TemplateView
Expand Down Expand Up @@ -936,7 +935,36 @@ def create_measures(self, data):

def done(self, form_list, **kwargs):
bulk_create_edit.apply_async()
# call new celery task here, passing in id of CreateMeasures object

def get_all_serialized_cleaned_data(self):
"""
Returns a merged dictionary of all step cleaned_data.
If a step contains
a `FormSet`, the key will be prefixed with 'formset-' and contain a list
of the formset cleaned_data dictionaries, as expected in
`create_measures()`.
Note: This patched version of `super().get_all_cleaned_data()` takes advantage of retrieving previously-saved
cleaned_data by summary page to avoid revalidating forms unnecessarily.
"""

# TODO: not currently used, but will be used once the form-based
# serialisation is in place.

all_cleaned_data = {}
# for form_key in self.get_form_list():
# cleaned_data = self.get_cleaned_data_for_step(form_key)
for form_key in [self.MEASURE_DETAILS]:
cleaned_data = self.get_serialized_cleaned_data_for_step(form_key)
if isinstance(cleaned_data, (tuple, list)):
all_cleaned_data.update(
{
f"formset-{form_key}": cleaned_data,
},
)
else:
all_cleaned_data.update(cleaned_data)
return all_cleaned_data

def get_all_cleaned_data(self):
"""
Expand Down

0 comments on commit afb2280

Please sign in to comment.