refactor(schema): add a new column for storing large connector transaction IDs

[kashif-m]

Type of Change

• Bugfix
• New feature
• Enhancement
• Refactoring
• Dependency updates
• Documentation
• CI/CD

Description

This PR introduces below changes around Worldpay connector

Core changes

• add new columns (TEXT) for storing the JWE based connector transaction ID
  • payment_attempt - processor_transaction_data
  • refund - connector_transaction_data and connector_refund_data
  • captures - connector_capture_data

Connector integration changes (Worldpay only)

• make line1 and city non optional in payment request
• add zip in dynamic required fields for Worldpay cards

QA

• add cypress test cases for dynamic fields for Worldpay

Additional Changes

• This PR modifies the API contract
• This PR modifies the database schema
• This PR modifies application configuration/environment variables

Motivation and Context

This PR keeps Worldpay's payments integration intact.

How did you test it?

1. Route a payment through Worldpay

cURL

curl --location --request POST 'http://localhost:8080/payments' \
    --header 'Content-Type: application/json' \
    --header 'Accept: application/json' \
    --header 'api-key: dev_Wc9lMS2KJ6l35ehmlSsR3dwwiOvGZaWLmkk3BqRLRe6vqa1rkiNJ3mVza3U1EdWh' \
    --data-raw '{"amount":100,"currency":"USD","confirm":true,"capture_method":"automatic","capture_on":"2022-09-10T10:11:12Z","customer_id":"cus_JgZZwioWal2CoKgsLXEb","email":"[email protected]","name":"John Doe","phone":"999999999","profile_id":"pro_NyQDoa1SRAC4rZ5QjDv9","phone_country_code":"+65","description":"Its my first payment request","authentication_type":"no_three_ds","return_url":"https://hyperswitch.io","setup_future_usage":"off_session","customer_acceptance":{"acceptance_type":"online","accepted_at":"1963-05-03T04:07:52.723Z","online":{"ip_address":"127.0.0.1","user_agent":"amet irure esse"}},"connector":["worldpay"],"payment_method":"card","payment_method_data":{"card":{"card_number":"4917610000000000","card_exp_month":"03","card_exp_year":"2030","card_cvc":"737","nick_name":"JD"},"billing":{"email":"[email protected]","address":{"first_name":"John","last_name":"US"}}},"billing":{"address":{"line1":"1467","line2":"Harrison Street","line3":"Harrison Street","city":"San Fransico","state":"California","zip":"94122","country":"US","first_name":"John","last_name":"US"}},"statement_descriptor_name":"joseph","statement_descriptor_suffix":"JS","metadata":{"udf1":"value1","new_customer":"true","login_date":"2019-09-10T10:11:12Z"},"browser_info":{"ip_address":"129.0.0.1","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36","accept_header":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","language":"en-US","color_depth":32,"screen_height":1117,"screen_width":1728,"time_zone":-330,"java_enabled":true,"java_script_enabled":true}}'

Response

{"payment_id":"pay_wY8cJE6MgvO13EE0vIyx","merchant_id":"merchant_1736497521","status":"succeeded","amount":100,"net_amount":100,"shipping_cost":null,"amount_capturable":0,"amount_received":100,"connector":"worldpay","client_secret":"pay_wY8cJE6MgvO13EE0vIyx_secret_42yawtMDmMbJf15nP8TP","created":"2025-01-10T09:35:51.576Z","currency":"USD","customer_id":"cus_JgZZwioWal2CoKgsLXEb","customer":{"id":"cus_JgZZwioWal2CoKgsLXEb","name":"John Doe","email":"[email protected]","phone":"999999999","phone_country_code":"+65"},"description":"Its my first payment request","refunds":null,"disputes":null,"mandate_id":null,"mandate_data":null,"setup_future_usage":"off_session","off_session":null,"capture_on":null,"capture_method":"automatic","payment_method":"card","payment_method_data":{"card":{"last4":"0000","card_type":"CREDIT","card_network":"Visa","card_issuer":"BANKPOLSKAKASAOPIEKIS.A.(BANKPEKAOSA)","card_issuing_country":"POLAND","card_isin":"491761","card_extended_bin":null,"card_exp_month":"03","card_exp_year":"2030","card_holder_name":"John US","payment_checks":null,"authentication_data":null},"billing":{"address":{"city":null,"country":null,"line1":null,"line2":null,"line3":null,"zip":null,"state":null,"first_name":"John","last_name":"US"},"phone":null,"email":"[email protected]"}},"payment_token":null,"shipping":null,"billing":{"address":{"city":"San Fransico","country":"US","line1":"1467","line2":"Harrison Street","line3":"Harrison Street","zip":"94122","state":"California","first_name":"John","last_name":"US"},"phone":null,"email":null},"order_details":null,"email":"[email protected]","name":"John Doe","phone":"999999999","return_url":"https://hyperswitch.io/","authentication_type":"no_three_ds","statement_descriptor_name":"joseph","statement_descriptor_suffix":"JS","next_action":null,"cancellation_reason":null,"error_code":null,"error_message":null,"unified_code":null,"unified_message":null,"payment_experience":null,"payment_method_type":"credit","connector_label":null,"business_country":null,"business_label":"default","business_sub_label":null,"allowed_payment_method_types":null,"ephemeral_key":{"customer_id":"cus_JgZZwioWal2CoKgsLXEb","created_at":1736501751,"expires":1736505351,"secret":"epk_d47f022a1de246bebe54d6898c408836"},"manual_retry_allowed":false,"connector_transaction_id":"eyJrIjoiazNhYjYzMiIsImxpbmtWZXJzaW9uIjoiNi4wLjAifQ==.sN:g8wd64bwkbrp0md+bPxcanBnk2zLdsIqSa1pR99GGg8fCNQpPLoWNslSzWNPFBM5Tpa8tW7EFI5onKINsgChMHeJVoeH2lrBWCRyjZYT6h+lbqfJa+1BSoKFSY8HLJDIunO0DhjaUi3niPBfv98:VUQcO2wyJ4EGywR:l4ssCpZY8cZeCeMgt1ol:GABqQG6u+T:u3sfHv3ezSOJxioRTixsThPEhzFW4ZZ6mObj3CK0rnFndcM0swvZMqgQwSEj5tBsydfzM4XKX20O6Nme96ha9twqxTSQIfr1rtl9V3q7fw3w5O9UZT4vQi9BMyCcaHkWSD:RbCWCcmiQqa5EPu9d6p6Bt0lCYl:Np4aWZ0pJCEaV4g9uGTAlKDtig0jKl6MU3ArZWtDWm7LKjrd+hQ9RR:Z491q0vLPK+etFBNqgwjiceXMDUvG7lafYHHR5Ak5lI+Sa7YaesgGkXXzgiDTOQzNK9UH+W9rzqFt6t5VcedliH8B9MKI3IKaPyVavYo4YzMmVfRID7seWtakAYMzGq+YEqwioSKkuPXbo=","frm_message":null,"metadata":{"udf1":"value1","login_date":"2019-09-10T10:11:12Z","new_customer":"true"},"connector_metadata":null,"feature_metadata":null,"reference_id":"95f4583f-a8ee-426c-acd8-f475d85b0bef","payment_link":null,"profile_id":"pro_NyQDoa1SRAC4rZ5QjDv9","surcharge_details":null,"attempt_count":1,"merchant_decision":null,"merchant_connector_id":"mca_jJbWUjL9iGOEfQo8Uw8P","incremental_authorization_allowed":null,"authorization_count":null,"incremental_authorizations":null,"external_authentication_details":null,"external_3ds_authentication_attempted":false,"expires_on":"2025-01-10T09:50:51.576Z","fingerprint":null,"browser_info":{"language":"en-US","time_zone":-330,"ip_address":"129.0.0.1","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36","color_depth":32,"java_enabled":true,"screen_width":1728,"accept_header":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","screen_height":1117,"java_script_enabled":true},"payment_method_id":"pm_xeVtmDykRQKg3pvt4EI5","payment_method_status":"active","updated":"2025-01-10T09:35:52.981Z","split_payments":null,"frm_metadata":null,"merchant_order_reference_id":null,"order_tax_amount":null,"connector_mandate_id":"https://try.access.worldpay.com/tokens/eyJrIjoxLCJkIjoiWndnc0NNQjJXNkZFTkl4TUZUZFBKdzhQdm9OS3JjYnFnT0haOThuOVE4RT0ifQ"}

2. Create a payment link without passing in details

cURL

curl --location --request POST 'http://localhost:8080/payments' \
    --header 'Content-Type: application/json' \
    --header 'Accept: application/json' \
    --header 'api-key: dev_Wc9lMS2KJ6l35ehmlSsR3dwwiOvGZaWLmkk3BqRLRe6vqa1rkiNJ3mVza3U1EdWh' \
    --data-raw '{"authentication_type":"three_ds","customer_id":"cus_JgZZwioWal2CoKgsLXEb","profile_id":"pro_NyQDoa1SRAC4rZ5QjDv9","amount":100,"currency":"USD","payment_link":true,"connector":["worldpay"],"description":"This is my description of why this payment was requested.","capture_method":"automatic","billing":{"address":{"line2":"Harrison Street","line3":"Harrison Street","state":"CA","first_name":"John","last_name":"Doe"},"phone":{"number":"8056594427","country_code":"+91"}},"email":"[email protected]","session_expiry":100000,"return_url":"https://example.com","payment_link_config":{"theme":"#0E103D","logo":"https://hyperswitch.io/favicon.ico","seller_name":"Hyperswitch Inc.","enabled_saved_payment_method":true,"hide_card_nickname_field":true,"show_card_form_by_default":true,"transaction_details":[{"key":"Policy Number","value":"938478327648","ui_configuration":{"is_key_bold":false,"is_value_bold":false,"position":2}},{"key":"Business Country","value":"Germany","ui_configuration":{"is_key_bold":false,"is_value_bold":false,"position":1}},{"key":"Product Name","value":"Life Insurance Elite","ui_configuration":{"is_key_bold":false,"is_value_bold":false,"position":3}}]}}'

• Open payment link

3. Dynamic field tests

Checklist

• I formatted the code cargo +nightly fmt --all
• I addressed lints thrown by cargo clippy
• I reviewed the submitted code
• I added unit tests for my changes where possible

[semanticdiff-com]

Review changes with  

Changed Files

File	Status
crates/hyperswitch_connectors/src/connectors/worldpay/requests.rs	59% smaller
crates/hyperswitch_connectors/src/connectors/worldpay/transformers.rs	48% smaller
crates/diesel_models/src/schema_v2.rs	26% smaller
crates/router/src/db/capture.rs	18% smaller
crates/common_enums/Cargo.toml	Unsupported file format
crates/common_utils/src/types.rs	0% smaller
crates/diesel_models/Cargo.toml	Unsupported file format
crates/diesel_models/src/capture.rs	0% smaller
crates/diesel_models/src/payment_attempt.rs	0% smaller
crates/diesel_models/src/query/capture.rs	0% smaller
crates/diesel_models/src/refund.rs	0% smaller
crates/diesel_models/src/schema.rs	0% smaller
crates/diesel_models/src/user/sample_data.rs	0% smaller
crates/hyperswitch_domain_models/src/payments/payment_attempt.rs	0% smaller
crates/router/src/configs/defaults/payment_connector_required_fields.rs	0% smaller
crates/router/src/core/payments/operations/payment_response.rs	0% smaller
crates/router/src/core/payments/transformers.rs	0% smaller
crates/router/src/core/refunds.rs	0% smaller
crates/router/src/core/webhooks/incoming.rs	0% smaller
crates/router/src/db/refund.rs	0% smaller
crates/router/src/types/storage/payment_attempt.rs	0% smaller
crates/router/src/utils/user/sample_data.rs	0% smaller
crates/storage_impl/src/payments/payment_attempt.rs	0% smaller
cypress-tests/cypress/e2e/PaymentUtils/WorldPay.js	0% smaller
migrations/2025-01-09-135057_add_processor_transaction_data/down.sql	Unsupported file format
migrations/2025-01-09-135057_add_processor_transaction_data/up.sql	Unsupported file format
v2_migrations/2024-08-28-081721_add_v2_columns/down.sql	Unsupported file format
v2_migrations/2024-08-28-081721_add_v2_columns/up.sql	Unsupported file format
v2_migrations/2025-01-13-081847_drop_v1_columns/down.sql	Unsupported file format
v2_migrations/2025-01-13-081847_drop_v1_columns/up.sql	Unsupported file format