Standardized NIP-47 metadata for payer/payee in list_transactions?

[danieldaquino]

We are implementing a wallet view with a list of transactions from NWC wallets, by using this standard call: https://github.com/nostr-protocol/nips/blob/master/47.md#list_transactions

The list_transactions request returns a response in this format:

{
    "result_type": "list_transactions",
    "result": {
        "transactions": [
            {
               "type": "incoming", // "incoming" for invoices, "outgoing" for payments
               "invoice": "string", // encoded invoice, optional
               "description": "string", // invoice's description, optional
               "description_hash": "string", // invoice's description hash, optional
               "preimage": "string", // payment's preimage, optional if unpaid
               "payment_hash": "string", // Payment hash for the payment
               "amount": 123, // value in msats
               "fees_paid": 123, // value in msats
               "created_at": unixtimestamp, // invoice/payment creation time
               "expires_at": unixtimestamp, // invoice expiration time, optional if not applicable
               "settled_at": unixtimestamp, // invoice/payment settlement time, optional if unpaid
               "metadata": {} // generic metadata that can be used to add things like zap/boostagram details for a payer name/comment/etc.
           }
        ],
    },
}

However, I noticed that there does not seem to be a standardized way to communicate information about the sender or receiver of the payment. For example: if it's a zap, it does not indicate the person who was zapped (for outgoing zaps) or the person who sent the zap (for incoming zaps).

The "metadata" field indicates it might hold this information, but it does not specify the dictionary keys that would be used for such purposes.

Is there currently any standardized way to get that information? Should one be specified in NIP-47?

[danieldaquino]

CC @rolznz @asoltys, would you happen to have any insights around this topic? Thank you very much in advance!

[asoltys]

Hi @danieldaquino yeah I wasn't sure what to put in there exactly either so just left it empty for now. I think you can get zap details for incoming payments from the description field. We're not storing the description for outgoing payments but now I wonder if maybe we should be or else yeah I'm happy to fill in the metadata field with something too. I'll wait to hear from @rolznz on that.

If it happens to be a zap between coinos users you could get the bolt11 of the outgoing payment and then call lookup_invoice to get the description which would have the zap metadata but that won't work for external invoices so we probably need another solution.

[rolznz]

@danieldaquino @asoltys we haven't proposed this yet, but in Alby Hub we support setting a metadata field for make_invoice and pay_invoice which will be stored locally by Alby Hub. This currently gets set when a zap or LNURL payment is made to an Alby lightning address.

Currently we have designed it to support a few open protocols like LNURL (LUD-12 comments and LUD-18 payer data) and NIP-57 (zaps). What do you think?

  metadata?: {
    comment?: string; // LUD-12
    payer_data?: {
      email?: string;
      name?: string;
      pubkey?: string;
    }; // LUD-18
    nostr?: {
      pubkey: string;
      tags: string[][];
      // ... other zap fields
    }; // NIP-57
  } & Record<string, unknown>;

for outgoing zaps then you would need to set the metadata field inside the pay_invoice payload e.g.

{
    "method": "pay_invoice",
    "params": {
        "invoice": "lnbc50n1...", // bolt11 invoice from the zap
        "metadata": {"nostr": {...nostr zap here...}}
    }
}

[jb55]

the way zaps are designed is that the zap request is put as the description hash. on CLN you can recover the description from invoices generated this way, which is what I use in my CLN zapper. I would have assumed other nodes also hold onto the zap request and would include it in the description field, but it seems not?

[danieldaquino]

 metadata?: {
   comment?: string; // LUD-12
   payer_data?: {
     email?: string;
     name?: string;
     pubkey?: string;
   }; // LUD-18
   nostr?: {
     pubkey: string;
     tags: string[][];
     // ... other zap fields
   }; // NIP-57
 } & Record<string, unknown>;

@rolznz I like this! It seems like the general approach taken here would be to allocate some top-level metadata keys (e.g. comment, payer_data, nostr) to specific standards (e.g. LUD-12, LUD-18, NIP-57) — correct? If that's the case, I like the approach, and would be happy with this format!

[danieldaquino]

the way zaps are designed is that the zap request is put as the description hash. on CLN you can recover the description from invoices generated this way, which is what I use in my CLN zapper. I would have assumed other nodes also hold onto the zap request and would include it in the description field, but it seems not?

@jb55, are you referring to the description tag in a zap receipt event? I am not sure I understand how the sender/receiver information would be retrieved from an NWC transaction entry in this case.

[jb55]

no I am not... I am referring to the description of the zap lightning invoice. lightning invoices have either description or description_hash. zaps use description_hash (technically it also allows description invoices if it fits). When creating a zap invoice the zap request is stored in the description(hash). When creating deschash invoices with CLN the original zap request (description) is still retained on the lightning node side. Zappers then include this in the description tag of the zap.

If I was building a NWC implementation I would just include this zap request in the description response on the NWC transaction, but it doesn't look like coinos is doing this. not sure about other implementations

it's also confusing to me why we would need a metadata field in this case

[danieldaquino]

no I am not... I am referring to the description of the zap lightning invoice. lightning invoices have either description or description_hash. zaps use description_hash (technically it also allows description invoices if it fits). When creating a zap invoice the zap request is stored in the description(hash). When creating deschash invoices with CLN the original zap request (description) is still retained on the lightning node side. Zappers then include this in the description tag of the zap.

If I was building a NWC implementation I would just include this zap request in the description response on the NWC transaction, but it doesn't look like coinos is doing this. not sure about other implementations

Oh, I see what you mean, @jb55. So in the NIP-47's list_transactions response, the "description" and "description_hash" fields are not independent fields for a separate purpose — but instead those are the decoded pieces of information from the BOLT-11 lightning invoice itself — which according to NIP-57, correspond to the kind:9734 zap request Nostr event, and its SHA-256 hash, respectively (in the case of a Zap transaction).

My concern with this approach would then be that — as you mentioned — the full kind:9734 zap request JSON data does not always fit inside the description field in a BOLT-11-encoded invoice, and we need the full zap request event to retrieve sender/receiver information.

In those cases where it does not fit, only a description_hash is present in the BOLT-11 invoice, which would not be enough information to reconstruct the full kind:9734 event — necessary to find the sender/recipient. Further, such description hash is not a queryable piece of information in the Nostr network as far as I have tried (I could show a real example if needed to illustrate this point).

To make this work for NWC clients, it seems that it would be necessary to require the inclusion of the kind:9734 event in the "description" field of the NWC transaction even if the BOLT-11-encoded invoice does not contain such description. Is this what you had in mind, @jb55? @rolznz, @asoltys, from an NWC provider standpoint, is this feasible?

it's also confusing to me why we would need a metadata field in this case

I believe it's partially due to some gaps I had in my understanding of the NIP-47 (NWC) and NIP-57 (Zap) specs until now, and some inconsistencies I found with real data during troubleshooting.

From @rolznz's comment, it might be that non-zap transactions can also contain useful receiver/sender data, presumably related to the LUD-12 and LUD-18 specs, but I am not familiar with those at all to know if such information could also be obtained directly from the BOLT-11 invoice itself.

I suppose the metadata field would be needed to supplement whatever data is missing from the other fields.

[danieldaquino]

To make this work for NWC clients, it seems that it would be necessary to require the inclusion of the kind:9734 event in the "description" field of the NWC transaction even if the BOLT-11-encoded invoice does not contain such description.

Actually, I noticed this already seems to be the case with Coinos — at least in some of the cases.

@rolznz, can you please confirm if this is already the case for Alby Hub as well?

We're not storing the description for outgoing payments but now I wonder if maybe we should be or else yeah I'm happy to fill in the metadata field with something too.

Sorry, @asoltys, I should have paid more attention to this before — this explains the data I have seen during my testing. I had mostly outgoing payments in my test wallet (thus, empty descriptions), which led me to research into the specification.

Would there be any issues/limitations in adding kind:9734 events to the description field in outgoing payments as well?

Thank you all for the valuable information and apologize for my confusion!

[rolznz]

@danieldaquino no, we do not return the zap request in the description field (I think only CLN by default supports the behavior @jb55 mentioned above).

We cannot rely on the zap request fitting in the description field of the invoice itself. Therefore the payer has to store the zap request alongside the payment. Currently in the NIP-47 spec pay_invoice does not have a way to save arbitrary data. I think adding a metadata field here allows for many use-cases for attaching different types of metadata to outgoing payments.

[danieldaquino]

@danieldaquino no, we do not return the zap request in the description field (I think only CLN by default supports the behavior @jb55 mentioned above).

We cannot rely on the zap request fitting in the description field of the invoice itself. Therefore the payer has to store the zap request alongside the payment. Currently in the NIP-47 spec pay_invoice does not have a way to save arbitrary data. I think adding a metadata field here allows for many use-cases for attaching different types of metadata to outgoing payments.

That's a good point @rolznz, if a client sent a zap using an NWC wallet — and assuming the zap request does not fit in the lightning invoice, there is currently no reliable way that it would be able to find the zap request.

sequenceDiagram
    participant NWC Provider
    participant Client
    participant Recipient LNURL server
    participant Relays listed on zap request
    Client->>Recipient LNURL server: Verify support and recipient info
    Recipient LNURL server->>Client: Support and recipient info
    Client->>Recipient LNURL server: NIP-57 `kind:9734` zap request
    Recipient LNURL server->>Client: Lightning Invoice (description hash only)
    Client->>NWC Provider: NIP-47 `pay_invoice` command (Invoice with description hash only)
    NWC Provider->>Recipient LNURL server: Pay invoice
    Recipient LNURL server->>Relays listed on zap request: NIP-57 `kind:9735` zap receipt (w/ embedded zap request)
    Relays listed on zap request->>Client: NIP-57 `kind:9735` zap receipt (w/ embedded zap request)

Loading

In that case, it would make sense to add that data to the pay_invoice command as you mentioned, via either:

• a metadata field, or
• the "description" + "description_hash" field — with the same sort of requirement to add the zap request even if it does not fit in the lightning invoice itself.

I am now leaning towards using the metadata field as the primary source of truth for zap request info. The "description" field is defined as the description in the invoice, and fitting a zap request in there regardless of whether it is in the actual BOLT-11 payload feels a bit hacky, it breaks some the interface's definitions to fit the missing metadata. The metadata field seems to be more consistent interface.

@jb55, thoughts?

[danieldaquino]

@rolznz, side question, can you please confirm whether Alby Hub already supports this metadata field on the format you specified on pay_invoice and list_invoice?

If so, I could implement support for that, and fallback to reading the "description" field when the metadata field is not present (for maximum compatibility).

[rolznz]

@danieldaquino yes, we support setting metadata on pay_invoice and make_invoice, and also returning metadata in all methods that return transaction data (make_invoice, lookup_invoice, list_transactions) and notifications.

[jb55]

I am now leaning towards using the metadata field as the primary source of truth for zap request info. The "description" field is defined as the description in the invoice, and fitting a zap request in there regardless of whether it is in the actual BOLT-11 payload feels a bit hacky, it breaks some the interface's definitions to fit the missing metadata. The metadata field seems to be more consistent interface.

I think its fine but if they have the data to put it in the metadata then I don't understand why they wouldn't just put it in the description since the same data is the description_hash preimage... but doesn't bother me either way.