Fail to connect to short PAN ID (TZ-1021)

[kgkask]

Question

how to make end devices join the correct Zigbee network based on a manually set serial number in multiple Zigbee network environment?

Additional context.

I am using three ESP32-H2 and one ESP32-C6. I want to implement a mechanism for joining a network based on a manually set serial number on each device. I was able to do this part where I can check it from the end device and remove it from the network based on that. However, now I want to check it in the existence of multiple networks. So I used two H2 and one C6 as coordinators; one of them has the correct serial number, and the others do not. At the beginning of the communication, I join any network normally and exchange the serial number. If it matches, done. Otherwise, the end device is removed from the network, scans all available networks once using 'esp_zb_zdo_active_scan_request', saves the result in a stack, and then checks if the current short PAN ID 'esp_zb_get_pan_id' matches one of the available ones. It removes it from the stack and sets the last network PAN ID in the stack using 'esp_zb_set_pan_id'.
I am not using 'esp_zb_set_extended_pan_id' because the scan result shows all extended PAN IDs are the same.

I referred to #378 #81 and see that setting the PAN does not work, and even for me, the behavior sometimes meets my expectations where it must iterate and join all networks in the stack one by one, but this did not happen.

I am not sure if there is a shortcut for this, but here is my implementation for the callback function of the scan procedure.

`void zigbee_scan_available_nwk_callback(esp_zb_zdp_status_t zdo_status, uint8_t count, esp_zb_network_descriptor_t *nwk_descriptor) {

// get current nwk PAN id before leaving it
uint16_t current_pan_id = esp_zb_get_pan_id();

//#################### Leave nwk serial number did not match ###########
// Initialize the leave request parameter structure
esp_zb_zdo_mgmt_leave_req_param_t leave_req;
esp_zb_ieee_addr_t ieee_addr;
uint16_t short_addr = esp_zb_get_short_address(); // Example network address

// Copy the retrieved IEEE address to the leave request structure
memcpy(leave_req.device_address, ieee_addr, sizeof(esp_zb_ieee_addr_t));
leave_req.dst_nwk_addr = short_addr;
leave_req.reserved = 0;
leave_req.remove_children = 1; // Set to 1 if children should be removed
leave_req.rejoin = 0; // Set to 1 if the device should rejoin after leaving

// Call the leave request function
esp_zb_lock_acquire(portMAX_DELAY);
//esp_zb_zdo_device_leave_req(&leave_req, NULL, NULL);
esp_zb_bdb_reset_via_local_action();
esp_zb_lock_release();
//########################################################################

if (!has_scanned) {
// Perform network scan
if (zdo_status != ESP_ZB_ZDP_STATUS_SUCCESS) {
log_i("Zigbee network scan failed with status: %d\n", zdo_status);
}

  log_i("Zigbee network scan completed. Number of networks found: %d\n", count);

  for (uint8_t i = 0; i < count && network_count < MAX_NETWORKS; ++i) {
      available_networks[network_count].short_pan_id = nwk_descriptor[i].short_pan_id;
      available_networks[network_count].permit_joining = nwk_descriptor[i].permit_joining;
      memcpy(available_networks[network_count].extended_pan_id, nwk_descriptor[i].extended_pan_id, sizeof(esp_zb_ieee_addr_t));
      network_count++;

      log_i("Network %d:\n", i + 1);
      log_i("  PAN ID: 0x%04X\n", nwk_descriptor[i].short_pan_id);
      log_i("  Permit Joining: %d\n", nwk_descriptor[i].permit_joining);
      log_i("  Extended PAN ID: 0x%016llX\n", nwk_descriptor[i].extended_pan_id);
  }
has_scanned = true;

}

//check the current_pan_id
log_i("###########################################1 0x%08x", current_pan_id);

// Check if the current extended PAN ID is in the list of available networks
for (int i = 0; i < network_count; i++) {
    log_i("Checking network %d with short PAN ID 0x%08x", i, available_networks[i].short_pan_id);
    if (current_pan_id == available_networks[i].short_pan_id) {
        log_i("Match found for extended PAN ID 0x%08x", current_pan_id);
        // Remove the current network from the list
        for (int j = i; j < network_count; j++) {
            log_i("Removing network with extended PAN ID 0x%08x", available_networks[j].short_pan_id);
            available_networks[j] = available_networks[j + 1];
            log_i("Moved network with extended PAN ID 0x%08x to position %d", available_networks[j].short_pan_id, j);
        }
        network_count--;


        break;
    }
}

// Log remaining networks
for (int i = 0; i < network_count; i++) {
    log_i("0x%08x", available_networks[i].short_pan_id);
}

if (network_count > 0) {
    // Set the extended PAN ID to the last network found
    esp_zb_set_pan_id(available_networks[network_count - 1].short_pan_id);
    log_i("Set Extended PAN ID to the last network found: 0x%016X\n", available_networks[network_count - 1].short_pan_id);        
}
// Starting the Zigbee interface
//ESP_ERROR_CHECK(esp_zb_start(false));
//esp_zb_main_loop_iteration();

}`

here is scan result:

[xieqinan]

@kgkask ，

The Extended PAN ID is defined that it is the globally unique 64-bit PAN identifier of the network. This identifier should be
unique among the PAN overlapping in a given area. This identifier is used to avoid PAN ID conflicts between distinct
networks.It may not be a typical network environment in your test. The esp-zigbee-sdk supports distinguishing different networks using only the Extended PAN ID.

[kgkask]

@xieqinan
First thanks for your quick reply,

This is what I expect it, and as work around I thought that using the short PAN ID would work and it happens sometimes after multiple attempts to join the wrong network. So I want to look for a way to connect to the setted network without using "esp_zb_bdb_start_top_level_commissioning()" function since it does not follow the behavior I am looking for.

[kgkask]

@xieqinan Is there other way to check if connecting or setting the extended PAN ID is working ?

[kgkask]

Is there an update to solve this issue?

[xieqinan]

@kgkask ,

If you intend to connect to the network without using esp_zb_bdb_start_top_level_commissioning(), additional network parameters need to be set on the end device, such as the network key, TC-link key, PANID, extended PANID, and a valid short address, among others. There are no APIs available to retrieve these parameters from the coordinator, as doing so would compromise security. I recommend using the extended PAN ID to distinguish between different networks and utilizing esp_zb_bdb_start_top_level_commissioning() to join the network. Additionally, have you called esp_zb_set_long_address() or esp_zb_set_extended_pan_id() to set the same long address or extended panid on the coordinator devices? It is not advisable to use the same extended PANID in overlapping network areas.

[kgkask]

@xieqinan
Thanks for the reply

The use of esp_zb_bdb_start_top_level_commissioning() keep the ED connecting randomly to any network, and using esp_zb_set_pan_id() & esp_zb_set_extended_pan_id() used but they do not behave as I expect it, I will try with esp_zb_set_long_address() if a scan is present for that.

[xieqinan]

@kgkask ，

I think there might be a misunderstanding. The issue you're encountering, where the End Devices (EDs) randomly connect to any network, is likely caused by the use of the same Extended PAN ID across multiple networks in your test environment. This behavior occurs because the same Extended PAN ID is being used due to calling esp_zb_set_extended_pan_id() or esp_zb_set_long_address() on each coordinator, which is not compliant with Zigbee specifications.

Could you please explain why you need the same Extended PAN ID across multiple networks in your environment? To address this issue, we should use esp_zb_zdo_active_scan_request() to scan the networks and ensure that the Extended PAN ID is unique when the coordinator creates the network. If the Extended PAN ID is unique, the call to esp_zb_set_extended_pan_id() will correctly join the specified network on the End Device side.

[kgkask]

@xieqinan first thanks for the reply

As you can see when I scan the extended Pan ID is not unique at all not sure why? , so as work around I used the short Pan ID.

if there is a way to start each coordinator with different extended Pan ID it might be helpful but as you can see below:

[xieqinan]

@kgkask ,

If you have not changed the example in the SDK, the extended PAN ID will be set using the device's MAC address. To resolve this issue, I need to confirm the following questions:

• Which example of the SDK are you using as the coordinator?
• What is the version of ESP-IDF?
• Could you please share a patch of your project?

[kgkask]

The feature I want to add non, but the basic behavior light_switch example acta the same. I am using a way changed code for my project so it might me confusing, for the main differences they are stated in the issue. I am using Arduino version for this project, which is 3.0.4.

…

On Wed, Aug 7, 2024, 4:16 PM xieqinan ***@***.***> wrote: @kgkask <https://github.com/kgkask> , Which example of SDK used by you acts as the coordinator? — Reply to this email directly, view it on GitHub <#388 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BIMZNFL5BW7CH426RUO226LZQIF2DAVCNFSM6AAAAABLN76DB6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZTGMZTENZSGU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[xieqinan]

@kgkask

The feature I want to add non, but the basic behavior light_switch example act the same.

I suggest you try using the default light_switch example without making any changes for your test. I believe this will prevent the issue where all networks use the same extended PAN ID.

[kgkask]

@xieqinan

Good idea I will try it and let you know. Thanks

[kgkask]

I have tried the scan function with the Light_switch example and even for the network where the light_bulb is connected to does not appear in the result for the scan :

I will double check again, however the extended PAN ID is similar as the one used to appear when multiple networks where detected.

here is the code to scan for the networks :

void zigbee_scan_available_nwk_callback(uint8_t count, esp_zb_network_descriptor_t *nwk_descriptor) {
  
    // Log the number of networks found
    log_i("Zigbee network scan completed. Number of networks found: %d\n", count);

    // Iterate over the available networks and log their details
    for (uint8_t i = 0; i < count; ++i) {
        log_i("Network %d:\n", i + 1);
        log_i("  PAN ID: 0x%04X\n", nwk_descriptor[i].short_pan_id);
        log_i("  Permit Joining: %d\n", nwk_descriptor[i].permit_joining);
        log_i("  Extended PAN ID: 0x%016llX\n", nwk_descriptor[i].extended_pan_id);
    }
}

[kgkask]

I have tried the scan function with the Light_switch example and even for the network where the light_bulb is connected to does not appear in the result for the scan :

I will double check again, however the extended PAN ID is similar as the one used to appear when multiple networks where detected.

here is the code to scan for the networks :

void zigbee_scan_available_nwk_callback(uint8_t count, esp_zb_network_descriptor_t *nwk_descriptor) {
  
    // Log the number of networks found
    log_i("Zigbee network scan completed. Number of networks found: %d\n", count);

    // Iterate over the available networks and log their details
    for (uint8_t i = 0; i < count; ++i) {
        log_i("Network %d:\n", i + 1);
        log_i("  PAN ID: 0x%04X\n", nwk_descriptor[i].short_pan_id);
        log_i("  Permit Joining: %d\n", nwk_descriptor[i].permit_joining);
        log_i("  Extended PAN ID: 0x%016llX\n", nwk_descriptor[i].extended_pan_id);
    }
}

I have revised the function implementation and find out that I did a silly mistake, here is the result for the correct implementation:

There is 3 Coordinators around however it shows the result for 2 only not sure why?! anyhow it shows that there is an issue with the Extended PAN ID in the main code I am trying to develop. unfortunately I stopped looking into it for long now and there is a weird error I'm encountering now, I will try to fix it ASAP.

HERE IS THE CORRECT IMPLEMENTATION OF THE FUNCTION TO SCAN AVAILABLE ZIGBEE NETWORKS:
1st calling the following function for the API:
esp_zb_zdo_active_scan_request(ESP_ZB_PRIMARY_CHANNEL_MASK, 4, zigbee_scan_available_nwk_callback);
which calls back

// Global variables or constants
#define MAX_NETWORKS 10
esp_zb_network_descriptor_t available_networks[MAX_NETWORKS];

// Your implementation of the callback function
void zigbee_scan_available_nwk_callback(esp_zb_zdp_status_t zdo_status, uint8_t count, esp_zb_network_descriptor_t *nwk_descriptor) {
    if (zdo_status != ESP_ZB_ZDP_STATUS_SUCCESS) {
        ESP_LOGI("Zigbee", "Zigbee network scan failed with status: %d", zdo_status);
        return;
    }

    ESP_LOGI("Zigbee", "Zigbee network scan completed. Number of networks found: %d", count);

    for (uint8_t i = 0; i < count; ++i) {
        ESP_LOGI("Zigbee", "Network %d:", i + 1);
        ESP_LOGI("Zigbee", "  PAN ID: 0x%04X", nwk_descriptor[i].short_pan_id);
        ESP_LOGI("Zigbee", "  Permit Joining: %d", nwk_descriptor[i].permit_joining);
        ESP_LOGI("Zigbee", "  Extended PAN ID: 0x%016llX", nwk_descriptor[i].extended_pan_id);
    }
}

[kgkask]

@xieqinan

As an update for this issue, the extended_pan_id in my code was wrongly shown and then I fixed it and I can get a unique ID I need now try to connect to the desired network.

and

[xieqinan]

@kgkask ,

Good job! You can call esp_zb_set_extended_pan_id() to set the extended panid for device to join the desired network now.

[kgkask]

@xieqinan

after long debugging, I reached to what I expect to be the source of the issue which is memcpy() func in Arduino which I think it copies the extended_PAN_ID in wrong way so when I use it to connect to certain it fails and I do not know since esp_zb_set_extended_pan_id() returns void.

the below image shows the error appears from copying the PAN ID.

here is the simple light_switch example I used to reproduce the issue.
EXT_PAN_ID_issue.zip

NOTE: I could not find a simple example to try if esp_zb_set_extended_pan_id() is working or not !!!

[xieqinan]

@kgkask ,

Please ensure the esp_zb_set_extended_pan_id() is called after the esp_zb_init() and before esp_zb_start().

[kgkask]

@xieqinan

I according to my case I need it to be used within the Zigbee loop, which is after calling esp_zb_init() in order to switch between the available networks.

Anyhow, I will make sure that it works first in the same manner you mentioned and then try it in other cases.

one question, the argument passed to the function is of type "esp_zb_ieee_addr_t" which is originally an uint8_t array, so is it possible to pass it a constant PAN ID since its the same as the mac of the MCU and it wont be changed ? in order to check the format of the input which might be the cause of the error.

basically, I mean what is the format of the argument passed to the function ? ?

Much thanks

[kgkask]

@xieqinan

I called the function esp_zb_set_extended_pan_id(ext_pan_id); where uint8_t ext_pan_id[8] and I make sure that the address is one of the available network, and I run ESP_ERROR_CHECK(esp_zb_start(false)); as suggested, however it resets but still connected to the old network although esp_zb_nvram_erase_at_start(true); is not commented.

Here is the output of the code:

at the beginning the ED connected to "74:4d:bd:ff:fe:60:31:13" and the scanned all available networks and then copied the first scanned network which is "40:4c:ca:ff:fe:47:ad:c0" to ext_pan_id. and then when reboots it reconnect to the old one as mentioned above.

SO!!! basically esp_zb_set_extended_pan_id() has no effect.

here is the code if someone wanna verify
set_ext_pan_id_issue.zip

[xieqinan]

@kgkask ,

SO!!! basically esp_zb_set_extended_pan_id() has no effect.

From the logs, it appears that the available networks are being scanned after the device has already joined a network, so the esp_zb_set_extended_pan_id() function has no effect.

I have modified the example you provided and attached it here. I have not the environment to test the code, but you can search for // >> to see where the changes have been made.

set_ext_pan_id_issue.zip

[kgkask]

@kgkask ,

SO!!! basically esp_zb_set_extended_pan_id() has no effect.

From the logs, it appears that the available networks are being scanned after the device has already joined a network, so the esp_zb_set_extended_pan_id() function has no effect.

I have modified the example you provided and attached it here. I have not the environment to test the code, but you can search for // >> to see where the changes have been made.

set_ext_pan_id_issue.zip

I have checked the code Thanks for that.

the issue is I need to connect to the ZCR based on certain value that I need to fetch from the ED first then decided, not only scan for available networks.

anyway, your code answers the question of how to used esp_zb_set_extended_pan_id().
I have nothing in mind for now to work around this issue maybe I need to look for another approach.

Much thanks <3 @xieqinan

[chshu]

@kgkask feel free to reopen if any follow up questions.