Multi Query

[vladigg-ruse]

Hi friends, I need help with new library.
How to make Async stream connection with ESP8266 to RTDB, when node is changed, get value from another node and push unique name with value = server timestamp .
I make 2x aClient (aClient, aClient2): first for stream, and second for get. But when use aClient2 to push data, ESP crash. If I make aClient3 like aClient2 and use it for push, give me error TCP connecton.

What is correct way to do this 3 jobs?

Regards!

[mobizt]

In case of crash, library does not cause the crash based on your information.

You can call push, set, get, patch functions as you want without problems because there is no chance of crash as all tasks will be stored in FIFO queue.

The possible crash is because dangling pointer problem and you should read the Readme carefully for using this library correctly otherwise you should use sync functions instead.

In case of using many async clients, you have to check your device free heap to avoid out of memory situation as the WiFIClientSecure cannot work in case of too low memory (<7k).

[mobizt]

You also have to read the Readme for choosing the proper memory option for ESP8266.

[vladigg-ruse]

How many clients should I use in my case? One for stream, second for get, set, push, etc. or many for each operation. I've read documentation and examples but it's not very clear when you have several different requests how to execute them

[mobizt]

It is reasonable for you to understand like that If you don't have much experienced in ESP8266 with WiFiClientSecure and HTTPClient library programming.

As you 've seen from documentation, it wrote that about 1k of memory consumed for each async/sync task stored in a async client's queue that is quite low amount of memory.

Your device may lose only 3k for running 3 async/sync tasks at a time which this excludes the server response payload that can be large or small memory to be additionally consumed.

And when it comes into the real work that external SSL Client (WiFiClientSecure) was used as a client for async client, the usage memory is mostly dependent of the SSL Client usage as it required for large SSL data handleing.

If you look at all examples, you will see these lines of codes.

FirebaseClient/examples/RealtimeDatabase/Async/Get/Get.ino

Lines 96 to 101 in 3bfc8ff

	#if defined(ESP32) || defined(ESP8266) || defined(PICO_RP2040)
	ssl_client.setInsecure();
	#if defined(ESP8266)
	ssl_client.setBufferSizes(4096, 1024);
	#endif
	#endif

Which one is the ESP8266 WiFIClientSecure memory buffer allocation setting which takes into account for memory usage which is adjustable.

In ESP32, WiFIClientSecure consumes up to 50k with non-adjustable when it connected to server.

Finally, the server response that need to be store in the async result for further processing by user takes some amount of RAM.

In Raspberry Pi Pico which memory approx. 200k is available, few kb used by WiFIClientSecure, the tenth of async client can be possible.

In ESP32, 50k used by WiFIClientSecure, the only three or four asyn client can be possible.
 
There is not possible for any library that can concisely calculation the amount of memory usage in the https client/server application.

Then you have to do your own experiment for measuring the limitation based on your device and application.

[mobizt]

when you have several different requests how to execute them

It's a memory availability dependent.

In case of using many async clients, you have to check your device free heap to avoid out of memory situation as the WiFIClientSecure cannot work in case of too low memory (<7k).

[vladigg-ruse]

Make new project with include onli FirebaseClient and WiFiManager last version. Here is the code.
Everything work normally until "Database.push<object_t>(aClient2, "/log/open", ts_json, asyncCB);" .

When I change "/last" node, stream print to console "type, data, event and path", after that Get "/garage/relay/relayType" and print to console, if relayType is 1, it's ok, but if 2 - ESP crash when try to push to "/log/open".

Where I'm wrong? Here is my code:

#include <WiFiManager.h>
#include <FirebaseClient.h>


#define API_KEY "####################"
#define USER_EMAIL "##@##.com"
#define USER_PASSWORD "######"
#define DATABASE_URL "https://#######################"

void asyncCB(AsyncResult &aResult);

void printResult(AsyncResult &aResult);

DefaultNetwork network; // initilize with boolean parameter to enable/disable network reconnection

UserAuth user_auth(API_KEY, USER_EMAIL, USER_PASSWORD);

FirebaseApp app;

#if defined(ESP32) || defined(ESP8266) || defined(PICO_RP2040)
#include <WiFiClientSecure.h>
WiFiClientSecure ssl_client;
#elif defined(ARDUINO_ARCH_SAMD)
#include <WiFiSSLClient.h>
WiFiSSLClient ssl_client;
#endif

// In case the keyword AsyncClient using in this example was ambigous and used by other library, you can change
// it with other name with keyword "using" or use the class name AsyncClientClass directly.

using AsyncClient = AsyncClientClass;

AsyncClient aClient(ssl_client, getNetwork(network));

#if defined(ESP32) || defined(ESP8266) || defined(PICO_RP2040)
WiFiClientSecure ssl_client2;
#elif __has_include(<WiFiSSLClient.h>)
WiFiSSLClient ssl_client2;
#endif

AsyncClient aClient2(ssl_client2, getNetwork(network));

RealtimeDatabase Database;

AsyncResult aResult_no_callback;

unsigned long ms = 0;

void setup()
{

    Serial.begin(115200);
    WiFiManager wm;
    bool res;
    res = wm.autoConnect("AutoConnectAP","password");
    if(!res) {
        Serial.println("Failed to connect");
        // ESP.restart();
    } 
    else {
        //if you get here you have connected to the WiFi    
        Serial.println("connected...yeey :)");
    }
    // WiFi.begin(WIFI_SSID, WIFI_PASSWORD);

    Serial.print("Connecting to Wi-Fi");
    unsigned long ms = millis();
    while (WiFi.status() != WL_CONNECTED)
    {
        Serial.print(".");
        delay(300);
    }
    Serial.println();
    Serial.print("Connected with IP: ");
    Serial.println(WiFi.localIP());
    Serial.println();

    Firebase.printf("Firebase Client v%s\n", FIREBASE_CLIENT_VERSION);

    Serial.println("Initializing app...");

#if defined(ESP32) || defined(ESP8266) || defined(PICO_RP2040)
    ssl_client.setInsecure();
    ssl_client2.setInsecure();
#if defined(ESP8266)
    ssl_client.setBufferSizes(4096, 1024);
    ssl_client2.setBufferSizes(4096, 1024);
#endif
#endif

    app.setCallback(asyncCB);

    initializeApp(aClient2, app, getAuth(user_auth));

    // Waits for app to be authenticated.
    // For asynchronous operation, this blocking wait can be ignored by calling app.loop() in loop().
    ms = millis();
    while (app.isInitialized() && !app.ready() && millis() - ms < 120 * 1000);

    app.getApp<RealtimeDatabase>(Database);

    Database.url(DATABASE_URL);

    Database.get(aClient, "/garage", asyncCB, true /* SSE mode */);
}

void loop()
{
  
    app.loop();

    Database.loop();
}
void asyncCB(AsyncResult &aResult)
{

    printResult(aResult);
}

void printResult(AsyncResult &aResult)
{
    if (aResult.appEvent().code() > 0)
    {
        Firebase.printf("Event msg: %s, code: %d\n", aResult.appEvent().message().c_str(), aResult.appEvent().code());
    }

    if (aResult.isDebug())
    {
        Firebase.printf("Debug msg: %s\n", aResult.debug().c_str());
    }

    if (aResult.isError())
    {
        Firebase.printf("Error msg: %s, code: %d\n", aResult.error().message().c_str(), aResult.error().code());
    }

    if (aResult.available())
    {
        
        RealtimeDatabaseResult &RTDB = aResult.to<RealtimeDatabaseResult>();
        if (RTDB.isStream())
        {
            Firebase.printf("event: %s\n", RTDB.event().c_str());
            Firebase.printf("path: %s\n", RTDB.dataPath().c_str());
            Firebase.printf("data: %s\n", RTDB.to<const char *>());
            Firebase.printf("type: %d\n", RTDB.type());
            if (strcmp(RTDB.dataPath().c_str(), "/last") == 0) {
              Database.get(aClient2, "/garage/relay/relayType", asyncCB);
            }
        }
        else
        {
            Firebase.printf("payload: %s\n", aResult.c_str());
            int v2 = RTDB.to<int>();
            if (v2 == 1) {
                Serial.println(v2);
                 
            } else if (v2 == 2) {
                Serial.println(v2);
                object_t ts_json;
                JsonWriter writer;
                writer.create(ts_json, ".sv", "timestamp"); /
                Database.push<object_t>(aClient2, "/log/open", ts_json, asyncCB);
            }
        }
    }
}

[mobizt]

Your device stack memory is not enough for running any memory consuming code inside the callback and ISR.

By default, esp8266 has only 2k stack memory.

This is the critical and most important when you program the microcontroller which has limited memory especially area that reserved for stack.

You have to remove all of this from your printResult because the printResult was called by async callback.

Database.get(aClient2, "/garage/relay/relayType", asyncCB);
Database.push<object_t>(aClient2, "/log/open", ts_json, asyncCB);

You can run your code in printResult safely if it was called from the static function i.e. setup or loop.

To avoid using callback in such low memory device, library provides the option to use AsyncResult (aResult_no_callback) instead of AsyncCallback which your code in printResult now can run normally without stack issue because it was called from loop (and it is now use heap).

FirebaseClient/examples/RealtimeDatabase/Async/Get/Get.ino

Line 72 in 3bfc8ff

AsyncResult aResult_no_callback;

You need to learn the basic esp8266 programming to improve your knowledge.

[vladigg-ruse]

Thank you vary match, I make this changes:
in setup function:

Database.get(aClient, "/garage", aResult_no_callback, true /* SSE mode */);

in loop function added this line:

printResult(aResult_no_callback);

in printResult:

RealtimeDatabaseResult &RTDB = aResult.to<RealtimeDatabaseResult>();
        if (RTDB.isStream())
        {
            Firebase.printf("event: %s\n", RTDB.event().c_str());
            Firebase.printf("path: %s\n", RTDB.dataPath().c_str());
            Firebase.printf("data: %s\n", RTDB.to<const char *>());
            Firebase.printf("type: %d\n", RTDB.type());

            // The stream event from RealtimeDatabaseResult can be converted to the the values as following.
            bool v1 = RTDB.to<bool>();
            int v2 = RTDB.to<int>();
            float v3 = RTDB.to<float>();
            double v4 = RTDB.to<double>();
            String v5 = RTDB.to<String>();
            if (strcmp(RTDB.dataPath().c_str(), "/last") == 0) {
              Database.get(aClient2, "/garage/relay/relayType", aResult_no_callback);
            }
        }
        else
        {
            Firebase.printf("payload: %s\n", aResult.c_str());
            int v2 = RTDB.to<int>();
            if (v2 == 1) {
                Serial.println(v2);
                 
            } else if (v2 == 2) {
                Serial.println(v2);
                object_t ts_json;
                JsonWriter writer;
                writer.create(ts_json, ".sv", "timestamp"); // -> {".sv": "timestamp"}
                Database.push<object_t>(aClient2, "/log/open", ts_json, aResult_no_callback);
            }
        }

and now everything work without callback. If I want to use callback for stream, how to use streamed data outside callback function. I know for globally variable, but I want when node is changed, turn on or off relay based on streamed value in node?

Regards!

[mobizt]

The stream changes event can be stored in the global defined bool var that can be checked in the loop.

For the data to use outside the callback, you can copy the async result in the callback to another async result that defined globally using = operator or just using global defined String to copy only payload.

[vladigg-ruse]

Thank you very very much for your library and for support!