[BUG] mapping values in gun.user does not work properly

[gustavcorpas]

TLDR;

You cannot reliably map data on an authenticated gun.user in the same way you can when using just gun.

Why is this bad. A practical example

Say you have some users. These users have groups. The groups have certain attributes.
You may want to map through the users groups, and do stuff with them. Currently this is not possible to do reliably.

What is going on?

When you do gun.get("groups").map().once(cb) this will iterate all the groups.
All of the group's attributes will be available in the callback.

When you do user.get("groups").map().once(cb) this will iterate the groups, if it is in the mood to do so.
Sometimes all the group's attributes will be available, sometimes only a subset of them.

Some numbers and variations

In the code example below we iterate the groups, and await the attributes. If we can await all of them we count that as a success.
I tested a few variations of the code, here are the results:

Testing on gun yields expected results:

This table shows expected results for groups with two and three attributes

number of attributes	iterations	iterations with all attributes present
2	1	1
2	2	2
2	3	3
2	4	4
2	5	5
2	100	100
3	1	1
3	2	2
3	3	3
3	4	4
3	5	5
3	100	100

Testing on gun.user yields unexpected results!

This table shows seemingly random behaviour.
Sometimes some values may be present. See images attached below.

number of attributes	iterations	iterations with all attributes present
2	1	1
2	2	2
2	3	3
2	4	4
2	5	5
2	20	11
2	100	(4, 11 or 12, ...)
3	1	1
3	2	2
3	3	3
3	4	2
3	5	2
3	20	2
3	100	2

A code example / test

        //
        // TEST SETUP
        // 
        const iterations = 10; 

         // test on gun, test function returns number of correctly mapped items.
        console.log("RUNNING GUN TESTS");
        console.log(await test_consecutive(gun, iterations));

        // test on authenticated gun.user(). 
        console.log("RUNNING USER TESTS");
        user.create("alice", "alicepass", () => {
            user.auth("alice", "alicepass", async () => {
                console.log(await test_consecutive(user, iterations));
            });
        });


        //
        // TEST FUNCTION
        // 
        async function test_consecutive(gun_test, iterations) {

            let num = 0;

            gun_test.get("groups").map().once(async function() {
                console.log(await this);
                await this.get("one");
                await this.get("two");
                await this.get("three");
                num += 1;
            })

            for(let i = 0; i < iterations; i++){

                const group_pair = await SEA.pair();
                const user_group = gun_test.get("groups").get(group_pair.pub);

                user_group.get("one").put(Date.now());
                user_group.get("two").put(Date.now());
                user_group.get("three").put(Date.now()); // removing this line will make gun.user() be slightly less bad.

            }

            await new Promise(resolve => setTimeout(() => {resolve(true)}, 2000));

            return num;

        }

Images

The consistent behaviour of gun.map looks like this:

The inconsistent behaviour of gun.user() looks like this:

[gustavcorpas]

It seems that using await of any kind will cause this error. Again, only when using gun.user -> normal gun does not care.
Without the await in the following example, gun.user will behave as expected.

Example

            for(let i = 0; i < iterations; i++){

                await delay(0) // await for zero seconds, will cause issues as the operation is async
                const group_pair = crypto.randomUUID();
                const user_group = gun_test.get("groups").get(group_pair);

                user_group.get("one").put(Date.now());
                user_group.get("two").put(Date.now());
                user_group.get("three").put(Date.now()); // removing this line will make gun.user() be slightly less bad.

            }

[bmatusiak]

your above code in the last comment has a problem

const group_pair = crypto.randomUUID();                
console.log(typeof group_pair.pub)

[gustavcorpas]

your above code in the last comment has a problem

const group_pair = crypto.randomUUID();                
console.log(typeof group_pair.pub)

You're right. I corrected it. The bigger issue still stands thogh.

[bmatusiak]

so.. im not 100% sure, but i think Map/Set are BEST used together and put/on(ce) are BEST used together.. map() use to list out keys ... to map out more data from the graph

https://gun.eco/docs/Frozen#update-immutable-links-to-mutable-user-content

so im asking.. can you check and see if Set and Map fail too?

[gustavcorpas]

From my understanding map just maps all the values in a node. set is just a wrapper for put, that will give each item a unique key. What I think is odd in this case, is that normal gun maps everything correctly, but gun.user has problems mapping stuff.

Looking at your link, if I understand correctly, the idea is that we put a reference to the user private node a in the normal public gun. Then use gun.user to put our stuff, but gun to map over the stuff. This is cool! <3

Im not sure that this adresses this issue completely. I will have to try, but doing

gun.get(`~${user._.sea.pub}`).map();

will still not work. I think the issue is with the way that gun.put works when authenticated.

[bmatusiak]

ok cool... there is like small handful of places where this problem could be. because SEA adds in stuff to check/validate user profiles and hashed content. and that code starts here , https://github.com/amark/gun/blob/master/sea.js#L1321

[gustavcorpas]

Cool, I will take a look at that. In the mean time for anyone else stumbling upon this, what I am doing currently is having all functions that make use of .put be synchronous. If they depend on awaited variables the functions can still await those internally kinda like so:

            user.get("root").get("foo").map().once(res => console.log(res)); // will log 100 times

            for(let i = 0; i < 100; i++){
                put_something(); // function is synchronous
            }

            function put_something() {
                const prepare = (async () => { return {id: (await SEA.pair()).pub } })();
                prepare.then(({id}) => { user.get("root").get("foo").get(id).put("hi!") });
            }

[amark]

@gustavcorpas this is helpful for helping hunt/track down the bug, thank you. Anyone willing to do a screen call to debug with me?

[gustavcorpas]

@amark - I've seen you talk about an "intro to gun internals" a few years? ago. Did this ever happen? I would help, but I find it really difficult to find my way though the code.

[bitnom]

@gustavcorpas this is helpful for helping hunt/track down the bug, thank you. Anyone willing to do a screen call to debug with me?

@amark - I've seen you talk about an "intro to gun internals" a few years? ago. Did this ever happen? I would help, but I find it really difficult to find my way though the code.

This is a good point. I've been a fan of this project for many years. The codebase is difficult to approach though, so I never contributed directly.