confused about duplicate() and repeating transducers etc.

[pixelpusher]

Hi, I'm fuzzy on some basic things. I'm trying to traverse a grid using the grid iterators forwards then backwards in this example: https://editor.p5js.org/eraskgold/sketches/SV_7YjRnX

Forwards is not a problem but trying to add the backwards to the forwards and cycle through them infinitely is a problem. After the forwards part it gets undefined when it jumps to the next value.

Additionally, I thoguht it would be easier to draw lines between the points if I duplicated all the points in the array so I would have pairs of 2 points to use in the line() function, but the comp( duplicate (1) ... etc) on line 29 produces unexpected results. Is that because comp() applied functions from right to left wheras transduce() does it from right to left? I'm a little stuck.

Thanks!

[postspectacular]

Hey & Welcome @pixelpusher 👋 - first off, I don't have an p5js account, so here's an updated/refactored/simplified version and some more comments/explanations below:

/**
 * Demo of integrating p5js as a module and thi.ng grid iterators
 * with a forward/reverse and warp pipeline
 *
 * by Evan Raskob -- https://post.lurk.org/@BITPRINT
 *
 * Might be buggy, but hey!
 */

import { zigzagColumns2d } from 'https://cdn.skypack.dev/@thi.ng/grid-iterators';
import { comp, concat, cycle, map, mapIndexed, partition, push, reverse, transduce } from 'https://cdn.skypack.dev/@thi.ng/transducers';
import 'https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.6.0/p5.min.js';

const NB = 30; // grid points -- 45 is nice too

let grid; // 2D grid from thi.ng

window.setup = function() {
  createCanvas(600, 600);
  background(220);
  
  const BW = ceil(width / (NB+1));
  const BH = ceil(height / (NB+1));
  
  const forwardgrid = transduce(
    // compose processing pipeline of 3 steps:
    comp(
      
      // 1) modulate raw grid coordinates
      mapIndexed((i, v) => {
        let [x, y] = v;
        //x = x+0.33*cos((1 + sin(PI*i/NB))*NB*PI*i/10);
        x = x+0.33*cos(5*NB*PI*i/10);
        y = y+0.5*cos(6*PI*i/NB);
        return [x,y];
      }),
      
      // 2) project into screen space (pixel coords)
      map(([x, y]) => [(x+1)*(BW-1), (y+1)*(BH-1)]),
      
      // 3) repartition into overlapping (point) pairs
      // more generally: [1, 2, 3, 4] => [1,2], [2,3], [3,4]
      partition(2, 1)
    ),
    
    // collect results as array
    push(),
    
    // grid iterator as data source
    zigzagColumns2d({ cols: NB })
  );
  
  // grid in reverse
  const reversegrid = reverse(forwardgrid);
  // or alternatively...
  // const reversegrid = forwardgrid.slice().reverse();
  
  // create infinite repetition of the combined fwd & reverse sequence
  grid = cycle(concat(forwardgrid, reversegrid));

  frameRate(30);
};


window.draw = function() {
  try {
    // read next point pair (aka line segment) from (endless) iterator
    let [[x1,y1], [x2,y2]] = grid.next().value;
    strokeWeight(2);
    stroke(`hsla(${floor(frameCount/10) % 360}, 80%, 80%, 0.8)`);
    line(x1, y1, x2, y2);
    ellipse(x1,y1,4,4);
    ellipse(x2,y2,4,4);
  } catch (err) {
    console.error(`DRAW(next):${err.message}`);
  }
};

As for the complications and resulting confusions in your OG code:

1. The comp() function to use for composing transducers is part of the @thi.ng/transducers package (not the one from @thi.ng/compose). In your case this didn't make any difference and it's in principle the same function, but there're some other preparation steps which can make a difference in other use cases...

2. I couldn't really follow your argument re: using duplicate() for what you described as your plan (nor could I find any unexpected results with it - what are they?), so I refactored the transducer pipeline to pre-calculate the screen coordinates and then form point pairs (aka line segments) using overlapping partition() instead. This then also massively simplifies the main draw() function... Let me know if that short explanation in the comments makes sense. Here are also the docs for this function...

3. I also wasn't sure what you were trying to achieve with your use of iterate() to create the reverse order of the forward grid sequence. This function does (and expects) something very different than what you gave it and/or expected from it... So I've included 2 possible solutions too. The difference between using reverse() as iterator version vs. array.slice().reverse() is that for the latter we need to create a copy of the array first, whereas the iterator version is non-destructive and also works with non-array inputs (all just FYI)

4. Having now already pre-computed everything during setup(), the draw() function only has to pull new line segments from the iterator and draw it. I updated your color() logic to use stroke() instead (also somehow the hue part needed a floor() or else you'd only get black, not sure if that's P5 issue or canvas API...)

Anyhow, hope these comments are helpful and make the whole thing a bit less confusing! I'd still be interested about your thought patterns leading to points 2 & 3. With the latter, I guess one issue is that iterate() and iterator() are v. similar but conceptually very different... not sure what I can do about that though 🤷

[pixelpusher]

I see, that makes a lot of sense. Thanks for the example!

1. I see, thanks for clarifying -- this was my first attempt with transducers so I think I used a combo of functions from your older blog posts, didn't realise there were different versions!

2. Partition is a nice idea, works well here! But I still don't see why duplicate(1) at the start of the comp(...) doesn't have the same effect, shouldn't that duplicate every element of the array in order? In my original code, I had duplicate() at the end because I was confused about the ordering of comp() vs. transduce() since the former applies arguments from left to right and the latter from right to left. So I would expect this to work, but it fails to compile:

`const forwardgrid = transduce(
// compose processing pipeline of 3 steps:
comp(
// 0) duplicate all original points in order
duplicate(1),

  // 1) modulate raw grid coordinates
  mapIndexed((i, v) => {
    let [x, y] = v;
    //x = x+0.33*cos((1 + sin(PI*i/NB))*NB*PI*i/10);
    x = x+0.33*cos(5*NB*PI*i/10);
    y = y+0.5*cos(6*PI*i/NB);
    return [x,y];
  }),
  
  // 2) project into screen space (pixel coords)
  map(([x, y]) => [(x+1)*(BW-1), (y+1)*(BH-1)])
  
),

// collect results as array
push(),

// grid iterator as data source
zigzagColumns2d({ cols: NB })

);`

3. I see, I tried iterator originally but was having trouble locating which package it was part of and which was the correct usage (there's a lot of functionality in thi.ng to sort through!) My thought was that reverse() might reverse the original array, which is why I copied it, but it make a lot of sense that it just acts as a non-destructive iterator. Much simpler.

4. Great! Yes, likely that their parser only takes ints not floats for hue angles??

Thanks for taking the time to explain this and clean it up, very helpful!

[postspectacular]

Hi @pixelpusher - sorry for the delay... Just tried to look once more how you tried to use the duplicate() transducer in your original code, but it's already replaced with the new version. Can you maybe explain what your intention was with using that? Because I'm still not sure what result your were trying to achieve, so it's also a bit hard to analyze... were you after the kind of polyline result as in my edited version or something else?

[nkint]

I highly subscribe every word that Karsten wrote ☝️, and I wasn't sure what you want to achieve as well.
I super like Karsten's example and the simplicity with which he composes things (as usual).
As a side not, i love partition() I'm using it all the time.

In my opinion the most important is the point 3., iterate and iterator are very different things, and in this case maybe typescript would have helped you to understand it.

As a general (and very personal) note, for me the smell in your code was this line:

const reversegrid = iterate([...forwardgrid].reverse());

because when you "materialise" an iterator into an array (by [...itr] or itr.slice) you are "loosing part of its power" and you can just use reverse

Said that, this is a version of your script with less change and less refactor, hope I go what you want to achieve, but please use Karsten's version as a reference 🙂

import "https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.6.0/p5.min.js";
import {
  cycle,
  iterate,
  mapIndexed,
  push,
  transduce,
  duplicate,
  reverse,
  repeat,
  concat,
  take,
  comp,
} from "https://cdn.skypack.dev/@thi.ng/transducers";
import { zigzagColumns2d } from "https://cdn.skypack.dev/@thi.ng/grid-iterators";

const NB = 9;

let grid; // 2D grid from thi.ng
let lastPoints = [0, 0];

window.setup = function () {
  createCanvas(150, 150);
  background(220);

  const forwardgrid = transduce(
    comp(
      mapIndexed((i, v) => {
        let [x, y] = v;
        x = x + 0.33 * cos((5 * NB * PI * i) / 10);
        y = y + 0.5 * cos((6 * PI * i) / NB);
        return [x, y];
      })
    ),
    push(),
    zigzagColumns2d({ cols: NB })
  );

  const reversegrid = reverse(forwardgrid);

  grid = cycle(concat(forwardgrid, reversegrid));

  lastPoints = [width / (NB + 1), height / (NB + 1)];
  frameRate(30);
};

window.draw = function () {
  fill(220, 100);
  rect(0, 0, width, height);

  const BW = ceil(width / (NB + 1));
  const BH = ceil(height / (NB + 1));
  try {
    let b = grid.next();
    let [x, y] = [(b.value[0] + 1) * (BW - 1), (b.value[1] + 1) * (BH - 1)];
    
    stroke(10, 10, 10);
    line(lastPoints[0], lastPoints[1], x, y);

    ellipse(x, y, 4, 4);
    ellipse(lastPoints[0], lastPoints[1], 4, 4);

    lastPoints = [x, y];
  } catch (err) {
    console.error(`DRAW(next):${err.message}`);
  }
};

just my two cents.
ciao 👋🙂

[postspectacular]

Thanks, @nkint - I like your version (also with the bg fade) and your comments are spot on!

Here're a few more points to know/mention re: iterators vs. arrays:

It's good to think about iterators as generalized, lazy data sources, but we always have to consider how they're being used and be aware of their stateful nature (in most cases). Knowing about their finiteness is very important too! Here's an example which might seem counter-intuitive at first:

// generator function creating sequence 1,2,3
function* a() { yield 1; yield 2; yield 3; }

// instantiate
const b = a();

If we now try to use this like in Evan's example to create a concatenation of b and its reverse, we get this:

[...concat(b, reverse(b))]
// [1,2,3]

Even worse, if we do the same one more time:

[...concat(b, reverse(b))]
// []

The reason for this is that b is finite, meaning after it yielded 3, it's done and by the time concat is attempting to pull values from its 2nd argument (aka reverse(b), which itself is an iterator and therefore lazy), there're no more values left to give and hence that first concat only yields the above result. Likewise, trying do it one more time (with the same b) produces no values whatsoever because b already is exhausted...

This is maybe a contrived example, but hopefully shows why it's sometimes important to cache iterables (as an array or otherwise), e.g. by changing it to the following, the above issues all disappear:

// cache a's sequence into an array
b = [...a()];

// produces expected result
[...concat(b, reverse(b))]
// [ 1, 2, 3, 3, 2, 1 ]

// again...
[...concat(b, reverse(b))]
// [ 1, 2, 3, 3, 2, 1 ]

q.e.d. 😄

The flipside of this is ignoring the infinite nature of some iterators (e.g. cycle(), but also iterate(), repeat(), repeatedly(), some versions of range() etc.) - attempting to cache those into an array will simply blow up the JS runtime and these kind of iterators should always be used such that they're being limited in some other way. Again, in Evan's demo, we're only ever pulling a single value per frame from the infinite cycle(), so that's no problem, only this kind of approach will lead to epic, instant failure: [...cycle([1,2,3])] => goodbye VM!

[pixelpusher]

Ah, thanks for this, that's actually super helpful. I wrote a bunch of iterator functions for my other project, and at one point blew up the runtime because I missed this.

So to be clear, even though concat(forwardgrid, reversegrid)); is essentially concat(forwardgrid, reverse(forwardgrid)); it's ok in this case because cycle() isn't exhausting the underlying iterators? That's what I was worried about in the original version with the copy, which is more memory/resource-intensive but feels a little safer because it's harder to blow things up with it.

(This is one of my major gripes with JS... so easy to blow everything up! That's where linting/pre-checking comes in I suppose)