Autopilot — am I doing it right?

[xanoni]

Context: I create more outbound channels than people create channels to me — guess my node is just not that popular :-( — so I try to create some inbound liquidity on those channels by borrowing from channels that have inbound liquidity. I have the below running on my machine in an endless loop. Is this how rebalance-lnd is supposed to be used, or is there an easier way?

Initially I didn't have the if statement, so it would try to massage both incoming and outgoing liquidity ... however, then I realized that it was just moving Sats from left to right and then back left (?) in an endless loop, which probably cost me some Sats.

BELOW IS OUTDATED, PLEASE SEE NEWER POSTS INSTEAD!!

#! /usr/bin/env -S bash -e

THRESH_RATIO=45
PCT_INIT=100
PCT_DROP=15
SLEEP_MINS=5

cd "${HOME}/rebalance-lnd" || exit
PCT_CUR=${PCT_INIT}

while true; do
    echo -e "\nStarting cycle with PCT_CUR=${PCT_CUR} ..."

    if [ "${1}" == "to" ]; then
        in_num_chans=$(./rebalance.py -r ${THRESH_RATIO} -l -i | grep -c "Channel ID:")

        for i in $(seq 1 "${in_num_chans}"); do
            echo "In: $i ..."
            nice -n 19 ./rebalance.py --econ-fee -t "${i}" -p "${PCT_CUR}" &
        done
    else
        out_num_chans=$(./rebalance.py -r ${THRESH_RATIO} -l -o | grep -c "Channel ID:")

        for i in $(seq 1 "${out_num_chans}"); do
            echo "Out: $i ..."
            nice -n 19 ./rebalance.py --econ-fee -f "${i}" -p "${PCT_CUR}" &
        done
    fi

    echo "Waiting for processes to finish ..."
    wait
    echo "Sleeping for ${SLEEP_MINS} minutes ..."
    sleep $((60 * SLEEP_MINS))

    if [ ${PCT_CUR} -gt 5 ]; then
        PCT_CUR=$((PCT_CUR * (100-PCT_DROP) / 100))
    else
        PCT_CUR=${PCT_INIT} # reset to start
    fi
done

[C-Otto]

With --econ-fee you shouldn't be able to move in both directions.

[C-Otto]

I don't think there are hard rules on how to rebalance. I suggest you have a look at #170. Instead of rebalancing to 50/50, you might be better off having enough liquidity on each side of the channel. If you're running the script all the time, you could also use lower amounts (say: 100k sats) in each attempt, and just repeat this until you're satisfied (note: compare the amount with the "imbalance" amount before you do this!).

[xanoni]

With --econ-fee you shouldn't be able to move in both directions.

you are right, I only added that after my observation, so will test again

[xanoni]

By the way, this is what I'm using with the latest version that no longer needs explicit specification of --econ-fee, etc.

Do I still need to gradually decrease the -p parameter, or will the script automatically probe how much can be achieved?

https://gist.github.com/xanoni/430ead6d11e9caaed8b020b11bc3a75c

BELOW IS OUTDATED, PLEASE SEE NEWER POSTS INSTEAD!!

#! /usr/bin/env bash

PCT_INIT=100
PCT_MIN=10
PCT_DROP=15

MIN_AMOUNT=10000 # don't rebalance below this amount
MIN_LOCAL=1000000 # min outbound sats per chan
MIN_REMOTE=1000000 # min inbound sats per chan

SLEEP_MINS=5
NICENESS='+19'

cd "${HOME}/rebalance-lnd" || exit
PCT_CUR="${PCT_INIT}"

while true; do
    echo -e "\nStarting cycle with PCT_CUR=${PCT_CUR} ..."

    # Increase outbound capacity
    if [ -z "${1}" ] || [ "${1}" == "outbound" ] || [ "${1}" == "all" ]; then
        num_chans="$(nice -n"${NICENESS}" ./rebalance.py \
            --list-candidates --incoming --min-local="${MIN_LOCAL}" \
            | grep -c "Channel ID:")"

        echo "Trying to increase outbound capacity for ${num_chans} channels:"

        for i in $(seq 1 "${num_chans}"); do
            echo ".. chan $i/${num_chans} ..."
            nice -n"${NICENESS}" ./rebalance.py \
                --to="${i}" \
                --min-amount="${MIN_AMOUNT}" \
                --min-local="${MIN_LOCAL}" \
                --percentage="${PCT_CUR}" &
        done
    fi

    #  Increase inbound capacity
    if [ -z "${1}" ] || [ "${1}" == "inbound" ] || [ "${1}" == "all" ]; then
        num_chans="$(nice -n"${NICENESS}" ./rebalance.py \
            --list-candidates --outgoing --min-remote="${MIN_REMOTE}" \
            | grep -c "Channel ID:")"

        echo "Trying to increase inbound capacity for ${num_chans} channels:"

        for i in $(seq 1 "${num_chans}"); do
            echo ".. chan $i/${num_chans} ..."
            nice -n"${NICENESS}" ./rebalance.py \
                --from="${i}" \
                --min-amount="${MIN_AMOUNT}" \
                --min-remote="${MIN_REMOTE}" \
                --percentage="${PCT_CUR}" &
        done
    fi

    echo "Waiting for processes to finish ..." && wait

    echo "Sleeping for ${SLEEP_MINS} minutes ..." && sleep "$((60 * SLEEP_MINS))"

    if [ "${PCT_CUR}" -gt "${PCT_MIN}" ]; then
        PCT_CUR="$((PCT_CUR * (100-PCT_DROP) / 100))"
    else
        PCT_CUR="${PCT_INIT}" # reset to start
    fi
done

[C-Otto]

Thanks for the udpate! There's no probing, the amount is used as specified (or computed once, without any probing).

[xanoni]

@C-Otto I just noticed that the script no longer works, because it seems that I can no longer specify the channel by index (e.g., --to=5 for the 5th candidate). Was this removed on purpose?

[C-Otto]

Yes, I removed the feature, mainly it made the code rather complex, but also because the targeted channel might appear at another position (with another index) in between running the two commands. Users also were confused because the index depended on values given for --ratio.

[xanoni]

Gotcha. I bet awk can handle it.

[TrezorHannes]

Gotcha. I bet awk can handle it.

Have you by any chance successfully added the awk grabber for your autopilot script?

[xanoni]

Gotcha. I bet awk can handle it.

Have you by any chance successfully added the awk grabber for your autopilot script?

Nah I haven't gotten to that, for now I'm using below lazy workaround :) ... it think it works well enough. (It just picks channels randomly, so channels are potentially selected more than once in each round.)

#! /usr/bin/env bash

PCT_INIT=100
PCT_MIN=25
PCT_DROP=25

MIN_AMOUNT=10000 # don't rebalance below this amount (def: 10k)
MIN_LOCAL=1000000 # min outbound sats per chan (def: 1M)
MIN_REMOTE=1000000 # min inbound sats per chan (def: 1M)
FEE_FACTOR=0.95 # (def: 1.0)

# channels to be rebalanced per run/cycle
CHANS_PER_ITER_TO=30
CHANS_PER_ITER_FROM=50

# `nice` process value (+19 = lowest prio)
NICENESS='+19'

# time to sleep between runs/cycles, in minutes (picked randomly)
SLEEP_MIN=20
SLEEP_MAX=60

# workaround for 32-Bit Raspbian Bullseye
LD_PRELOAD="/usr/lib/arm-linux-gnueabihf/libatomic.so.1.2.0"

cd "${HOME}/rebalance-lnd" || exit 1
PCT_CUR="${PCT_INIT}"

while true; do
    echo -en "\nStarting cycle with PCT_CUR=${PCT_CUR} ...\n\n"

    for _ in $(seq 1 "${CHANS_PER_ITER_TO}"); do
        # increase outbound capacity
        LD_PRELOAD="${LD_PRELOAD}" nice -n"${NICENESS}" ./rebalance.py \
            --to="-1" \
            --min-amount="${MIN_AMOUNT}" \
            --min-local="${MIN_LOCAL}" \
            --percentage="${PCT_CUR}" \
            --fee-factor="${FEE_FACTOR}"
    done

    for _ in $(seq 1 "${CHANS_PER_ITER_FROM}"); do
        # increase inbound capacity
        LD_PRELOAD="${LD_PRELOAD}" nice -n"${NICENESS}" ./rebalance.py \
            --from="-1" \
            --min-amount="${MIN_AMOUNT}" \
            --min-remote="${MIN_REMOTE}" \
            --percentage="${PCT_CUR}" \
            --fee-factor="${FEE_FACTOR}"
    done
    
    # pick random timeframe from interval and sleep
    SLEEP_MINS=$((SLEEP_MIN + RANDOM % (SLEEP_MAX-SLEEP_MIN)))
    echo -en "\n\nSleeping for ${SLEEP_MINS} minutes ...\n" && sleep "$((60 * SLEEP_MINS))"

    # modify "--percentage" parameter before next run/cycle
    if [ "${PCT_CUR}" -gt "${PCT_MIN}" ]; then
        PCT_CUR="$((PCT_CUR * (100-PCT_DROP) / 100))"
    else
        PCT_CUR="${PCT_INIT}" # reset to start
    fi
done