What basic knowledge is needed?

[zfhxi]

Hello, I am a second year software engineering master student and I am poor at math.

I have some questions about the equations. In section 3.2, "(Ll)−1 i,j describes the correlation of vli and vlj at the equilibrium status", I don't understand the relationship between "inverse of Laplacian matrix" and "equilibrium status". Besides, In the appendix, I don't know how to get Eq.(4) from Eq.(2).

I wonder which books or papers should I refer to for understanding this equations better? Could you provide us with the information?
Thanks.

[hbai98]

Hi!

In the appendix, Eq.(2) is a linear system equation for the rate of fluid change in the graph system $G$(See Fig.1), where $\boldsymbol{I(t)}\in\mathbb{R}^N$ , $\boldsymbol{L}\in\mathbb{R}^{N\times N}$, and $N$ is the number of patches,

$$\hat{\boldsymbol{I}(t)} = \frac{d \boldsymbol{I}(t)}{d t} = \boldsymbol{L}\boldsymbol{I}(t) + u(t)\Gamma(\boldsymbol{F}).$$

So, we see that the rate of change is related to $\boldsymbol{L}$, the Laplacian matrix that transforms the amount of fluid $\boldsymbol{I}(t)$ to the rate. Each $\boldsymbol{L}_{i, j}$ denotes the extent the rate $\hat{\boldsymbol{I}_i(t)}$ should rely on the amount $\boldsymbol{I}_j(t)$.

This approach is based on the theorem that can be found in Newton's Law of cooling, and $\boldsymbol{L}$ plays a similar role as the coefficient in heat transfer.

Furthermore, Eq.(4) aims to integrate the change over time (0, t), showing the fluid's overall distribution at time t[Some typo in the previous appendix, now is fixed and released into the arxiv! ].

$$\boldsymbol{I}(t) = \int_{t^{{\prime}}=0}^{t} e^{\boldsymbol{L}(t-t^{\prime})}\Gamma(\boldsymbol{F})u(t^{\prime})dt^{\prime},$$

where $\boldsymbol{F}\in \mathbb{R}^{H\times W}$ is the attention map from Transformer, $\Gamma$ is a flatten operation, that $\Gamma(\boldsymbol{F})\in \mathbb{R}^{N}$.

To derive Eq.(4), it's a little tricky since the suffix $u(t)\Gamma(\boldsymbol{F})$, the influx of $G$, makes the original ODE function $\hat{\boldsymbol{I}(t)}=\boldsymbol{L}\boldsymbol{I}(t)$ in Eq.(2) not straightforward to represent as $C_1e^{\lambda x}V_0+C_2$.

I found this link, which helps me understand the nonautonomous(non-homogeneous) linear system like Eq.(2) given the variation of parameters method. Furthermore, if you are interested in the derivation on Eq.(4), please check the Linear Diffentiable Eqaution.

Given the integration factor $A(t)$, satisfying $-A(t)\boldsymbol{L}=A'(t)$, Eq.(2) multiplied by $A(t)$ in both sides is,

$$A(t)\hat{\boldsymbol{I}(t)} + A'(t)\boldsymbol{I}(t) = (A(t)\boldsymbol{I}(t))'= A(t)u(t)\Gamma(\boldsymbol{F}).$$

Integrate both sides, given the constant c,

$$A(t)\boldsymbol{I}(t) + c = \int A(t)u(t)\Gamma(\boldsymbol{F}) dt.$$

Then,

$$\boldsymbol{I}(t) = \frac{\int A(t)u(t)\Gamma(\boldsymbol{F}) dt + c}{A(t)}.$$

Given $-A(t)\boldsymbol{L}=A'(t)$ and another constant k, we can determine $A(t)$,

$$-\boldsymbol{L}=\frac{A'(t)}{A(t)}=(ln(A(t))'.$$ $$-\int \boldsymbol{L} dt + k =ln(A(t)).$$ $$A(t) = e^{ -\int \boldsymbol{L} dt + k} = k e^{ -\int \boldsymbol{L} dt}.$$

Given the initial condition that there's no fluid at time $\boldsymbol{I}(0) = 0$, we eliminate the constant when taking it back to the $\boldsymbol{I}(t)$,

$$\boldsymbol{I}(t) = \int_{t^{{\prime}}=0}^{t} e^{\boldsymbol{L}(t-t^{\prime})}\Gamma(\boldsymbol{F})u(t^{\prime})dt^{\prime},$$

where $u(t^{\prime})$ is a unit step function(a scale parameter, we can drop it), and when $t\to \infty$,

$$\lim_{t \to \infty}\boldsymbol{I}(t) =\lim_{t \to \infty} \boldsymbol{L}^{-1}(1-e^{-\boldsymbol{L}t})\Gamma(\boldsymbol{F}) = \boldsymbol{L}^{-1}\Gamma(\boldsymbol{F}),$$

[zfhxi]

Gosh, thanks for the detailed reply!

[hbai98]

All visitors are welcome to keep commenting on this issue if there are questions about the design.