Graph execution error

[chang-change]

Is the problem shown in the figure below the reason for too many collocation points? When I reduce the collocation points, the error disappears. I wonder if there is any other workaround other than reducing configuration points? The GPU I am using is RTX 3090.

[lululxvi]

Smaller network? or use mini-batch.

[chang-change]

Thanks for your reply. My network is net = dde.nn.FNN( [2] + [16]*3 + [1] , 'sin', "Glorot uniform"), and I don't think the network too big. The collocation points are $10^6$. During the network training, I use 'model.train(iterations=iterations,batch_size=10)', but the error don't disappear.

[praksharma]

First of all using too much collocation points is not going to help you. A better strategy is to increase the point cloud density in regions with high PDE loss.

The source code is always helpful. Check this. You need to use dde.callbacks.PDEResidualResampler. Now check this to understand how it works.

In short, you can check dde.data.pde.resample_training_points() if you are looking to modify the source code for your own implementation.

[chang-change]

Sorry for replying so late! I have finished looking at the source code you provided, but I still have some questions.

First, In this line, I don't know how to get the training points? Where could I find the source code of self.geom.uniform_points?

Second, I used dde.callbacks.PDEResidualResampler(period = 100), but the error is still here. The reason of using too much collocation points is in order to calculate the integration by Monte Carlo integration.

[praksharma]

self.geom is an instance of deepxde.geometry.Geometry. This is the class definition of Geometry class.

So when you type self.geom.uniform_points , this function is executed.

However in temporal problems. You use the GeometryXTime class. So, in time dependent problems this function is used to sample the uniform points.

I guess your are using Monte Carlo integration implementation of the loss function. Why not use low discrepancy sequences instead of costly uniform sampling. How complex is your geometry? Because 10^6 collocation points are just too much. I have access to a supercomputer with lots of NVIDIA A100 40GB. If you want I can run it for you. Just email me that source code. [email protected]

[chang-change]

@praksharma
Thanks for your reply! I checked this function and I don't know how this function does the sampling? It just returns random_points. In the random_points function, I only found the comments.

Why not use low discrepancy sequences instead of costly uniform sampling.

How should I use low discrepancy sequences instead of costly uniform sampling? Just as the discussion, my sampling code is following.

def sample_func(xmin,xmax,ymin,ymax,num = num):

    x_points = tf.random.uniform((tf.shape(xmin)[0],num),xmin,xmax)   # x_points_shape = (n,num)
    y_points = tf.random.uniform((tf.shape(ymin)[0],num),ymin,ymax)   # y_points_shape = (n,num)
    
    x_ = tf.reshape(x_points,[tf.shape(xmin)[0]*num,1])    # x_shape = (n*num,1)
    y_ = tf.reshape(y_points,[tf.shape(ymin)[0]*num,1])    # y_shape = (n*num,1)
                    
    x_y = tf.concat([x_,y_],axis = 1)   # x_y_shape  = (n*num,2)
    
    return x_y 

I want to do a Monte Carlo integration using numerous sampling points around the collocation point.

[praksharma]

I didn't saw that. So, uniform sample is not implemented in DeepXDE. You can use LHS, Sobol or Halton sample. By default DeepXDE use Sobol. You can try this in Tensorflow.

You need to sample the collocation points inside a CSG geometry and calculate the area for Monte Carlo integration. In past I have used PySDF to do Monte Carlo integration on a STL geometry (non CSG geometry). The only problem is you need to generate a STL file for each BC and PDE which needs ANSYS ACT or Hypermesh.

[lululxvi]

Why do you need numerical integration? Do you solve an integral differential equation?

[chang-change]

Yes. I need to solve a differential-integral equation in which there are some partial derivative terms and a double integral term. My idea was to use the Monte Carlo integration approximation, which requires many sample points. This resulted in a memory error. Do you give some suggestion to solve double integral in neural network? Thanks!

[lululxvi]

Use quadrature, see https://doi.org/10.1137/18M1229845