Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Memory leak in node process #520

Open
Nialito opened this issue Jan 21, 2025 · 12 comments
Open

[Bug]: Memory leak in node process #520

Nialito opened this issue Jan 21, 2025 · 12 comments
Labels
p2-bug Something isn't working

Comments

@Nialito
Copy link

Nialito commented Jan 21, 2025

Environments

  • playwright-go Version: 0.4901.0
  • Browser: Chromium
  • OS and version: Windows 11 / Ubuntu 24.04

Bug description
When running my program that uses playwright-go for an extended period, the node process starts consuming more and more RAM.

An hour after launching my program, the node process consumes the following amount of memory:
Image

After 4 days of continuous operation, the memory consumption increases:
Image

Based on the RSS field, after one hour of running the program, the node process was consuming approximately 73 MB of memory (RSS: 75,244 KB). After 4 days of operation, it increased to approximately 296 MB (RSS: 302,844 KB).
This indicates a steady increase in memory usage by the node process, which may suggest a memory leak.

To Reproduce
Please consider the minimal code example that demonstrates the issue:

package main

import (
	"fmt"
	"log"
	"time"

	"github.com/playwright-community/playwright-go"
)

type PlaywrightService struct {
	PlaywrightInstance *playwright.Playwright
	BrowserInstance    playwright.Browser
}

func main() {
	playwrightService := &PlaywrightService{}

	playwrightInstance, err := playwright.Run()
	if err != nil {
		log.Fatalf("Failed to start Playwright: %v", err)
	} else {
		playwrightService.PlaywrightInstance = playwrightInstance
	}
	defer playwrightService.PlaywrightInstance.Stop()

	browserInstance, err := playwrightInstance.Chromium.Launch(playwright.BrowserTypeLaunchOptions{
		Headless: playwright.Bool(true),
	})
	if err != nil {
		log.Fatalf("Failed to launch Chromium: %v", err)
	} else {
		playwrightService.BrowserInstance = browserInstance
	}
	defer playwrightService.BrowserInstance.Close()

	for {
		if err := playwrightService.OpenURL("http://www.msftconnecttest.com/connecttest.txt"); err != nil {
			log.Printf("Failed to open URL: %v", err)
		}
		time.Sleep(10 * time.Second)
	}
}

func (ps *PlaywrightService) OpenURL(url string) error {
	context, err := ps.BrowserInstance.NewContext()
	if err != nil {
		return fmt.Errorf("creating browser context: %v", err)
	}
	defer context.Close()

	page, err := context.NewPage()
	if err != nil {
		return fmt.Errorf("creating new page: %v", err)
	}
	defer page.Close()

	if _, err := page.Goto(url, playwright.PageGotoOptions{
		Timeout: playwright.Float(5000),
	}); err != nil {
		return fmt.Errorf("navigating to %s: %v", url, err)
	}

	return nil
}

Additional context
In this code, the program continuously opens the specified URL in an infinite loop with a 10-second interval. Each cycle creates a new browser context and a new page, which are then closed using defer. Despite this, there is a constant increase in memory consumption by the node process. This may indicate a possible memory leak in playwright-go or an issue in the interaction with Node.js.
I would appreciate any help in resolving this problem.

@Nialito Nialito added the p2-bug Something isn't working label Jan 21, 2025
@wade-liwei
Copy link

me too

#525

@canstand
Copy link
Collaborator

Please log if there is an error when closing BrowserContext.

defer func() {
		err := context.Close()
		if err != nil {
			log.Printf("Failed to close browser context: %v", err)
		}
	}()

@canstand
Copy link
Collaborator

And microsoft/playwright#34230

@Nialito
Copy link
Author

Nialito commented Jan 25, 2025

@canstand, thank you for your suggestion!

Please log if there is an error when closing BrowserContext.

defer func() {
err := context.Close()
if err != nil {
log.Printf("Failed to close browser context: %v", err)
}
}()

I modified my code to log any errors when closing the context as you recommended. However, after running the program with this change, no errors are being logged when closing the context. The memory consumption of the node process continues to increase over time.

And microsoft/playwright#34230

I also tried adding init: true to my docker-compose.yml, but this did not help. I've observed memory leaks not only inside the Docker container (Ubuntu 24.04) but also on my main system running Windows 11. Moreover, I did not see any daemon processes in the list of processes. In my screenshot, you can see how cli.js run-driver is consuming more and more RAM.

Updated method:

func (ps *PlaywrightService) OpenURL(url string) error {
	context, err := ps.BrowserInstance.NewContext()
	if err != nil {
		return fmt.Errorf("creating browser context: %v", err)
	}
	defer func() {
		err := context.Close()
		if err != nil {
			log.Printf("Failed to close browser context: %v", err)
		} else {
			log.Println("Browser context successfully closed")
		}
	}()

	page, err := context.NewPage()
	if err != nil {
		return fmt.Errorf("creating new page: %v", err)
	}
	defer func() {
		err := page.Close()
		if err != nil {
			log.Printf("Failed to close page: %v", err)
		} else {
			log.Println("Page successfully closed")
		}
	}()

	if _, err := page.Goto(url, playwright.PageGotoOptions{
		Timeout: playwright.Float(5000),
	}); err != nil {
		return fmt.Errorf("navigating to %s: %v", url, err)
	}

	return nil
}

Output:

Image

@wade-liwei
Copy link

Page takes too many memory,

You do not need to close the browser, use only one browser,

make sure to close the page, and increase access step by step.

@wade-liwei
Copy link

I think you are from China or Asia.

@Nialito
Copy link
Author

Nialito commented Jan 26, 2025

@wade-liwei,
Thank you for your response, but I didn't quite understand it.
In my code, I create a new browser context for each request to ensure full isolation between them. This is necessary so that each request has its own cookies, localStorage, and other data. Otherwise, sharing a single context or page would lead to shared state between requests.
At the end of each request, I close both the page and the browser context using defer context.Close() and defer page.Close(). This should release any memory associated with them, regardless of how much memory the page consumes during its operation. Despite this, I'm observing a steady increase in memory usage by the Node process over time, which suggests there might be a memory leak.

@wade-liwei
Copy link

I am an English beginner,
I understand your situation now, but my situation is different from yours.

@wade-liwei
Copy link

why do you need different browser context for each request?

@Nialito
Copy link
Author

Nialito commented Jan 26, 2025

@wade-liwei,

but my situation is different from yours

I think we might be experiencing the same issue, as I'm observing the same increase in memory usage as shown in your screenshot from your issue (#525).

why do you need different browser context for each request?

I need to create a new browser context for each request to ensure that cookies and localStorage are isolated. This isolation is crucial to prevent shared state between requests. As I mentioned earlier, I'm closing both the page and the browser context using defer context.Close() and defer page.Close(). In theory, this should release all associated resources, and the memory usage should remain stable. However, despite properly closing these resources, I'm still observing a steady increase in memory usage by the Node process over time.

@wade-liwei
Copy link

wade-liwei commented Jan 27, 2025

got it.

Your problem may be Golang GC, do you use docker to limit memory resources?
or Out Of Memory?

and headless use less memory.

@wade-liwei
Copy link

right now, my container_memory_usage_bytes.

Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
p2-bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants