[BUG] Playwright-core - Timeout while waiting for event \"download\"\nNote: #3726

osmenia · 2020-09-02T10:42:10Z

Context:

Playwright Version: 1.3.0
Operating System: AWS Lambda
Node.js version: 12
Browser: [Chromium]
Extra: playwright-aws-lambda

Hi Team,

I am testing playwright-core with AWS lambda.

there is problem with page.waitForEvent("download") in playwright-code 1.3.0.

page.waitForEvent("download") works fine with playwright-code 1.0.2 but with playwright-code 1.3.0 does not.

Error:

"TimeoutError: page.waitForEvent: Timeout 30000ms exceeded.",
"Note: use DEBUG=pw:api environment variable and rerun to capture Playwright logs.",
" at ProgressController.run (/opt/node_modules/playwright-core/lib/progress.js:71:30)",
" at Page.waitForEvent (/opt/node_modules/playwright-core/lib/page.js:243:35)",
" at Page.waitForEvent (/opt/node_modules/playwright-core/lib/helper.js:80:31)",

Code Snippet

 await page.goto(
    "https://file-examples.com/index.php/sample-documents-download/sample-xls-download/"
  );

  const handles = await page.$$("#table-files .file-link");

  for (let num of handles) {
    const [download] = await Promise.all([
      page.waitForEvent("download"), // wait for download to start
      num.click("a.btn"),
    ]);
    const path = await download.path();
}

mxschmitt · 2020-09-05T13:28:45Z

Since num is a ElementHandle, the click method does not accept a new selector. You have to use $ to get the new element to click on it.

Example:

// @ts-check
const playwright = require("playwright");

(async () => {
  const browser = await playwright.chromium.launch()
  const context = await browser.newContext({
    acceptDownloads: true
  })
  const page = await context.newPage()
  await page.goto(
    "https://file-examples.com/index.php/sample-documents-download/sample-xls-download/"
  );

  const handles = await page.$$("#table-files .file-link");

  for (let num of handles) {
    const [download] = await Promise.all([
      page.waitForEvent("download"), // wait for download to start
      (await num.$("a.btn")).click(),
    ]);
    const path = await download.path();
    console.log(path)
  }
  await browser.close();
})();

Interactive: https://try.playwright.tech/?s=gd4r9

osmenia · 2020-09-07T11:44:59Z

Hi Max,

Thanks you very much for your help.

I tried your solution. It does not work on aws lambda with playwright-core 1.3.0, but with PLW 1.0.2 works fine.

Error:
"TimeoutError: page.waitForEvent: Timeout 30000ms exceeded.",

@mxschmitt is it possible to test playwright-core local?

dgozman · 2020-10-01T07:38:19Z

Could you please run with DEBUG=pw:api,pw:browser* environment variable and post the logs here? This seems like an AWS lambda specific issue.

osmenia · 2020-10-01T14:02:41Z

@dgozman
To be honest, i do not know how to run DEBUG=pw:api,pw:browser* on aws.

dgozman · 2020-10-01T14:25:24Z

@dgozman
To be honest, i do not know how to run DEBUG=pw:api,pw:browser* on aws.

I think this should be possible directly in Node, if you use javascript lambda:

// Set the env variable.
process.env.DEBUG = 'pw:api,pw:browser*';
// Use playwright.
const browser = await require('playwright').chromium.launch({});

jknooks583 · 2020-10-01T19:15:19Z

@osmenia Just out of curiosity, is this working fine locally and just not working with AWB lambda?

osmenia · 2020-10-01T19:22:54Z

@jknooks583

Yes, Code works locally fine. all good
Code also works fine on AWS Lambda with playwright-code 1.0.2 but with playwright-code 1.3.0 does not (TimeoutError). It is the same code.

I did not test with playwright-code 1.4

jknooks583 · 2020-10-01T21:05:17Z

@osmenia Hey just wanted to let you know that I tested with playwright 1.4 and it is working for me if you want to try that out

osmenia · 2020-10-03T20:08:36Z

@jknooks583 How did you do test? For me does not work :(

Here ist code:

const playwright = require('playwright-aws-lambda');

exports.handler = async(event, context) => {
  let browser = null;

  try {
    const browser = await playwright.launchChromium();
    const context = await browser.newContext({ acceptDownloads: true });
    const page = await context.newPage();

    await page.goto(
      "https://file-examples.com/index.php/sample-documents-download/sample-xls-download/"
    );
    const handles = await page.$$("#table-files .file-link");

    for (let num of handles) {
      const [download] = await Promise.all([
        page.waitForEvent("download"), // wait for download to start
        num.click("a.btn"),
      ]);
      const path = await download.path();
      console.log(path);
    }
  }
  catch (error) {
    throw error;
  }
  finally {
    if (browser !== null) {
      await browser.close();
    }
  }
};

I create Layer with Playwright 1.0.2

package.json
 "author": "",
  "license": "ISC",
  "dependencies": {
    "playwright-aws-lambda": "^0.6.0",
    "playwright-core": "^1.0.2"
  }

This is result:
see Working Download.log file


START RequestId: 9c809345-8fa0-492f-8246- Version: $LATEST
2020-10-03T14:48:10.382Z	-8fa0-492f-8246-	INFO	/tmp/playwright_downloads-dcZRq6/f6f9961c-7b52-4250-830d-e0cf49e9f838
2020-10-03T14:48:10.667Z	-8fa0-492f-8246-	INFO	/tmp/playwright_downloads-dcZRq6/48d9a174-8574-4069-8a2b-2ff0ab1182cc
2020-10-03T14:48:10.862Z	-8fa0-492f-8246-	INFO	/tmp/playwright_downloads-dcZRq6/648e89ec-e534-4fce-b093-994031a92004
2020-10-03T14:48:11.249Z	-8fa0-492f-8246-	INFO	/tmp/playwright_downloads-dcZRq6/da7f8dd3-0ad8-496e-b5ff-74e26fa92df1
2020-10-03T14:48:11.422Z	-8fa0-492f-8246-	INFO	/tmp/playwright_downloads-dcZRq6/9cd62ad0-5e38-4740-8e81-dc9e0c2eff90
2020-10-03T14:48:11.598Z	-8fa0-492f-8246-	INFO	/tmp/playwright_downloads-dcZRq6/11f48922-4704-4ff7-9ebe-d8d981e02a0d
2020-10-03T14:48:11.770Z	-8fa0-492f-8246-	INFO	/tmp/playwright_downloads-dcZRq6/1551df6c-10cb-4d8c-83f5-1638ea70b58e
2020-10-03T14:48:11.922Z	-8fa0-492f-8246-	INFO	/tmp/playwright_downloads-dcZRq6/aac320bf-0fd7-461d-9c74-de104cf0382c
2020-10-03T14:48:12.088Z	-8fa0-492f-8246-	INFO	/tmp/playwright_downloads-dcZRq6/2d5f03ce-bb21-4f06-9877-af1d50243abd
2020-10-03T14:48:12.261Z	-8fa0-492f-8246-	INFO	/tmp/playwright_downloads-dcZRq6/c625ad80-f8be-472e-9b39-8bea07fcef11

Files are downloaded with playwright 1.0.2. Download works all good.

I createt Layer with Playwright 1.3.0

package.json
 "author": "",
  "license": "ISC",
  "dependencies": {
    "playwright-aws-lambda": "^0.6.0",
    "playwright-core": "^1.3.0"
  }

I am getting Error:

{
  "errorType": "TimeoutError",
  "errorMessage": "page.waitForEvent: Timeout 30000ms exceeded.\nNote: use DEBUG=pw:api environment variable and rerun to capture Playwright logs.",
  "trace": [
    "TimeoutError: page.waitForEvent: Timeout 30000ms exceeded.",
    "Note: use DEBUG=pw:api environment variable and rerun to capture Playwright logs.",
    "    at ProgressController.run (/opt/nodejs/node_modules/playwright-core/lib/progress.js:71:30)",
    "    at Page.waitForEvent (/opt/nodejs/node_modules/playwright-core/lib/page.js:243:35)",
    "    at Page.waitForEvent (/opt/nodejs/node_modules/playwright-core/lib/helper.js:80:31)",
    "    at Runtime.exports.handler (/var/task/index.js:19:14)"
  ]
}

This is layer with plawright 1.4.0

package.json
 "author": "",
  "license": "ISC",
  "dependencies": {
    "playwright-aws-lambda": "^0.6.0",
    "playwright-core": "^1.4.0"
  }

Error Message:

{
  "errorType": "Error",
  "errorMessage": "elementHandle.click: : expected object, got string",
  "trace": [
    "elementHandle.click: : expected object, got string",
    "    at ElementHandle._wrapApiCall (/opt/nodejs/node_modules/playwright-core/lib/client/channelOwner.js:76:15)",
    "    at ElementHandle.click (/opt/nodejs/node_modules/playwright-core/lib/client/elementHandle.js:89:21)",
    "    at Runtime.exports.handler (/var/task/index.js:20:13)",
    "    at runMicrotasks (<anonymous>)",
    "    at runNextTicks (internal/process/task_queues.js:62:5)",
    "    at processImmediate (internal/timers.js:429:9)"
  ]
}

**I replaced "num.click("a.btn")," with "(await num.$("a.btn")).click()," Thanks @mxschmitt **

Then I got this Error:

{
  "errorType": "TimeoutError",
  "errorMessage": "Timeout while waiting for event \"download\"\nNote: use DEBUG=pw:api environment variable and rerun to capture Playwright logs.",
  "trace": [
    "TimeoutError: Timeout while waiting for event \"download\"",
    "Note: use DEBUG=pw:api environment variable and rerun to capture Playwright logs.",
    "    at /opt/nodejs/node_modules/playwright-core/lib/client/waiter.js:40:51",
    "    at async Waiter.waitForPromise (/opt/nodejs/node_modules/playwright-core/lib/client/waiter.js:48:28)",
    "    at async Page.waitForEvent (/opt/nodejs/node_modules/playwright-core/lib/client/page.js:285:24)",
    "    at async Promise.all (index 0)",
    "    at async Runtime.exports.handler (/var/task/index.js:18:26)"
  ]
}

Please find attached Log Files. @dgozman @mxschmitt
Working Download.log is Log file from plw 1.0.2
Error_Download.log is Log file from plw 1.4.0
Working Download.log
Error_Download.log

dgozman · 2020-11-09T23:30:23Z

Sorry for the slow response 😞 I have looked at the logs, and unfortunately I don't see where the problem comes from.

Two ideas:

Variable browser is declared twice (once with let, another with const), so the browser.close() code is never executed. Not sure whether not closing a browser is a problem with AWS lambda.
Perhaps the site is blocking your IP from repeated access? Some kind of anti-bot protection?
Maybe more detailed logs would help us, if you are willing to run again with process.env.DEBUG = 'pw:api,pw:browser*,pw:protocol*';.

dgozman · 2020-11-23T22:20:36Z

Closing because we cannot reproduce and there is no additional information to make progress.

osmenia · 2020-11-24T09:47:47Z

Sorry for the slow response 😞 I have looked at the logs, and unfortunately I don't see where the problem comes from.

Two ideas:

Variable browser is declared twice (once with let, another with const), so the browser.close() code is never executed. Not sure whether not closing a browser is a problem with AWS lambda.

Perhaps the site is blocking your IP from repeated access? Some kind of anti-bot protection?

Maybe more detailed logs would help us, if you are willing to run again with process.env.DEBUG = 'pw:api,pw:browser*,pw:protocol*';.

Hi Gozman,
thnx very much
i will test it

osmenia · 2021-01-03T10:10:08Z

Hi Team,

first of all I would like to wish you a happy new year!

I am runing code on aws lambda.
Here is code:

const playwright = require('playwright-aws-lambda');
 exports.handler = async(event, context) => {
  
  try {
    console.log("Start here -> debug");
    const browser = await playwright.launchChromium();
    const context = await browser.newContext({ acceptDownloads: true });
    const page = await context.newPage();
    await page.goto(
      "https://file-examples.com/index.php/sample-documents-download/sample-xls-download/"
    );
    const handles = await page.$$("#table-files .file-link");
    for (let num of handles) {
      console.log("This is num output ", num);
      const [download] = await Promise.all([
        page.waitForEvent("download"), // wait for download to start
        (await num.$("a.btn")).click(),
      ]);
      const path = await download.path();
      console.log("This is file path: ",path);
    }
    await browser.close();
  }
  catch (error) {
    throw error;
  }
};

Config:


  "dependencies": {
    "playwright-aws-lambda": "^0.6.0",
    "playwright-core": "^1.7.1"
  }

run with param: process.env.DEBUG = 'pw:api,pw:browser,pw:protocol'**

Error:

{ "errorType": "TimeoutError", "errorMessage": "Timeout while waiting for event \"download\"\nNote: use DEBUG=pw:api environment variable and rerun to capture Playwright logs.", "trace": [ "TimeoutError: Timeout while waiting for event \"download\"", "Note: use DEBUG=pw:api environment variable and rerun to capture Playwright logs.", " at /opt/nodejs/node_modules/playwright-core/lib/client/waiter.js:40:51", " at async Waiter.waitForPromise (/opt/nodejs/node_modules/playwright-core/lib/client/waiter.js:48:28)", " at async Page.waitForEvent (/opt/nodejs/node_modules/playwright-core/lib/client/page.js:306:24)", " at async Promise.all (index 0)", " at async Runtime.exports.handler (/var/task/index.js:21:26)" ] }

Please find attached log file.
Log_AWS_03012021.zip

I tried also with another page with download, did not work on aws. Local works fine.

  await page.goto("https://speed.hetzner.de/");
  const [download] = await Promise.all([
    page.waitForEvent("download"), // wait for download to start
    page.click("body > p:nth-child(2) > a"),
  ]);   const path = await download.path();
  console.log(path);

Can you pls help
@dgozman @mxschmitt

dgozman · 2021-01-04T20:59:47Z

@osmenia I do not know much about aws lambda environment, but it seems that playwright-aws-lambda does some unzipping of Chromium binary to /tmp/chromium. Perhaps you should pass downloadsPath pointing somewhere in /tmp?

imhashir · 2021-01-21T06:12:51Z

Any progress here? I am getting the same error. Code is working fine when running locally but not when deployed via serverless deploy:

imhashir · 2021-01-22T21:08:55Z

@dgozman
I tried enabling the logs as you suggested above and turns out, the event is not firing up in lambda. I can see the download logs and network activity logs as below:

The download was completed but I did not receive the completion callback. I used two approaches:

1st Approach:

  const [download] = await Promise.all([
    page.waitForEvent('download'),
    page.click(`#${BTN_ID}`),
  ]);
  const downloadStream = await download.createReadStream();

Received timeout error

2nd Approach:

const downloadFile = async (page, element) => {
  return new Promise((resolve, reject) => {
    page.on('download', (download) => {
      console.log('File downloaded!');
      resolve(download);
    });
    return page.click(element);
  });
};
const download = await downloadFile(page, `#${BTN_ID}`);

Did not receive any timeout error and the lambda function kept running, and ultimately, timed out.

I came up with the second approach to check if the on download event is emitted, which isn't. So something's up with the events on lambda. I have a project due, very soon, so any urgent help would be really appreciated.

This is the package I am using for chrome on lambda: https://github.com/JupiterOne/playwright-aws-lambda
I guess this is the only difference between local and lambda execution. Locally, we use local installation of chrome and on lambda, we use this package. So Is it possible that this chrome is not emitting the on-download event? (but I can see the download completion in network logs)

dgozman · 2021-01-26T02:54:04Z

@imhashir Thank you for digging into this issue. Looking at playwright-aws-lambda's built-in chromium, it seems to be 8 months old. It is quite possible that recent Playwright does not work with downloads of 8-months old Chromium, because the project is quickly evolving. What is your Playwright version?

osmenia · 2021-01-26T08:07:23Z

@imhashir @dgozman
Try with Playwright Version 1.0.2

you can try this:
JupiterOne/playwright-aws-lambda#27 (comment)

imhashir · 2021-01-26T22:13:44Z

@dgozman
Yeah, I suspected that as well. My playwright-core version is 1.8.0.

@osmenia
You are legend! Thanks man. I just downgraded to 1.0.2 and it worked both locally and on lambda. I've been stuck at it for days, so glad it worked. I'll try the solution provided on that issuecomment as well, but I'll have to look a bit deeper into docker-lambda thing.

Now the only issue with downgrade approach is that the suggestedFilename function is not in 1.0.2. Is there any workaround for that at the moment?

osmenia · 2021-01-27T05:34:58Z

@imhashir

maybe this can help:

.....
//dawnloaded file path
  const path = await download.path();

//read dawnloaded file to buffer
  var data = fs.readFileSync(path);
 
//write buffer to file
 fs.writeFileSync('../data/helloworld${uuid}.xls', data);

TheAPIguys · 2022-11-20T00:04:59Z

Hello Guys,

I having the timeout error as below:
{
"errorType": "TimeoutError",
"errorMessage": "page.waitForEvent: Timeout 30000ms exceeded while waiting for event "download"\n=========================== logs ===========================\nwaiting for event "download"\n============================================================",
"trace": [
"page.waitForEvent: Timeout 30000ms exceeded while waiting for event "download"",
"=========================== logs ===========================",
"waiting for event "download"",
"============================================================",
" at Runtime.exports.handler (/var/task/index.js:54:11)"
]
}

my version is as follows:
"dependencies": {
"playwright-aws-lambda": "^0.9.0",
"playwright-core": "^1.28.0"
}

there is any way to solve this issue?

pavelfeldman assigned dgozman Oct 20, 2020

dgozman closed this as completed Nov 23, 2020

osmenia changed the title ~~[BUG] Playwright-core - page.waitForEvent: Timeout 30000ms exceeded~~ [BUG] Playwright-core - Timeout while waiting for event \"download\"\nNote: Jan 3, 2021

imhashir mentioned this issue Jan 23, 2021

Download event not caught and always times out JupiterOne/playwright-aws-lambda#27

Open

Mathspy mentioned this issue Feb 3, 2021

Improve documentation for DEBUG variable #5279

Closed

NikkTod mentioned this issue Mar 29, 2022

[Question] ERR_ABORTED while trying to download a file inside TeamCity CI machine within Chrome microsoft/playwright-java#863

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] Playwright-core - Timeout while waiting for event \"download\"\nNote: #3726

[BUG] Playwright-core - Timeout while waiting for event \"download\"\nNote: #3726

osmenia commented Sep 2, 2020 •

edited

Loading

mxschmitt commented Sep 5, 2020

osmenia commented Sep 7, 2020 •

edited

Loading

dgozman commented Oct 1, 2020 •

edited

Loading

osmenia commented Oct 1, 2020

dgozman commented Oct 1, 2020

jknooks583 commented Oct 1, 2020

osmenia commented Oct 1, 2020

jknooks583 commented Oct 1, 2020

osmenia commented Oct 3, 2020

dgozman commented Nov 9, 2020

dgozman commented Nov 23, 2020

osmenia commented Nov 24, 2020

osmenia commented Jan 3, 2021

dgozman commented Jan 4, 2021

imhashir commented Jan 21, 2021

imhashir commented Jan 22, 2021 •

edited

Loading

dgozman commented Jan 26, 2021

osmenia commented Jan 26, 2021 •

edited

Loading

imhashir commented Jan 26, 2021

osmenia commented Jan 27, 2021 •

edited

Loading

TheAPIguys commented Nov 20, 2022

[BUG] Playwright-core - Timeout while waiting for event \"download\"\nNote: #3726

[BUG] Playwright-core - Timeout while waiting for event \"download\"\nNote: #3726

Comments

osmenia commented Sep 2, 2020 • edited Loading

mxschmitt commented Sep 5, 2020

osmenia commented Sep 7, 2020 • edited Loading

dgozman commented Oct 1, 2020 • edited Loading

osmenia commented Oct 1, 2020

dgozman commented Oct 1, 2020

jknooks583 commented Oct 1, 2020

osmenia commented Oct 1, 2020

jknooks583 commented Oct 1, 2020

osmenia commented Oct 3, 2020

dgozman commented Nov 9, 2020

dgozman commented Nov 23, 2020

osmenia commented Nov 24, 2020

osmenia commented Jan 3, 2021

dgozman commented Jan 4, 2021

imhashir commented Jan 21, 2021

imhashir commented Jan 22, 2021 • edited Loading

dgozman commented Jan 26, 2021

osmenia commented Jan 26, 2021 • edited Loading

imhashir commented Jan 26, 2021

osmenia commented Jan 27, 2021 • edited Loading

TheAPIguys commented Nov 20, 2022

osmenia commented Sep 2, 2020 •

edited

Loading

osmenia commented Sep 7, 2020 •

edited

Loading

dgozman commented Oct 1, 2020 •

edited

Loading

imhashir commented Jan 22, 2021 •

edited

Loading

osmenia commented Jan 26, 2021 •

edited

Loading

osmenia commented Jan 27, 2021 •

edited

Loading