Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SlimerJs and CasperJs captureContent property issue #690

Open
veterkh opened this issue Mar 30, 2018 · 3 comments
Open

SlimerJs and CasperJs captureContent property issue #690

veterkh opened this issue Mar 30, 2018 · 3 comments

Comments

@veterkh
Copy link

veterkh commented Mar 30, 2018

versions

  • SlimerJS: Innophi SlimerJS 1.0.0
  • CasperJS: 1.1.4
  • Firefox: 59.0.2
  • Operating system: Ubuntu 16.04

Steps to reproduce the issue

I'm using casperjs with slimerjs in order to retrieve response body, cause phantomjs didn't support it. I'm bit confused where to place captureContent property, and tried all the variants, but still can't get body of response in callback.

var casper = require('casper').create({
    pageSettings: {
        captureContent: [ '/.*/' ]
    },
    captureContent: [ '/.*/' ]
});

casper.start();
casper.captureContent = [ '/.*/' ];
casper.page.captureContent = [ '/.*/' ];

casper.thenOpen('https://google.com', function() {
    casper.on('resource.received', function onKeywordResponseReceived(resource) {
        if (resource.stage === 'end') {
            console.log(resource.url);
            console.log(JSON.stringify(resource));
            console.log(JSON.stringify(resource.body));
        }
    });
    
    this.wait(10000, function(){});
});

casper.run(function runCasper() {
    casper.exit();
});

Run it in the following way:

casperjs --engine=slimerjs downloadTest.js

Actual results:

https://ssl.gstatic.com/gb/js/sem_7e6c732dccf2b05588c7c99408d0d47f.js 
{"id":10,"url":"https://ssl.gstatic.com/gb/js/sem_7e6c732dccf2b05588c7c99408d0d47f.js","time":"2018-03-30T18:12:51.820Z","headers":[{"name":"accept-ranges","value":"bytes"},{"name":"vary","value":"Accept-Encoding, Origin"},{"name":"content-encoding","value":"gzip"},{"name":"content-type","value":"text/javascript"},{"name":"content-length","value":"20670"},{"name":"date","value":"Wed, 14 Mar 2018 07:03:49 GMT"},{"name":"expires","value":"Thu, 14 Mar 2019 07:03:49 GMT"},{"name":"last-modified","value":"Fri, 09 Mar 2018 04:15:00 GMT"},{"name":"x-content-type-options","value":"nosniff"},{"name":"server","value":"sffe"},{"name":"x-xss-protection","value":"1; mode=block"},{"name":"cache-control","value":"public, max-age=31536000"},{"name":"age","value":"1422542"},{"name":"alt-svc","value":"hq=\":443\"; ma=2592000; quic=51303432; quic=51303431; quic=51303339; quic=51303335,quic=\":443\"; ma=2592000; v=\"42,41,39,35\""},{"name":"X-Firefox-Spdy","value":"h2"}],"bodySize":56811,"contentType":"text/javascript","contentCharset":"","redirectURL":null,"stage":"end","status":200,"statusText":"OK","referrer":"https://www.google.com/","isFileDownloading":false,"body":"","httpVersion":{"major":2,"minor":0}} 
"" 

Expected results:

https://ssl.gstatic.com/gb/js/sem_7e6c732dccf2b05588c7c99408d0d47f.js 
{"id":10,"url":"https://ssl.gstatic.com/gb/js/sem_7e6c732dccf2b05588c7c99408d0d47f.js","time":"2018-03-30T18:12:51.820Z","headers":[{"name":"accept-ranges","value":"bytes"},{"name":"vary","value":"Accept-Encoding, Origin"},{"name":"content-encoding","value":"gzip"},{"name":"content-type","value":"text/javascript"},{"name":"content-length","value":"20670"},{"name":"date","value":"Wed, 14 Mar 2018 07:03:49 GMT"},{"name":"expires","value":"Thu, 14 Mar 2019 07:03:49 GMT"},{"name":"last-modified","value":"Fri, 09 Mar 2018 04:15:00 GMT"},{"name":"x-content-type-options","value":"nosniff"},{"name":"server","value":"sffe"},{"name":"x-xss-protection","value":"1; mode=block"},{"name":"cache-control","value":"public, max-age=31536000"},{"name":"age","value":"1422542"},{"name":"alt-svc","value":"hq=\":443\"; ma=2592000; quic=51303432; quic=51303431; quic=51303339; quic=51303335,quic=\":443\"; ma=2592000; v=\"42,41,39,35\""},{"name":"X-Firefox-Spdy","value":"h2"}],"bodySize":56811,"contentType":"text/javascript","contentCharset":"","redirectURL":null,"stage":"end","status":200,"statusText":"OK","referrer":"https://www.google.com/","isFileDownloading":false,"body":"BODY OF RESPONSE WITH JS FILE","httpVersion":{"major":2,"minor":0}} 
"BODY OF RESPONSE WITH JS FILE" 

Js files was given for example, in test workaround.

@veterkh
Copy link
Author

veterkh commented Mar 30, 2018

Parser eats the text, off course captureContent is set to [ '/.NO_SPACE*NO_SPACE/' ] just remove NO_SPACE

@veterkh
Copy link
Author

veterkh commented Apr 5, 2018

Ok, the reason of empty body was too stupid. Just need to remove single quotes , cause we need to bypass clear JS regex in array, like this: [ /./ ].
And by the way, proper way to set captureContent settings is the following:
casper.page.captureContent = [ /.
/ ]
and put it AFTER casper.start(); not obvious, really

@laurentj
Copy link
Owner

@veterkh Does your last message mean that your problem is fixed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants