JsJaws Internet access

[eljeffeg]

JsJaws tried to reach out to the internet for jquery-3.3.1.js. My environment limits this..

 File "/var/lib/assemblyline/.local/lib/python3.11/site-packages/assemblyline_v4_service/common/base.py", line 181, in handle_task
    self.execute(request)
  File "/opt/al_service/jsjaws.py", line 935, in execute
    file_path, file_content = self._handle_filtered_code(file_path, file_content)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/al_service/jsjaws.py", line 762, in _handle_filtered_code
    filtered_file_path, filtered_file_content, lib_path = self._extract_filtered_code(file_content)
                                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/al_service/jsjaws.py", line 4215, in _extract_filtered_code
    resp = get(lib_path, timeout=15)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/assemblyline/.local/lib/python3.11/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/assemblyline/.local/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/assemblyline/.local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/assemblyline/.local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/assemblyline/.local/lib/python3.11/site-packages/requests/adapters.py", line 698, in send
    raise SSLError(e, request=request)
SSLError: HTTPSConnectionPool(host='code.jquery.com', port=443): Max retries exceeded with url: /jquery-3.3.1.js

[eljeffeg]

I see this does have internet access. I confused external with internet. Do we have a list of domains that would need to be whitelisted for this to work? Perhaps it can default to an internal copy of the latest jquery if it can't access the internet.

[gdesmar]

The JsJaws service is looking at the value of docker_config.allow_internet_access before trying to fetch a file from the internet.
From reading the code, I can see that this configuration is different from allow_download_from_internet, but probably not explained clearly.
The allow_download_from_internet is a service variable that will impact the argument given to malwarejail and boxjs, two of the internal JsJaws tool, so that they don't try to go online to download content.
The docker_config.allow_internet_access is not a service variable, but will be found in the JsJaws configuration, under the container tab, by clicking on the Container image box that should say Internet access: [yes|no]. In there, you should be able to modify the value under Allow outside access. This configuration is having an impact on the deployment of the container in your cluster, and which network will be connected to the container.
The default value for the first is False, and the second is True. If you want a fully offline instance, you can set them both to False.
For the case at hand, the jquery download is made by our service directly, and not an internal tool. This is done to clean malicious file that are built by interweaving the malicious code in a well known benign file. We are bundling a few of them (lodash, d3, underscore, ...) but I assume that for jquery, there were too many, or they were releasing new versions too fast?
@cccs-jh I believe we could change the Dockerfile so that at build time, the buildhost would fetch a few of the latest versions of jquery and have them locally in the container. The code could check if the right version of jquery is there first, before trying to download, which would allow offline instances to do better?
I just looked at the d3 version (1.29.5) that is bundled in the service and it was released August 18, 2011. Do we know if it's still the version that malware authors are using?
Regarding a list of domains that are used there is only one for now: code.jquery.com.

[eljeffeg]

Thanks, I was a little confused as I thought internet meant defining it as external. Maybe the Service Management column could make a distinction for Internal Services that have internet access vs External Services. If it is an internal service that uses internet, it would be great to be able to see what sites it is allowed to access.