All extensions show as unsigned by the Extension Marketplace preventing auto installation in VSCode 1.9.4+

[angrycub]

Problem Statement

It seems that VS Code 1.94+ is not compatible with extensions hosted in the code-marketplace. They all have this signature warning (Screenshot 1) which prevents them from being installed in the standard way i.e. blue Install button, or automatically if you’ve enabled auto updates. You can still install via the cog wheel if you proceed passed the warning (Screenshots 2 & 3).

🖥️ Screenshots

Screenshot 1

Screenshot 2

Screenshot 3

Potentially related issues

• Extension .sigzip assets does not match zip format #922

  VS Code added a simple format verification for the signature asset for extension, it has to be a zip file and contain a .signature.p7s (the content does not matter), the signature file returned from open-vsx fails this verification resulting in any extension failing to install.

[janLo]

We've experienced the very same issue. The signature seems not to be contained in the actual VSXI package. Instead, the extensionquery-API provides it as a separate asset for a given version.

Our solution is to download it separately (we have a mirroring mechanism which uses the extensionquery-API to fetch the version information of the extensions and passes new version assets to code-marketplace add) and put it manually next to the extension in our Artifactory repository. The reverse proxy in front of the marketplace then mangles the manifest response via embedded Lua to inject the signature asset in the response.

This way, VS Code can download the signature and stops complaining.

[code-asher]

Bringing over my notes from #67:

I think we will need to implement https://github.com/filiptronicek/node-ovsx-sign in Go. We generate what we need when an extension is added, or on demand for existing extensions for backwards compatibility.

This should also allow adding your own signatures since it will only generate if one does not already exist.

[Kira-Pilot]

duplicated by https://github.com/coder/customers/issues/702

[p1r4t3-s4il0r]

Hello @janLo

Could you give me more information about have you download the signature ? I'm experiencing the same issue with code-server 1.91.1

Thanks.

[janLo]

@p1r4t3-s4il0r we have a downloader that does it all for us and a bit of infrastructure to use it on the other side.

This is the code that downloads a list of extensions and places them into artifactory:

Downloader

The method process_extensions gets a list of extension ids and fetches the latest version and handles the vsxi files and signatures.

import asyncio
import os
import tempfile

import orjson
import typing
import aiohttp
import logging
import urllib.parse
import concurrent.futures
import pprint
import subprocess


MARKETPLACE_URL = "https://marketplace.visualstudio.com/_apis/public/gallery/extensionquery?api-version=3.0-preview.1"

_log = logging.getLogger(__name__)

PLATFORMS = {"linux-x64", "web", "alpine-x64", "universal", "unkknown", "undefined"}


class ExtensionInfo(typing.NamedTuple):
    extension_id: str
    extension_name: str
    extension_publisher: str
    extension_version: str
    targetPlatform: str | None

    vsxi_url: str
    signature_url: str


def make_filter(extensions: list[str]) -> bytes:
    return orjson.dumps(
        {
            "filters": [
                {
                    "criteria": [{"filterType": 7, "value": ext} for ext in extensions],
                    "pageNumber": 1,
                    "pageSize": len(extensions),
                    "sortBy": 0,
                    "sortOrder": 0,
                }
            ],
            "flags": 147,
        }
    )


def _filter_pre_release(version: dict) -> bool:
    return next(
        iter(
            prop["value"] != "true"
            for prop in version["properties"]
            if prop["key"] == "Microsoft.VisualStudio.Code.PreRelease"
        ),
        True,
    )


class ManifestMetadata(typing.NamedTuple):
    extension_id: str
    extension_publisher: str
    extension_version: str


def _get_asset_file_url(files: list[dict[str, str]], asset_type: str) -> str | None:
    return next(
        iter((file["source"] for file in files if file["assetType"] == asset_type)),
        None,
    )


class PluginDownloader(object):
    def __init__(
        self,
        artifactory_url: str,
        artifactory_repo: str,
        artifactory_token: str,
        code_marketplace_bin: str,
    ):
        self._artifactory_url = artifactory_url
        self._artifactory_repo = artifactory_repo
        self._artifactory_token = artifactory_token
        self._marketplace_url = MARKETPLACE_URL
        self._code_marketplace_bin = code_marketplace_bin

        self._session: aiohttp.ClientSession | None = None
        self._pool = concurrent.futures.ThreadPoolExecutor(max_workers=4)

    @property
    def session(self) -> aiohttp.ClientSession:
        if self._session is None:
            self._session = aiohttp.ClientSession(trust_env=True)
        return self._session

    async def _fech_manifest_metadata(self, manifest_url) -> ManifestMetadata | None:
        async with self.session.get(manifest_url) as resp:
            if not resp.ok:
                _log.warning("Could not fetch manifest metadata: %s", manifest_url)
                return None

            data = orjson.loads(await resp.text())
            return ManifestMetadata(
                extension_id=data["name"],
                extension_publisher=data["publisher"],
                extension_version=data["version"],
            )

    async def _fetch_marketplace_info(
        self, extensions: list[str]
    ) -> typing.AsyncGenerator[ExtensionInfo, None]:
        version_data = []

        async with self.session.post(
            self._marketplace_url,
            headers={"Content-Type": "application/json"},
            data=make_filter(extensions),
        ) as resp:
            if not resp.ok:
                _log.error("Could not fetch marketplace info for %s", extensions)
            data = orjson.loads(await resp.text())

            for result in data.get("results", []):
                for extension in result.get("extensions", []):
                    publisher = extension["publisher"]
                    ext_id = (
                        f"{publisher["publisherName"]}.{extension['extensionName']}"
                    )
                    all_versions = extension.get("versions", [])
                    first_version = next(
                        iter(
                            version
                            for version in all_versions
                            if _filter_pre_release(version)
                        ),
                        None,
                    )
                    if first_version is None:
                        _log.warning("No version found for extension %s", ext_id)
                        continue

                    if "targetPlatform" in first_version:
                        try:
                            versions = [
                                version
                                for version in all_versions
                                if version["version"] == first_version["version"]
                                and (
                                    "targetPlatform" not in version
                                    or version["targetPlatform"] in PLATFORMS
                                )
                                and _filter_pre_release(version)
                            ]
                        except KeyError:
                            _log.exception(
                                "Version broken for %s?\n%s\n",
                                ext_id,
                                pprint.pformat(
                                    [
                                        v
                                        for v in all_versions
                                        if v["version"] == first_version["version"]
                                    ],
                                    indent=4,
                                ),
                            )
                            raise

                    else:
                        versions = [first_version]

                    _log.debug(
                        "Found %d versions for extension %s", len(versions), ext_id
                    )

                    version_data.extend(versions)

        _log.debug("Processing %d version information requests", len(version_data))

        pending: set[asyncio.Future[ExtensionInfo | None]] = set()
        for version in version_data:
            if len(pending) > 6:
                finished, pending = await asyncio.wait(
                    pending, return_when=asyncio.FIRST_COMPLETED
                )

                while finished:
                    res = await finished.pop()
                    if res is not None:
                        yield res

            pending.add(asyncio.ensure_future(self._fetch_version_info(version)))

        while pending:
            finished, pending = await asyncio.wait(
                pending, return_when=asyncio.FIRST_COMPLETED
            )

            while finished:
                res = await finished.pop()
                if res is not None:
                    yield res

    async def _fetch_version_info(
        self, version: dict[str, typing.Any]
    ) -> ExtensionInfo | None:
        files = version["files"]

        manifest = _get_asset_file_url(files, "Microsoft.VisualStudio.Code.Manifest")
        if manifest is None:
            _log.warning("No manifest url for extension")
            return None

        metadata = await self._fech_manifest_metadata(manifest)
        if metadata is None:
            return None

        ext_id = f"{metadata.extension_publisher}.{metadata.extension_id}"

        vsxi = _get_asset_file_url(files, "Microsoft.VisualStudio.Services.VSIXPackage")
        if vsxi is None:
            _log.warning("No vsxi package found for extension %s", ext_id)
            return None

        sig = _get_asset_file_url(
            files, "Microsoft.VisualStudio.Services.VsixSignature"
        )
        if sig is None:
            _log.warning("No signature found for extension %s", ext_id)
            return None

        return ExtensionInfo(
            extension_id=ext_id,
            extension_publisher=metadata.extension_publisher,
            extension_name=metadata.extension_id,
            extension_version=metadata.extension_version,
            targetPlatform=version.get("targetPlatform"),
            vsxi_url=vsxi,
            signature_url=sig,
        )

    def _artifactory_signature_url(self, extension: ExtensionInfo) -> str:
        version = (
            extension.extension_version
            if extension.targetPlatform in (None, "universal", "unknown", "undefined")
            else f"{extension.extension_version}@{extension.targetPlatform}"
        )
        return urllib.parse.urljoin(
            self._artifactory_url,
            "/".join(
                [
                    self._artifactory_repo,
                    extension.extension_publisher,
                    extension.extension_name,
                    version,
                    "signature",
                ]
            ),
        )

    def _artifactory_auth_header(self) -> dict[str, str]:
        return {"Authorization": f"Bearer {self._artifactory_token}"}

    async def _plugin_already_present(
        self,
        extension: ExtensionInfo,
    ) -> bool:
        async with self.session.head(
            url=self._artifactory_signature_url(extension),
            headers=self._artifactory_auth_header(),
        ) as resp:
            return 100 < resp.status < 400

    def _do_fetch_plugin(
        self, extension_url: str, description: str | None = None
    ) -> bool:
        proc = subprocess.Popen(
            [
                self._code_marketplace_bin,
                "add",
                extension_url,
                "--artifactory",
                self._artifactory_url,
                "--repo",
                self._artifactory_repo,
            ],
            env={"ARTIFACTORY_TOKEN": self._artifactory_token} | os.environ,
        )
        try:
            proc.wait(1800)
        except:  # noqa
            _log.warning(
                "Failed to fetch plugin %s (download did not finish)",
                f"{extension_url} ({description})"
                if description is not None
                else extension_url,
                exc_info=True,
            )
            return False
        if proc.returncode != 0:
            _log.warning("Failed to add extension %s", extension_url)
            return False
        return True

    def _fetch_manual_download(self, extension_url: str) -> bool:
        with tempfile.NamedTemporaryFile() as fn:
            curl_proc = subprocess.Popen(
                ["/usr/bin/curl", "-s", "-f", "-o", fn.name, extension_url],
                env=os.environ,
            )
            try:
                curl_proc.wait(240)
            except:  # noqa
                _log.warning(
                    "Failed to fetch plugin %s (download did not finish)",
                    extension_url,
                    exc_info=True,
                )
                return False

            if curl_proc.returncode != 0:
                _log.warning(
                    "Failed to fetch plugin %s (download failed)", extension_url
                )
                return False

            return self._do_fetch_plugin(fn.name, extension_url)

    async def _fetch_plugin(
        self,
        extension: ExtensionInfo,
    ) -> bool:
        _log.debug(
            "Fetch vsxi package for %s from %s",
            extension.extension_id,
            extension.vsxi_url,
        )

        loop = asyncio.get_running_loop()
        async with self.session.head(extension.vsxi_url) as resp:
            if not resp.ok:
                _log.warning(
                    "Failed to fetch extension info for %s", extension.extension_id
                )
                return False

            if (
                "Content-Length" in resp.headers
                and int(resp.headers["Content-Length"]) > 90 * 1024 * 1024
            ):
                return await loop.run_in_executor(
                    self._pool, self._fetch_manual_download, extension.vsxi_url
                )

        return await loop.run_in_executor(
            self._pool, self._do_fetch_plugin, extension.vsxi_url
        )

    async def _fetch_signature(self, extension: ExtensionInfo) -> bool:
        async with self.session.get(extension.signature_url) as sig_resp:
            if not sig_resp.ok:
                _log.warning(
                    "Failed to fetch signature for extension %s", extension.extension_id
                )
                return False

            data = await sig_resp.read()

        async with self.session.put(
            self._artifactory_signature_url(extension),
            data=data,
            headers=self._artifactory_auth_header(),
        ) as upload_resp:
            if not upload_resp.ok:
                _log.warning(
                    "Failed to upload signature for extension %s",
                    extension.extension_id,
                )
                return False

            return True

    async def _process_extension(self, extension: ExtensionInfo):
        if await self._plugin_already_present(extension):
            _log.debug("Extension %s already present", extension.extension_id)
            return False

        if await self._fetch_plugin(extension):
            if await self._fetch_signature(extension):
                _log.info("Extension %s downloaded", extension.extension_id)
                return True

        return True

    async def _process_batch(
        self, extensions: list[str]
    ) -> typing.AsyncGenerator[asyncio.Task, None]:
        _log.info("Fetching info for %d extensions", len(extensions))

        dl_cont = 0

        async for info in self._fetch_marketplace_info(extensions):
            if await self._plugin_already_present(info):
                _log.debug(
                    "Extension %s (%s, %s) already present",
                    info.extension_id,
                    info.extension_version,
                    info.targetPlatform,
                )
                continue

            dl_cont += 1
            yield asyncio.ensure_future(self._process_extension(info))

        _log.info(
            "Finished processing information for %d extensions, %d artifacts will be downloaded",
            len(extensions),
            dl_cont,
        )

    async def process_extensions(self, extensions: list[str]):
        chunk_size = 10
        chunks = [
            extensions[i : i + chunk_size]
            for i in range(0, len(extensions), chunk_size)
        ]

        _log.info("Processing %d extensions in %d chunks", len(extensions), len(chunks))

        tasks = []

        for chunk in chunks:
            async for task in self._process_batch(chunk):
                tasks.append(task)

        res = await asyncio.gather(*tasks)

        _log.info(
            "Downloaded %d extensions, %d failed",
            len(res),
            len([x for x in res if not x]),
        )

        await self.session.close()
        self._session = None

And then I have a bit of LUA magic in our reverse proxy in front of the code-marketplace, that adds the signature:

Nginx config

init_by_lua_block { require "cjson" }

server {
    listen 443 ssl;

    server_name marketplace.visualstudio.com;

    ssl_certificate           /etc/nginx/certs/marketplace.visualstudio.com.crt;
    ssl_certificate_key       /etc/nginx/certs/marketplace.visualstudio.com.key;
    ssl_dhparam               /etc/nginx/certs/dhparam.pem;


    ssl_session_cache  builtin:1000  shared:SSL:10m;
    ssl_protocols  TLSv1 TLSv1.1 TLSv1.2;

    # ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
    ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
    ssl_prefer_server_ciphers on;



    # proxy settings
    # proxy_set_header   Host                 $http_host;
    proxy_set_header   X-Forwarded-Host $http_host;
    proxy_set_header   X-Real-IP            $remote_addr;
    proxy_set_header   X-Forwarded-Proto    $scheme;
    proxy_set_header   X-Forwarded-For      $remote_addr;
    proxy_redirect     off;

    set $config_name "marketplace.visualstudio.com";


    location /_apis/public/gallery/ {

        location ~ ^/.*/extensionquery$ {
            if ($request_method = POST ) {
                content_by_lua_block {
                    local cjson = require "cjson"
                    local dest = ngx.re.sub(ngx.var.uri, "^/_apis/public/gallery/?(.*)$", "/api/$1", "o")
                    ngx.req.read_body()
                    local capt = ngx.location.capture(dest, {method = ngx.HTTP_POST, body =ngx.req.get_body_data()})
                    local content = cjson.decode(capt.body)

                    for r_idx, res in ipairs(content.results) do
                        for e_idx, ext in ipairs(res.extensions) do
                            for v_idx, ver in ipairs(ext.versions) do
                                table.insert(ver.files, {assetType = "Microsoft.VisualStudio.Services.VsixSignature", source = "https://artifactory.example.com/artifactory/code-marketplace-generic/" .. ext.publisher.publisherId .. "/".. ext.extensionName .."/" .. ver.version .. "/signature"})
                            end
                        end
                    end

                    local final = cjson.encode(content)

                    for k, v in pairs(capt.header) do
                        ngx.header[k] = v
                    end
                    ngx.header.content_length = #final

                    ngx.print(final)
                }
            }
            rewrite ^/_apis/public/gallery/?(.*)$ /api/$1 break;
            proxy_pass https://code-marketplace.example.com;
        }
        rewrite ^/_apis/public/gallery/?(.*)$ /api/$1 break;
        proxy_pass https://code-marketplace.example.com;
    }

    location /api/ {
            proxy_pass https://code-marketplace.example.com;
    }

    location /items {
            rewrite ^/items/?(.*)$ /item/$1 break;
            proxy_pass https://code-marketplace.example.com;
    }

    location / {
        location ~ ^/assets/(.*)/Microsoft.VisualStudio.Services.VsixSignature$ {
            rewrite ^/assets/(.*)/Microsoft.VisualStudio.Services.VsixSignature$ https://artifactory.example.com/artifactory/code-marketplace-generic/$1/signature break;
            proxy_pass https://artifactory.example.com;
        }
        proxy_pass https://code-marketplace.example.com;
    }

    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }
}

[DataOps7]

Just finished setting up the code-marketplace with Artifactory and getting the same error.
In my case, clicking Install in the cog wheel generates an error:

Unable to verify the first certificate

[janLo]

That might be because vscode does not trust the server certificate of the marketplace service

[DataOps7]

I am using coder with code-server, the pod running the coder workspace is trusting the domain,
Is there a different certificate configuration for code-server?

[janLo]

With vscode desktop I had to put the ca cert of the ca that issued the Code-Server certificate to the chrome trust store.

(Separate trust stores for different software instances are certainly an invention from hell. They're just there to ruin your day 😉)

[DataOps7]

That sounds terrible!
@code-asher Do you know how to configure code-server to trust the code-marketplace?
I'm working in an air-gapped environment and want to use code-server with code-marketplace and Artifactory.

[code-asher]

If you mean extension signing, there is no way to do that currently as far as I know aside from the workarounds above. It needs to be implemented here, and disabling signature verification in code-server appears to have no effect from what I read (I have not tried it myself though, so maybe it does work).

If you mean trust as in a TLS certificate, then likely you need to add your CA to both the local machine (some requests are made from the browser) and the remote machine (other requests are made from the server).

Edit: oh I missed the conversation above, you definitely mean the TLS cert. Yeah you have to trust your CA on both machines.

[DataOps7]

That's exactly what I was thinking, both the client machine and the code-server pod trust the CA (curl works just fine with HTTPS)
But still I'm getting the Unable to verify the first certificate error when trying to install from the cog wheel install button.
I have uploaded the extension to Artifactory using the the code-marketplace CLI and a VSIX downloaded from Microsoft's store.
Any other ideas?

Edit 1

When opening the dev tools (F12) on the code-server browser it looks like all requests to the code-marketplace domain are HTTPS and work well (200 OK), for example, fetching the README.md. But the error looks internal to VSCode, these are the logs in the VSCode output console:

Error: unable to verify the first certificate

at TLSSocket.onConnectSecure  (node:_tls_wrap:1076:8)
...

Edit 2

These are the logs in the code-server log file:

Getting Manifest... <extension-name>
#1 <https-marketplace-url>/assets/<extension-publisher>/<extension-name>/<version>/Microsoft.VisualStudio.Code.Manifest - error GET unable to verify the first certificate

Tried compiling my own extension and pushing to the registry and got the same error.
When using CURL on the same URL printed, from the pod running the code-server, it works fine, and return a redirect to package.json