[Bug] README is not correctly shown on verdaccio using `yarn npm publish`

[ikatyang]

Describe the bug

README is not correctly shown on self-hosted verdaccio using yarn npm publish.

To Reproduce

(require docker to launch verdaccio so as to reproduce)

Reproduction

mkdir yarn-verdaccio-readme
cd yarn-verdaccio-readme
yarn set version berry
echo '
npmPublishRegistry: http://localhost:4873
unsafeHttpWhitelist: [localhost]
' >> .yarnrc.yml
echo '
{
  "name": "yarn-verdaccio-readme",
  "version": "0.1.0"
}
' > package.json
echo '# yarn-verdaccio-readme' > README.md
yarn
docker run --rm -p 4873:4873 -d verdaccio/verdaccio:4.8.1
echo 'input some random username/password'
yarn npm login --publish
yarn npm publish
echo 'open http://localhost:4873/-/web/detail/yarn-verdaccio-readme then you will see "ERROR: No README data found!".'

Screenshots

$ yarn npm publish
➤ YN0000: README.md
➤ YN0000: package.json
➤ YN0000: Package archive published
➤ YN0000: Done in 0.91s

Environment

• Yarn version 2.1.1

Additional context

• README.md is uploaded correctly.
• No issue with npm publish.

[arcanis]

It's difficult to say since the npm protocols aren't documented, but I'd tend to say that it's probably a verdaccio issue, since the README are in the archives.

[ikatyang]

I found the issue is that the readme field is missing in the publish body:

berry/packages/plugin-npm-cli/sources/commands/npm/publish.ts

Lines 164 to 199 in bedc0cf

	return {
	_id: name,
	_attachments: {
	[tarballName]: {
	[`content_type`]: `application/octet-stream`,
	data: buffer.toString(`base64`),
	length: buffer.length,
	},
	},
	name,
	access,
	[`dist-tags`]: {
	[tag]: version,
	},
	versions: {
	[version]: {
	...raw,
	_id: `${name}@${version}`,
	name,
	version,
	dist: {
	shasum,
	integrity,
	// the npm registry requires a tarball path, but it seems useless 🤷
	tarball: tarballURL.toString(),
	},
	},
	},
	};

Adding the readme field can fix the issue:

 return {
+  readme: "# yarn-verdaccio-readme\n",
   _id: name, 
   _attachments: { 

I'm not sure if this should be fixed on the verdaccio side or the yarn side, but I'll report this to the verdaccio repo as well.

[juanpicado]

I found the issue is that the readme field is missing in the publish body:

berry/packages/plugin-npm-cli/sources/commands/npm/publish.ts

Lines 164 to 199 in bedc0cf

	return {
	_id: name,
	_attachments: {
	[tarballName]: {
	[`content_type`]: `application/octet-stream`,
	data: buffer.toString(`base64`),
	length: buffer.length,
	},
	},
	name,
	access,
	[`dist-tags`]: {
	[tag]: version,
	},
	versions: {
	[version]: {
	...raw,
	_id: `${name}@${version}`,
	name,
	version,
	dist: {
	shasum,
	integrity,
	// the npm registry requires a tarball path, but it seems useless 🤷
	tarball: tarballURL.toString(),
	},
	},
	},
	};

Adding the readme field can fix the issue:

 return {
+  readme: "# yarn-verdaccio-readme\n",
   _id: name, 
   _attachments: { 

I'm not sure if this should be fixed on the verdaccio side or the yarn side, but I'll report this to the verdaccio repo as well.

Yeah, the readme comes as string as one more element of the body, if is not there it is just ignored. The readme prop is missing in the makePublishBody function. I'd like to fix it 🙃 but need bit of help how to read the README file properly in order to include it into the body.

[davidroeca]

Just stumbled across this issue myself. Unfamiliar with why it's not working like yarn 1 or npm, but I did do some digging into some potentially related issues:

yarnpkg/website#1054
https://github.com/npm/libnpmpublish/issues/12

Looks like libnpmpublish references a readme field in the manifest, which is nowhere to be seen in @yarnpkg/core's manifest nor the referenced publish body in this issue. Looks like readmeFilename is another field that could be provided as well.

EDIT:
Confirmed that manifests that were published via lerna + yarn 1 contain readmeFilename, while manifests published via yarn berry contain neither readme nor readmeFilename

[yarnbot]

Hi! 👋

This issue looks stale, and doesn't feature the reproducible label - which implies that you didn't provide a working reproduction using Sherlock. As a result, it'll be closed in a few days unless a maintainer explicitly vouches for it or you edit your first post to include a formal reproduction (you can use the playground for that).

Note that we require Sherlock reproductions for long-lived issues (rather than standalone git repositories or similar) because we're a small team. Sherlock gives us the ability to check which bugs are still affecting the master branch at any given point, and decreases the amount of code we need to run on our own machines (thus leading to faster bug resolutions). It helps us help you! 😃

If you absolutely cannot reproduce a bug on Sherlock (for example because it's a Windows-only issue), a maintainer will have to manually add the upholded label. Thanks for helping us triaging our repository! 🌟

[ikatyang]

Can someone in the Yarn team add the upholded label to this issue? This is definitely a bug, though I'm not sure if this should be fixed on the yarn side or on the verdaccio side, but since there is no official response after I found that the issue is caused by the missing readme field and the verdaccio maintainer would like to fix this issue and asked for the help, I'd say this issue is still active.

[juanpicado]

It's difficult to say since the npm protocols aren't documented, but I'd tend to say that it's probably a verdaccio issue, since the README are in the archives.

The thing is verdaccio does not unpack tarballs by design, at least the default storage, nexus does not care since has no UI, artifactory I'd guess the same (at least the last time I used it) and npmjs publich I think use the readme file. Verdaccio relies on the readme property which comes with the packument on publish. Something new is npm7 doe snot send that property in the root body anymore verdaccio/verdaccio#1905 (comment), only in the version is being published, same that pnpm does. Maybe for consistency we could do the same here.

I'd be willing to fix it if you like the idea and I get some instructions how to do it, I already checked the code but I don't know how to start.

[artola]

@arcanis I got into this issue today, we publish using Yarn v3 to Azure without problem to see the README but the same against Verdaccio does not work. Could you please give us some hint on this topic? Surely @juanpicado is the go to person on Verdaccio side.

[juanpicado]

Just a small notes from my side, pnpm had the same issue recently on v6 which was fixed a few weeks ago https://github.com/pnpm/pnpm/releases/tag/v6.24.1 pnpm/pnpm#4117 . From verdaccio side, expects the body to contain a readme property with a string of the readme content. Also checking npm8 still sends the readme https://github.com/npm/read-package-json/blob/main/read-json.js#L295