Building plugins with docker results in a successful build with no plugins installed

[bendemeyer]

I'm not sure if this issue is more appropriate here or in answer-plugins, since it relates to plugins but seems like it might be an issue with answer build generally. Anyway, I'm putting it in both and I can delete one if it's in the wrong place.

I'm trying to get answer running with some plugins, using the docker build process described here: https://answer.apache.org/docs/plugins/#docker-build

When I do this the answer build step completes successfully, but when I run answer plugin with the new binary I get an empty output. If I run the server and access the admin area, it shows no plugins are installed.

To troubleshoot, I set up a builder image with this basic Dockerfile:

FROM apache/answer AS answer-source

FROM golang:1.22-alpine AS golang-builder

COPY --from=answer-source /usr/bin/answer /usr/bin/answer

RUN apk --no-cache add build-base git bash nodejs npm go
RUN npm install -g [email protected]

And within that I run

answer build --with github.com/apache/answer-plugins/render-markdown-codehighlight --output /usr/bin/answer_new

That process runs, compiles, and appears to complete successfully. However, the plugins aren't actually installed in the output binary.

/go # answer_new plugin
/go # 

I can also see that the newly built binary is a little smaller than the original:

/usr/bin # ls -lah | grep answer
-rwxr-xr-x    1 root     root       74.3M Nov 22 06:56 answer
-rwxr-xr-x    1 root     root       68.5M Jan 23 15:05 answer_new

I'm running this on an M1 mac, and I've replicated this result in both arm & amd64 container architectures. I've also tried upgrading the build container to the latest version of pnpm to see if that made a difference, and it did not.

This is the (mostly) complete console output I get when I run answer build, with some of the big-long-lists-of-things truncated for the sake of making it more readable:

/go # answer build --with github.com/apache/answer-plugins/render-markdown-codehighlight --output /usr/bin/answer_new
try to build a new answer with plugins:
github.com/apache/answer-plugins/render-markdown-codehighlight
[build] build dir: /go/answer_build4202246818
[go mod tidy]
go: finding module for package github.com/apache/incubator-answer/cmd
go: finding module for package github.com/apache/answer-plugins/render-markdown-codehighlight
go: downloading github.com/apache/answer-plugins v1.2.1
go: downloading github.com/apache/answer-plugins/render-markdown-codehighlight v1.0.0
go: downloading github.com/apache/incubator-answer v1.4.1
go: found github.com/apache/answer-plugins/render-markdown-codehighlight in github.com/apache/answer-plugins/render-markdown-codehighlight v1.0.0
go: found github.com/apache/incubator-answer/cmd in github.com/apache/incubator-answer v1.4.1
go: downloading github.com/apache/answer-plugins/util v1.0.3-0.20250107030257-cf94ebc70954
go: downloading github.com/apache/answer v1.4.2-RC1.0.20250107023923-061894735091
go: downloading github.com/gin-gonic/gin v1.10.0
<TRUNCATED: "go: downloading {something}">
[go mod vendor]
[go list -mod=mod -m -f {{.Dir}} github.com/apache/incubator-answer]
try to copy dir from /go/answer_build4202246818/vendor/github.com/apache/incubator-answer-plugins to /go/answer_build4202246818/vendor/github.com/apache/incubator-answer/ui/src/plugins
[pnpm pre-install]

> [email protected] pre-install /go/answer_build4202246818/vendor/github.com/apache/incubator-answer/ui
> node ./scripts/importPlugins.js && pnpm install && node ./scripts/preinstall.js 

Lockfile is up to date, resolution step is skipped
Packages: +1591
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

   ╭──────────────────────────────────────────────────────────────────╮
   │                                                                  │
   │                Update available! 8.9.2 → 9.15.4.                 │
   │   Changelog: https://github.com/pnpm/pnpm/releases/tag/v9.15.4   │
   │                Run "pnpm add -g pnpm" to update.                 │
   │                                                                  │
   │      Follow @pnpmjs for updates: https://twitter.com/pnpmjs      │
   │                                                                  │
   ╰──────────────────────────────────────────────────────────────────╯

Downloading registry.npmjs.org/typescript/4.9.5: 11.62 MB/11.62 MB, done
Progress: resolved 1591, reused 0, downloaded 1589, added 1591, done
node_modules/.pnpm/[email protected]/node_modules/core-js-pure: Running postinstall script, done in 34ms
node_modules/.pnpm/[email protected]/node_modules/core-js: Running postinstall script, done in 32ms

dependencies:
+ @codemirror/lang-markdown 6.2.5
+ @codemirror/language-data 6.5.1
+ @codemirror/state 6.4.1
+ @codemirror/view 6.26.3
+ axios 1.7.7
+ bootstrap 5.3.2
<TRUNCATED: "+ {dependency}">

devDependencies:
+ @commitlint/cli 17.1.2
+ @commitlint/config-conventional 17.2.0
+ @fullhuman/postcss-purgecss 4.1.3
+ @testing-library/dom 8.18.1
+ @testing-library/jest-dom 4.2.4
<TRUNCATED: "+ {dependency}">

. prepare$ pnpm build:packages
│ > [email protected] build:packages /go/answer_build4202246818/vendor/github.com/apache/incubator-answer/ui
│ > pnpm -r --filter=./src/plugins/* run build
│ No projects matched the filters in "/go/answer_build4202246818/vendor/github.com/apache/incubator-answer/ui"
└─ Done in 555ms
Done in 9.9s
[pnpm build]

> [email protected] build /go/answer_build4202246818/vendor/github.com/apache/incubator-answer/ui
> node ./scripts/env.js && react-app-rewired build

Creating an optimized production build...
Browserslist: caniuse-lite is outdated. Please run:
  npx update-browserslist-db@latest
  Why you should do it regularly: https://github.com/browserslist/update-db#readme
Browserslist: caniuse-lite is outdated. Please run:
  npx update-browserslist-db@latest
  Why you should do it regularly: https://github.com/browserslist/update-db#readme
Compiled with warnings.

Module not found: Error: Can't resolve 'buffer' in '/go/answer_build4202246818/vendor/github.com/apache/incubator-answer/ui/node_modules/.pnpm/[email protected]/node_modules/js-yaml/lib/js-yaml/type'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "buffer": require.resolve("buffer/") }'
	- install 'buffer'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "buffer": false }

Search for the keywords to learn more about each warning.
To ignore, add // eslint-disable-next-line to the line before.

File sizes after gzip:

  102.29 kB  build/static/js/codemirror.8ce63365.js
  70.83 kB   build/static/js/lezer.5966669a.js
  65.01 kB   build/static/js/chunk-nodesInitial.2237608c.chunk.js
  49.13 kB   build/static/js/main.dadb9922.js
  48.88 kB   build/static/js/chunk-mix2.e38b5d14.chunk.js
  <TRUNCATED: "{size} {filename}">

The project was built assuming it is hosted at /.
You can control this with the homepage field in your package.json.

The build folder is ready to be deployed.
You may serve it with a static server:

  npm install -g serve
  serve -s build

Find out more about deployment here:

  https://cra.link/deployment

try to merge i18n files
i18n dir:  /go/answer_build4202246818/vendor/github.com/apache/answer-plugins/render-markdown-codehighlight/i18n
[go build -ldflags -X github.com/apache/incubator-answer/cmd.Version=1.4.1 -X github.com/apache/incubator-answer/cmd.Revision=c2ad8c6 -X github.com/apache/incubator-answer/cmd.Time=1732256436 -o /usr/bin/answer_new .]
build new answer successfully /usr/bin/answer_new

[LinkinStars]

@bendemeyer Due to recent adjustments to the repo name during the graduation process, this issue has arisen. There are two possible solutions: if version 1.4.1 is being used, then lower-version plugins must be specified. If the latest version 1.4.2 is being used, the latest version plugins can be utilized.

Use the latest version to build. For docker

FROM apache/answer:1.4.2-RC2 as answer-source

........

If you want to use the old version, you need to choose the plugin's old version specifically and use the old repo name (incubator-answer-plugins).

FROM apache/answer as answer-builder

........

RUN answer build \
    --with github.com/apache/incubator-answer-plugins/[email protected] \
    --output /usr/bin/new_answer

[bendemeyer]

@LinkinStars Thanks for the explanation!

One of the goals I'm aiming for in the way I set up my instance is to minimize long-term maintenance overhead, including things like explicit version management. Ideally I'd like to be able to update to the latest release just by re-running the build with my existing Dockerfile.

Do you think it would be possible to establish a versioning/tagging pattern between the apache/answer and apache/answer-plugin projects that would guarantee (or at least as close to that as is possible in software) compatibility between dynamically referenced objects? Maybe something like @stable and @latest tags?

I don't know much about these projects yet, so I'm not sure how much overhead would go into implementing & maintaining something like that. If it's doable, I'd be happy to contribute if I can get some pointers on where to start.

[bendemeyer]

@LinkinStars One follow-up note on this as well, the second approach you outlined (using the older plugin versions with [email protected]) worked exactly as described, but the first (using [email protected] with the newest plugins) needed some tweaking.

In a golang-builder container based on the apache/answer:1.4.2-RC2 image, when I tried to run this build command:

answer build --with github.com/apache/answer-plugins/render-markdown-codehighlight --output /usr/bin/answer_new

It errored with the following result:

try to build a new answer with plugins:
github.com/apache/answer-plugins/render-markdown-codehighlight
[build] build dir: /go/answer_build4074786202
[go mod tidy]
go: finding module for package github.com/apache/answer/cmd
go: finding module for package github.com/apache/answer-plugins/render-markdown-codehighlight
go: found github.com/apache/answer-plugins/render-markdown-codehighlight in github.com/apache/answer-plugins/render-markdown-codehighlight v1.0.0
go: found github.com/apache/answer/cmd in github.com/apache/answer v1.4.1
go: answer imports
        github.com/apache/answer/cmd: github.com/apache/[email protected]: parsing go.mod:
        module declares its path as: github.com/apache/incubator-answer
                but was required as: github.com/apache/answer

I noticed it was still trying to pull [email protected] as its starting point and figured that was the cause of the problem. I found this comment in the build script which suggests this is probably expected behavior since 1.4.2 hasn't actually been released yet:

// If user specify a module replacement, use it. Otherwise, use the latest version.

I was then able to specify a module replacement using the ANSWER_MODULE environment variable, and that ultimately did the trick. I updated my Dockerfile like so and it works now:

FROM apache/answer:1.4.2-RC2 AS answer-source

FROM golang:1.22-alpine AS golang-builder

ENV ANSWER_MODULE="github.com/apache/[email protected]"

......

[LinkinStars]

@bendemeyer Great!

The issue is due to a recent repository name change. The name was incubator-answer modified to answer since the project had not graduated. Thank you for your efforts and we apologize for the unnecessary trouble this issue has caused you. After the name is confirmed, this problem will not occur again.

Do you think it would be possible to establish a versioning/tagging pattern between the apache/answer and apache/answer-plugin projects that would guarantee (or at least as close to that as is possible in software) compatibility between dynamically referenced objects? Maybe something like @stable and @latest tags?

For plugins, we support specifying the latest version of the plugin. For example:
github.com/apache/answer-plugins/render-markdown-codehighlight@latest

Then for you to mention establishing a correlation about the version between the two is a good idea. Maybe in the plugin, we can set a minimum version of Answer to be supported. Because usually, higher versions are compatible with lower versions. This part needs to be considered more. 🤔

[bendemeyer]

Maybe in the plugin, we can set a minimum version of Answer to be supported.

Something like this seems like a great idea to me! If the plugin "releases" can hold metadata about which versions of core answer they support, then the answer build process could potentially account for that, and install the latest version of each requested plugin that's compatible with whichever version of answer that's being built.

Another possible change that occurred to me is about the default version of answer that gets built if ANSWER_MODULE isn't defined. Would it make more sense to have it default to the same version as the current binary, as opposed to the latest release? The advantage to that change would be simplifying the docker build process, as whichever container you start with is the one you'll build. The biggest disadvantage I see is that it would make upgrading an existing installation in-place a little more complicated.

After poking around in the build code, I think I have a pretty good idea of how that second change might be implemented. If it would be desirable, I can test it out and open a PR.