Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate documentation page with JSDoc #2

Closed
11 tasks
eulertour opened this issue Dec 15, 2023 · 9 comments
Closed
11 tasks

Generate documentation page with JSDoc #2

eulertour opened this issue Dec 15, 2023 · 9 comments

Comments

@eulertour
Copy link
Owner

eulertour commented Dec 15, 2023

Use JSDoc to generate a documentation page in svelte so that it can be added to the documentation page here. The goal is to have something similar to the one here. The webpage should:

  • Have a menu on the left that links to all pages
  • Contain 1-2 freeform pages that aren't generated from a template (placeholder text)
  • Include a page for each class within
  • Each page should have
    • The name of the class
    • A summary taken from the docstring (placeholder text)
    • An enumeration of the class arguments and types

Don't worry about the content of the website or complex styling, the main goal is just to be able to generate the site with templated pages for the classes. I'll also want a walkthrough on the tools used to generate the website and how I can extend/customize it with new classes or default values.

@niktek
Copy link

niktek commented Dec 18, 2023

Status update:

So the two options of JSDoc and Sveld for getting the docs stuff out quickly and easily are not going to work in this situation. JSdoc is only going to work on .js files or TS that has been transpiled like you would get in a Vite/esbuild pipeline and likewise Sveld runs as a scanner of Svelte components which is not really applicable in this pure TS environment.

So the obvious route forward is to look into TSDoc with it's API extractor https://api-extractor.com/pages/overview/intro/ (the doc model) and the typedoc project https://typedoc.org - both of which would give us a json summary of the classes/doc comments for displaying in the pages.

I've setup both of them and they both fail. They start failing on:

build/bundle.d.ts:523:86 - error TS2304: Cannot find name 'WidthSetupConfig'.

523     constructor(UserScene: Class<StudioScene>, canvasRef: HTMLCanvasElement, config: WidthSetupConfig | HeightSetupConfig | undefined);
                                                                                         ~~~~~~~~~~~~~~~~

build/bundle.d.ts:523:105 - error TS2304: Cannot find name 'HeightSetupConfig'.

523     constructor(UserScene: Class<StudioScene>, canvasRef: HTMLCanvasElement, config: WidthSetupConfig | HeightSetupConfig | undefined);

And then continues with a lot more errors. Basically the @ts-nocheck might stop things being annoying during dev, but the errors it is reporting are real and need to be solved if we want to use any of the TS tooling to assist us.

I don't know if this is a quick fix where fixing 1 or 2 references/types or something like that solves the bulk of the 27 issues reported in geometry.ts :
image

So for now I'm going to pause on looking any further into this until I get some feedback on the TS issues.

@niktek
Copy link

niktek commented Dec 20, 2023

Hi,

Just pushed the feat/doc-generation branch as a status update.

You can run <package manager> api-extractor run which will generate /temp/studio.api.json
The next step will be to build out the genDocs script which uses api-extractor-model to summarise the generated .json file to create the output needed for the docsite and delete the intermediate files/folders that api-extractor has generated.

I had hoped that it would be able to do parsing with just a file or two, but because it is also doing full type checking, it has to consume all the referenced files, so it encountered some more TS issues. I've made some modifications to the source files and annotated them with TODO: to show where those changes are - as well as a hackish change on the bundle.d.ts imports that will get overwritten the next time the build script is run.

As another test I also added typedoc (<package manager> typedoc build/bundle.d.ts) to see if was a bit more lenient on the generation, but it also had issues and doesn't even get to the generated output stage.

These changes are purely to get the api extraction working so that I can use the outputted .json file. They are not the changes that would actually be needed for the project. When you get a chance, it would probably be best to bring a copy of /src/* from the main branch over the version in this branch and resolve the TS issues.

Cheers,
Nik

@ghostdevv
Copy link
Collaborator

@niktek
Copy link

niktek commented Dec 21, 2023

Hey @ghostdevv !

Yeah that looks great for the next step (how it has zero stars is beyond me), but I think all the tools are kind of stuck till the TS issues are resolved.

~/code/studio.js bun extractinator extract src/ temp/
{
  dts_path: '/Users/nik/code/studio.js/.extractinator/dts-1703126283078/MeshLineGeometry.d.ts',
  path: '/Users/nik/code/studio.js/src/MeshLine/MeshLineGeometry.ts'
}
file:///Users/nik/code/studio.js/node_modules/extractinator/dist/cli.js:345
      throw new Error(`Unable to find dts path for "${input_path}"`);
            ^

Error: Unable to find dts path for "/Users/nik/code/studio.js/src/MeshLine/MeshLineGeometry.ts"
    at emit_dts (file:///Users/nik/code/studio.js/node_modules/extractinator/dist/cli.js:345:13)
    at async extractinator (file:///Users/nik/code/studio.js/node_modules/extractinator/dist/cli.js:362:15)
    at async file:///Users/nik/code/studio.js/node_modules/extractinator/dist/cli.js:419:27

Node.js v21.4.0
error: "extractinator" exited with code 1 (SIGHUP)

@eulertour
Copy link
Owner Author

I just pushed fixes for the TS errors and circular imports. api-extractor still seems to just fail outright:

> @eulertour/[email protected] apiExtract
> api-extractor run --local --verbose


api-extractor 7.39.0  - https://api-extractor.com/

Using configuration from ./config/api-extractor.json
Analysis will use the bundled TypeScript version 5.3.3

ERROR: Internal Error: Unable to analyze the export "SceneController" in
/Users/devinneal/GitHub/studio.js/build/bundle.d.ts
You have encountered a software defect. Please consider reporting the issue to the maintainers of this application.

And the gendocs.ts script seems to dislike something about the files in dist/, though I don't know how those were created/how to recreate them.

Still, it feels good for those stubborn TS errors to be gone from VsCode. Let me know if there's more I need to do to unblock you.

@niktek
Copy link

niktek commented Dec 21, 2023

So I just want to preface this update, this is turning about to be quite a different approach than what we took with the Skeleton docs where we basically used a vite plugin to extract the docs and we were also extracting from svelte components. Having said that - this is still relevant to the stuff I'm working on and I'm happy to keep exploring this.

Thanks for getting the TS issues fixed! There is still something going on with the d.ts being generated into build/bundle.d.ts - the latest commit has the slight correction in it, but lets walk through what's happening.

Running bun run build to generate the bundle.d.ts file will show some errors on lines 787-788 where it is trying to export things unnecessarily because they have already been exported via the Scene namespace. Changing those lines to be:

export { Geometry, Animation, Text, Scene, THREE, Utils, Diagram, Constants, setCameraDimensions, setCanvasViewport };
//export type { StudioScene, AnimationRepresentation };

Will now allow api-extractor run to go ahead and do it's thing: the main one of interest being the DocModel (/temp/studio.api.json). I have temporarily left on it's d.ts rollup functionality which is responsible for the /dist/studio.d.ts file for comparisons sake - it is differently formatted, but it's hard to say whether it is for better or worse spec wise.

At this point I recommend clear ing the console to make it easier to see. We can now run bun genDocs which ingests the /temp/studio.api.json docmodel via @microsoft/api-extractor-model where I have got it displaying the full APIEntity hierarchy, types and nested members. To save us all from console spam hell, I have adjusted it to only show the inner data available to us for the first occurrence of each type that is found, so if you scroll towards the top you will see the json outputs interspersed.

At the top of the console you will see that I've added some dummy TSDoc annotations to the Animation constructor to see how they would come through. And whilst it comes through in the docComment field - none of the tags have been split out or presented in any useful manner.

@ghostdevv I noticed that Extractinator uses the @microsoft/tsdoc and @micrsoft/tsdoc-config packages directly, which I had avoided due to the probably misleading blurb from Microsoft. Are you able to shed any light on whether they get a better representation of the Docs themselves ?

I've also stepped back and just tried to find any other tools that are oriented towards this problem and haven't really turned up much. But I'm completely open to any suggestions of other routes to look into - but my inclination is that sticking close to the TSDoc packages makes the most sense.

Tomorrow i'll continue to explore the tsdoc packages as well and check out how Extractinator works too.

@eulertour
Copy link
Owner Author

I'm glad you were able to make some more progress! And that you got me to start taking those type errors more seriously 😅.

You both have more experience with this than I do, but I'm getting the feeling that this task is for some reason more difficult with this codebase than it is on average. Does it seem that way to you? And if so, do you have any idea what might be causing it?

When I started writing this I mostly just picked whatever typescript settings would get me up and running the fastest, so I don't know if it'd be better if I was to use "alwaysStrict": true or replace rollup with api-extractor for the bundling, or something like that.

@ghostdevv
Copy link
Collaborator

how it has zero stars is beyond me

Thanks! We only started making it recently

Yeah that looks great for the next step (how it has zero stars is beyond me), but I think all the tools are kind of stuck till the TS issues are resolved.

Could you try with version 0.3.0?

@niktek
Copy link

niktek commented Dec 23, 2023

Running Extractinator on this repo seems to get pretty far now with the following output:
image
Along with the emits of the d.ts files in the .extractinator folder.

@eulertour eulertour closed this as not planned Won't fix, can't repro, duplicate, stale Feb 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants