Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: validate type comments and generate .d.ts #204

Open
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

voxpelli
Copy link
Member

@voxpelli voxpelli commented Mar 8, 2024

This is an alternative to #60 which also fixes #150

As this PR doesn't rewrite to TS its easier to keep up to date as other changes happen.

No files has been renamed, the build script remains largely the same, the code as well mostly.

The generated type file
import * as estree from 'estree';
import * as eslint from 'eslint';

type Node = estree.Node | estree.Expression;

declare const READ: unique symbol;
declare const CALL: unique symbol;
declare const CONSTRUCT: unique symbol;
declare const ESM: unique symbol;
declare class ReferenceTracker {
    constructor(globalScope: eslint.Scope.Scope, { mode, globalObjectNames, }?: {
        mode?: "legacy" | "strict" | undefined;
        globalObjectNames?: string[] | undefined;
    } | undefined);
    variableStack: eslint.Scope.Variable[];
    globalScope: eslint.Scope.Scope;
    mode: "legacy" | "strict";
    globalObjectNames: string[];
    iterateGlobalReferences(traceMap: TraceMap): IterableIterator<Reference>;
    iterateCjsReferences(traceMap: TraceMap): IterableIterator<Reference>;
    iterateEsmReferences(traceMap: TraceMap): IterableIterator<Reference>;
    _iterateVariableReferences(variable: eslint.Scope.Variable, path: string[], traceMap: TraceMap, shouldReport: boolean): IterableIterator<Reference>;
    _iteratePropertyReferences(rootNode: RichNode, path: string[], traceMap: TraceMap): IterableIterator<Reference>;
    _iterateLhsReferences(patternNode: RichNode, path: string[], traceMap: TraceMap): IterableIterator<Reference>;
    _iterateImportReferences(specifierNode: RichNode, path: string[], traceMap: TraceMap): IterableIterator<Reference>;
}
declare namespace ReferenceTracker {
    export { READ };
    export { CALL };
    export { CONSTRUCT };
    export { ESM };
}
type ReferenceType = typeof READ | typeof CALL | typeof CONSTRUCT;
type TraceMap = {
    [key: string]: TraceMap;
} & Partial<Record<ReferenceType, boolean>>;
type RichNode = eslint.Rule.Node | Node;
type Reference = {
    node: RichNode;
    path: string[];
    type: ReferenceType;
    info: unknown;
};

declare function findVariable(initialScope: eslint.Scope.Scope, nameOrNode: string | Node): eslint.Scope.Variable | null;

declare function getFunctionHeadLocation(node: Extract<eslint.Rule.Node, {
    type: 'FunctionDeclaration' | 'FunctionExpression' | 'ArrowFunctionExpression';
}>, sourceCode: eslint.SourceCode): eslint.AST.SourceLocation | null;

declare function getFunctionNameWithKind(node: Extract<eslint.Rule.Node, {
    type: 'FunctionDeclaration' | 'FunctionExpression' | 'ArrowFunctionExpression';
}>, sourceCode?: eslint.SourceCode | undefined): string;

declare function getInnermostScope(initialScope: eslint.Scope.Scope, node: Node): eslint.Scope.Scope;

declare function getPropertyName(node: Extract<Node, {
    type: 'MemberExpression' | 'Property' | 'MethodDefinition' | 'PropertyDefinition';
}>, initialScope?: eslint.Scope.Scope | undefined): string | null;

declare function getStaticValue(node: Node, initialScope?: eslint.Scope.Scope | null | undefined): {
    value: unknown;
    optional?: never;
} | {
    value: undefined;
    optional?: true;
} | null;

declare function getStringIfConstant(node: Node, initialScope?: eslint.Scope.Scope | null | undefined): string | null;

declare function hasSideEffect(node: eslint.Rule.Node, sourceCode: eslint.SourceCode, { considerGetters, considerImplicitTypeConversion }?: VisitOptions | undefined): boolean;
type VisitOptions = {
    considerGetters?: boolean | undefined;
    considerImplicitTypeConversion?: boolean | undefined;
};

declare function isArrowToken(token: eslint.AST.Token): boolean;
declare function isCommaToken(token: eslint.AST.Token): boolean;
declare function isSemicolonToken(token: eslint.AST.Token): boolean;
declare function isColonToken(token: eslint.AST.Token): boolean;
declare function isOpeningParenToken(token: eslint.AST.Token): boolean;
declare function isClosingParenToken(token: eslint.AST.Token): boolean;
declare function isOpeningBracketToken(token: eslint.AST.Token): boolean;
declare function isClosingBracketToken(token: eslint.AST.Token): boolean;
declare function isOpeningBraceToken(token: eslint.AST.Token): boolean;
declare function isClosingBraceToken(token: eslint.AST.Token): boolean;
declare function isCommentToken(token: eslint.AST.Token): boolean;
declare function isNotArrowToken(token: eslint.AST.Token): boolean;
declare function isNotCommaToken(token: eslint.AST.Token): boolean;
declare function isNotSemicolonToken(token: eslint.AST.Token): boolean;
declare function isNotColonToken(token: eslint.AST.Token): boolean;
declare function isNotOpeningParenToken(token: eslint.AST.Token): boolean;
declare function isNotClosingParenToken(token: eslint.AST.Token): boolean;
declare function isNotOpeningBracketToken(token: eslint.AST.Token): boolean;
declare function isNotClosingBracketToken(token: eslint.AST.Token): boolean;
declare function isNotOpeningBraceToken(token: eslint.AST.Token): boolean;
declare function isNotClosingBraceToken(token: eslint.AST.Token): boolean;
declare function isNotCommentToken(token: eslint.AST.Token): boolean;

declare function isParenthesized(timesOrNode: number, nodeOrSourceCode: eslint.Rule.Node, optionalSourceCode: eslint.SourceCode): boolean;
declare function isParenthesized(timesOrNode: eslint.Rule.Node, nodeOrSourceCode: eslint.SourceCode): boolean;

declare class PatternMatcher {
    constructor(pattern: RegExp, { escaped }?: {
        escaped?: boolean | undefined;
    } | undefined);
    execAll(str: string): IterableIterator<RegExpExecArray>;
    test(str: string): boolean;
    [Symbol.replace](str: string, replacer: string | ((...strs: (string | number)[]) => string)): string;
}

declare namespace _default {
    export { CALL };
    export { CONSTRUCT };
    export { ESM };
    export { findVariable };
    export { getFunctionHeadLocation };
    export { getFunctionNameWithKind };
    export { getInnermostScope };
    export { getPropertyName };
    export { getStaticValue };
    export { getStringIfConstant };
    export { hasSideEffect };
    export { isArrowToken };
    export { isClosingBraceToken };
    export { isClosingBracketToken };
    export { isClosingParenToken };
    export { isColonToken };
    export { isCommaToken };
    export { isCommentToken };
    export { isNotArrowToken };
    export { isNotClosingBraceToken };
    export { isNotClosingBracketToken };
    export { isNotClosingParenToken };
    export { isNotColonToken };
    export { isNotCommaToken };
    export { isNotCommentToken };
    export { isNotOpeningBraceToken };
    export { isNotOpeningBracketToken };
    export { isNotOpeningParenToken };
    export { isNotSemicolonToken };
    export { isOpeningBraceToken };
    export { isOpeningBracketToken };
    export { isOpeningParenToken };
    export { isParenthesized };
    export { isSemicolonToken };
    export { PatternMatcher };
    export { READ };
    export { ReferenceTracker };
}

export { CALL, CONSTRUCT, ESM, PatternMatcher, READ, ReferenceTracker, _default as default, findVariable, getFunctionHeadLocation, getFunctionNameWithKind, getInnermostScope, getPropertyName, getStaticValue, getStringIfConstant, hasSideEffect, isArrowToken, isClosingBraceToken, isClosingBracketToken, isClosingParenToken, isColonToken, isCommaToken, isCommentToken, isNotArrowToken, isNotClosingBraceToken, isNotClosingBracketToken, isNotClosingParenToken, isNotColonToken, isNotCommaToken, isNotCommentToken, isNotOpeningBraceToken, isNotOpeningBracketToken, isNotOpeningParenToken, isNotSemicolonToken, isOpeningBraceToken, isOpeningBracketToken, isOpeningParenToken, isParenthesized, isSemicolonToken };

@voxpelli voxpelli self-assigned this Mar 8, 2024
@voxpelli voxpelli requested a review from a team March 8, 2024 22:09
@ota-meshi
Copy link
Member

That's a good idea!
Could you please fix the CI error?

@scagood
Copy link

scagood commented Apr 3, 2024

Following up from this comment: eslint-community/eslint-plugin-n#169 (comment)

I made a couple of utility changes to the .d.ts file I stole from your description 😄
If this is not the right place for these comments please point me in the right direction!

TraceMap

I made a couple of changes to TraceMap:

  1. I added a configurable parameter
type TraceMap<Info extends unknown> =
  & { [key: string]: TraceMap<Info>; }
  & Partial<Record<ReferenceType, Info>>;
  1. I unwrapped the TraceMap to be one object with all the properties as optional.
type TraceMap<Info extends unknown> = {
  [READ]?: Info;
  [CALL]?: Info;
  [CONSTRUCT]?: Info;
  [key: string]: TraceMap<Info>;
}
This second one confuses me a little, as I know it works, but I dont know why! Before I would get a small army of these kinds of error:

88 errors to be exact 😓

image

Object literal may only specify known properties, and 'alloc' does not exist in type 'Partial<Record<ReferenceType, SupportInfo>>'

Reference

Reference also got its own argument:

type Reference<Info extends unknown> = {
  node: RichNode;
  path: string[];
  type: ReferenceType;
  // Previously:
  // info: unknown;
  info: Info;
};

ReferenceTracker

Because TraceMap and Reference now have inferable arguments, we make make the iterateXXXReferences functions a little more friendly by adding the Info argument. This allows for simple type inference

  iterateGlobalReferences<Info extends unknown>(traceMap: TraceMap<Info>): IterableIterator<Reference<Info>>;
  iterateCjsReferences<Info extends unknown>(traceMap: TraceMap<Info>): IterableIterator<Reference<Info>>;
  iterateEsmReferences<Info extends unknown>(traceMap: TraceMap<Info>): IterableIterator<Reference<Info>>;

This allows the following to just work out of the box:

/**
 * @typedef CustomInfo
 * @property {string[]} experimental
 * @property {string[]} supported
 * @property {string[]} deprecated
 */

/** @type {TraceMap<CustomInfo>} */
const traceMap = {}

for (const reference of tracker.iterateGlobalReferences(traceMap)) {
  console.info(reference.info.supported);
}

For the sake of simplicity I threw together an example of this on the TS playground:
https://www.typescriptlang.org/play?#code/ ...


I am happy to contibute where ever is needed as long as I'm not step on anyones toes 😁

@voxpelli
Copy link
Member Author

voxpelli commented Apr 7, 2024

I'll get around to update this PR real soon.

One challenge right now is that this module support quite old versions of Node.js.

I would prefer to align with eslint-plugin-n, [email protected] etc the engine range they use: ^18.18.0 || ^20.9.0 || >=21.1.0

That's a separate issue though, but one I would want to deal with before spending work on backporting changes to those older versions.


cc @JoshuaKGoldberg, would be lovely with some typescript-eslint feedback on this PR eventually as well :)

.eslintrc.js Outdated Show resolved Hide resolved
Co-authored-by: Josh Goldberg ✨ <[email protected]>
rules: {
semi: ["error", "never"],
"semi-spacing": ["error", { before: false, after: true }],

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's something deeply funny to me about an eslint-community project using ESLint for formatting.

"coverage": "opener ./coverage/lcov-report/index.html",
"docs:build": "vitepress build docs",
"docs:watch": "vitepress dev docs",
"format": "npm run -s format:prettier -- --write",
"format:prettier": "prettier .",
"format:check": "npm run -s format:prettier -- --check",
"lint": "eslint .",
"lint:eslint": "eslint .",
"lint:tsc": "tsc",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naming: I wouldn't call TypeScript a "lint" task. Even if it acts essentially as a linter in this setup, I've found it confusing for newer contributors to see it named as one.

const operations = Object.freeze({
ArrayExpression(node, initialScope) {
if (node.type !== "ArrayExpression") {
return null
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 are these ifs actually necessary, or just to appease a type checker that doesn't understand what's happening?

If it's the latter, the general right thing to do would be to fix TypeScript's understanding of the types so it doesn't yell at you.

For posterity: I tried adding /** @param {import("estree").ArrayExpression} node */ to the ArrayExpression one and got a type error on operations:

Type 'Readonly<{ ArrayExpression(node: ArrayExpression, initialScope: Scope | undefined): { value: any[]; } | null; AssignmentExpression(node: Node, initialScope: Scope | undefined): StaticValue | null; ... 14 more ...; UnaryExpression(node: Node, initialScope: Scope | undefined): { ...; } | ... 3 more ... | null; }>' is not assignable to type 'Readonly<Partial<Record<"CatchClause" | "ClassBody" | "Identifier" | "Literal" | "MethodDefinition" | "PrivateIdentifier" | "Program" | "Property" | "PropertyDefinition" | "SpreadElement" | ... 60 more ... | "VariableDeclaration", VisitorCallback>>>'.
  Types of property 'ArrayExpression' are incompatible.
    Type '(node: ArrayExpression, initialScope: Scope | undefined) => { value: any[]; } | null' is not assignable to type 'VisitorCallback'.ts(2322)

Tricky.

I got around it by:

  1. Renaming types.mjs to types.d.ts, so I could use "real" TypeScript syntax and not the annoying JSDoc sludge
  2. Making the VisitorCallback type generic:
export type VisitorCallback<InNode extends Node> = (
    node: InNode,
    initialScope: eslint.Scope.Scope | undefined,
) => StaticValue | null
  1. Using a type annotation on the ArrayExpression member:
/** @type {Readonly<Partial<Record<import('eslint').Rule.NodeTypes, import("./types.js").VisitorCallback<any>>>>} */
const operations = Object.freeze({
    /** @type {import("./types.js").VisitorCallback<import("estree").ArrayExpression>} */
    ArrayExpression(node, initialScope) {

YMMV.

@@ -74,10 +72,11 @@ function replaceS(matcher, str, replacement) {
* Replace a given string by a given matcher.
* @param {PatternMatcher} matcher The pattern matcher.
* @param {string} str The string to be replaced.
* @param {(...strs[])=>string} replace The function to replace each matched part.
* @param {(...strs: (string|number)[])=>string} replace The function to replace each matched part.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How far is this from being merged? Would it be possible to cherry pick this change explicitly?

Context:
I'm introducing an Eslint plugin to our workflow that is relying on this package. Our typechecking currently depends on maxNodeModuleJsDepth since we have a monorepo including old js-with-types-in-jsdoc modules, resulting in this package being blocked in CI because of invalid syntax.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a person doing lots of js-with-types and have used maxNodeModuleJsDepth, I would say: There's never an excuse to use maxNodeModuleJsDepth anymore, since one can now compile type declarations from JSDoc declarations.

In a monorepo – either have TypeScript handle the entire monorepo as a single entity (thus allowing JSDoc in all parts) and handle individual module paths through paths in tsconfig.json or use something like @monorepo-utils/workspaces-to-typescript-project-references to set up project references to compile type declarations for the different parts (haven't tried this specific approach with JS)

Published JSDoc comments are not meant for consumption by TypeScript – only published type declarations are

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How far is this from being merged?

On this topic, see #232 :)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@voxpelli Thank you for the reply, I agree with this completely

Published JSDoc comments are not meant for consumption by TypeScript – only published type declarations are

We've not figured out how to adopt a better typescript config yet but some blockers were resolved in TS 5.5 so maybe we should give it another try, thank you for the pointers!

I'll be patching this on our end for now.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GNRSN Feel free to pick my brain on the types-in-js approach over at https://github.com/voxpelli/types-in-js :)

@voxpelli
Copy link
Member Author

I would prefer to align with eslint-plugin-n, [email protected] etc the engine range they use: ^18.18.0 || ^20.9.0 || >=21.1.0

That's a separate issue though, but one I would want to deal with before spending work on backporting changes to those older versions.

Created #234 to move this along, and #235 as a companion to that

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Consider adding typescript declarations?
5 participants