Added findtext and xpath #47

dispread · 2024-04-12T12:30:40Z

I dont know why they werent already there so I added them.

Peace out

github-actions · 2024-04-12T12:31:01Z

lxml-stubs/html/_element.pyi

    def findall(
        self, path: _ElemPathArg, namespaces: _NSMapArg | None = None
    ) -> list[HtmlElement]: ...
    def iterfind(
        self, path: _ElemPathArg, namespaces: _NSMapArg | None = None
    ) -> Iterator[HtmlElement]: ...
+    def xpath(


🚫 [mypy] _{reported by reviewdog 🐶}
Signature of "xpath" incompatible with supertype "_Element" [override]

github-actions · 2024-04-12T12:31:01Z

lxml-stubs/html/_element.pyi

+        self,
+        _path: _AnyStr,
+        *,
+        namespaces: _NonDefaultNSMapArg | None = None,


🚫 [mypy] _{reported by reviewdog 🐶}
Name "_NonDefaultNSMapArg" is not defined [name-defined]

github-actions · 2024-04-12T12:31:01Z

lxml-stubs/html/_element.pyi

+        _path: _AnyStr,
+        *,
+        namespaces: _NonDefaultNSMapArg | None = None,
+        extensions: _XPathExtFuncArg | None = None,


🚫 [mypy] _{reported by reviewdog 🐶}
Name "_XPathExtFuncArg" is not defined [name-defined]

github-actions · 2024-04-12T12:31:01Z

lxml-stubs/html/_element.pyi

+        namespaces: _NonDefaultNSMapArg | None = None,
+        extensions: _XPathExtFuncArg | None = None,
+        smart_strings: bool = True,
+        **_variables: _XPathVarArg


🚫 [mypy] _{reported by reviewdog 🐶}
Name "_XPathVarArg" is not defined [name-defined]

github-actions · 2024-04-12T12:31:01Z

lxml-stubs/html/_element.pyi

+        extensions: _XPathExtFuncArg | None = None,
+        smart_strings: bool = True,
+        **_variables: _XPathVarArg
+    ) -> list[HTMLElement]: ...


🚫 [mypy] _{reported by reviewdog 🐶}
Name "HTMLElement" is not defined [name-defined]

github-actions · 2024-04-12T12:31:18Z

lxml-stubs/html/_element.pyi

@@ -195,12 +195,27 @@ class HtmlElement(etree.ElementBase):
    def find(
        self, path: _ElemPathArg, namespaces: _NSMapArg | None = None
    ) -> HtmlElement | None: ...
+    def findtext(


🚫 [pyright] _{reported by reviewdog 🐶}
Method "findtext" overrides class "_Element" in an incompatible manner
Parameter "namespaces" is missing in override
Return type mismatch: base method returns type "str | None", override returns type "str | _T@findtext" (reportIncompatibleMethodOverride)

github-actions · 2024-04-12T12:31:18Z

lxml-stubs/html/_element.pyi

+        self,
+        _path: _AnyStr,
+        *,
+        namespaces: _NonDefaultNSMapArg | None = None,


🚫 [pyright] _{reported by reviewdog 🐶}
Type of parameter "namespaces" is partially unknown
Parameter type is "Unknown | None" (reportUnknownParameterType)

github-actions · 2024-04-12T12:31:18Z

lxml-stubs/html/_element.pyi

+        self,
+        _path: _AnyStr,
+        *,
+        namespaces: _NonDefaultNSMapArg | None = None,


🚫 [pyright] _{reported by reviewdog 🐶}
"_NonDefaultNSMapArg" is not defined (reportUndefinedVariable)

github-actions · 2024-04-12T12:31:18Z

lxml-stubs/html/_element.pyi

+        _path: _AnyStr,
+        *,
+        namespaces: _NonDefaultNSMapArg | None = None,
+        extensions: _XPathExtFuncArg | None = None,


🚫 [pyright] _{reported by reviewdog 🐶}
Type of parameter "extensions" is partially unknown
Parameter type is "Unknown | None" (reportUnknownParameterType)

github-actions · 2024-04-12T12:31:18Z

lxml-stubs/html/_element.pyi

+        _path: _AnyStr,
+        *,
+        namespaces: _NonDefaultNSMapArg | None = None,
+        extensions: _XPathExtFuncArg | None = None,


🚫 [pyright] _{reported by reviewdog 🐶}
"_XPathExtFuncArg" is not defined (reportUndefinedVariable)

github-actions · 2024-04-12T12:31:19Z

lxml-stubs/html/_element.pyi

+        namespaces: _NonDefaultNSMapArg | None = None,
+        extensions: _XPathExtFuncArg | None = None,
+        smart_strings: bool = True,
+        **_variables: _XPathVarArg


🚫 [pyright] _{reported by reviewdog 🐶}
Type of parameter "_variables" is unknown (reportUnknownParameterType)

github-actions · 2024-04-12T12:31:19Z

lxml-stubs/html/_element.pyi

+        namespaces: _NonDefaultNSMapArg | None = None,
+        extensions: _XPathExtFuncArg | None = None,
+        smart_strings: bool = True,
+        **_variables: _XPathVarArg


🚫 [pyright] _{reported by reviewdog 🐶}
"_XPathVarArg" is not defined (reportUndefinedVariable)

github-actions · 2024-04-12T12:31:19Z

lxml-stubs/html/_element.pyi

+        extensions: _XPathExtFuncArg | None = None,
+        smart_strings: bool = True,
+        **_variables: _XPathVarArg
+    ) -> list[HTMLElement]: ...


🚫 [pyright] _{reported by reviewdog 🐶}
"HTMLElement" is not defined (reportUndefinedVariable)

dispread · 2024-04-12T12:33:10Z

Guess i found out

Silly mistakes

github-actions · 2024-04-12T12:53:59Z

lxml-stubs/html/_element.pyi

    def cssselect(
        self,
        expr: str,
        *,
        translator: _CSSTransArg = "xml",
-    ) -> list[HtmlElement]: ...
+    ) -> _XPathObject: ...


🚫 [mypy] _{reported by reviewdog 🐶}
Name "_XPathObject" is not defined [name-defined]

github-actions · 2024-04-12T12:54:07Z

lxml-stubs/html/_element.pyi

    def cssselect(
        self,
        expr: str,
        *,
        translator: _CSSTransArg = "xml",
-    ) -> list[HtmlElement]: ...
+    ) -> _XPathObject: ...


🚫 [pyright] _{reported by reviewdog 🐶}
"_XPathObject" is not defined (reportUndefinedVariable)

Hopefully works now

github-actions · 2024-04-12T12:57:58Z

lxml-stubs/html/_element.pyi

@@ -195,18 +198,33 @@ class HtmlElement(etree.ElementBase):
    def find(
        self, path: _ElemPathArg, namespaces: _NSMapArg | None = None
    ) -> HtmlElement | None: ...
+    def findtext(


🚫 [mypy] _{reported by reviewdog 🐶}
Signature of "findtext" incompatible with supertype "_Element" [override]

github-actions · 2024-04-12T12:58:06Z

lxml-stubs/html/_element.pyi

@@ -195,18 +198,33 @@ class HtmlElement(etree.ElementBase):
    def find(
        self, path: _ElemPathArg, namespaces: _NSMapArg | None = None
    ) -> HtmlElement | None: ...
+    def findtext(


🚫 [pyright] _{reported by reviewdog 🐶}
Method "findtext" overrides class "_Element" in an incompatible manner
  Positional parameter count mismatch; base method has 3, but override has 4
  Parameter "namespaces" is missing in override
  Return type mismatch: base method returns type "str | None", override returns type "str | _T@findtext" (reportIncompatibleMethodOverride)

so thats why it didnt work

I swear I'm not retarded, just tired

dispread · 2024-04-12T13:11:45Z

I swear I'm not retarded, just tired

abelcheung · 2024-04-12T16:00:05Z

@dispread No sweat, please do it at your own pace. I'll check again tomorrow, in case you have additional commits related to this change.

BTW I want to caution on something:

dont know why they werent already there

When a method is absent in a subclass, this means the corresponding method in superclass would be used (etree._Element in this case).

abelcheung · 2024-04-14T01:17:57Z

@dispread If these changes are all you intended to submit, then they are not needed. As I said before, if the methods don't exist in HtmlElement, the corresponding definition of findtext() and xpath() in its superclasses (etree._Element in this case) would be picked up.

Feel free to reopen this PR if you have other changes in mind.

dispread · 2024-04-15T12:20:50Z

@abelcheung
Just out of curiosity, what IDE are you testing this on? I am using Pycharm and it doesn't seem to recognize the inherited methods hence my commits. Even on my branch it defaults to showing the autogenerated stubs, which is better than original (no hint on inherited Html element methods).
Is there something I'm missing?

Thanks in advance 👍

abelcheung · 2024-04-15T16:14:19Z

I see. I'm using vscode with pylance extension, and everything just works automatically. If pycharm really needs a manual override of methods in order to show up, that could indicate either oversight of setting or a missing feature. Even in terms of pure stub itself (disregarding any IDE support), this manual override shouldn't be needed at all. Guess I need to spend some time to get #8 going then.

dispread · 2024-04-15T16:50:05Z

If I'll have some free time tomorrow/wednesday I can look into whether it's just my config or something reproducible.

abelcheung · 2024-04-16T08:17:02Z

I can reproduce the problem you have faced with a default installation of Pycharm 2024.1. The autocompletion popup manages to show ElementBase.__init__() (The direct superclass of HtmlElement), but doesn't go beyond that. My current bet is that, _Element is deemed a private class, so none of its methods would show up in public class like HtmlElement. Same applies for ElementBase._init().

I manage to find a temporary hack but it's quite ugly. Using the example below:

from lxml import html
elem = html.Element('div')
result = elem.  # <- no xpath() in autocomplete

At this point, try to enforce the type of elem to _Element temporarily, like:

from lxml.etree import _Element
elem: _Element = html.Element('div')

Then the signature of xpath() would show up. After using, delete the temp enforced type.

I Will try to continue investigating later on, for now perhaps you can also try out if vscode works fine for you.

dispread added 2 commits April 12, 2024 13:54

Added findtext() and xpath()

31b0ec3

Specify arg types

4a4c76e

github-actions bot reviewed Apr 12, 2024

View reviewed changes

dispread marked this pull request as draft April 12, 2024 12:38

dispread added 2 commits April 12, 2024 14:47

Fixed some stuff

8632479

Updated stuff

76e546f

Silly mistakes

github-actions bot reviewed Apr 12, 2024

View reviewed changes

Update _element.pyi

38543c9

Hopefully works now

github-actions bot reviewed Apr 12, 2024

View reviewed changes

dispread added 4 commits April 12, 2024 15:03

added overload to findtext

ed2cd1b

so thats why it didnt work

Update _element.pyi

423234f

Update _element.pyi

a5b5443

I swear I'm not retarded, just tired

Update _element.pyi

b596b7c

dispread marked this pull request as ready for review April 12, 2024 13:11

abelcheung closed this Apr 14, 2024

abelcheung mentioned this pull request May 4, 2024

Alternative type checker support #8

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added findtext and xpath #47

Added findtext and xpath #47

dispread commented Apr 12, 2024

github-actions bot Apr 12, 2024

github-actions bot Apr 12, 2024

github-actions bot Apr 12, 2024

github-actions bot Apr 12, 2024

github-actions bot Apr 12, 2024

github-actions bot Apr 12, 2024

github-actions bot Apr 12, 2024

github-actions bot Apr 12, 2024

github-actions bot Apr 12, 2024

github-actions bot Apr 12, 2024

github-actions bot Apr 12, 2024

github-actions bot Apr 12, 2024

github-actions bot Apr 12, 2024

dispread commented Apr 12, 2024

github-actions bot Apr 12, 2024

github-actions bot Apr 12, 2024

github-actions bot Apr 12, 2024

github-actions bot Apr 12, 2024

dispread commented Apr 12, 2024

abelcheung commented Apr 12, 2024 •

edited

Loading

abelcheung commented Apr 14, 2024

dispread commented Apr 15, 2024

abelcheung commented Apr 15, 2024

dispread commented Apr 15, 2024

abelcheung commented Apr 16, 2024 •

edited

Loading

Added findtext and xpath #47

Added findtext and xpath #47

Conversation

dispread commented Apr 12, 2024

github-actions bot Apr 12, 2024

Choose a reason for hiding this comment

github-actions bot Apr 12, 2024

Choose a reason for hiding this comment

github-actions bot Apr 12, 2024

Choose a reason for hiding this comment

github-actions bot Apr 12, 2024

Choose a reason for hiding this comment

github-actions bot Apr 12, 2024

Choose a reason for hiding this comment

github-actions bot Apr 12, 2024

Choose a reason for hiding this comment

github-actions bot Apr 12, 2024

Choose a reason for hiding this comment

github-actions bot Apr 12, 2024

Choose a reason for hiding this comment

github-actions bot Apr 12, 2024

Choose a reason for hiding this comment

github-actions bot Apr 12, 2024

Choose a reason for hiding this comment

github-actions bot Apr 12, 2024

Choose a reason for hiding this comment

github-actions bot Apr 12, 2024

Choose a reason for hiding this comment

github-actions bot Apr 12, 2024

Choose a reason for hiding this comment

dispread commented Apr 12, 2024

github-actions bot Apr 12, 2024

Choose a reason for hiding this comment

github-actions bot Apr 12, 2024

Choose a reason for hiding this comment

github-actions bot Apr 12, 2024

Choose a reason for hiding this comment

github-actions bot Apr 12, 2024

Choose a reason for hiding this comment

dispread commented Apr 12, 2024

abelcheung commented Apr 12, 2024 • edited Loading

abelcheung commented Apr 14, 2024

dispread commented Apr 15, 2024

abelcheung commented Apr 15, 2024

dispread commented Apr 15, 2024

abelcheung commented Apr 16, 2024 • edited Loading

abelcheung commented Apr 12, 2024 •

edited

Loading

abelcheung commented Apr 16, 2024 •

edited

Loading