Improve Screener docs #2207

ValueRaider · 2025-01-05T17:21:37Z

The main change is converting the predefined bodies to use EquityQuery. Also simplify the queries, Yahoo has made them longer than necessary. This will make Screener doc page shorter and clearer.

@ericpien what is purpose of Screener.patch_body()? It's not documented well.

Begin converting predefined queries to use EquityQuery Add IS-IN operator Disable Screen.patch_body() because: what is use case? Easier to get to Screener doc page

R5dan · 2025-01-05T19:20:44Z

yfinance/screener/screener.py

+PREDEFINED_SCREENER_BODY_MAP = {
+    # 'aggressive_small_caps': {"offset":0,"size":25,"sortField":"eodvolume","sortType":"desc","quoteType":"equity","query":{"operator":"and","operands":[{"operator":"or","operands":[{"operator":"eq","operands":["exchange","NMS"]},{"operator":"eq","operands":["exchange","NYQ"]}]},{"operator":"or","operands":[{"operator":"LT","operands":["epsgrowth.lasttwelvemonths",15]}]}]},"userId":"","userIdType":"guid"},
+    'aggressive_small_caps': {"offset":0,"size":25,"sortField":"eodvolume","sortType":"desc","quoteType":"equity","userId":"","userIdType":"guid",
+                            "query":EQy('and', [EQy('is-in', ['exchange', 'NMS', 'NYQ']), EQy('lt', ["epsgrowth.lasttwelvemonths", 15])])},
+    'day_gainers': {"offset":0,"size":25,"sortField":"percentchange","sortType":"DESC","quoteType":"EQUITY","query":{"operator":"AND","operands":[{"operator":"gt","operands":["percentchange",3]},{"operator":"eq","operands":["region","us"]},{"operator":"or","operands":[{"operator":"BTWN","operands":["intradaymarketcap",2000000000,10000000000]},{"operator":"BTWN","operands":["intradaymarketcap",10000000000,100000000000]},{"operator":"GT","operands":["intradaymarketcap",100000000000]}]},{"operator":"gte","operands":["intradayprice",5]},{"operator":"gt","operands":["dayvolume",15000]}]},"userId":"","userIdType":"guid"},
+    'day_losers': {"offset":0,"size":25,"sortField":"percentchange","sortType":"ASC","quoteType":"EQUITY","query":{"operator":"AND","operands":[{"operator":"lt","operands":["percentchange",-2.5]},{"operator":"eq","operands":["region","us"]},{"operator":"or","operands":[{"operator":"BTWN","operands":["intradaymarketcap",2000000000,10000000000]},{"operator":"BTWN","operands":["intradaymarketcap",10000000000,100000000000]},{"operator":"GT","operands":["intradaymarketcap",100000000000]}]},{"operator":"gte","operands":["intradayprice",5]},{"operator":"gt","operands":["dayvolume",20000]}]},"userId":"","userIdType":"guid"},
+    'growth_technology_stocks': {"offset":0,"size":25,"sortField":"eodvolume","sortType":"desc","quoteType":"equity","query":{"operator":"and","operands":[{"operator":"or","operands":[{"operator":"BTWN","operands":["quarterlyrevenuegrowth.quarterly",50,100]},{"operator":"GT","operands":["quarterlyrevenuegrowth.quarterly",100]},{"operator":"BTWN","operands":["quarterlyrevenuegrowth.quarterly",25,50]}]},{"operator":"or","operands":[{"operator":"BTWN","operands":["epsgrowth.lasttwelvemonths",25,50]},{"operator":"BTWN","operands":["epsgrowth.lasttwelvemonths",50,100]},{"operator":"GT","operands":["epsgrowth.lasttwelvemonths",100]}]},{"operator":"eq","operands":["sector","Technology"]},{"operator":"or","operands":[{"operator":"eq","operands":["exchange","NMS"]},{"operator":"eq","operands":["exchange","NYQ"]}]}]},"userId":"","userIdType":"guid"},
+    'most_actives': {"offset":0,"size":25,"sortField":"dayvolume","sortType":"DESC","quoteType":"EQUITY","query":{"operator":"AND","operands":[{"operator":"eq","operands":["region","us"]},{"operator":"or","operands":[{"operator":"BTWN","operands":["intradaymarketcap",10000000000,100000000000]},{"operator":"GT","operands":["intradaymarketcap",100000000000]},{"operator":"BTWN","operands":["intradaymarketcap",2000000000,10000000000]}]},{"operator":"gt","operands":["dayvolume",5000000]}]},"userId":"","userIdType":"guid"},
+    'most_shorted_stocks': {"size":25,"offset":0,"sortField":"short_percentage_of_shares_outstanding.value","sortType":"DESC","quoteType":"EQUITY","topOperator":"AND","query":{"operator":"AND","operands":[{"operator":"or","operands":[{"operator":"EQ","operands":["region","us"]}]},{"operator":"gt","operands":["intradayprice",1]},{"operator":"gt","operands":["avgdailyvol3m",200000]}]},"userId":"","userIdType":"guid"},
+    'small_cap_gainers': {"offset":0,"size":25,"sortField":"eodvolume","sortType":"desc","quoteType":"equity","query":{"operator":"and","operands":[{"operator":"lt","operands":["intradaymarketcap",2000000000]},{"operator":"or","operands":[{"operator":"eq","operands":["exchange","NMS"]},{"operator":"eq","operands":["exchange","NYQ"]}]}]},"userId":"","userIdType":"guid"},
+    'undervalued_growth_stocks': {"offset":0,"size":25,"sortType":"DESC","sortField":"eodvolume","quoteType":"EQUITY","query":{"operator":"and","operands":[{"operator":"or","operands":[{"operator":"BTWN","operands":["peratio.lasttwelvemonths",0,20]}]},{"operator":"or","operands":[{"operator":"LT","operands":["pegratio_5y",1]}]},{"operator":"or","operands":[{"operator":"BTWN","operands":["epsgrowth.lasttwelvemonths",25,50]},{"operator":"BTWN","operands":["epsgrowth.lasttwelvemonths",50,100]},{"operator":"GT","operands":["epsgrowth.lasttwelvemonths",100]}]},{"operator":"or","operands":[{"operator":"eq","operands":["exchange","NMS"]},{"operator":"eq","operands":["exchange","NYQ"]}]}]},"userId":"","userIdType":"guid"},
+    'undervalued_large_caps': {"offset":0,"size":25,"sortField":"eodvolume","sortType":"desc","quoteType":"equity","query":{"operator":"and","operands":[{"operator":"or","operands":[{"operator":"BTWN","operands":["peratio.lasttwelvemonths",0,20]}]},{"operator":"lt","operands":["pegratio_5y",1]},{"operator":"btwn","operands":["intradaymarketcap",10000000000,100000000000]},{"operator":"or","operands":[{"operator":"eq","operands":["exchange","NMS"]},{"operator":"eq","operands":["exchange","NYQ"]}]}]},"userId":"","userIdType":"guid"},
+    'conservative_foreign_funds': {"offset":0,"size":25,"sortType":"DESC","sortField":"fundnetassets","quoteType":"MUTUALFUND","query":{"operator":"and","operands":[{"operator":"or","operands":[{"operator":"EQ","operands":["categoryname","Foreign Large Value"]},{"operator":"EQ","operands":["categoryname","Foreign Large Blend"]},{"operator":"EQ","operands":["categoryname","Foreign Large Growth"]},{"operator":"EQ","operands":["categoryname","Foreign Small/Mid Growth"]},{"operator":"EQ","operands":["categoryname","Foreign Large Blend"]},{"operator":"EQ","operands":["categoryname","Foreign Small/Mid Blend"]},{"operator":"EQ","operands":["categoryname","Foreign Small/Mid Value"]},{"operator":"EQ","operands":["categoryname","Foreign Small/Mid Blend"]},{"operator":"EQ","operands":["categoryname","Foreign Small/Mid Value"]},{"operator":"EQ","operands":["categoryname","Foreign Small/Mid Blend"]},{"operator":"EQ","operands":["categoryname","Foreign Small/Mid Value"]},{"operator":"EQ","operands":["categoryname","Foreign Small/Mid Blend"]},{"operator":"EQ","operands":["categoryname","Foreign Small/Mid Value"]}]},{"operator":"or","operands":[{"operator":"EQ","operands":["performanceratingoverall",4]},{"operator":"EQ","operands":["performanceratingoverall",5]}]},{"operator":"lt","operands":["initialinvestment",100001]},{"operator":"lt","operands":["annualreturnnavy1categoryrank",50]},{"operator":"or","operands":[{"operator":"EQ","operands":["riskratingoverall",1]},{"operator":"EQ","operands":["riskratingoverall",3]},{"operator":"EQ","operands":["riskratingoverall",2]}]},{"operator":"or","operands":[{"operator":"eq","operands":["exchange","NAS"]}]}]},"userId":"","userIdType":"guid"},
+    'high_yield_bond': {"offset":0,"size":25,"sortType":"DESC","sortField":"fundnetassets","quoteType":"MUTUALFUND","query":{"operator":"and","operands":[{"operator":"or","operands":[{"operator":"EQ","operands":["performanceratingoverall",4]},{"operator":"EQ","operands":["performanceratingoverall",5]}]},{"operator":"lt","operands":["initialinvestment",100001]},{"operator":"lt","operands":["annualreturnnavy1categoryrank",50]},{"operator":"or","operands":[{"operator":"EQ","operands":["riskratingoverall",1]},{"operator":"EQ","operands":["riskratingoverall",3]},{"operator":"EQ","operands":["riskratingoverall",2]}]},{"operator":"or","operands":[{"operator":"EQ","operands":["categoryname","High Yield Bond"]}]},{"operator":"or","operands":[{"operator":"eq","operands":["exchange","NAS"]}]}]},"userId":"","userIdType":"guid"},
+    'portfolio_anchors': {"offset":0,"size":25,"sortType":"DESC","sortField":"fundnetassets","quoteType":"MUTUALFUND","query":{"operator":"and","operands":[{"operator":"or","operands":[{"operator":"EQ","operands":["categoryname","Large Blend"]}]},{"operator":"or","operands":[{"operator":"EQ","operands":["performanceratingoverall",4]},{"operator":"EQ","operands":["performanceratingoverall",5]}]},{"operator":"lt","operands":["initialinvestment",100001]},{"operator":"lt","operands":["annualreturnnavy1categoryrank",50]},{"operator":"or","operands":[{"operator":"eq","operands":["exchange","NAS"]}]}]},"userId":"","userIdType":"guid"},
+    'solid_large_growth_funds': {"offset":0,"size":25,"sortType":"DESC","sortField":"fundnetassets","quoteType":"MUTUALFUND","query":{"operator":"and","operands":[{"operator":"or","operands":[{"operator":"EQ","operands":["categoryname","Large Growth"]}]},{"operator":"or","operands":[{"operator":"EQ","operands":["performanceratingoverall",5]},{"operator":"EQ","operands":["performanceratingoverall",4]}]},{"operator":"lt","operands":["initialinvestment",100001]},{"operator":"lt","operands":["annualreturnnavy1categoryrank",50]},{"operator":"or","operands":[{"operator":"eq","operands":["exchange","NAS"]}]}]},"userId":"","userIdType":"guid"},
+    'solid_midcap_growth_funds': {"offset":0,"size":25,"sortType":"DESC","sortField":"fundnetassets","quoteType":"MUTUALFUND","query":{"operator":"and","operands":[{"operator":"or","operands":[{"operator":"EQ","operands":["categoryname","Mid-Cap Growth"]}]},{"operator":"or","operands":[{"operator":"EQ","operands":["performanceratingoverall",5]},{"operator":"EQ","operands":["performanceratingoverall",4]}]},{"operator":"lt","operands":["initialinvestment",100001]},{"operator":"lt","operands":["annualreturnnavy1categoryrank",50]},{"operator":"or","operands":[{"operator":"eq","operands":["exchange","NAS"]}]}]},"userId":"","userIdType":"guid"},
+    'top_mutual_funds': {"offset":0,"size":25,"sortType":"DESC","sortField":"percentchange","quoteType":"MUTUALFUND","query":{"operator":"and","operands":[{"operator":"gt","operands":["intradayprice",15]},{"operator":"or","operands":[{"operator":"EQ","operands":["performanceratingoverall",5]},{"operator":"EQ","operands":["performanceratingoverall",4]}]},{"operator":"gt","operands":["initialinvestment",1000]},{"operator":"or","operands":[{"operator":"eq","operands":["exchange","NAS"]}]}]},"userId":"","userIdType":"guid"}
+}


Why does aggressive_small_caps of these use EquityQuery and some are just dictionaries/lists? (Sorry just seen new commit, I presume you are changing them all)

Why are these now here instead of const.py? Are they not constant?

R5dan · 2025-01-05T20:21:59Z

yfinance/screener/screener_query.py

@@ -69,7 +77,9 @@ def __init__(self, operator: str, operand: Union[numbers.Real, str, List['Equity
        if len(operand) <= 0:
            raise ValueError('Invalid field for Screener')

-        if operator in {'OR','AND'}: 
+        if operator == 'IS-IN':


Why is there a is-in operand?
Wouldn't it make more sense to make it a function as it is not an actual operand?

I wonder about this from design perspective as well. I tried to retain as much "likeliness" as possible to Yahoo's operator types. Since there is no "is-in" operator in yahoo's own query, it could get unnecessarily difficult to debug / extend in the future. Is there a reason to take on that cost?

Yes, that was what I was trying to get across.
I think it would be better to have a function for is_in if at all:

def is_in(key, *values): operands = [] for oper in values: operands.append(EquityQuery("EQ", oper)) return EquityQuery("OR", operands)

Similar to how the to_dict works, just earlier.

The primary goal is to simplify users' task of writing queries, and part of that is length.

EquityQuery('IS-IN', ['exchange', 'NMS', 'NYQ'])

vs

EquityQuery('OR', [EquityQuery('EQ, ['exchange', 'NMS'], EquityQuery('EQ, ['exchange', 'NYQ']])

is in is Pythonic, yfinance is Pythonic

is in is Pythonic, yfinance is Pythonic

Im not saying don't have an is-in but make it have a clear separation from EquityQuery. A custom function or class, like how I defined some in my PR

ericpien · 2025-01-05T22:59:01Z

The main change is converting the predefined bodies to use EquityQuery

Is the advantage of this cleaner documentation?

@ericpien what is purpose of Screener.patch_body()? It's not documented well.

It is meant to update just the provided portion of the body. i.e. user wants to repeatedly use the same Screener but just update the _body.offset by 200. Instead of reconstructing the whole Query or Screener, user can just update that value by passing the dict to the method then iterate in a loop.

ericpien · 2025-01-05T23:12:41Z

yfinance/const.py

+        "performanceratingoverall",
+        "initialinvestment", 
+        "annualreturnnavy1categoryrank", 
+        "riskratingoverall"}, 


Are these valid fields to screen Equity? or is it reserved for Mutual Funds?

Good point. Screener needs a new FundQuery aka FundOperation class.

ValueRaider · 2025-01-06T20:24:26Z

The main change is converting the predefined bodies to use EquityQuery

Is the advantage of this cleaner documentation?

It's the only reason, to shorten the docs. Next task is collapsing the long lists.

Also now Screener has been completely redone, it did not "feel right" before.

Begin improving Screener docs

a086a14

Begin converting predefined queries to use EquityQuery Add IS-IN operator Disable Screen.patch_body() because: what is use case? Easier to get to Screener doc page

R5dan mentioned this pull request Jan 5, 2025

Redo Screener #2190

Draft

Fix GTE & LTE operators. Convert more predefineds

7c2dbd7

R5dan suggested changes Jan 5, 2025

View reviewed changes

Finish query converts, fix some bugs

545752e

ValueRaider changed the title ~~Begin improving Screener docs~~ Improve Screener docs Jan 5, 2025

ericpien reviewed Jan 5, 2025

View reviewed changes

ValueRaider added 2 commits January 6, 2025 20:06

Split Query into: Operation & Query

050c7fd

Fix ruff & sphinx RST

ee65aae

ValueRaider added 2 commits January 6, 2025 21:21

Add 'FundOperation', add concept of common fields

a6e6f0a

Fix ruff

835172b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve Screener docs #2207

Improve Screener docs #2207

ValueRaider commented Jan 5, 2025

R5dan Jan 5, 2025 •

edited

Loading

R5dan Jan 5, 2025

ericpien Jan 5, 2025

R5dan Jan 5, 2025

ValueRaider Jan 6, 2025 •

edited

Loading

R5dan Jan 6, 2025

ericpien commented Jan 5, 2025 •

edited

Loading

ericpien Jan 5, 2025

ValueRaider Jan 6, 2025

ValueRaider commented Jan 6, 2025

Improve Screener docs #2207

Are you sure you want to change the base?

Improve Screener docs #2207

Conversation

ValueRaider commented Jan 5, 2025

R5dan Jan 5, 2025 • edited Loading

Choose a reason for hiding this comment

R5dan Jan 5, 2025

Choose a reason for hiding this comment

ericpien Jan 5, 2025

Choose a reason for hiding this comment

R5dan Jan 5, 2025

Choose a reason for hiding this comment

ValueRaider Jan 6, 2025 • edited Loading

Choose a reason for hiding this comment

R5dan Jan 6, 2025

Choose a reason for hiding this comment

ericpien commented Jan 5, 2025 • edited Loading

ericpien Jan 5, 2025

Choose a reason for hiding this comment

ValueRaider Jan 6, 2025

Choose a reason for hiding this comment

ValueRaider commented Jan 6, 2025

R5dan Jan 5, 2025 •

edited

Loading

ValueRaider Jan 6, 2025 •

edited

Loading

ericpien commented Jan 5, 2025 •

edited

Loading