Recent helpful findings: crum manager, additional routes, etc. #43

roblav96 · 2017-06-10T17:13:13Z

Hey man, love what you're doing here. I've been reverse engineering Yahoo's API recently and thought I'd provide some of my findings.

Crum Manager

sstrickx/yahoofinance-api
@sstrickx wrote a crum manager in Java.

Recommended Symbols

GET https://query1.finance.yahoo.com/v6/finance/recommendationsbysymbol/PLUG

Daily Gainers

GET https://query2.finance.yahoo.com/v1/finance/screener/predefined/saved?formatted=false&lang=en-US&region=US&scrIds=day_gainers&count=5&corsDomain=finance.yahoo.com

Search Autocomplete

GET https://autoc.finance.yahoo.com/autoc?query=PLUG&region=1&lang=en

Symbol Conversations

GET https://finance.yahoo.com/_finance_doubledown/api/resource/canvass.getMessageList;count=20;oauthConsumerKey=finance.oauth.client.canvass.prod.consumerKey;oauthConsumerSecret=finance.oauth.client.canvass.prod.consumerSecret;query=namespace = "yahoo_finance" and (tag = "PLUG");region=US;sortBy=createdAt;userActivity=true

Symbol News

GET http://feeds.finance.yahoo.com/rss/2.0/headline?s=PLUG&region=US&lang=en-US
Returns in RSS format, couldn't figure out how to get JSON.

Multiple Symbol Quote

GET https://query2.finance.yahoo.com/v7/finance/quote?symbols=NVDA,PLUG,AMD

Multiple Symbol Historical Charts

GET https://query1.finance.yahoo.com/v7/finance/spark?symbols=PLUG,AMD&range=1d&formated=true
Only returns close data though =/

Trending Symbols

GET https://query1.finance.yahoo.com/v1/finance/trending/US?lang=en-US&region=US&count=5&corsDomain=finance.yahoo.com

Live Streamer

import * as Promise from 'bluebird'
import * as Nstream from 'stream'
import * as Nhttp from 'http'
import * as Nnet from 'net'
import * as Axios from 'axios'

const YH_IDK_KEY = {
	a00: 'Ask',
	a50: 'AskSize',
	b00: 'Bid',
	b60: 'BidSize',
	c10: 'Change',
	c63: 'ChangeRealtime',
	c64: 'DisputedChangeRealtimeAfterHours',
	c85: 'ChangeRealtimeAfterHours',
	c86: 'PercentChangeRealtimeAfterHours',
	g53: 'DayLow',
	h53: 'DayHigh',
	j10: 'MarketCap',
	l84: 'PriceRealtime',
	l86: 'PriceRealtimeAfterHours',
	p20: 'PercentChange',
	p43: 'PercentChangeRealtime',
	p44: 'PercentChangeRealtimeAfterHours',
	t53: 'DisputedTimestampForCommodities',
	t54: 'DisputedTimestampForStocks',
	v53: 'Volume',
	v00: 'Volume2',
	l10: 'LastSalePrice',
	t10: 'LastSaleTime',
	l90: 'EcnQuoteLastValue',
}
const YH_KEY_IDK = {
	Ask: 'a00',
	AskSize: 'a50',
	Bid: 'b00',
	BidSize: 'b60',
	Change: 'c10',
	ChangeRealtime: 'c63',
	DisputedChangeRealtimeAfterHours: 'c64',
	ChangeRealtimeAfterHours: 'c85',
	PercentChangeRealtimeAfterHours: 'p44',
	DayLow: 'g53',
	DayHigh: 'h53',
	MarketCap: 'j10',
	PriceRealtime: 'l84',
	PriceRealtimeAfterHours: 'l86',
	PercentChange: 'p20',
	PercentChangeRealtime: 'p43',
	DisputedTimestampForCommodities: 't53',
	DisputedTimestampForStocks: 't54',
	Volume: 'v53',
	Volume2: 'v00',
	LastSalePrice: 'l10',
	LastSaleTime: 't10',
	EcnQuoteLastValue: 'l90',
}

let total = 0
let stream = null as Nhttp.IncomingMessage
function startYahooStream() {
	return Promise.resolve().then(function() {
		let keys = [
			YH_KEY_IDK.Ask,
			YH_KEY_IDK.AskSize,
			YH_KEY_IDK.Bid,
			YH_KEY_IDK.BidSize,
			YH_KEY_IDK.Volume,
			YH_KEY_IDK.Volume2,
			YH_KEY_IDK.PriceRealtime,
			YH_KEY_IDK.PriceRealtimeAfterHours,
			YH_KEY_IDK.LastSalePrice,
			YH_KEY_IDK.EcnQuoteLastValue,
			YH_KEY_IDK.LastSaleTime,
			YH_KEY_IDK.DisputedTimestampForStocks,
		]
		return Axios.default.get('https://streamerapi.finance.yahoo.com/streamer/1.0', {
			params: {
				s: 'NVDA,PLUG,AMD',
				k: keys.join(','),
				r: 0,
				callback: 'parent.yfs_u1f',
				mktmcb: 'parent.yfs_mktmcb',
				gencallback: 'parent.yfs_gencb',
				mu: 1,
				lang: 'en-US',
				region: 'US',
				localize: 0,
			},
			responseType: 'stream',
		}).then<any>(function({ data }) {
			return new Promise<string>(function(resolve, reject) {
				stream = data as Nhttp.IncomingMessage

				stream.on('data', function(chunk) {
					try {
						let input = chunk.toString()
						let jsons = [] as Array<{ [key: string]: any }>

						let split = input.split('yfs_u1f(')
						split.shift()
						total = total + split.length
						split.forEach(function(v) {
							let i = v.indexOf(');}')
							let parsed = v.substring(0, i).replace(/(['"])?([a-zA-Z0-9_]+)(['"])?:/g, '"$2": ')
							if (!parsed) return;
							jsons.push(JSON.parse(parsed))
						})

						let sendi = {}
						jsons.forEach(function(s) {
							let symbol = Object.keys(s)[0]
							if (!sendi[symbol]) sendi[symbol] = {};
							let v = s[symbol]
							Object.keys(v).forEach(function(k) {
								let n = parseFloat(v[k])
								if (n == 0) return;
								let key = YH_IDK_KEY[k] as string
								sendi[symbol][key] = n
							})
						})
						console.log('sendi >', JSON.stringify(sendi, null, 4))

					} catch (error) {
						console.error('quotes.startYahooStream > catch > error', error)
					}
				})

				stream.on('end', resolve)
				stream.on('error', reject)
			})
		})

	}).catch(function(error) {
		console.error('quotes.startYahooStream > error', error)
		return Promise.resolve()

	}).finally(function() {
		console.warn('quotes.stream > FINALLY')
		if (stream) {
			stream.removeAllListeners()
			stream.destroy()
			stream = null
		}
		return startYahooStream()
	})
}
startYahooStream()

console.log('sendi >', JSON.stringify(sendi, null, 4))

sendi >
{
    NVDA: {
        LastSalePrice: 149.6,
        Volume2: 92,
        Ask: 149.5,
        AskSize: 1800,
        Bid: 148.2,
        BidSize: 100,
        PriceRealtime: 149.6
    }
}

Hope you find this helpful!

The text was updated successfully, but these errors were encountered:

pilwon · 2017-06-10T19:17:36Z

@roblav96 Thanks for the excellent info. It might help @gadicc with his PR #42!

36bits · 2017-11-02T19:37:23Z

@roblav96 Just wanted to second the thanks for this info. The Yahoo stock quote CSV API at http://download.finance.yahoo.com/d/quotes.csv?s=FOO,BAR has been giving a '999 Request Denied' error since yesterday. The 'Multiple Symbol Quote' URL above was invaluable in getting my quotes going again.