Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How does the simulation work? #608

Open
Foxius opened this issue Nov 10, 2024 · 9 comments
Open

How does the simulation work? #608

Foxius opened this issue Nov 10, 2024 · 9 comments
Labels
Question Further information is requested

Comments

@Foxius
Copy link

Foxius commented Nov 10, 2024

I want to fork your repository and ran into the following problem:
After I select a streamer to watch and start sending HEAD requests to the video player (as it was in your code), when checking the drop progress (based on DropCurrentSession), it outputs the following:

[2024-11-09 08:08:53,172] - TwitchSimulator - DEBUG -GQL Current Drop Response: [{'data': {'currentUser': {'id': '967038334', 'dropCurrentSession': {'channel': None, 'game': None, 'currentMinutesWatched': 0, 'requiredMinutesWatched': 0, 'dropID': '', '__typename': 'DropCurrentSession'}, '__typename': 'User'}}, 'extensions': {'durationMilliseconds': 57, 'operationName': 'DropCurrentSessionContext', 'requestID': '01JC7EAMEHGHS0RHHC4F8TXADS'}}]

That is, it does not count (as I understand it) viewing. Can you help with this? I'm not asking you to write code, I just want to know: what else besides head queries do you use to simulate watching?

@Foxius
Copy link
Author

Foxius commented Nov 10, 2024

I forgot to clarify - if I go to the stream MYSELF (+- for a minute) before starting the script and then run the script, then it will work correctly

[2024-11-09 08:08:53,180] - TwitchSimulator - DEBUGGING - [OAuth 75a7] GQL's current response to deletion is: [{'data': _BOS_'Current user': _BOS_' ID': '975208803', 'dropCurrentSession': _BOS_' channel': _BOS_'id': '418513309', 'name': 'cok1es', 'Display name': 'Cok1es', "__typename": "Channel"}, "game": _BOS_"ID": "516866", "Display name": "STALCRAFT: X", "__typename": "Game"}, "Current views in minutes": 600, "Required views in minutes": 630, 'dropID': '6e82a343-96ad-11ef-b7bd-0a58a9feac02', '__typename': 'DropCurrentSession'}, '__typename': 'User'}}, 'extensions': {'Duration of milliseconds': 87, 'Operation name': 'DropCurrentSessionContext', 'RequestId': '01JC7EAMDWSRCRPGSSQB81CDA4'}}]

@DevilXD DevilXD added the Question Further information is requested label Nov 10, 2024
@DevilXD
Copy link
Owner

DevilXD commented Nov 10, 2024

Hello,

I want to fork your repository and ran into the following problem:

[2024-11-09 08:08:53,172] - TwitchSimulator

I'm sorry, but I don't understand the problem here. This application isn't named "TwitchSimulator", and it doesn't look like anything my code would print out. "Forking" a repository involves starting with the existing code at base. If you're changing the existing code, you're kinda on your own when it comes to writing, using and debugging it.

However, if you're making something of your own, that works similar to the miner, then all I can tell you, is that Twitch tends to be buggy in how the progress is being reported. Expect it to not be reported at all, the response being complete nonsense (what you've got), the response looking okay, but pointing to a drop that you're not currently actually mining right now, the drop ID being correct but the rest of the response not making any sense, or actually getting something that looks like a good response and progress. In my miner, for the DropCurrentSessionContext operation and response, if dropID in the returned response doesn't point towards any drop that's currently tracked by the miner, or points towards a drop that cannot be mined on the currently watched channel, then the miner falls back to "pretend mining" mode, where the most likely drop to be mined has their progress increased. That's it.

@Foxius
Copy link
Author

Foxius commented Nov 10, 2024

Yes. I didn't say it right, sorry. I'm not making a fork, I'm making my own miner based on yours.
I understand correctly - if it is not possible to get the number of minutes that are left until the end of viewing, it looks at how many minutes are needed and outputs itself? And in principle I would like to understand more about “pretend mining” mode....
Also, as I understand, it counts minutes but just does not show them?

@Foxius
Copy link
Author

Foxius commented Nov 10, 2024

i tested it now. My results:

5 minutes sending head requests and get that response from DropCurrentSessionContext:

[2024-11-10 20:34:13,863] - TwitchSimulator - DEBUG - [OAuth g078] GQL Current Drop Response: [{'data': {'currentUser': {'id': '775030133', 'dropCurrentSession': {'channel': None, 'game': None, 'currentMinutesWatched': 0, 'requiredMinutesWatched': 0, 'dropID': '', '__typename': 'DropCurrentSession'}, '__typename': 'User'}}, 'extensions': {'durationMilliseconds': 58, 'operationName': 'DropCurrentSessionContext', 'requestID': '01JCBBC5DBHFGJ66715T34HHND'}}]

i join to stream nmplol and see this in my console:

[2024-11-10 20:34:55,630] - TwitchSimulator - DEBUG - [OAuth g078] GQL Current Drop Response: [{'data': {'currentUser': {'id': '775030133', 'dropCurrentSession': {'channel': {'id': '21841789', 'name': 'nmplol', 'displayName': 'Nmplol', '__typename': 'Channel'}, 'game': {'id': '263490', 'displayName': 'Rust', '__typename': 'Game'}, 'currentMinutesWatched': 1, 'requiredMinutesWatched': 60, 'dropID': 'fe9601ff-9d1b-11ef-a7ae-0a58a9feac02', '__typename': 'DropCurrentSession'}, '__typename': 'User'}}, 'extensions': {'durationMilliseconds': 73, 'operationName': 'DropCurrentSessionContext', 'requestID': '01JCBBDE6JYJG2CKP1ZD9WDRX5'}}]

So he didn't count the five minutes of viewing time until I logged on to the stream :(

@DevilXD
Copy link
Owner

DevilXD commented Nov 10, 2024

And in principle I would like to understand more about “pretend mining” mode.... Also, as I understand, it counts minutes but just does not show them?

The part of the code responsible for that can be found here:

TwitchDropsMiner/twitch.py

Lines 922 to 934 in d4cfcb5

# Solution 2: If GQL fails, figure out which drop we're most likely mining
# right now, and then bump up the minutes on that drop
if not handled:
if (drop := self.get_active_drop(channel)) is not None:
drop.bump_minutes()
drop_text = (
f"{drop.name} ({drop.campaign.game}, "
f"{drop.current_minutes}/{drop.required_minutes})"
)
logger.log(CALL, f"Drop progress from active search: {drop_text}")
handled = True
else:
logger.log(CALL, "No active drop could be determined")

get_active_drop just iterates over all campaigns that are tracked, and calls can_earn(watching_channel) on them. If a campaign can be earned on the currently watched channel (game matches, ACL matches or not present, etc.) All drops that can be earned from such campaigns are put into a list of options to choose from. The miner then sorts the list and outputs the drop that has the least amount of remaining minutes left. That drop then has it's progress "artificially" increased by 1 minute. That's about it. Most of the time, these theoretical progress increases match the progress you can see on the website. The progress will "resync" back to it's true value on the next inventory refresh, or when Twitch decides to actually return proper progress in one of the later responses.

5 minutes sending head requests
So he didn't count the five minutes of viewing time

There's actually a 5 minutes period of "uncertainty" that I've noticed happens on Twitch side, each and every time you deviate from steady mining on a single channel. The progress is misreported during that period the most, and is the main reason the "pretend mining" mode even exists. The period starts every time you start mining a particular channel, so it also happens after channel switches.

After those 5 minutes of steadily mining the channel pass, the progress reporting goes more or less back to normal. You have the option to either ignore it, or mask it with "pretend mining" like I did.

As long as the progress keeps increasing as reported by the inventory page, you're doing everything right, even without proper progress reporting.

@Foxius
Copy link
Author

Foxius commented Nov 10, 2024

What if it doesn't show up in my inventory either? I logged into a new account and sent requests to view it. the inventory was empty.
Here's the code I'm using (it's test code, not from the project I threw above, but it's literally copied into the project so they work the same way).

import asyncio
import aiohttp
from typing import Dict
import logging
from exceptions import GQLException, MinerException

logger = logging.getLogger("TwitchDropsSimulator")
logger.setLevel(logging.DEBUG)

console_handler = logging.StreamHandler()
console_handler.setLevel(logging.DEBUG)

formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
console_handler.setFormatter(formatter)

logger.addHandler(console_handler)

import requests

class TwitchSimulator:
    def __init__(self, oauth_token: str, username: str):
        self.oauth_token = oauth_token
        self.username = username
        self.session: aiohttp.ClientSession | None = None
        self.api_url = "https://gql.twitch.tv/gql"

    async def connect(self):
        self.session = aiohttp.ClientSession(headers={
                "Authorization": f"OAuth {self.oauth_token}",
                "Client-ID": "kimne78kx3ncx6brgo4mv6wki5h1ko",
            })

    async def close(self):
        if self.session:
            await self.session.close()

    async def gql_request(self, data: str) -> Dict:
        async with self.session.post(self.api_url, data=data) as response:
            if response.status != 200:
                logger.error(f"GQL request failed with status {response.status}")
                raise GQLException(f"GQL request failed with status {response.status}")
            data = await response.json()
            if "errors" in data:
                logger.error(f"GQL request returned errors: {data['errors']}")
                raise GQLException(f"GQL request returned errors: {data['errors']}")
            return data

    async def get_stream_url(self) -> str:
        data = '''[{
            "operationName": "PlaybackAccessToken_Template",
            "query": "query PlaybackAccessToken_Template($login: String!, $isLive: Boolean!, $vodID: ID!, $isVod: Boolean!, $playerType: String!, $platform: String!) {  streamPlaybackAccessToken(channelName: $login, params: {platform: $platform, playerBackend: \\"mediaplayer\\", playerType: $playerType}) @include(if: $isLive) {    value    signature   authorization { isForbidden forbiddenReasonCode }   __typename  }  videoPlaybackAccessToken(id: $vodID, params: {platform: $platform, playerBackend: \\"mediaplayer\\", playerType: $playerType}) @include(if: $isVod) {    value    signature   __typename  }}",
            "variables": {
                "isLive": true,
                "login": "''' + self.username + '''",
                "isVod": false,
                "vodID": "",
                "playerType": "site",
                "platform": "web"
            }
        }]'''
        
        print(data)
        response = await self.gql_request(data)
        logger.debug(f"GQL Stream URL Response: {response}")

        token_data = response[0]["data"]["streamPlaybackAccessToken"]
        playback_url = f'https://usher.ttvnw.net/api/channel/hls/{self.username}.m3u8?sig={token_data["signature"]}&token={token_data["value"]}'
        
        async with self.session.get(playback_url) as resp:
            m3u8_data = await resp.text()

        # Найти последний сегмент потока
        lines = m3u8_data.splitlines()
        for line in reversed(lines):
            if line.startswith("http"):
                return line

        raise MinerException("No stream segments found")


    async def simulate_watch(self):
        try:
            stream_segment_url = await self.get_stream_url()
            logger.info(f"Simulating watch on segment: {stream_segment_url}")

            while True:
                async with self.session.head(stream_segment_url) as resp:
                    if resp.status == 200:
                        logger.info("HEAD request successful, simulating watch.")
                    else:
                        logger.warning(f"HEAD request failed with status {resp.status}")
                await asyncio.sleep(20)  # Отправка HEAD запроса каждые 20 секунд
        except Exception as e:
            logger.error(f"Simulation error: {e}")

    async def get_drop_progress(self) -> Dict:
        data = f'''[{{"operationName":"DropCurrentSessionContext","variables":{{"channelLogin":"{self.username}"}}, "extensions":{{"persistedQuery":{{"version":1,"sha256Hash":"4d06b702d25d652afb9ef835d2a550031f1cf762b193523a92166f40ea3d142b"}}}}}}]'''
        
        response = await self.gql_request(data)
        logger.debug(f"GQL Current Drop Response: {response}")

        # drop_session = response[0].get("data", {}).get("currentUser", {}).get("dropCurrentSession", {})
        # if not drop_session:
            # raise MinerException("No active drop session available")

        # return drop_session

    async def run(self):
        try:
            await self.connect()

            stream_segment_url = await self.get_stream_url()
            logger.info(f"Simulating initial watch on segment: {stream_segment_url}")

            # for i in range(5):
                # async with self.session.head(stream_segment_url) as resp:
                    # if resp.status == 200:
                        # logger.info(f"HEAD request #{i + 1} successful.")
                    # else:
                        # logger.warning(f"HEAD request #{i + 1} failed with status {resp.status}")
                # await asyncio.sleep(20)

            # drop_progress = await self.get_drop_progress()
            # logger.info(f"Drop session data: {drop_progress}")

            # required_minutes = drop_progress.get("requiredMinutesWatched", 0)
            # current_minutes = drop_progress.get("currentMinutesWatched", 0)
            # remaining_minutes = required_minutes - current_minutes

            # logger.info(f"Remaining minutes to drop: {remaining_minutes}")

            # watch_task = asyncio.create_task(self.simulate_watch())
            # remaining_minutes = 123012321321092133213128231890231890
            while True:
                # logger.info(f"Remaining minutes: {remaining_minutes}")
                await asyncio.sleep(60)
                remaining_minutes -= 1

            watch_task.cancel()
            logger.info("Drop progress completed!")
        finally:
            await self.close()



if __name__ == "__main__":
    OAUTH_TOKEN = ""
    USERNAME = "nmplol"
    
    simulator = TwitchSimulator(OAUTH_TOKEN, USERNAME)
    asyncio.run(simulator.run())

@DevilXD
Copy link
Owner

DevilXD commented Nov 10, 2024

What if it doesn't show up in my inventory either?

You can "prime" a drop to appear in the inventory, by watching a single minute of a stream. Not-started campaigns are only available on the campaigns page otherwise. Once it shows up in the inventory, you can track it until it gets finished and disappears off there.

I logged into a new account and sent requests to view it. the inventory was empty.

Well, you're doing something wrong then. I can't really help you develop the entire thing, my project took weeks (or even months) of research to arrive at the point it's currently at now. If you're not getting the progress, then it can be anything. The most troublesome issue I can see with the code you've posted, is using the Web version Client ID (starts with "kimne"), as the Twitch web client has been protected by the integrity system by Twitch themselves, in response to this and some other projects starting to automate drop acquisition. Using it usually won't lead you anywhere. I recommend trying again with either the SmartBox or Mobile client IDs instead.

@Foxius
Copy link
Author

Foxius commented Nov 10, 2024

SmartBox или Mobile.

Can I find them in your code?

@DevilXD
Copy link
Owner

DevilXD commented Nov 10, 2024

Yes. Most of the answers for your issues can be solved by just studying the existing code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants