Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: saves twitter cookies to envs #598

Merged
merged 4 commits into from
Dec 17, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion electron/main.js
Original file line number Diff line number Diff line change
Expand Up @@ -311,7 +311,8 @@ const createMainWindow = async () => {

try {
await scraper.login(username, password, email);
return { success: true };
const cookies = await scraper.getCookies();
return { success: true, cookies };
Comment on lines 313 to +315
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the call to scraper.login always use the password? I think we first should check whether we already have the cookies.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably we can check if the cookies already exists in that path (TWIKIT_COOKIES_PATH), but I guess (correct me if I'm wrong) this logic is only executed once, at the time of registering?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's what it should happen, but not what it is happening. On validation failure it tries again. Or what if the user closes the app before starting the agent?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is FE code only @dvilelaf. this is for the form where a user first time enters their credentials, we only once validate them (by using scraper.login) and write the cookies recieved. we don't allow to change that, so it won't be called ever again from FE

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that. So if a user:

  • Opens Pearl for the first time
  • Fills in the form and data is validated
  • User closes the app
  • User re-opens the app

The login wont be called again?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I say this because this morning I saw 2 succesfull logins being made from the validation form.

Copy link
Collaborator Author

@Tanya-atatakai Tanya-atatakai Dec 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not from FE side. but I assume from BE it will - when starting an agent. and here @OjusWiZard might need to update that (or correct me)

Comment on lines 313 to +315
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the call to scraper.login always use the password? I think we first should check whether we already have the cookies.

} catch (error) {
console.error('Twitter login error:', error);
return { success: false, error: error.message };
Expand Down
22 changes: 14 additions & 8 deletions frontend/components/SetupPage/SetupYourAgent/SetupYourAgent.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -118,18 +118,20 @@ const SetupYourAgentForm = ({ serviceTemplate }: SetupYourAgentFormProps) => {

// validate the twitter credentials
setSubmitButtonText('Validating Twitter credentials...');
const isTwitterCredentialsValid = electronApi?.validateTwitterLogin
? await validateTwitterCredentials(
values.xEmail,
values.xUsername,
values.xPassword,
electronApi.validateTwitterLogin,
)
: false;
const { isValid: isTwitterCredentialsValid, cookies } =
electronApi?.validateTwitterLogin
? await validateTwitterCredentials(
values.xEmail,
values.xUsername,
values.xPassword,
electronApi.validateTwitterLogin,
)
: { isValid: false };
setTwitterCredentialsValidationStatus(
isTwitterCredentialsValid ? 'valid' : 'invalid',
);
if (!isTwitterCredentialsValid) return;
if (!cookies) return;

// wait for agent setup to complete
setSubmitButtonText('Setting up agent...');
Expand All @@ -151,6 +153,10 @@ const SetupYourAgentForm = ({ serviceTemplate }: SetupYourAgentFormProps) => {
...serviceTemplate.env_variables.TWIKIT_PASSWORD,
value: values.xPassword,
},
TWIKIT_COOKIES: {
...serviceTemplate.env_variables.TWIKIT_COOKIES,
value: cookies,
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will write it in the format of:

'{"guest_id_marketing":"111","guest_id_ads":"111","personalization_id":"111","guest_id":"111","kdt":"111","twid":"111","ct0":"111","auth_token":"111","att":"111"}'

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's only a key named _twitter_sess missing when I compare it to the cookies that are generated by the BE validation. @dvilelaf do you know if that will cause any issues?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is needed, but I'm not 100% sure. I definitely have it in my cookies

},
GENAI_API_KEY: {
...serviceTemplate.env_variables.GENAI_API_KEY,
value: values.geminiApiKey,
Expand Down
28 changes: 19 additions & 9 deletions frontend/components/SetupPage/SetupYourAgent/validation.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { ServiceTemplate } from '@/client';
import { StakingProgramId } from '@/enums/StakingProgram';
import { ServicesService } from '@/service/Services';
import { XCookie } from '@/types/Cookies';

/**
* Validate the Google Gemini API key
Expand All @@ -21,6 +22,14 @@ export const validateGeminiApiKey = async (apiKey: string) => {
}
};

const formatXCookies = (cookiesArray: XCookie[]) => {
const cookiesObject: Record<string, string> = {};
cookiesArray.forEach((cookie) => {
cookiesObject[cookie.key] = cookie.value;
});
return JSON.stringify(cookiesObject);
};

/**
* Validate the Twitter credentials
*/
Expand All @@ -36,25 +45,26 @@ export const validateTwitterCredentials = async (
email: string;
username: string;
password: string;
}) => Promise<{ success: boolean }>,
) => {
if (!email || !username || !password) return false;
}) => Promise<{ success: boolean; cookies?: XCookie[] }>,
): Promise<{ isValid: boolean; cookies?: string }> => {
if (!email || !username || !password) return { isValid: false };

try {
const isValidated = await validateTwitterLogin({
const result = await validateTwitterLogin({
username,
password,
email,
});
if (isValidated.success) {
return true;

if (result.success && result.cookies) {
return { isValid: true, cookies: formatXCookies(result.cookies) };
}

console.error('Error validating Twitter credentials:', isValidated);
return false;
console.error('Error validating Twitter credentials:', result);
return { isValid: false };
} catch (error) {
console.error('Unexpected error validating Twitter credentials:', error);
return false;
return { isValid: false };
}
};

Expand Down
8 changes: 7 additions & 1 deletion frontend/constants/serviceTemplates.ts
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ export const SERVICE_TEMPLATES: ServiceTemplate[] = [
provision_type: EnvProvisionType.COMPUTED,
},
CELO_LEDGER_RPC: {
name: 'Base ledger RPC',
name: 'Celo ledger RPC',
description: '',
value: '',
provision_type: EnvProvisionType.COMPUTED,
Expand All @@ -212,6 +212,12 @@ export const SERVICE_TEMPLATES: ServiceTemplate[] = [
value: '',
provision_type: EnvProvisionType.USER,
},
TWIKIT_COOKIES: {
name: 'Twitter cookies',
description: '',
value: '',
provision_type: EnvProvisionType.USER,
},
GENAI_API_KEY: {
name: 'Gemini api key',
description: '',
Expand Down
3 changes: 2 additions & 1 deletion frontend/context/ElectronApiProvider.tsx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { get } from 'lodash';
import { createContext, PropsWithChildren } from 'react';

import { XCookie } from '@/types/Cookies';
import { ElectronStore, ElectronTrayIconStatus } from '@/types/ElectronApi';

type ElectronApiContextProps = {
Expand Down Expand Up @@ -40,7 +41,7 @@ type ElectronApiContextProps = {
username: string;
password: string;
email: string;
}) => Promise<{ success: boolean }>;
}) => Promise<{ success: boolean; cookies?: XCookie[] }>;
};

export const ElectronApiContext = createContext<ElectronApiContextProps>({
Expand Down
14 changes: 14 additions & 0 deletions frontend/types/Cookies.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
/**
* Cookies returned by agent-twitter-client
* when log in
*/
export type XCookie = {
key: string;
value: string;
domain?: string;
path?: string;
secure?: boolean;
httpOnly?: boolean;
sameSite?: string;
expires?: string;
};
2 changes: 2 additions & 0 deletions operate/services/manage.py
Original file line number Diff line number Diff line change
Expand Up @@ -575,6 +575,7 @@ def _deploy_service_onchain_from_safe( # pylint: disable=too-many-statements,to
"TWIKIT_USERNAME",
"TWIKIT_EMAIL",
"TWIKIT_PASSWORD",
"TWIKIT_COOKIES",
"TWIKIT_COOKIES_PATH",
]
):
Expand All @@ -591,6 +592,7 @@ def _deploy_service_onchain_from_safe( # pylint: disable=too-many-statements,to
username=service.env_variables["TWIKIT_USERNAME"]["value"],
email=service.env_variables["TWIKIT_EMAIL"]["value"],
password=service.env_variables["TWIKIT_PASSWORD"]["value"],
cookies=service.env_variables["TWIKIT_COOKIES"]["value"],
cookies_path=cookies_path,
),
"TWIKIT_COOKIES_PATH": str(cookies_path),
Expand Down
19 changes: 14 additions & 5 deletions operate/services/utils/memeooorr.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ def await_for_cookies() -> dict:


async def async_get_twitter_cookies(
username: str, email: str, password: str, cookies_path: Path
username: str, email: str, password: str, cookies: str, cookies_path: Path
) -> Optional[str]:
"""Verifies that the Twitter credentials are correct and get the cookies"""

Expand All @@ -59,13 +59,22 @@ async def async_get_twitter_cookies(
valid_cookies = False
cookies_path.parent.mkdir(exist_ok=True, parents=True)

# If cookies exist, try with those and validate
if cookies_path.exists():
if not valid_cookies and cookies:
print("Checking the provided cookies")
client.set_cookies(json.loads(cookies))
user = await client.user()
print(f"User from cookies: {user.screen_name}")
valid_cookies = user.screen_name == username

# If cookies file exist, try with those and validate
if not valid_cookies and cookies_path.exists():
print("Checking the cookies file")
with open(cookies_path, "r", encoding="utf-8") as cookies_file:
cookies = json.load(cookies_file)
client.set_cookies(cookies)

user = await client.user()
print(f"User from cookies file: {user.screen_name}")
valid_cookies = user.screen_name == username

if not valid_cookies:
Expand All @@ -89,9 +98,9 @@ async def async_get_twitter_cookies(


def get_twitter_cookies(
username: str, email: str, password: str, cookies_path: Path
username: str, email: str, password: str, cookies: str, cookies_path: Path
) -> Optional[str]:
"""get_twitter_cookies"""
return asyncio.run(
async_get_twitter_cookies(username, email, password, cookies_path)
async_get_twitter_cookies(username, email, password, cookies, cookies_path)
)
Loading