Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
ddb0bad
feat: add Flutter integration tests and BrowserStack CI pipeline
teodorciuraru Sep 2, 2025
3d04c79
feat: add comprehensive Flutter multi-platform CI and Ditto Cloud syn…
teodorciuraru Sep 2, 2025
3fe9c66
fix: resolve Flutter analyzer failures and .env asset issues
teodorciuraru Sep 2, 2025
e0a1e82
fix: create .env file directly in flutter_app directory for CI
teodorciuraru Sep 2, 2025
2892527
feat: add iOS BrowserStack app testing with unsigned .ipa builds
teodorciuraru Sep 2, 2025
fee2c9f
fix: repair broken YAML structure in Flutter BrowserStack workflow
teodorciuraru Sep 2, 2025
403f8a6
fix: move iOS BrowserStack test script to separate file to resolve YA…
teodorciuraru Sep 2, 2025
7c083f9
feat: completely rewrite Flutter BrowserStack for real device testing
teodorciuraru Sep 2, 2025
f86f256
feat: simplify Flutter BrowserStack integration to run existing integ…
teodorciuraru Sep 2, 2025
4c250f5
fix: use Flutter version 3.x instead of 'stable' to match working pr-…
teodorciuraru Sep 2, 2025
54debc9
fix: rewrite Flutter BrowserStack workflow using iOS pattern
teodorciuraru Sep 2, 2025
f78135c
feat: add Flutter iOS BrowserStack testing alongside Android
teodorciuraru Sep 2, 2025
bd48e45
feat: complete Flutter BrowserStack Ditto Sync testing with Web support
teodorciuraru Sep 2, 2025
4db17a0
simplify Flutter BrowserStack CI to follow JavaScript Web pattern
teodorciuraru Sep 2, 2025
71a9acb
fix Flutter BrowserStack CI: use unit tests instead of integration tests
teodorciuraru Sep 2, 2025
a61287a
restore real Flutter BrowserStack testing for Android, iOS, and Web
teodorciuraru Sep 2, 2025
52151ec
fix Flutter BrowserStack workflow YAML syntax
teodorciuraru Sep 2, 2025
85a35e6
fix Flutter BrowserStack YAML syntax with heredoc approach
teodorciuraru Sep 2, 2025
3636b13
fix Flutter BrowserStack YAML with direct Python heredoc execution
teodorciuraru Sep 2, 2025
03518f0
separate Flutter BrowserStack jobs properly - Android, Web, iOS
teodorciuraru Sep 2, 2025
42cff2b
fix: add missing BrowserStack credentials to Flutter workflow env
teodorciuraru Sep 2, 2025
63bb0d0
fix: resolve iOS and Web BrowserStack testing issues
teodorciuraru Sep 2, 2025
34f6986
fix: resolve iOS IPA signing and Web BrowserStack Local tunnel issues
teodorciuraru Sep 2, 2025
2888b98
fix: use Swift BrowserStack unsigned IPA pattern for Flutter iOS
teodorciuraru Sep 2, 2025
b434413
fix: replace iOS Python/Appium with native Flutter testing framework
teodorciuraru Sep 2, 2025
11b5722
fix: add array bounds checking to prevent IndexError exceptions
teodorciuraru Sep 2, 2025
a6215f8
docs: improve document ID parsing with shared utility and documentation
teodorciuraru Sep 2, 2025
317fe5d
feat: implement real Flutter integration tests on BrowserStack devices
teodorciuraru Sep 3, 2025
4aae774
fix: resolve Python externally-managed-environment error for Appium
teodorciuraru Sep 3, 2025
83c642f
fix: add missing flutter_driver dependency for BrowserStack Appium in…
teodorciuraru Sep 3, 2025
271c835
feat: adopt proven Flutter BrowserStack pattern from PR #134
teodorciuraru Sep 3, 2025
e90b518
feat: implement separate BrowserStack jobs for iOS, Android and Web p…
teodorciuraru Sep 3, 2025
d20771b
fix: resolve iOS device build path and BrowserStack project organization
teodorciuraru Sep 3, 2025
f1c5291
feat: implement proper BrowserStack Flutter integration testing API
teodorciuraru Sep 3, 2025
80bb46f
fix: correct BrowserStack Flutter API parameters for project creation
teodorciuraru Sep 3, 2025
43bd4fa
fix: use compiled test APK for Android BrowserStack integration tests
teodorciuraru Sep 3, 2025
903a8bb
fix: add Gradle assembleDebugAndroidTest to build instrumentation tes…
teodorciuraru Sep 3, 2025
e32ed42
fix: improve BrowserStack API error handling and debugging
teodorciuraru Sep 3, 2025
792ae19
fix: correct BrowserStack Flutter API endpoints
teodorciuraru Sep 3, 2025
76a78c5
fix: correct BrowserStack build endpoint from /builds to /build
teodorciuraru Sep 3, 2025
870c4ef
feat: implement correct Flutter iOS integration testing with artifact…
teodorciuraru Sep 3, 2025
cc45adf
fix: move iOS build to macOS runner
teodorciuraru Sep 4, 2025
f38f57e
fix: add pod install for iOS dependencies
teodorciuraru Sep 4, 2025
ff420ef
feat: add BrowserStack Local testing support
teodorciuraru Sep 4, 2025
89f40e1
fix: disable iOS code signing for BrowserStack test builds
teodorciuraru Sep 4, 2025
5a550f8
debug: add detailed logging for iOS test package creation
teodorciuraru Sep 4, 2025
b77b81e
fix: correct iOS test package file path resolution
teodorciuraru Sep 4, 2025
291517b
debug: add iOS test package location detection
teodorciuraru Sep 4, 2025
db2b847
fix: correct BrowserStack iOS test package path
teodorciuraru Sep 4, 2025
bacf87b
fix: remove unsupported local testing from iOS Flutter BrowserStack i…
teodorciuraru Sep 4, 2025
ed3c01b
fix: remove local testing from Android BrowserStack integration
teodorciuraru Sep 4, 2025
2f54ec0
sync test working
cameron1024 Sep 9, 2025
4097909
fixup other tests
cameron1024 Sep 9, 2025
8e01b2a
BP tests
cameron1024 Sep 11, 2025
a42394f
fix
cameron1024 Sep 11, 2025
7d1a724
Merge branch 'cm/flutter-browserstack' of https://github.com/getditto…
cameron1024 Sep 11, 2025
48c4f8e
bidirectional big peer sync working
cameron1024 Sep 11, 2025
14e3bd0
add build script
cameron1024 Sep 11, 2025
e38e224
add flutter linux to github actions
cameron1024 Sep 11, 2025
d3a4468
cd flutter_app
cameron1024 Sep 11, 2025
4e7def0
remove assets
cameron1024 Sep 11, 2025
a0c9370
add linux project
cameron1024 Sep 11, 2025
73ce64b
more dependencies
cameron1024 Sep 11, 2025
307c756
add test script
cameron1024 Sep 12, 2025
0c34820
try macos
cameron1024 Sep 12, 2025
795d44d
create macos projects
cameron1024 Sep 12, 2025
78dcb22
try with android emulator
cameron1024 Sep 12, 2025
7d4301a
set version
cameron1024 Sep 12, 2025
b49521a
ls
cameron1024 Sep 12, 2025
1fbbbf3
trying linux again
cameron1024 Sep 12, 2025
4060a5c
fix script
cameron1024 Sep 12, 2025
cd2dbce
add assertions to tests
cameron1024 Sep 12, 2025
7df01f7
fix(cameron): my basic reading comprehension
cameron1024 Sep 12, 2025
541300f
inject secrets properly
cameron1024 Sep 12, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
292 changes: 292 additions & 0 deletions .github/scripts/flutter-browserstack-test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,292 @@
#!/usr/bin/env python3
"""
BrowserStack cross-browser testing script for Ditto Flutter Web application.
This script runs automated tests on multiple browsers using BrowserStack to verify
the basic functionality of the Ditto Tasks Flutter web application.
"""
import time
import json
import sys
import os
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.chrome.options import Options as ChromeOptions
from selenium.webdriver.firefox.options import Options as FirefoxOptions

def parse_run_id_from_doc_id(doc_id):
"""
Extract the run ID from a document ID.
Expected formats:
- github_test_RUNID_RUNNUMBER (index 2 contains RUNID)
- github_test_web_RUNID_RUNNUMBER (index 3 contains RUNID)
Returns RUNID if format matches, else returns the original doc_id.
"""
parts = doc_id.split('_')
if len(parts) >= 4 and parts[2] == 'web':
return parts[3] # Web format
elif len(parts) >= 3:
return parts[2] # Standard format
else:
return doc_id

def wait_for_sync_document(driver, doc_id, max_wait=30):
"""Wait for a specific document to appear in the task list."""
print(f"Waiting for document '{doc_id}' to sync...")
# Extract the run ID from the document ID using shared parsing logic
run_id = parse_run_id_from_doc_id(doc_id)
print(f"Looking for GitHub Run ID: {run_id}")

start_time = time.time()

while (time.time() - start_time) < max_wait:
try:
# Flutter web renders tasks differently than React - look for text content
# Search for elements containing the run ID
task_elements = driver.find_elements(By.XPATH, f"//*[contains(text(), '{run_id}')]")

# Check each element for our GitHub test task
for element in task_elements:
try:
element_text = element.text.strip()
# Check if the run ID appears in the text and it's our GitHub test task
if run_id in element_text and "GitHub Test Task" in element_text:
print(f"✓ Found synced document: {element_text}")
return True
except:
continue

except Exception as e:
# Only log errors occasionally to reduce noise
pass

time.sleep(1) # Check every second

print(f"❌ Document not found after {max_wait} seconds")
return False

def run_test(browser_config):
"""Run automated test on specified browser configuration."""
print(f"Starting test on {browser_config['browser']} {browser_config['browser_version']} on {browser_config['os']}")

# Set up BrowserStack options
bs_options = {
'browserVersion': browser_config['browser_version'],
'os': browser_config['os'],
'osVersion': browser_config['os_version'],
'sessionName': f"Ditto Flutter Tasks Test - {browser_config['browser']} {browser_config['browser_version']}",
'buildName': f"Ditto Flutter Web Build #{os.environ.get('GITHUB_RUN_NUMBER', '0')}",
'projectName': 'Ditto Flutter Web',
'local': 'true',
'debug': 'true',
'video': 'true',
'networkLogs': 'true',
'consoleLogs': 'info'
}

# Create browser-specific options
if browser_config['browser'].lower() == 'chrome':
options = ChromeOptions()
options.set_capability('bstack:options', bs_options)
elif browser_config['browser'].lower() == 'firefox':
options = FirefoxOptions()
options.set_capability('bstack:options', bs_options)
else:
# Fallback to Chrome for other browsers
options = ChromeOptions()
options.set_capability('bstack:options', bs_options)

driver = None
try:
# Initialize WebDriver with modern options
driver = webdriver.Remote(
command_executor=f"https://{os.environ['BROWSERSTACK_USERNAME']}:{os.environ['BROWSERSTACK_ACCESS_KEY']}@hub.browserstack.com/wd/hub",
options=options
)

# Set additional session info
driver.execute_script('browserstack_executor: {"action": "setSessionName", "arguments": {"name": "Ditto Flutter Tasks Web Test"}}')

# Navigate to the application
print("Navigating to Flutter web application...")
driver.get("http://localhost:3000")

# Wait for page to load
WebDriverWait(driver, 30).until(
lambda d: d.execute_script("return document.readyState") == "complete"
)

print("Page loaded, waiting for Flutter app initialization...")

# Wait for Flutter to initialize - look for the app title
try:
WebDriverWait(driver, 30).until(
EC.presence_of_element_located((By.XPATH, "//*[contains(text(), 'Ditto Tasks')]"))
)
print("Flutter app title found")
except:
print("Flutter app title check failed, continuing with extended wait...")
time.sleep(10)

# Wait a bit more for Ditto initialization
print("Waiting for Ditto initialization...")
time.sleep(5)

# Check for GitHub test document
github_doc_id = os.environ.get('GITHUB_TEST_DOC_ID')
if github_doc_id:
print(f"Checking for GitHub test document: {github_doc_id}")
if wait_for_sync_document(driver, github_doc_id):
print("✓ GitHub test document successfully synced from Ditto Cloud")
else:
print("❌ GitHub test document did not sync within timeout period")
raise Exception("Failed to sync test document from Ditto Cloud")
else:
print("⚠ No GitHub test document ID provided, skipping sync verification")

# Verify key UI elements are present
print("Verifying UI elements...")

# Check for app title
app_title = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.XPATH, "//*[contains(text(), 'Ditto Tasks')]"))
)
print("✓ App title found")

# Check for FloatingActionButton (FAB) - Flutter web renders this as a button
try:
fab = driver.find_element(By.XPATH, "//button[contains(@class, 'floating') or @aria-label='Add Task' or contains(@title, 'Add')]")
print("✓ FloatingActionButton found")
except:
# Fallback: look for any button that might be the FAB
try:
fab = driver.find_element(By.XPATH, "//button[last()]")
print("✓ Potential FAB found (fallback)")
except:
print("⚠ FAB not found using standard selectors")
fab = None

# Check for Sync toggle - look for switch or checkbox elements
try:
sync_toggle = driver.find_element(By.XPATH, "//*[contains(text(), 'Sync') or contains(text(), 'Active')]")
print(f"✓ Sync element found: {sync_toggle.text}")
except:
print("⚠ Sync toggle not found")

# Test basic functionality - add a task if FAB is available
if fab:
print("Testing task creation...")

try:
fab.click()
time.sleep(2) # Wait for dialog to appear

# Look for text input field in dialog
text_field = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.XPATH, "//input[@type='text'] | //textarea"))
)

text_field.clear()
text_field.send_keys("Test Task from BrowserStack Flutter")

# Look for Add button in dialog
add_button = driver.find_element(By.XPATH, "//button[contains(text(), 'Add') or contains(text(), 'Submit')]")
add_button.click()

# Wait a bit for the task to appear
time.sleep(3)

# Check if task appeared in list
try:
task_item = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.XPATH, "//*[contains(text(), 'Test Task from BrowserStack Flutter')]"))
)
print("✓ Task created successfully and appears in list")
except:
print("⚠ Task may not have appeared in list")

except Exception as e:
print(f"⚠ Task creation test failed: {str(e)}")
else:
print("⚠ FAB not found, skipping task creation test")

# Take a screenshot for verification
driver.save_screenshot(f"flutter_test_screenshot_{browser_config['browser']}.png")
print(f"✓ Screenshot saved for {browser_config['browser']}")

# Report success to BrowserStack
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "All tests passed successfully"}}')
print("✓ Reported success status to BrowserStack")

print(f"✅ Test completed successfully on {browser_config['browser']}")
return True

except Exception as e:
print(f"❌ Test failed on {browser_config['browser']}: {str(e)}")
if driver:
try:
driver.save_screenshot(f"flutter_error_screenshot_{browser_config['browser']}.png")
print(f"Error screenshot saved for {browser_config['browser']}")

# Report failure to BrowserStack
driver.execute_script(f'browserstack_executor: {{"action": "setSessionStatus", "arguments": {{"status":"failed", "reason": "Test failed: {str(e)[:100]}"}}}}')
print("✓ Reported failure status to BrowserStack")
except:
print("⚠ Failed to save screenshot or report status")
return False

finally:
if driver:
driver.quit()

def main():
"""Main function to run all browser tests."""
# Browser configurations to test
browsers = [
{
'browser': 'Chrome',
'browser_version': '120.0',
'os': 'Windows',
'os_version': '11'
},
{
'browser': 'Firefox',
'browser_version': '121.0',
'os': 'Windows',
'os_version': '11'
}
]

# Run tests on all browsers
results = []
for browser_config in browsers:
success = run_test(browser_config)
results.append({
'browser': f"{browser_config['browser']} {browser_config['browser_version']}",
'os': f"{browser_config['os']} {browser_config['os_version']}",
'success': success
})

# Print summary
print("\n=== Test Summary ===")
passed = 0
total = len(results)
for result in results:
status = "✅ PASSED" if result['success'] else "❌ FAILED"
print(f"{result['browser']} on {result['os']}: {status}")
if result['success']:
passed += 1

print(f"\nOverall: {passed}/{total} tests passed")

# Exit with appropriate code
if passed == total:
print("🎉 All tests passed!")
sys.exit(0)
else:
print("💥 Some tests failed!")
sys.exit(1)

if __name__ == "__main__":
main()
62 changes: 62 additions & 0 deletions .github/scripts/ios-browserstack-test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
#!/usr/bin/env python3
import os
import time
from appium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

def test_ios_app():
# BrowserStack capabilities for iOS
desired_caps = {
'platformName': 'iOS',
'platformVersion': '17',
'deviceName': 'iPhone 15',
'app': os.environ.get('IOS_APP_URL'),
'browserstack.user': os.environ.get('BROWSERSTACK_USERNAME'),
'browserstack.key': os.environ.get('BROWSERSTACK_ACCESS_KEY'),
'project': 'Ditto Flutter iOS',
'build': f"iOS Build #{os.environ.get('GITHUB_RUN_NUMBER', '0')}",
'name': 'Ditto Flutter iOS Integration Test'
}

driver = webdriver.Remote('https://hub-cloud.browserstack.com/wd/hub', desired_caps)

try:
# Wait for app to load
print("Waiting for app to initialize...")
time.sleep(10)

# Try to find key UI elements
try:
# Look for app title or main UI elements
title_element = WebDriverWait(driver, 30).until(
lambda d: d.find_element(By.XPATH, "//*[contains(@name, 'Ditto') or contains(@label, 'Tasks')]")
)
print(f"✓ Found app UI element: {title_element.get_attribute('name') or title_element.get_attribute('label')}")
except:
print("⚠ Could not find specific app title, checking for any interactive elements...")

# Fallback - look for any button or interactive element
elements = driver.find_elements(By.XPATH, "//XCUIElementTypeButton | //XCUIElementTypeTextField | //XCUIElementTypeSwitch")
if elements:
print(f"✓ Found {len(elements)} interactive elements - app loaded successfully")
else:
print("❌ No interactive elements found - app may not have loaded properly")
raise Exception("App did not load interactive elements")

# Mark test as passed
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "iOS app loaded and basic UI verified"}}')
print("✅ iOS app test completed successfully")
return True

except Exception as e:
print(f"❌ iOS app test failed: {str(e)}")
driver.execute_script(f'browserstack_executor: {{"action": "setSessionStatus", "arguments": {{"status":"failed", "reason": "Test failed: {str(e)[:100]}"}}}}')
return False
finally:
driver.quit()

if __name__ == "__main__":
success = test_ios_app()
exit(0 if success else 1)
Loading
Loading