feat(webapi): batch task submission via dedicated endpoint

[yaugenst-flex]

batch = Batch(simulations=sims, use_batch_endpoint=True)
batch_data = batch.run()

Greptile Summary

This PR introduces a new batch task submission feature that allows submitting multiple simulations via a dedicated server endpoint instead of individual API calls. The implementation adds a new BatchTask class and refactors the existing task management system into a cleaner class hierarchy.

Key Changes:

• New Batch Endpoint: Introduces upload_batch() function and BatchTask class that can create multiple simulation tasks in a single API call via the new /batch-tasks endpoint
• Class Hierarchy Refactor: Creates BaseTask abstract base class and renames SimulationTask to Task (with backward compatibility alias), establishing a cleaner inheritance structure
• Batch Types: Adds BatchType enum with different batch processing types (BATCH, INVDES, PERMUTATION, PARALLEL, MONTE_CARLO, RF_SWEEP) for server-side optimization
• Container Integration: Updates Batch class with use_batch_endpoint flag and batch_type field, allowing users to opt into the new batch submission method
• Performance Optimization: Reduces HTTP overhead by creating all tasks in a single batch request, then uploading simulation files individually

The feature maintains full backward compatibility by defaulting use_batch_endpoint=False, ensuring existing code continues to work unchanged. The implementation follows established patterns in the codebase with proper validation, error handling, and HTTP communication through existing utilities.

Important Files Changed

Click to expand file changes

Filename	Score	Overview
tidy3d/web/core/task_core.py	4/5	Adds BatchTask class and refactors SimulationTask into BaseTask hierarchy with batch creation methods
tidy3d/web/core/types.py	5/5	Adds BatchType enum for server-side batch processing optimization with minimal, focused changes
tidy3d/web/api/container.py	4/5	Introduces batch endpoint configuration and splits job creation logic between individual and batch approaches
tidy3d/web/api/webapi.py	4/5	Implements new upload_batch() function and refactors from SimulationTask to Task throughout the module
tests/test_web/test_webapi.py	5/5	Adds comprehensive test coverage for batch endpoint functionality and backward compatibility

Confidence score: 4/5

• This PR is generally safe to merge with robust backward compatibility and comprehensive test coverage
• Score reflects some unimplemented server-side functionality and extensive TODO comments indicating ongoing development
• Files need attention: tidy3d/web/core/task_core.py for placeholder methods and tidy3d/web/api/container.py for mixed batch/individual handling patterns

Sequence Diagram

sequenceDiagram
    participant User
    participant Batch
    participant BatchTask
    participant webapi
    participant Server
    participant Task

    User->>Batch: create Batch(simulations, use_batch_endpoint=True)
    User->>Batch: access .jobs property
    
    Batch->>Batch: _create_jobs_from_batch()
    Batch->>webapi: upload_batch(simulations, folder_name, ...)
    
    webapi->>webapi: validate simulations
    webapi->>BatchTask: create(simulations, folder_name, ...)
    
    BatchTask->>Server: POST /projects/{folder_id}/batch-tasks
    Server-->>BatchTask: {batchId, tasks: [{taskId, taskName}]}
    
    BatchTask-->>webapi: (batch_id, task_ids)
    webapi-->>Batch: (batch_id, task_ids)
    
    loop for each simulation
        Batch->>Task: get(task_id)
        Task-->>Batch: task instance
        Batch->>Task: upload_simulation(stub, ...)
        Task->>Server: upload simulation file
    end
    
    Batch-->>User: jobs dict with pre-assigned task_ids
    
    User->>Batch: start()
    loop for each job
        Batch->>Task: start()
        Task->>Server: POST /tasks/{task_id}/submit
    end
    
    User->>Batch: monitor()
    loop monitoring
        Batch->>Task: get status
        Task->>Server: GET /tasks/{task_id}/detail
        Server-->>Task: task status
        Task-->>Batch: status
    end
    
    User->>Batch: load(path_dir)
    loop for each job
        Batch->>Task: download(path)
        Task->>Server: download simulation data
        Server-->>Task: simulation data
    end
    Batch-->>User: BatchData

Loading

[greptile-apps]

_{5 files reviewed, 7 comments}

_{Edit Code Review Bot Settings | Greptile}

[greptile-apps]

logic: Individual task deletion in loop could leave batch in inconsistent state if some deletions fail - consider error handling and rollback strategy

[greptile-apps]

logic: Individual task submission in loop could leave batch partially submitted if some fail - consider atomic batch submission or proper error handling

[greptile-apps]

style: Inconsistent naming convention: 'Permutation' and 'Parallel' use title case while others use uppercase. Consider using 'PERMUTATION' and 'PARALLEL' for consistency.

[greptile-apps]

style: The PIE796 noqa comment suppresses a 'prefer using a unique enum value' warning, but having DEFAULT = RF_SWEEP creates an alias that could cause confusion. Consider using a separate method like get_default() instead.

[greptile-apps]

style: Redundant import of json module - already imported at line 4

Suggested change

	import json
	def batch_request_matcher(request):
	json_data = json.loads(request.body)
	assert "tasks" in json_data
	assert "batchType" in json_data
	assert "groupName" in json_data
	for task in json_data["tasks"]:
	assert "groupName" in task
	assert task["groupName"] == json_data["groupName"]
	return True, None

[greptile-apps]

style: Accessing _cached_properties directly may be fragile - consider using a public method if this internal access pattern is common

[greptile-apps]

logic: Modifying simulations dict during iteration could cause unexpected behavior. Consider creating a new dict or processing modifications separately.

[greptile-apps]

_{5 files reviewed, 1 comment}

_{Edit Code Review Bot Settings | Greptile}

[greptile-apps]

logic: The batch_id returned from _submit_batch is not stored or used anywhere, potentially missing functionality for batch tracking