Readme Updated

.markdownlint.json

@@ -4,6 +4,7 @@
   "MD045": false,
   "MD003": false,
   "MD013": {
-    "code_blocks": false
+    "code_blocks": false,
+    "line_length": 120
   }
 }

.prettierignore

@@ -7,3 +7,4 @@ pathfinder/
 orchestrator_venv/
 build/
 node_modules/
+scripts/

CHANGELOG.md

@@ -6,6 +6,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## Added
 
+- readme: setup instructions added
 - tests: http_client tests added
 - Added Atlantic proving service integration
 - setup functions added for cloud and db

README.md

@@ -4,11 +4,42 @@ The Madara orchestrator is designed to be an additional service which runs in
 parallel to Madara and handles various critical jobs that ensure proper block
 processing, proof generation, data submission and state transitions.
 
+> 📝 **Note**: These instructions are verified for Ubuntu systems. While most steps remain similar
+> for macOS, some package names and installation commands may differ.
+
+## Table of Contents
+
+- [Overview](#-overview)
+- [Architecture](#️-architecture)
+  - [Job Processing Model](#job-processing-model)
+  - [Queue Structure](#queue-structure)
+  - [Workflow](#workflow)
+- [Technical Requirements](#️-technical-requirements)
+  - [System Dependencies](#system-dependencies)
+  - [Core Dependencies](#core-dependencies)
+- [Installation & Setup](#-installation--setup)
+  - [Building from Source](#building-from-source)
+  - [Local Development Setup](#local-development-setup)
+  - [Setup Mode](#setup-mode)
+  - [Run Mode](#run-mode)
+  - [Command Line Options](#command-line-options)
+- [Configuration](#️-configuration)
+  - [AWS Configuration](#aws-configuration)
+  - [Prover Configuration](#prover-configuration)
+  - [Database Configuration](#database-configuration)
+- [Testing](#-testing)
+  - [Local Environment Setup](#local-environment-setup)
+  - [Types of Tests](#types-of-tests)
+  - [Running Tests](#running-tests)
+- [Monitoring](#-monitoring)
+- [Error Handling](#-error-handling)
+- [Additional Resources](#additional-resources)
+
 ## 📋 Overview
 
 The Madara Orchestrator coordinates and triggers five primary jobs in sequence,
-managing their execution through a centralized queue system, alowing
-for multiple orchestrator to run together!
+managing their execution through a centralized queue system, allowing
+for multiple orchestrators to run together!
 
 1. **SNOS (Starknet OS) Job** 🔄
 
@@ -73,26 +104,150 @@ The system uses dedicated queues for managing different job phases:
 
 ## 🛠️ Technical Requirements
 
-### Prerequisites
+### System Dependencies
+
+> For macOS users, use `brew install` instead of `apt install` for these dependencies.
+
+- Build essentials (`build-essential`)
+- OpenSSL (`libssl-dev`)
+- Package config (`pkg-config`)
+- Python 3.9 with development files
+- GMP library (`libgmp-dev`)
+
+### Core Dependencies
+
+- [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
+- [Rust](https://www.rust-lang.org/tools/install)
+- [Madara Node](https://github.com/madara-alliance/madara)
+  - Required for block processing
+  - Follow setup instructions at [Madara Documentation](https://github.com/madara-alliance/madara)
+- Prover Service (ATLANTIC)
+- MongoDB for job management
+- AWS services (or Localstack for local development):
+  - SQS for queues
+  - S3 for data storage
+  - SNS for alerts
+  - EventBridge for scheduling
+
+> 🚨 **Important Note**: Currently, Madara doesn't support the `get_storage_proof` endpoint.
+>
+> Therefore, you need to run Pathfinder alongside Madara:
+>
+> - Madara will run in sequencer mode
+> - Pathfinder will sync with Madara
+> - The orchestrator will use Pathfinder's RPC URL for SNOS and state update fetching
+>
+> This setup ensures proper functionality until the `get_storage_proof` endpoint is implemented in Madara.
 
-1. **Madara Node**
+## 🚀 Installation & Setup
 
-   - Required for block processing
-   - Follow setup instructions at [Madara Documentation](https://github.com/madara-alliance/madara)
+### Building from Source
 
-2. **Prover Service**
+1. **Install System Dependencies**
 
-   - ATLANTIC must be running
+   ```bash
+   # Ubuntu/Debian
+   sudo apt-get update
+   sudo apt install build-essential openssl pkg-config libssl-dev
+   sudo apt install python3.9 python3.9-venv python3.9-distutils libgmp-dev python3.9-dev
 
-3. **Infrastructure Dependencies**
-   - MongoDB for job management
-   - AWS services (or Localstack for local development):
-     - SQS for queues
-     - S3 for data storage
-     - SNS for alerts
-     - EventBridge for scheduling
+   # For macOS
+   brew install openssl pkg-config gmp [email protected]
+   ```
 
-## 🚀 Installation & Setup
+2. **Install Rust** (Cross-platform)
+
+   ```bash
+   curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
+   source ~/.bashrc  # Or source ~/.zshrc for macOS
+   ```
+
+3. **Clone Repository**
+
+   ```bash
+   git clone https://github.com/madara-alliance/madara-orchestrator.git
+   cd madara-orchestrator
+   git submodule update --init
+   ```
+
+4. **Build SNOS**
+
+   ```bash
+   make snos
+   ```
+
+5. **Build Project**
+
+   ```bash
+   cargo build --release
+   ```
+
+### Local Development Setup
+
+1. **Install Docker** (Cross-platform)
+   Follow the official installation guides:
+
+   - [Ubuntu Installation Guide](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository)
+   - [macOS Installation Guide](https://docs.docker.com/desktop/install/mac-install/)
+
+2. **Install Foundry** (Cross-platform)
+
+   ```bash
+   curl -L https://foundry.paradigm.xyz | bash
+   foundryup
+   ```
+
+3. **Start Local Services**
+
+   ```bash
+   # Start MongoDB
+   docker run -d -p 27017:27017 mongo
+
+   # Start Localstack
+   docker run -d -p 4566:4566 localstack/localstack@sha256:763947722c6c8d33d5fbf7e8d52b4bddec5be35274a0998fdc6176d733375314
+
+   # Start Anvil in a separate terminal
+   anvil --block-time 1
+   ```
+
+4. **Setup Mock Proving Service**
+
+   ```bash
+   # Create a simple HTTP server that always returns 200
+   # Example using Python
+   python3 -m http.server 8888 &
+
+   # Set the mock prover URL in your .env
+   MADARA_ORCHESTRATOR_ATLANTIC_SERVICE_URL=http://localhost:8888
+   ```
+
+5. **Deploy Mock Verifier Contract**
+
+   You can deploy the mock verifier contract in two ways:
+
+   a. **Using Test Implementation**
+
+   - Reference the implementation in `e2e-tests/tests.rs` which contains `AnvilSetup`
+   - This handles the deployment of both the StarkNet core contract and verifier contract
+
+   ```rust
+   // From e2e-tests/tests.rs
+   let anvil_setup = AnvilSetup::new();
+   let (starknet_core_contract_address, verifier_contract_address) = anvil_setup.deploy_contracts().await;
+   ```
+
+   b. **Manual Deployment**
+
+   - Alternatively, you can deploy the mock verifier directly using Foundry
+   - The contract should implement the same interface as used in the tests
+   - Set the deployed address in your environment:
+
+   ```bash
+   MADARA_ORCHESTRATOR_VERIFIER_ADDRESS=<deployed-contract-address>
+   ```
+
+Note: The mock services are intended for development and testing purposes only.
+In production, you'll need to use actual proving services and verifier contracts.
 
 ### Setup Mode
 
@@ -111,7 +266,15 @@ if the process fails mid-way.
 Run mode executes the orchestrator's job processing workflow. Example command:
 
 ```bash
-RUST_LOG=info cargo run --release --bin orchestrator run --atlantic --aws --settle-on-ethereum --aws-s3 --aws-sqs --aws-sns --da-on-ethereum --mongodb
+RUST_LOG=info cargo run --release --bin orchestrator run \
+    --atlantic \
+    --aws \
+    --settle-on-ethereum \
+    --aws-s3 \
+    --aws-sqs \
+    --aws-sns \
+    --da-on-ethereum \
+    --mongodb
 ```
 
 ### Command Line Options
@@ -171,7 +334,8 @@ AWS_SECRET_ACCESS_KEY=<your-secret>
 AWS_REGION=<your-region>
 ```
 
-Note: These configurations are also picked up from
+Note: These configurations are also picked up from your AWS credentials file (~/.aws/credentials)
+or environment variables if not specified in the .env file.
 
 ### Prover Configuration
 
@@ -213,9 +377,25 @@ It requires a `Otel-collector` url to be able to send metrics/logs/traces.
 - Failed jobs are tracked in the database for manual inspection after maximum retries
 - Integrated telemetry system for monitoring job failures
 
-## 🧪 Testing
+## 📓 Testing
+
+### Local Environment Setup
+
+Before running tests, ensure you have:
+
+1. **Required Services Running**:
 
-The Madara Orchestrator supports three types of tests:
+   - MongoDB on port 27017
+   - Localstack on port 4566
+   - Anvil (local Ethereum node)
+
+2. **Environment Configuration**:
+
+   ```bash
+   export MADARA_ORCHESTRATOR_ETHEREUM_SETTLEMENT_RPC_URL=<ethereum-rpc-url>
+   export MADARA_ORCHESTRATOR_RPC_FOR_SNOS=<snos-rpc-url>
+   export AWS_REGION=us-east-1
+   ```
 
 ### Types of Tests
 
@@ -226,23 +406,11 @@ The Madara Orchestrator supports three types of tests:
    - Uses mocked proving endpoints
 
 2. **Integration & Unit Tests** 🔌
+   - Tests component interactions
+   - Verifies individual functionalities
 
 ### Running Tests
 
-#### Required Services
-
-- MongoDB running on port 27017
-- Localstack running on port 4566
-- Anvil (local Ethereum node)
-
-#### Environment Configuration
-
-```bash
-export MADARA_ORCHESTRATOR_ETHEREUM_SETTLEMENT_RPC_URL=<ethereum-rpc-url>
-export MADARA_ORCHESTRATOR_RPC_FOR_SNOS=<snos-rpc-url>
-export AWS_REGION=us-east-1
-```
-
 #### Running E2E Tests
 
 ```bash
@@ -251,9 +419,6 @@ RUST_LOG=info cargo test --features testing test_orchestrator_workflow -- --noca
 
 #### Running Integration and Unit Tests
 
-The orchestrator uses LLVM coverage testing to ensure comprehensive test coverage
-of the codebase.
-
 ```bash
 RUST_LOG=debug RUST_BACKTRACE=1 cargo llvm-cov nextest \
     --release \
@@ -272,11 +437,19 @@ This command:
 - Excludes E2E tests from coverage analysis
 - Runs tests sequentially (single thread)
 - Continues testing even if failures occur
-- Enables debug logging and full backtraces for better error diagnosis
+- Enables debug logging and full backtraces for better error
+  diagnosis
 
 The coverage report (`lcov.info`) can be used with various code coverage
 visualization tools.
 
 ## 📓 More Information
 
 - Read the architecture present at `./docs/orchestrator_da_sequencer_diagram.png`
+
+## Additional Resources
+
+- Architecture Diagram: See `./docs/orchestrator_da_sequencer_diagram.png`
+- [Madara Documentation](https://github.com/madara-alliance/madara)
+- [LocalStack Documentation](https://docs.localstack.cloud/)
+- [Foundry Documentation](https://book.getfoundry.sh/)