Done ✅

.gitignore

@@ -0,0 +1,2 @@
+.vscode
+main

ASSIGNMENT.md

@@ -0,0 +1,52 @@
+#Degrees of Separation
+
+With cinema going global these days, every one of the [A-Z]ollywoods are now connected. Use the wealth of data available at [Moviebuff](http://www.moviebuff.com) to see how. 
+
+Write a Go program that behaves the following way:
+
+```
+$ degrees amitabh-bachchan robert-de-niro
+
+Degrees of Separation: 3
+
+1. Movie: The Great Gatsby
+Supporting Actor: Amitabh Bachchan
+Actor: Leonardo DiCaprio
+
+2. Movie: The Wolf of Wall Street
+Actor: Leonardo DiCaprio
+Director: Martin Scorsese
+
+3. Movie: Taxi Driver
+Director: Martin Scorsese
+Actor: Robert De Niro
+```
+
+Your solution should use the Moviebuff data available to figure out the smallest degree of separation between the two people. 
+All the inputs should be Moviebuff URLs for their respective people: For Amitabh Bachchan, his page is on http://www.moviebuff.com/amitabh-bachchan and his Moviebuff URL is `amitabh-bachchan`.
+
+Please do not attempt to scrape the Moviebuff website - All the data is available on an S3 bucket in an easy to parse JSON format here: `https://data.moviebuff.com/{moviebuff_url}`
+
+To solve the example above, your solution would fetch at least the following:
+
+http://data.moviebuff.com/amitabh-bachchan
+
+http://data.moviebuff.com/the-great-gatsby
+
+http://data.moviebuff.com/leonardo-dicaprio
+
+http://data.moviebuff.com/the-wolf-of-wall-street
+
+http://data.moviebuff.com/martin-scorsese
+
+http://data.moviebuff.com/taxi-driver
+
+##Notes
+* If you receive HTTP errors when trying to fetch the data, that might be the CDN throttling you. Luckily, Go has some very elegant idioms for rate limiting :)
+* There may be a discrepancy in some cases where a movie appears on an actor's list but not vice versa. This usually happens when we edit data while exporting it, so feel free to either ignore these mismatches or handle them in some way.
+
+Write a program in any language you want (If you're here from Gophercon, use Go :D) that does this. Feel free to make your own input and output format / command line tool / GUI / Webservice / whatever you want. Feel free to hold the dataset in whatever structure you want, but try not to use external databases - as far as possible stick to your langauage without bringing in MySQL/Postgres/MongoDB/Redis/Etc.
+
+To submit a solution, fork this repo and send a Pull Request on Github.
+
+For any questions or clarifications, raise an issue on this repo and we'll answer your questions as fast as we can.

Makefile

@@ -0,0 +1,10 @@
+.PHONY: build run dev
+
+build:
+	go build -o ./cmd/main ./cmd
+
+run:
+	go run ./cmd/main.go
+
+running:
+	CompileDaemon -build="go build -o ./cmd/main ./cmd" -command=./cmd/main

README.md

@@ -1,52 +1,133 @@
-#Degrees of Separation
+# 🎬 Degrees of Separation - Movie Industry Connections
 
-With cinema going global these days, every one of the [A-Z]ollywoods are now connected. Use the wealth of data available at [Moviebuff](http://www.moviebuff.com) to see how. 
+This Go application finds the degrees of separation between two people in the movie industry using data from Moviebuff. It implements an efficient graph traversal algorithm with concurrent data fetching to determine the shortest path between two industry professionals through their movie collaborations.
 
-Write a Go program that behaves the following way:
+## ✨ Features
 
+- 🚀 **Concurrent Data Fetching**: Efficiently fetches data from external APIs using goroutines
+- 💾 **In-Memory Caching**: Implements a thread-safe caching mechanism for person and movie data
+- 🛡️ **Rate Limiting**: Protects against API throttling with built-in rate limiting
+- 📊 **Performance Monitoring**: Includes pprof endpoints for runtime analysis
+- ⚡ **Graceful Error Handling**: Robust error handling for API failures and invalid inputs
+- ⚙️ **Configuration via Environment Variables**: Flexible configuration through environment variables
+- 🔄 **Resource Management**: Proper channel and goroutine lifecycle management
+
+## 🏗️ Architecture
+
+### 🌐 Data Fetching
+- Uses worker pools for concurrent data fetching from external APIs
+- Implements separate workers for person and movie data
+- Controlled concurrency with predefined worker counts
+
+### 📦 Caching
+- Thread-safe in-memory cache using maps
+- Implements `sync.RWMutex` for concurrent read/write operations
+- Caches both person and movie data after fetching
+
+### 📈 Performance & Monitoring
+- Pprof endpoints for runtime profiling and debugging
+- Periodic logging of goroutine statistics
+- Rate limiting to prevent API throttling
+
+### 🛠️ Error Handling & Resource Management
+- Context-based cancellation for cleanup
+- Proper channel closing mechanisms
+- Graceful error handling for API failures
+- Existence validation (of target person) to prevent long unnecessary searches
+
+## 🔌 API Endpoints
+
+### GET /separation
+Query Parameters:
+- `from`: Moviebuff URL of the first person
+- `to`: Moviebuff URL of the second person
+
+Example:
+```
+GET /separation?from=amitabh-bachchan&to=robert-de-niro
 ```
-$ degrees amitabh-bachchan robert-de-niro
 
-Degrees of Separation: 3
+Response:
+```json
+{
+    "separation": 3
+}
+```
 
-1. Movie: The Great Gatsby
-Supporting Actor: Amitabh Bachchan
-Actor: Leonardo DiCaprio
+## ⚙️ Configuration
 
-2. Movie: The Wolf of Wall Street
-Actor: Leonardo DiCaprio
-Director: Martin Scorsese
+The application can be configured using the following environment variables:
 
-3. Movie: Taxi Driver
-Director: Martin Scorsese
-Actor: Robert De Niro
+- `PORT`: Server port (default: 3001)
+- `PPROF_PORT`: Port for pprof endpoints
+- `LOG_LEVEL`: Logging level (debug/info)
+- `RATE_LIMIT`: API rate limit per minute
+- `PERSON_DATA_FETCH_WORKERS`: Number of concurrent person data fetchers
+- `MOVIE_DATA_FETCH_WORKERS`: Number of concurrent movie data fetchers
+
+## 📥 Getting Started
+
+1. Clone the repository:
+```bash
+git clone https://github.com/AbdulRahimOM/challenge2015.git
+cd challenge2015
 ```
 
-Your solution should use the Moviebuff data available to figure out the smallest degree of separation between the two people. 
-All the inputs should be Moviebuff URLs for their respective people: For Amitabh Bachchan, his page is on http://www.moviebuff.com/amitabh-bachchan and his Moviebuff URL is `amitabh-bachchan`.
+2. Install dependencies:
+```bash
+go mod download
+```
 
-Please do not attempt to scrape the Moviebuff website - All the data is available on an S3 bucket in an easy to parse JSON format here: `https://data.moviebuff.com/{moviebuff_url}`
+3. Copy the environment file and configure:
+```bash
+cp no-secrets.env .env
+# Edit .env with your preferred settings (or keep it as it is to run in default settings)
+```
 
-To solve the example above, your solution would fetch at least the following:
+## 🚀 Running the Application
 
-http://data.moviebuff.com/amitabh-bachchan
+1. Set up environment variables (optional)
+2. Run the application:
+```bash
+go run cmd/main.go
+```
 
-http://data.moviebuff.com/the-great-gatsby
+## 💪 Performance Considerations
 
-http://data.moviebuff.com/leonardo-dicaprio
+1. **Concurrent Data Fetching**
+   - 🔄 Optimized worker pools for API requests
+   - 👥 Separate workers for person and movie data
 
-http://data.moviebuff.com/the-wolf-of-wall-street
+2. **Caching**
+   - 📦 In-memory caching reduces API calls
+   - 🔒 Thread-safe read/write operations
 
-http://data.moviebuff.com/martin-scorsese
+3. **Resource Management**
+   - ⚡ Context-based cancellation
+   - 🧹 Proper cleanup of resources
+   - 🛡️ Rate limiting to prevent throttling
+
+## 📊 Monitoring
+
+### 🔍 Pprof Endpoints
+Access pprof endpoints at:
+```
+http://localhost:{PPROF_PORT}/debug/pprof/
+```
 
-http://data.moviebuff.com/taxi-driver
+Available profiles:
+- 🧵 Goroutine
+- 💾 Heap
+- 🔄 Thread
+- 🚫 Block
+- 📈 CPU profile
 
-##Notes
-* If you receive HTTP errors when trying to fetch the data, that might be the CDN throttling you. Luckily, Go has some very elegant idioms for rate limiting :)
-* There may be a discrepancy in some cases where a movie appears on an actor's list but not vice versa. This usually happens when we edit data while exporting it, so feel free to either ignore these mismatches or handle them in some way.
+## 🔮 Future Improvements
+- ⏰ Add cache expiration mechanism (Relevant, as new movies and persons are added)
+- 🔗 Show connection chain along with degree of seperation
 
-Write a program in any language you want (If you're here from Gophercon, use Go :D) that does this. Feel free to make your own input and output format / command line tool / GUI / Webservice / whatever you want. Feel free to hold the dataset in whatever structure you want, but try not to use external databases - as far as possible stick to your langauage without bringing in MySQL/Postgres/MongoDB/Redis/Etc.
+## 📚 Dependencies
 
-To submit a solution, fork this repo and send a Pull Request on Github.
+- 🚀 [Fiber](github.com/gofiber/fiber/v2) - Web framework
+- 📦 Standard Go libraries for concurrency and HTTP operations
 
-For any questions or clarifications, raise an issue on this repo and we'll answer your questions as fast as we can.

cmd/main.go

@@ -0,0 +1,56 @@
+package main
+
+import (
+	"test/internal/config"
+	"test/internal/tracer"
+	"time"
+
+	"net/http"
+	_ "net/http/pprof"
+
+	"github.com/gofiber/fiber/v2"
+	"github.com/gofiber/fiber/v2/log"
+	"github.com/gofiber/fiber/v2/middleware/limiter"
+)
+
+func main() {
+	setLogLevel()
+
+	app := fiber.New()
+	app.Use(limiter.New(limiter.Config{
+		Max:        config.RateLimit,
+		Expiration: 1 * time.Minute,
+	}))
+
+	app.Get("/seperation", func(c *fiber.Ctx) error {
+		from := c.Query("from")
+		to := c.Query("to")
+		if from == "" || to == "" {
+			return c.Status(fiber.StatusBadRequest).JSON(map[string]string{"error": "from and to query params are required"})
+		}
+
+		seperation, err := tracer.FindSeperation(from, to)
+		if err != nil {
+			return c.Status(fiber.StatusInternalServerError).JSON(map[string]string{"error": err.Error()})
+		}
+
+		return c.Status(fiber.StatusOK).JSON(map[string]interface{}{
+			"seperation": seperation,
+		})
+	})
+	go func() {
+		log.Fatal(http.ListenAndServe(":"+config.PprofPort, nil)) //for pprof, as fiber doesn't use net/http.
+	}()
+	log.Fatal(app.Listen(":" + config.Port))
+}
+
+func setLogLevel() {
+	switch config.LogLevel {
+	case "debug", "DEBUG":
+		log.SetLevel(log.LevelDebug)
+	case "info", "INFO":
+		log.SetLevel(log.LevelInfo)
+	default:
+		log.SetLevel(log.LevelInfo)
+	}
+}

go.mod

@@ -0,0 +1,24 @@
+module test
+
+go 1.23.2
+
+require (
+	github.com/gofiber/fiber/v2 v2.52.6
+	github.com/joho/godotenv v1.5.1
+)
+
+require (
+	github.com/andybalholm/brotli v1.1.0 // indirect
+	github.com/google/uuid v1.6.0 // indirect
+	github.com/klauspost/compress v1.17.11 // indirect
+	github.com/mattn/go-colorable v0.1.13 // indirect
+	github.com/mattn/go-isatty v0.0.20 // indirect
+	github.com/mattn/go-runewidth v0.0.16 // indirect
+	github.com/philhofer/fwd v1.1.3-0.20240916144458-20a13a1f6b7c // indirect
+	github.com/rivo/uniseg v0.2.0 // indirect
+	github.com/tinylib/msgp v1.2.5 // indirect
+	github.com/valyala/bytebufferpool v1.0.0 // indirect
+	github.com/valyala/fasthttp v1.51.0 // indirect
+	github.com/valyala/tcplisten v1.0.0 // indirect
+	golang.org/x/sys v0.28.0 // indirect
+)

go.sum

@@ -0,0 +1,33 @@
+github.com/andybalholm/brotli v1.1.0 h1:eLKJA0d02Lf0mVpIDgYnqXcUn0GqVmEFny3VuID1U3M=
+github.com/andybalholm/brotli v1.1.0/go.mod h1:sms7XGricyQI9K10gOSf56VKKWS4oLer58Q+mhRPtnY=
+github.com/gofiber/fiber/v2 v2.52.6 h1:Rfp+ILPiYSvvVuIPvxrBns+HJp8qGLDnLJawAu27XVI=
+github.com/gofiber/fiber/v2 v2.52.6/go.mod h1:YEcBbO/FB+5M1IZNBP9FO3J9281zgPAreiI1oqg8nDw=
+github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
+github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
+github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
+github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
+github.com/klauspost/compress v1.17.11 h1:In6xLpyWOi1+C7tXUUWv2ot1QvBjxevKAaI6IXrJmUc=
+github.com/klauspost/compress v1.17.11/go.mod h1:pMDklpSncoRMuLFrf1W9Ss9KT+0rH90U12bZKk7uwG0=
+github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
+github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
+github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
+github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
+github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
+github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc=
+github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
+github.com/philhofer/fwd v1.1.3-0.20240916144458-20a13a1f6b7c h1:dAMKvw0MlJT1GshSTtih8C2gDs04w8dReiOGXrGLNoY=
+github.com/philhofer/fwd v1.1.3-0.20240916144458-20a13a1f6b7c/go.mod h1:RqIHx9QI14HlwKwm98g9Re5prTQ6LdeRQn+gXJFxsJM=
+github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY=
+github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
+github.com/tinylib/msgp v1.2.5 h1:WeQg1whrXRFiZusidTQqzETkRpGjFjcIhW6uqWH09po=
+github.com/tinylib/msgp v1.2.5/go.mod h1:ykjzy2wzgrlvpDCRc4LA8UXy6D8bzMSuAF3WD57Gok0=
+github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
+github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
+github.com/valyala/fasthttp v1.51.0 h1:8b30A5JlZ6C7AS81RsWjYMQmrZG6feChmgAolCl1SqA=
+github.com/valyala/fasthttp v1.51.0/go.mod h1:oI2XroL+lI7vdXyYoQk03bXBThfFl2cVdIA3Xl7cH8g=
+github.com/valyala/tcplisten v1.0.0 h1:rBHj/Xf+E1tRGZyWIWwJDiRY0zc1Js+CV5DqwacVSA8=
+github.com/valyala/tcplisten v1.0.0/go.mod h1:T0xQ8SeCZGxckz9qRXTfG43PvQ/mcWh7FwZEA7Ioqkc=
+golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
+golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=