spark-service-MapOutputTrackerWorker.adoc

MapOutputTrackerWorker — MapOutputTracker for Executors

A MapOutputTrackerWorker is the MapOutputTracker for executors.

MapOutputTrackerWorker uses Java’s thread-safe java.util.concurrent.ConcurrentHashMap for mapStatuses internal cache and any lookup cache miss triggers a fetch from the driver’s MapOutputTrackerMaster.

Note	The only difference between MapOutputTrackerWorker and the base abstract class MapOutputTracker is that the mapStatuses internal registry is an instance of the thread-safe java.util.concurrent.ConcurrentHashMap.

Tip	Enable INFO or DEBUG logging level for org.apache.spark.MapOutputTrackerWorker logger to see what happens in MapOutputTrackerWorker. Add the following line to conf/log4j.properties: log4j.logger.org.apache.spark.MapOutputTrackerWorker=DEBUG Refer to Logging.