GoSpark testing problem

[pbandyop]

Hi,
I have trying to run this project successfully in my local computer - using both the scheduler and workers in same machine but using different terminal and ports of localhost.
Terminal - using scheduler
go test -run MasterMRLineCount -debug
runs fine and I get the following output:

Test: Master MapReduce Line Count...
master ip localhost port :4567
2015/03/30 11:22:10 method AssignJob has wrong number of ins: 5
2015/03/30 11:22:10 method KillWorkers has wrong number of ins: 1
2015/03/30 11:22:10 method Shutdown has wrong number of ins: 1
2015/03/30 11:22:10 method StartRegistrationServer has wrong number of ins: 1
2015/03/30 11:22:10 method WorkersAvailable has wrong number of ins: 1
2015/03/30 11:22:10 RegistrationServer: ready

but if I run the other command:
go test -run KMeans -timeout 1h
then I get the following errors:
`In NewScheduler: master ip localhost port :4567
2015/03/30 11:21:23 method AssignJob has wrong number of ins: 5
2015/03/30 11:21:23 method KillWorkers has wrong number of ins: 1
2015/03/30 11:21:23 method Shutdown has wrong number of ins: 1
2015/03/30 11:21:23 method StartRegistrationServer has wrong number of ins: 1
2015/03/30 11:21:23 method WorkersAvailable has wrong number of ins: 1
2015/03/30 11:21:23 Context [kmeans] is started.
x1:
Iter: 0
--- FAIL: TestKMeans (0.81s)
panic: runtime error: index out of range [recovered]
panic: runtime error: index out of range

goroutine 7 [running]:
testing.func·006()
/usr/local/go/src/testing/testing.go:441 +0x181
spark.(*RDD).Collect(0xc20805a800, 0x0, 0x0, 0x0)
/path_to_project/GoSpark/Spark/src/spark/rdd.go:107 +0x2a8
spark.TestKMeans(0xc20803c090)
/path_to_project/GoSpark/Spark/src/spark/kmeans_test.go:149 +0x8b6
testing.tRunner(0xc20803c090, 0x650cb8)
/usr/local/go/src/testing/testing.go:447 +0xbf
created by testing.RunTests
/usr/local/go/src/testing/testing.go:555 +0xa8b

goroutine 1 [chan receive]:
testing.RunTests(0x50fc98, 0x650ca0, 0xb, 0xb, 0xa88773a4036b1b01)
/usr/local/go/src/testing/testing.go:556 +0xad6
testing.(*M).Run(0xc208030910, 0x65e240)
/usr/local/go/src/testing/testing.go:485 +0x6c
main.main()
spark/_test/_testmain.go:72 +0x1d5

goroutine 8 [IO wait]:
net.(_pollDesc).Wait(0xc208010370, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(_pollDesc).WaitRead(0xc208010370, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(_netFD).accept(0xc208010310, 0x0, 0x724ff0, 0xc20802af30)
/usr/local/go/src/net/fd_unix.go:419 +0x40b
net.(_TCPListener).AcceptTCP(0xc20802e0c8, 0x713000, 0x0, 0x0)
/usr/local/go/src/net/tcpsock_posix.go:234 +0x4e
net.(_TCPListener).Accept(0xc20802e0c8, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/tcpsock_posix.go:244 +0x4c
spark.func·003()
/path_to_project/GoSpark/Spark/src/spark/master.go:121 +0x92
created by spark.(_Master).StartRegistrationServer
/path_to_project/GoSpark/Spark/src/spark/master.go:133 +0x451

goroutine 9 [IO wait]:
net.(_pollDesc).Wait(0xc2080101b0, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(_pollDesc).WaitRead(0xc2080101b0, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(_netFD).accept(0xc208010150, 0x0, 0x724ff0, 0xc20802af70)
/usr/local/go/src/net/fd_unix.go:419 +0x40b
net.(_TCPListener).AcceptTCP(0xc20802e0e0, 0x1abf94, 0x0, 0x0)
/usr/local/go/src/net/tcpsock_posix.go:234 +0x4e
net/http.tcpKeepAliveListener.Accept(0xc20802e0e0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/server.go:1976 +0x4c
net/http.(_Server).Serve(0xc208032300, 0x7277c0, 0xc20802e0e0, 0x0, 0x0)
/usr/local/go/src/net/http/server.go:1728 +0x92
net/http.(_Server).ListenAndServe(0xc208032300, 0x0, 0x0)
/usr/local/go/src/net/http/server.go:1718 +0x154
net/http.ListenAndServe(0x42deb0, 0x6, 0x726618, 0xc20802af50, 0x0, 0x0)
/usr/local/go/src/net/http/server.go:1808 +0xba
spark.func·001()
/path_to_project/GoSpark/Spark/src/spark/master.go:59 +0xc1
created by spark.MakeMaster
/path_to_project/GoSpark/Spark/src/spark/master.go:63 +0x300
exit status 2
FAIL spark 0.830s`

Also, if I run the worker commands then I get register RPC error. If I give the following command:
go test -run BasicWorker -timeout 1h
Test: Basic Worker...
master ip localhost port :4567
worker ip localhost port :5347
Register: RPC localhost:4567 register error
Register: RPC localhost:4567 register error
Register: RPC localhost:4567 register error
Register: RPC localhost:4567 register error
Register: RPC localhost:4567 register error

Also, if I give the following command:
go test -run WorkerSuperUnrel -timeout 1h -debug
I get the same error:

Test: Worker with Unreliable RPC and Crash Reboot ...
master ip localhost port :4567
worker ip localhost port :5911
2015/03/30 11:23:01 RunWorker localhost:5911
2015/03/30 11:23:01 worker: ready
Register: RPC localhost:4567 register error
Register: RPC localhost:4567 register error
Register: RPC localhost:4567 register error
Register: RPC localhost:4567 register error

Any idea if I am missing something or why am I getting these errors?

[peterkty]

Hi pbandyop, sorry for the bugs. This project was a course project, so we may not have tested well enough. And now we don't have much spare cycles to take care of this project. So you may need to debug by yourself by tracing the Go script. Feel free to fork it and patch the bugs you find. Thanks!