Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems indexing MOD09GQ.005 #32

Open
mrlathy opened this issue Mar 18, 2016 · 0 comments
Open

Problems indexing MOD09GQ.005 #32

mrlathy opened this issue Mar 18, 2016 · 0 comments

Comments

@mrlathy
Copy link

mrlathy commented Mar 18, 2016

Trying to import MOD09GQ.005 file in Spatial Hadoop 2.3 (installation shipped with apache hadoop).
File was taken from here:
http://e4ftl01.cr.usgs.gov/MOLT/MOD09GQ.005/2016.02.17/MOD09GQ.A2016048.h35v09.005.2016050064358.hdf

File was copied to HDFS as 1.hdf. Index command:
bin/shadoop index 1.hdf 1.grid mbr:0,0,100,100 sindex:rtree shape:nasapoint
returns the following error:

 bin/shadoop index 1.hdf 1.grid mbr:0,0,100,100 sindex:rtree shape:nasapoint
16/03/18 19:13:09 INFO util.NativeCodeLoader: Loaded the native-hadoop library
16/03/18 19:13:09 WARN snappy.LoadSnappy: Snappy native library not loaded
16/03/18 19:13:09 INFO mapred.FileInputFormat: No block filter specified
16/03/18 19:13:09 INFO mapred.FileInputFormat: Total input paths to process : 1
16/03/18 19:13:09 INFO operations.Repartition: Reading a sample of 10%
16/03/18 19:13:09 INFO mapred.FileInputFormat: No block filter specified
16/03/18 19:13:09 INFO mapred.FileInputFormat: Total input paths to process : 1
16/03/18 19:13:10 INFO mapred.JobClient: Running job: job_local161844959_0001
16/03/18 19:13:10 INFO mapred.LocalJobRunner: Waiting for map tasks
16/03/18 19:13:10 INFO mapred.LocalJobRunner: Starting task: attempt_local161844959_0001_m_000000_0
16/03/18 19:13:10 INFO util.ProcessTree: setsid exited with exit code 0
16/03/18 19:13:10 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5a4ca8a8
16/03/18 19:13:10 INFO mapred.MapTask: Processing split: file:/root/spatialhadoop-2.3-hadoop-1.2.1/1.hdf:0+71498628
16/03/18 19:13:10 INFO mapred.SpatialRecordReader: Open a SpatialRecordReader to file: file:/root/spatialhadoop-2.3-hadoop-1.2.1/1.hdf
16/03/18 19:13:10 INFO mapred.MapTask: numReduceTasks: 1
16/03/18 19:13:10 INFO mapred.MapTask: io.sort.mb = 100
16/03/18 19:13:10 INFO mapred.MapTask: data buffer = 79691776/99614720
16/03/18 19:13:10 INFO mapred.MapTask: record buffer = 262144/327680
16/03/18 19:13:10 INFO mapred.LocalJobRunner: Map task executor complete.
16/03/18 19:13:10 WARN mapred.LocalJobRunner: job_local161844959_0001
java.lang.Exception: java.lang.NumberFormatException: empty String
        at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:354)
Caused by: java.lang.NumberFormatException: empty String
        at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1011)
        at java.lang.Double.parseDouble(Double.java:540)
        at edu.umn.cs.spatialHadoop.io.TextSerializerHelper.consumeDouble(TextSerializerHelper.java:182)
        at edu.umn.cs.spatialHadoop.core.Point.fromText(Point.java:136)
        at edu.umn.cs.spatialHadoop.nasa.NASAPoint.fromText(NASAPoint.java:77)
        at edu.umn.cs.spatialHadoop.operations.Sampler$Map.map(Sampler.java:122)
        at edu.umn.cs.spatialHadoop.operations.Sampler$Map.map(Sampler.java:69)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
        at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:223)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
16/03/18 19:13:11 INFO mapred.JobClient:  map 0% reduce 0%
16/03/18 19:13:11 INFO mapred.JobClient: Job complete: job_local161844959_0001
16/03/18 19:13:11 INFO mapred.JobClient: Counters: 0
16/03/18 19:13:11 INFO mapred.JobClient: Job Failed: NA
java.io.IOException: Job failed!
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1357)
        at edu.umn.cs.spatialHadoop.operations.Sampler.sampleMapReduceWithRatio(Sampler.java:211)
        at edu.umn.cs.spatialHadoop.operations.Sampler.sample(Sampler.java:543)
        at edu.umn.cs.spatialHadoop.operations.Repartition.packInRectangles(Repartition.java:494)
        at edu.umn.cs.spatialHadoop.operations.Repartition.packInRectangles(Repartition.java:463)
        at edu.umn.cs.spatialHadoop.operations.Repartition.repartitionLocal(Repartition.java:590)
        at edu.umn.cs.spatialHadoop.operations.Repartition.repartition(Repartition.java:676)
        at edu.umn.cs.spatialHadoop.operations.Repartition.main(Repartition.java:721)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at edu.umn.cs.spatialHadoop.operations.Main.main(Main.java:115)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant