-
Notifications
You must be signed in to change notification settings - Fork 22
Errors and their possible solutions
Haider Iqbal edited this page Aug 9, 2024
·
3 revisions
Here we list some errors you may encounter in using OLS and their possible solutions.
This error happens during import of the CSV files into Neo4J. Here are some of the important snippets from and example error:
Error in input data
11:29:58 Caused by:ERROR in input
11:29:58 data source: BufferedCharSeeker[source:/hps/nobackup/parkinso/spot/ols4/prod/slurm_pipeline/./output_csv/dron_classes.csv, position:7784628224, line:723073
.
.
.
original error: Tried to read a field larger than buffer size 134217728. A common cause of this is that a field has an unterminated quote and so will try to seek until the next quote, which ever line it may be on. This should not happen if multi-line fields are disabled, given that the fields contains no new-line characters. ...
Increase the --read-buffer-size in ./dataload/load_into_neo4j.sh for data loads, or in ./dev-testing/load_test_into_neo4j.sh for test cases.
This error occurs during the import of CSV files into Neo4J. Error looks like below:
08:21:25 Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
08:21:25 at org.neo4j.csv.reader.SectionedCharBuffer.<init>(SectionedCharBuffer.java:69)
08:21:25 at org.neo4j.csv.reader.AutoReadingSource.<init>(AutoReadingSource.java:37)
08:21:25 at org.neo4j.csv.reader.CharSeekers.charSeeker(CharSeekers.java:53)
08:21:25 at org.neo4j.internal.batchimport.input.csv.EagerParserChunker.<init>(EagerParserChunker.java:52)
08:21:25 at org.neo4j.internal.batchimport.input.csv.CsvInputIterator.<init>(CsvInputIterator.java:74)
08:21:25 at org.neo4j.internal.batchimport.input.csv.CsvInputIterator.<init>(CsvInputIterator.java:88)
08:21:25 at org.neo4j.internal.batchimport.input.csv.CsvGroupInputIterator.next(CsvGroupInputIterator.java:89)
08:21:25 at org.neo4j.internal.batchimport.ExhaustingEntityImporterRunnable.run(ExhaustingEntityImporterRunnable.java:53)
08:21:25 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
08:21:25 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
08:21:25 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
08:21:25 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
08:21:25 at java.base/java.lang.Thread.run(Thread.java:829)
08:21:25 at org.neo4j.internal.helpers.NamedThreadFactory$2.run(NamedThreadFactory.java:110)
You would need to go to the Neo4J instance and update the headspace in the config file. It's usually placed under the conf folder inside the neo4J instance folder. The values that need to be updated are the following:
# Java Heap Size: by default the Java heap size is dynamically calculated based
# on available system resources. Uncomment these lines to set specific initial
# and maximum heap size.
dbms.memory.heap.initial_size=8g
dbms.memory.heap.max_size=8g
You can set the heap space as per your system resources. In the example above the heap space is set to 8 GB.
This error occurs during the import of CSV files into Neo4J. Error looks like below:
Import error: Node[2224677,used=false,created=false,rel=-1,prop=-1,labels=Inline(0x0:[]),light] not in use
10:09:19 Caused by:Node[2224677,used=false,created=false,rel=-1,prop=-1,labels=Inline(0x0:[]),light] not in use
10:09:19 org.neo4j.kernel.impl.store.InvalidRecordException: Node[2224677,used=false,created=false,rel=-1,prop=-1,labels=Inline(0x0:[]),light] not in use
10:09:19 at org.neo4j.kernel.impl.store.record.RecordLoad.verify(RecordLoad.java:141)
10:09:19 at org.neo4j.kernel.impl.store.CommonAbstractStore.verifyAfterReading(CommonAbstractStore.java:1074)
10:09:19 at org.neo4j.kernel.impl.store.CommonAbstractStore.readRecordFromPage(CommonAbstractStore.java:897)
10:09:19 at org.neo4j.kernel.impl.store.CommonAbstractStore.readIntoRecord(CommonAbstractStore.java:850)
10:09:19 at org.neo4j.kernel.impl.store.CommonAbstractStore.getRecordByCursor(CommonAbstractStore.java:830)
10:09:19 at org.neo4j.internal.batchimport.NodeSetFirstGroupStep.process(NodeSetFirstGroupStep.java:89)
10:09:19 at org.neo4j.internal.batchimport.NodeSetFirstGroupStep.process(NodeSetFirstGroupStep.java:39)
10:09:19 at org.neo4j.internal.batchimport.staging.ProcessorStep.lambda$receive$1(ProcessorStep.java:84)
10:09:19 at org.neo4j.internal.batchimport.executor.DynamicTaskExecutor$Processor.run(DynamicTaskExecutor.java:220)
10:09:19 at java.base/java.lang.Thread.run(Thread.java:829)
10:09:19 Suppressed: org.neo4j.internal.batchimport.executor.TaskExecutionPanicException: Executor has been shut down in panic
10:09:19 at org.neo4j.internal.batchimport.executor.DynamicTaskExecutor.assertHealthy(DynamicTaskExecutor.java:138)
10:09:19 at org.neo4j.internal.batchimport.executor.DynamicTaskExecutor.submit(DynamicTaskExecutor.java:123)
10:09:19 at org.neo4j.internal.batchimport.staging.ProcessorStep.receive(ProcessorStep.java:77)
10:09:19 at org.neo4j.internal.batchimport.staging.ProducerStep.sendDownstream(ProducerStep.java:74)
10:09:19 at org.neo4j.internal.batchimport.staging.PullingProducerStep.process(PullingProducerStep.java:53)
10:09:19 at org.neo4j.internal.batchimport.staging.ProducerStep.lambda$receive$0(ProducerStep.java:58)
10:09:19 ... 1 more
10:09:19 Caused by: [CIRCULAR REFERENCE: org.neo4j.kernel.impl.store.InvalidRecordException: Node[2224677,used=false,created=false,rel=-1,prop=-1,labels=Inline(0x0:[]),light] not in use]
10:09:19 Suppressed: org.neo4j.internal.batchimport.executor.TaskExecutionPanicException: Executor has been shut down in panic
10:09:19 at org.neo4j.internal.batchimport.executor.DynamicTaskExecutor.assertHealthy(DynamicTaskExecutor.java:138)
10:09:19 at org.neo4j.internal.batchimport.executor.DynamicTaskExecutor.submit(DynamicTaskExecutor.java:123)
10:09:19 at org.neo4j.internal.batchimport.staging.ProcessorStep.receive(ProcessorStep.java:77)
10:09:19 at org.neo4j.internal.batchimport.staging.Downstream.send(Downstream.java:57)
10:09:19 at org.neo4j.internal.batchimport.staging.SendDownstream.apply(SendDownstream.java:58)
10:09:19 at org.neo4j.internal.batchimport.staging.SendDownstream.apply(SendDownstream.java:26)
10:09:19 at org.neo4j.util.concurrent.WorkSync.doSynchronizedWork(WorkSync.java:222)
10:09:19 at org.neo4j.util.concurrent.WorkSync.tryDoWork(WorkSync.java:153)
10:09:19 at org.neo4j.util.concurrent.WorkSync.applyAsync(WorkSync.java:120)
10:09:19 at org.neo4j.internal.batchimport.staging.ProcessorStep.sendDownstream(ProcessorStep.java:176)
10:09:19 at org.neo4j.internal.batchimport.staging.ProcessorStep$Sender.send(ProcessorStep.java:254)
10:09:19 at org.neo4j.internal.batchimport.staging.ReadRecordsStep.process(ReadRecordsStep.java:108)
10:09:19 at org.neo4j.internal.batchimport.staging.ReadRecordsStep.process(ReadRecordsStep.java:41)
10:09:19 ... 3 more
10:09:19 Caused by: [CIRCULAR REFERENCE: org.neo4j.kernel.impl.store.InvalidRecordException: Node[2224677,used=false,created=false,rel=-1,prop=-1,labels=Inline(0x0:[]),light] not in use]
10:09:19 Suppressed: java.lang.RuntimeException: org.neo4j.internal.batchimport.executor.TaskExecutionPanicException: Executor has been shut down in panic
10:09:19 at org.neo4j.internal.batchimport.staging.AbstractStep.issuePanic(AbstractStep.java:148)
10:09:19 at org.neo4j.internal.batchimport.staging.AbstractStep.issuePanic(AbstractStep.java:140)
10:09:19 at org.neo4j.internal.batchimport.staging.ProcessorStep.sendDownstream(ProcessorStep.java:187)
10:09:19 at org.neo4j.internal.batchimport.staging.ProcessorStep$Sender.send(ProcessorStep.java:254)
10:09:19 at org.neo4j.internal.batchimport.staging.ReadRecordsStep.process(ReadRecordsStep.java:108)
10:09:19 at org.neo4j.internal.batchimport.staging.ReadRecordsStep.process(ReadRecordsStep.java:41)
10:09:19 ... 3 more
10:09:19 Caused by: [CIRCULAR REFERENCE: org.neo4j.internal.batchimport.executor.TaskExecutionPanicException: Executor has been shut down in panic]
Rerun this step in the data release pipeline.