Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve loading/indexing feedback #11

Open
leonardossz opened this issue May 12, 2017 · 1 comment
Open

Improve loading/indexing feedback #11

leonardossz opened this issue May 12, 2017 · 1 comment

Comments

@leonardossz
Copy link
Contributor

leonardossz commented May 12, 2017

The loading process can take quite a long time hence the user needs some kind of feedback if a query is issued while loader is still running.

@leonardossz
Copy link
Contributor Author

Also we need to store previous loader failures to let API users know that stargraph did not fulfill the intended request. Here is a clear case where this is happening though the user still receives a successful acknowledge. Remember that loader is asynchronous by design.

2017-07-20 18:10:21,610 INFO  [index] Loading dbpedia-2016.documents, [reset=true, limit=-1] 
2017-07-20 18:10:21,610 WARN  [index] Old data will be DELETED! 
2017-07-20 18:10:21,627 ERROR [core] Failed to load documents from file /home/lsouza/StarGraph/data/dbpedia-2016/documents/documents.json. 
2017-07-20 18:10:21,627 ERROR [index] Loader failure. 
net.stargraph.StarGraphException: Fail to initialize data provider: dbpedia-2016.documents
	at net.stargraph.core.Stargraph.createDataProvider(Stargraph.java:239)
	at net.stargraph.core.index.BaseIndexer.doBeforeLoad(BaseIndexer.java:177)
	at net.stargraph.core.index.BaseIndexer.lambda$doLoad$0(BaseIndexer.java:239)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused by: net.stargraph.StarGraphException: java.io.FileNotFoundException: File '/home/lsouza/StarGraph/data/dbpedia-2016/documents/documents.json' does not exist
	at net.stargraph.core.DocumentIterator.<init>(DocumentIterator.java:69)
	at net.stargraph.core.DocumentProviderFactory.create(DocumentProviderFactory.java:45)
	at net.stargraph.core.Stargraph.createDataProvider(Stargraph.java:230)
	... 7 common frames omitted
Caused by: java.io.FileNotFoundException: File '/home/lsouza/StarGraph/data/dbpedia-2016/documents/documents.json' does not exist
	at org.apache.commons.io.FileUtils.openInputStream(FileUtils.java:299)
	at org.apache.commons.io.FileUtils.lineIterator(FileUtils.java:1856)
	at net.stargraph.core.DocumentIterator.<init>(DocumentIterator.java:65)
	... 9 common frames omitted
2017-07-20 18:10:21,628 INFO  [index] Loader is finishing.. 
2017-07-20 18:10:21,628 INFO  [index] Waiting for transport to serialize all remaining documents. 
2017-07-20 18:10:21,628 INFO  [index] Optimizing index for reading.. 
2017-07-20 18:10:21,634 INFO  [index] Loader is done. 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant