Serposcope rely on these libraries (among many other).
- The web framework ninjaframework.
- Guice for dependency injection.
- The query builder querydsl allows autocompletion of SQL query, prevents SQL injections, etc.
You don't need to know these libraries to contribute to serposcope (you'll learn!) but they worth a check :)
If first build, initialize the querybuilder database :
./core/scripts/sqlcodegen.sh
(or .\core\scripts\sqlcodegen.bat
)
This script will create the structure of the database, it is sourced by querydsl to generate querybuilder classes.
You only need to run this script on first build or each time you change the structure of the database.
If you plan to edit SQL, table name must be in uppercase (allow compatibility of .sql files between h2 and MySQL).
When you code, you want to stay in superdevmode. It allows hot reloading of the classes when you edit code in your IDE (so you don't have to restart serposcope to see changes).
To start serposcope in superdevmode go in the serposcope/web/ folder and run the following command :
mvn ninja:run \
-Dninja.mainClass=serposcope.lifecycle.Daemon \
-Dninja.jvmArgs="-Dlogback.configurationFile=dev/logback-stdout.xml -Dserposcope.datadir=/var/tmp/serposcope -Dserposcope.db.options=;AUTO_SERVER=TRUE"
To build the package for production (compiled javascripts and single jar) :
mvn -Dsinglejar=true -Dminify=true -DskipTests clean install
Assuming no errors occurred, the .jar file will be located at web\target\serposcope.jar.
If you get errors such as "package com.serphacker.serposcope.querybuilder does not exist", ensure you have run sqlcodegen (.sh or .bat) first.
This fork does not contain installation files, so install from 2.15.0 (if required) and then do the following to upgrade.
- If Serposcope is already running, stop it. For example:
systemctl stop serposcope
- Overwrite the original .jar file with the newly built file. For example:
cp serposcope.jar /usr/share/serposcope/serposcope.jar
- Restart Serposcope. For example:
systemctl start serposcope
AS OF 2.16.0, TESTS ARE NO LONGER USED BUT ARE BEING KEPT HERE FOR POSTERITY
To run integration tests :
cd serposcope/core; mvn -Dtest=all failsafe:integration-test
cd serposcope/scraper; mvn -Dtest=all failsafe:integration-test
cd serposcope/web; mvn -Dtest=all -Dninja.standalone.class=serposcope.lifecycle.MyNinjaJetty failsafe:integration-test
Check also for testconfig.DEFAULT.properties in test resources if you want to run extensive integration tests relying on credentials or whatever