Skip to content

Experimental Database

Jump to bottom Edit New page
Kathryn Baker edited this page Aug 18, 2020 · 19 revisions

Wiki > The Backend System > System components > Experimental Database

The experimental database keeps track of:

  1. experiments (experiment table)
  2. experiment team members (experimentteams table)
  3. users in the team (user)
  4. roles in the team (role)

The data is populated centrally from the Experiment Database Populator (alternatively named RB number populator) which is run on control-svcs.

Architecture

The experiment database populator is a Python 3 program that is designed to run centrally and periodically update instrument databases. It does the following:

  • Monitors 'CS:INSTLIST' for instruments that are scheduled
  • Gathers 200 days worth of data from the web services hosted by business apps (Using the credentials stored in a git repository on the local share).
  • Reformats the data slightly to match the structure of the instrument databases
  • Pushes to the instrument database (Using the credentials stored in a git repository on the local share).
  • Optionally repeats every hour

Testing

  • If it is the first time running the populator you will need to install Python 3 (any distribution should do) and ensure that you pip install the requirements listed in requirements.txt
  • The populator has unit tests that are run in jenkins.
  • The DEBUG flag can be set in the main file to push instrument data to your local experiment database.

Deployment

The populator is installed on the linux server control-svcs. This can be accessed using the credentials in the usual place. The program is installed under /home/epics/RB_num_populator and is being run as a cron job every hour. Logs for the program are written into /home/epics/RB_number_populator/logs. Output from the cron job, which will show if the program is not working, is written to /tmp/rb_num_pop.out. So see cron job details you need to su to epics and type crontab -l but they are likely

20 * * * * sh /home/epics/RB_num_populator/rb_number_populator.sh > /tmp/rb_num_pop.out 2>&1

To update the populator, simply git pull in the above directory as epics.

Add a custom footer
Add a custom sidebar
Clone this wiki locally