This tools requires python3 python3-psutil python3-psycopg2 python3-six python3-requests python3-unicodecsv
Copy pgsf.example to ~/.pgsf and customize it with your Saleforce and Prosgres credentials.
The PostgreSQL database should already be created, with the schemas created. See install.sql
You need to set up each table one by one. For example:
./tabledesc.py Contact
will create a file mapping/Contact.csv with the list of fields you'll want to synchronize. The default is all fields but formulas and compound fields.
./query_bulk.py Contact
will create a bulk download job, and print a jobid.
./download.py <jobid>
will download all the csv from that job.
Before createtable, you have to update __sync table, and update status at 'error' to stop auto update.
./createtable.py Contact
will create a PostgreSQL version of a SF table
./csv_to_postgres.py <jobid>
will import the job csv files into a PostgreSQL table. After csv_to_postgres, you have to update __sync table, and update status at 'ready' to up auto update.
./query_poll_table.py Contact
will download only updates, and will import them in the PostgreSQL table.
Have a single command that runs query_bulk.py + download.py + csv_to_postgres
Check how many records would be fetch by a query_poll_table, and switch to query if it's above a certain limit.