Home
Welcome to the lastify wiki!
This is a brief documentation of the lastify programm.
Empty database is created by using the createDB function from createDB.py
For the design layout download test.db from the repository directory and open with
Lastify follows the general design patterns of relational databases
current file_name: lastivy_v6 makes optimized API requests to three different API and stores data in sqlite3 database
- pickle and depickle variables
- extract information from nested dictionaries
- convert json derived dict to DataFrame
- extract information from nested DataFrame
- one function for each API request
- functions for making authorized lastfm API requests (user.getRecommendedArtists). Desired API request is unfortunately offline, thus feature not in use
- json derived dicts are converted to DataFrame. Each API request returns different information, there is a enter_ function for each type of information.
- before entering information to the database the data needs to be matched against existing itemIDs. lookup_ID*, returnID,addIDcol fulfill that function.
-
Every request function has a corresponding Load function except for get_chart.
-
In order to limit database access each API request is coupled to a load function, that accumulate data from Request functions before using Database functions to write to database. Each load function needs to be equipped with a database lock which restricts database access to only one read or write access, to prevent database corruption or reading errors.
-
Load Functions further need to communicate with a corresponding Call_controler (see below) which logs requests and issues request permissions.
-
Load Functions also need to be equipped with a thread specific instance of an output variable registered to the Output_controler (see below), through which progress and errors are communicated.
Each Load function has thread class (singleton). Request functions are interdependent and must be made in a certain order. All requests depend on the user chart data. Therefore chart data is loaded in main thread.
- LOAD threads are started only once and run until finished.
- RESTART threads are continuously restarted until stopping condition is met.
There are several threads that control thread behavior and facilitate debugging
-
output_controler (singleton) A thread that collects output variables from each thread and generates output at a given period. Each thread can create its own instance of an output_variable which is registered to the output_controler. All output is saved in 'output_log.csv'
See docstring documentation of output_controler.py
Run: output_controler.py for an exemplary demonstration of its function
-
call_controler_v2 A thread controls the number of calls made in a given period of time to one API. Uses output_controler see docstring call_controler_v2 Run: call_controler_v2 for an exemplary demonstration of its function
-
CONTROLER A thread that takes a list of threading.events and a function (usually a function that blocks restarting behavior of one or more threads). When all events are set, ==True, function is executed
-
LOCKOWNER A thread that creates an output variable registered to output_controler that displays ownership of a given threading.Rlock. (useful for finding threads blocking the database lock)
current file_name: Filter.py and Writer.py this module contains a FilterManager which organises a number of filters that select specific data from the database.
All classes are using the database lock, theoretically filters can be applied and statistics calculated while database generation is still ongoing
-
A class that logs all existing filters, calculates a list of all common ID items of each filter, generates meaningful statistics on filtered and unfiltered data.
-
A Filter_manager is restricted to either trackIDs or albumIDs only those IDs are associated with spotifyIDs that are needed for playlist generation.
-
Upon installation the filter_manager will calculate and save the frequency of all tags and all release dates of the original data set. It will generate a TABLE and a corresponding VIEW of that table containing all relevant IDs. When filters are added the IDs in the TABLE will be updated.
-
It can calculate tag and release date frequency of the filtered data set.
-
More functions for statistical analysis will be added
Filters are classes that make a SELECT database request that returns either albumIDs or trackIDs. Which is indicated by the class name.
Each filter needs to be logged to FilterManager after installation Filters need to be deleted using FilterManager.del_filter()
FilterTag : Return IDs of items associated with a tag with a tag score greater than the given threshold
FilterPlays : Return IDs of items with a minimum and maximum number of plays by a specific user
FilterDate: Return IDs of items with release dates that fall in a given date range. Optional items not associated with a release date can be included.
Date database SELECT requests require complicated joins and grouping, mainly because release dates are only connected to albumIDs. To get release dates for tracks, tracks need to be connected to corresponding albumIDs. These connections are incomplete and many tracks can be found on multiple releases.
The function creates a temporary TABLE where all necessary TABLE JOINS are already integrated and a corresponding index.
These TEMP TABLES are used by Filter and FilterManager and Writer classes.
Writer.py
The writer uses the TEMP TABLES generated by the filtermanager to generate a playlist preview containing all sorting relevant data like release dates, plays and i_scores.
-
It can apply sorts and limit the number of items by a single artist in the database.
-
It finally writes 2 playslist one containing the item names and one with sportify code
The GUI has been designed using Qt designer 4.8, it can be found in gui.ui
controler.py The controler connects the GUI with the database generation and access elements
-
It transcribes gui.ui into python readable format gui.py
-
It combines the python threading used in lastify.py for database generation with QtThreads by putting database generation, filtermanager and writers into seperate working threads
for details see dosctrings
