Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/35 implement regex matching #44

Merged
merged 20 commits into from
Jan 29, 2024
Merged
Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
4d14a5e
Implemented regex matching, initial commit
PiaSchroeder Nov 28, 2023
8575422
Added credentials check for cubes and removed all references to set_db()
PiaSchroeder Dec 4, 2023
a47f437
Merge branch 'dev' into feat/35-implement-regex-matching
PiaSchroeder Dec 5, 2023
dd92bf4
Implemented regex matching, initial commit
PiaSchroeder Nov 28, 2023
d171ff8
Added credentials check for cubes and removed all references to set_db()
PiaSchroeder Dec 4, 2023
37c3366
fix tests
pmayd Dec 8, 2023
263da5f
refactoring Find and Result class to work with new database detection…
pmayd Dec 8, 2023
f897c77
fix find notebook and some errors related to db_name
pmayd Dec 8, 2023
07cca97
remove deprecated function
pmayd Dec 8, 2023
bea7e13
Renamed some functions and variables for consistency.
PiaSchroeder Dec 12, 2023
5d878b3
Merge branch 'dev' into feat/35-implement-regex-matching
PiaSchroeder Dec 12, 2023
76f60c5
Merged changes allowing to pass a db_name argument with find into loc…
PiaSchroeder Dec 12, 2023
380e078
Stored regex as dictionary and updated credential checks.
PiaSchroeder Dec 12, 2023
ae58504
fixed pre-commit issues
PiaSchroeder Dec 19, 2023
8d67630
fixed tests
PiaSchroeder Jan 7, 2024
0c0af52
add mocker for check_db_credentials so tests can run without proper c…
pmayd Jan 29, 2024
b856c48
fix pylint issues
pmayd Jan 29, 2024
3081fbc
properly fix pylint issues without breaking tests
pmayd Jan 29, 2024
eea0607
fix mypy issues, correct type hints
pmayd Jan 29, 2024
613a437
fix pylint issues
pmayd Jan 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Implemented regex matching, initial commit
PiaSchroeder authored and pmayd committed Dec 8, 2023
commit dd92bf4cf82759bfbd755fd1872691fac8a6ecc9
40 changes: 29 additions & 11 deletions nb/00_Setup.ipynb
Original file line number Diff line number Diff line change
@@ -6,9 +6,9 @@
"source": [
"# Setup `pystatis`\n",
"\n",
"You don't need to do much to use `pystatis`. Basically, the first time you import the package, it will create a `config.ini` file under `~/.pystatis`. This file is used for storing settings, for example your credentials fpr the supported databases.\n",
"You don't need to do much to use `pystatis`. Basically, the first time you import the package, it will create a `config.ini` file under `~/.pystatis`. This file is used for storing settings, for example your credentials for the supported databases.\n",
"\n",
"To set up your credentials, we need to ask you for your username and password. This is done by the `setup_credentials()` function. It will ask you interactivly for the credentials, or you can set the following environmental variables:\n",
"To set up your credentials, we need to ask you for your username and password. This is done by the `setup_credentials()` function. It will ask you interactively for the credentials, or you can set the following environment variables:\n",
"- `PYSTATIS_GENESIS_API_USERNAME`\n",
"- `PYSTATIS_GENESIS_API_PASSWORD`\n",
"- `PYSTATIS_ZENSUS_API_USERNAME`\n",
@@ -21,15 +21,33 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"`dotenv` is uses here to load a local `.env` file that contains the above mentioned environmental variables so we don't have to input them."
"`dotenv` is used here to load a local `.env` file that contains the above mentioned environment variables so we don't have to input them."
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 1,
"id": "888706f5-3a9e-4e0a-9ca6-fa430280bc03",
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"pystatis version: 0.1.5\n"
]
},
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from pathlib import Path\n",
"\n",
@@ -43,13 +61,13 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 2,
"id": "ee7969b6",
"metadata": {},
"outputs": [],
"source": [
"# only execute if you want to delete your config file for test purposes\n",
"# config.delete_config()"
"pystatis.config.delete_config()"
]
},
{
@@ -67,16 +85,16 @@
"source": [
"The only thing you have to do is to set up your user credentials.\n",
"\n",
"You can do so either by:\n",
"1. specifying the 4 environment variables `PYSTATIS_GENESIS_API_USERNAME|PASSWORD`, and `PYSTATIS_ZENSUS_API_USERNAME|PASSWORD`\n",
"You can do so by following either of these steps:\n",
"1. specifying the 6 environment variables `PYSTATIS_GENESIS_API_USERNAME|PASSWORD`, `PYSTATIS_ZENSUS_API_USERNAME|PASSWORD`, `PYSTATIS_REGIO_API_USERNAME|PASSWORD`\n",
"2. calling the function `setup_credentials()` which will guide you through the process\n",
"\n",
"Even if you do 1. please call `setup_credentials()` once as it will read out the environment variables and write the credentials to the `config.ini` in your config dir."
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 3,
"id": "6c301da5",
"metadata": {},
"outputs": [],
@@ -127,7 +145,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.6"
"version": "3.11.5"
}
},
"nbformat": 4,
4 changes: 2 additions & 2 deletions src/pystatis/cache.py
Original file line number Diff line number Diff line change
@@ -14,7 +14,7 @@

logger = logging.getLogger(__name__)

JOB_ID_PATTERN = r"\d+"
JOB_ID_PATTERN = r"_\d+"


def cache_data(
@@ -132,7 +132,7 @@ def normalize_name(name: str) -> str:
Returns:
str: The unique identifier without the optional job id.
"""
if len(re.findall(JOB_ID_PATTERN, name)) == 3:
if re.findall(JOB_ID_PATTERN, name):
PiaSchroeder marked this conversation as resolved.
Show resolved Hide resolved
name = name.split("_")[0]

return name
9 changes: 8 additions & 1 deletion src/pystatis/config.py
Original file line number Diff line number Diff line change
@@ -26,6 +26,9 @@
PKG_NAME = __name__.split(".", maxsplit=1)[0]
DEFAULT_CONFIG_DIR = str(Path().home() / f".{PKG_NAME}")
SUPPORTED_DB = ["genesis", "zensus", "regio"]
REGEX_DB = ["^((\d{5}-\d{4})|([0-9A-Z]{10}))$",
PiaSchroeder marked this conversation as resolved.
Show resolved Hide resolved
"^\d{4}[A-Z]-\d{4}$",
"^((\d{5}-.{1,2}($|-.*$))|(A.*$)|([0-9A-Z]{10}))"]

logger = logging.getLogger(__name__)
config = ConfigParser(interpolation=None)
@@ -124,8 +127,8 @@ def write_config() -> None:
def create_default_config() -> None:
"""Create a default config parser with empty credentials."""
config.add_section("settings")
config.set("settings", "active_db", "")
config.set("settings", "supported_db", ",".join(SUPPORTED_DB))
config.set("settings", "regex_db", ";".join(REGEX_DB))
PiaSchroeder marked this conversation as resolved.
Show resolved Hide resolved

config.add_section("genesis")
config.set(
@@ -178,6 +181,10 @@ def get_supported_db() -> list[str]:
"""Get a list of supported database names."""
return SUPPORTED_DB

def get_regex() -> list[str]:
PiaSchroeder marked this conversation as resolved.
Show resolved Hide resolved
"""Get a list of regex patterns matching item codes in the supported databases."""
return REGEX_DB


def get_cache_dir() -> str:
"""Get the cache directory."""
Loading