Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

merge to main #36

Merged
merged 55 commits into from
Oct 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
48aca31
set packages_dir to PyOdbDesignLib workspace subdir
nam20485 Jun 8, 2023
9871403
fix packages_dir
nam20485 Jun 8, 2023
d7677f7
disable python package publish for development branch
nam20485 Jun 8, 2023
26e76c0
keep a copy of the previous executable-based Dockerfile
nam20485 Jun 8, 2023
11c6bcd
use .sh instead of .ps1 script so we don't have to install PowerShell…
nam20485 Jun 8, 2023
c1793d7
add gunicorn to python requirements
nam20485 Jun 8, 2023
fd1472d
fix import since PyOdbDesignLib is inside of a directory package
nam20485 Jun 8, 2023
3566bf9
change Dockerfile to run Django server
nam20485 Jun 8, 2023
f3d56d7
split python build into its own presets
nam20485 Jun 11, 2023
0be51b1
put python settings behind cache variable so it can be used only in c…
nam20485 Jun 11, 2023
8a36e6f
add OS-specific swig scripts
nam20485 Jun 11, 2023
e3ab662
rename python build cache variable
nam20485 Jun 11, 2023
518b636
add post build command to copy library to name format expected for Py…
nam20485 Jun 11, 2023
6cb1bbb
test key password cacheing
nam20485 Jun 11, 2023
2553a5b
add include for std:::map and std::pair in SWIG module definition
nam20485 Jun 12, 2023
a7fe531
remove PowerShell swig script bc we have now have separate scripts fo…
nam20485 Jun 12, 2023
9f6e75a
add line for .sh files
nam20485 Jun 12, 2023
95b30a5
change to bash so source command is available
nam20485 Jun 12, 2023
dd7bcbb
add build presets for Python targets
nam20485 Jun 12, 2023
f35fe05
fix copy step on Linux
nam20485 Jun 12, 2023
36395be
update Dockerfile to use new python presets
nam20485 Jun 12, 2023
3e4c5b9
add OneDrive to linux paths
nam20485 Oct 4, 2023
5155402
minor change
nam20485 Oct 4, 2023
68a8f8e
change ouput
nam20485 Oct 4, 2023
481fa49
add extra line
nam20485 Oct 4, 2023
8bee3f8
link to libarchive for extracting .tgz files
nam20485 Oct 4, 2023
e47016f
add CLion (.idea) project directory
nam20485 Oct 4, 2023
4bf9273
add CMAKE_TOOLCHAIN_FILE variable for linux-* builds
nam20485 Oct 4, 2023
dc59b4d
use camel-case for LibArchive in find_project bc apparfently its case…
nam20485 Oct 4, 2023
45756bc
add and use ExitCode enum
nam20485 Oct 4, 2023
49ca00d
add support for extracting compressed archives
nam20485 Oct 5, 2023
18b3d2c
ignore vcpkg directory
nam20485 Oct 6, 2023
afc7943
add vcpkg manifest config files
nam20485 Oct 6, 2023
ce9ca6f
pass in destination ("parent") path to extract archive in
nam20485 Oct 6, 2023
924c0ce
pass in and extract archive to destination path when extracting
nam20485 Oct 6, 2023
13a1f28
add commented-out version of the toolchain variable in CMake presets …
nam20485 Oct 6, 2023
39afb27
parse the compressed .tgz files of the ODB++ archives (instead of the…
nam20485 Oct 6, 2023
649abe5
add try/catch around ParseFileModel()
nam20485 Oct 6, 2023
cf6ccbe
display output around extracting and parsing
nam20485 Oct 6, 2023
48c1059
link to Crow library
nam20485 Oct 6, 2023
de27720
support all types of archives regardless of extension- let libarchive…
nam20485 Oct 6, 2023
67caff6
enable support for all formats and filter in libarchive and enable se…
nam20485 Oct 6, 2023
f3efb82
add required implementation headers for library inclusion header
nam20485 Oct 6, 2023
666f401
use library include in app
nam20485 Oct 6, 2023
4463076
rename python OdbDesignServer to PyOdbDesignServer
nam20485 Oct 6, 2023
2ce2b34
fix warning
nam20485 Oct 6, 2023
bd08403
change LibArchive target_link_library to PUBLIC instead of PRIVATE
nam20485 Oct 6, 2023
6eb9f72
add comments
nam20485 Oct 6, 2023
7c846c7
add OdbDesignServer target
nam20485 Oct 6, 2023
290bc3b
add POST_BUILD command to copy templates/ dir to output directory
nam20485 Oct 6, 2023
c6a6476
add basic Crow server serving hello world html
nam20485 Oct 6, 2023
bad8e07
define _WIN32_WINNT for MSVC builds
nam20485 Oct 6, 2023
3b05ef5
disable force setting *_OUTPUT_* directories to _BINARY_DIR_ on MSVC …
nam20485 Oct 6, 2023
43a33b8
Update README.md
nam20485 Oct 6, 2023
ab5878f
Update README.md
nam20485 Oct 6, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@
###############################################################################
* text=auto

# Force bash scripts to always use LF line endings so that if a repo is accessed
# in Unix via a file share from Windows, the scripts will work.
*.sh text eol=lf

###############################################################################
# Set default behavior for command prompt diff.
#
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/python-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ on:
push:
branches:
- release
#- development

permissions:
contents: read
Expand Down Expand Up @@ -43,3 +44,4 @@ jobs:
with:
user: __token__
password: ${{ secrets.PYPI_API_TOKEN_PYODBDESIGNLIB }}
packages_dir: PyOdbDesignLib/dist
17 changes: 10 additions & 7 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,13 @@ PyOdbDesignLib/_PyOdbDesignLib.dll
PyOdbDesignLib/PyOdbDesignLib.py
PyOdbDesignLib/_PyOdbDesignLib.pyd
PyOdbDesignLib/__pycache__/
OdbDesignServer/Lib/
OdbDesignServer/OdbDesignServer/__pycache__/
OdbDesignServer/pyvenv.cfg
OdbDesignServer/db.sqlite3
OdbDesignServer/Scripts/
OdbDesignServer/api/__pycache__/
OdbDesignServer/PyOdbDesignLib/
PyOdbDesignServer/Lib/
PyOdbDesignServer/PyOdbDesignServer/__pycache__/
PyOdbDesignServer/pyvenv.cfg
PyOdbDesignServer/db.sqlite3
PyOdbDesignServer/Scripts/
PyOdbDesignServer/api/__pycache__/
PyOdbDesignServer/PyOdbDesignLib/
/cmake-build-debug
build/
/vcpkg_installed
8 changes: 8 additions & 0 deletions .idea/.gitignore

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions .idea/OdbDesign.iml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions .idea/misc.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions .idea/modules.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions .idea/vcs.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

24 changes: 14 additions & 10 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,15 @@ endif()

project ("OdbDesign")

if (MSVC)
# control where the static and shared libraries are built so that on windows
# we don't need to tinker with the path to run the executable
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY "${PROJECT_BINARY_DIR}")
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY "${PROJECT_BINARY_DIR}")
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY "${PROJECT_BINARY_DIR}")
endif()

#if (false AND MSVC)
# # control where the static and shared libraries are built so that on windows
# # we don't need to tinker with the path to run the executable
# set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY "${PROJECT_BINARY_DIR}")
# set(CMAKE_LIBRARY_OUTPUT_DIRECTORY "${PROJECT_BINARY_DIR}")
# set(CMAKE_RUNTIME_OUTPUT_DIRECTORY "${PROJECT_BINARY_DIR}")
#endif()

# increase warnings in a compiler-specific manner
if (MSVC)
# warning level 4
add_compile_options(/W4)
Expand All @@ -50,7 +51,10 @@ else()
endif()

# Include sub-projects.
add_subdirectory("OdbDesignApp")
add_subdirectory("OdbDesignLib")
add_subdirectory("OdbDesignApp")
add_subdirectory("OdbDesignServer")

target_link_libraries(OdbDesignApp PUBLIC OdbDesign)
# link to OdbDesign library in both executable targets
target_link_libraries(OdbDesignApp PUBLIC OdbDesign)
target_link_libraries(OdbDesignServer PUBLIC OdbDesign)
83 changes: 78 additions & 5 deletions CMakePresets.json
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,14 @@
"CMAKE_BUILD_TYPE": "Debug"
}
},
{
"name": "python-x64-debug",
"displayName": "Python x64 Debug",
"inherits": "x64-debug",
"cacheVariables": {
"PYTHON_MODULE_BUILD": true
}
},
{
"name": "x64-release",
"displayName": "x64 Release",
Expand All @@ -45,6 +53,14 @@
"CMAKE_BUILD_TYPE": "Release"
}
},
{
"name": "python-x64-release",
"displayName": "Python x64 Release",
"inherits": "x64-release",
"cacheVariables": {
"PYTHON_MODULE_BUILD": true
}
},
{
"name": "x86-debug",
"displayName": "x86 Debug",
Expand Down Expand Up @@ -72,7 +88,12 @@
"binaryDir": "${sourceDir}/out/build/${presetName}",
"installDir": "${sourceDir}/out/install/${presetName}",
"cacheVariables": {
"CMAKE_BUILD_TYPE": "Debug"
"CMAKE_BUILD_TYPE": "Debug",
"CMAKE_TOOLCHAIN_FILE": {
"value": "$env{HOME}/src/github/microsoft/vcpkg/scripts/buildsystems/vcpkg.cmake",
//"value": "$env{VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake",
"type": "FILEPATH"
}
},
"condition": {
"type": "equals",
Expand All @@ -92,7 +113,11 @@
"binaryDir": "${sourceDir}/out/build/${presetName}",
"installDir": "${sourceDir}/out/install/${presetName}",
"cacheVariables": {
"CMAKE_BUILD_TYPE": "Release"
"CMAKE_BUILD_TYPE": "Release",
"CMAKE_TOOLCHAIN_FILE": {
"value": "$env{HOME}/src/github/microsoft/vcpkg/scripts/buildsystems/vcpkg.cmake",
"type": "FILEPATH"
}
},
"condition": {
"type": "equals",
Expand All @@ -105,6 +130,22 @@
}
}
},
{
"name": "python-linux-release",
"displayName": "Python Linux Release",
"inherits": "linux-release",
"cacheVariables": {
"PYTHON_MODULE_BUILD": true
}
},
{
"name": "python-linux-debug",
"displayName": "Python Linux Debug",
"inherits": "linux-debug",
"cacheVariables": {
"PYTHON_MODULE_BUILD": true
}
},
{
"name": "linux-mingw-w64-debug",
"displayName": "Linux MinGW-w64 Debug",
Expand All @@ -115,7 +156,11 @@
"CMAKE_BUILD_TYPE": "Debug",
"CMAKE_C_COMPILER": "x86_64-w64-mingw32-gcc",
"CMAKE_CXX_COMPILER": "x86_64-w64-mingw32-g++",
"CMAKE_CXX_FLAGS": "-static-libgcc -static-libstdc++"
"CMAKE_CXX_FLAGS": "-static-libgcc -static-libstdc++",
"CMAKE_TOOLCHAIN_FILE": {
"value": "$env{HOME}/src/github/microsoft/vcpkg/scripts/buildsystems/vcpkg.cmake",
"type": "FILEPATH"
}
},
"condition": {
"type": "equals",
Expand All @@ -138,7 +183,11 @@
"CMAKE_BUILD_TYPE": "Release",
"CMAKE_C_COMPILER": "x86_64-w64-mingw32-gcc",
"CMAKE_CXX_COMPILER": "x86_64-w64-mingw32-g++",
"CMAKE_CXX_FLAGS": "-static-libgcc -static-libstdc++"
"CMAKE_CXX_FLAGS": "-static-libgcc -static-libstdc++",
"CMAKE_TOOLCHAIN_FILE": {
"value": "$env{HOME}/src/github/microsoft/vcpkg/scripts/buildsystems/vcpkg.cmake",
"type": "FILEPATH"
}
},
"condition": {
"type": "equals",
Expand Down Expand Up @@ -194,7 +243,7 @@
"displayName": "x64 Release",
"inherits": "windows-base",
"configurePreset": "x64-release"
},
},
{
"name": "x86-debug",
"displayName": "x86 Debug",
Expand All @@ -207,6 +256,18 @@
"inherits": "windows-base",
"configurePreset": "x86-release"
},
{
"name": "python-x64-debug",
"displayName": "Python x64 Debug",
"inherits": "windows-base",
"configurePreset": "python-x64-debug"
},
{
"name": "python-x64-release",
"displayName": "Python x64 Release",
"inherits": "windows-base",
"configurePreset": "python-x64-release"
},
{
"name": "linux-base",
"hidden": true,
Expand All @@ -229,6 +290,18 @@
"configurePreset": "linux-release",
"inherits": "linux-base"
},
{
"name": "python-linux-debug",
"displayName": "Python Linux Debug",
"configurePreset": "python-linux-debug",
"inherits": "linux-base"
},
{
"name": "python-linux-release",
"displayName": "Python Linux Release",
"configurePreset": "python-linux-release",
"inherits": "linux-base"
},
{
"name": "linux-mingw-w64-base",
"hidden": true,
Expand Down
72 changes: 22 additions & 50 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#FROM debian:bookworm-20230522-slim AS build
FROM debian:bookworm-20230522-slim AS build

# install dependencies
Expand All @@ -11,72 +12,43 @@ RUN apt-get update && \
cmake \
g++ \
ninja-build \
python3-dev \
#python3-pip \
#python3-virtualenv \
python3-dev \
# mingw-w64 \
swig

# add MS PowerShell repo
RUN curl https://packages.microsoft.com/keys/microsoft.asc | gpg --yes --dearmor --output /usr/share/keyrings/microsoft.gpg
RUN sh -c 'echo "deb [arch=amd64 signed-by=/usr/share/keyrings/microsoft.gpg] https://packages.microsoft.com/repos/microsoft-debian-bullseye-prod bullseye main" > /etc/apt/sources.list.d/microsoft.list'

# install PowerShell
RUN apt-get update && \
apt-get install -y --no-install-recommends \
powershell

# copy source
RUN mkdir -p /src/OdbDesign
COPY . /src/OdbDesign
WORKDIR /src/OdbDesign
COPY . .

# generate SWIG python bindings
RUN chmod +x scripts/generate-python-module.ps1
RUN scripts/generate-python-module.ps1
RUN chmod +x scripts/generate-python-module.sh
RUN ./scripts/generate-python-module.sh

# configure & build using presets
# linux-release
RUN cmake --preset linux-release
RUN cmake --build --preset linux-release
# # linux-debug
# RUN cmake --preset linux-debug
# RUN cmake --build --preset linux-debug

## build PyOdbDesignLib python package
#RUN python3 -m pip install -r PyOdbDesignLib/pkg-build-requirements.txt --break-system-packages
WORKDIR /src/OdbDesign/PyOdbDesignLib
# copy C++ wrapper library to a format that Python expects for extension modules
RUN cp /src/OdbDesign/out/build/linux-release/OdbDesignLib/libOdbDesign.so ./_PyOdbDesignLib.so
## build Python package
#RUN python3 -m build
RUN cmake --preset python-linux-release
RUN cmake --build --preset python-linux-release

# much smaller runtime image
FROM debian:bookworm-20230522-slim AS run
#FROM python:3.11.4-bullseye AS run
FROM debian:bookworm-20230522-slim as run

RUN mkdir /OdbDesign
WORKDIR /OdbDesign
# copy PyPyPyOdbDesignServer files
COPY --from=build /src/OdbDesign/PyOdbDesignServer PyOdbDesignServer

# copy binaries
RUN mkdir bin
COPY --from=build /src/OdbDesign/out/build/linux-release/OdbDesignLib/libOdbDesign.so ./bin/
COPY --from=build /src/OdbDesign/out/build/linux-release/OdbDesignApp/OdbDesignApp ./bin/
# copy Python files
COPY --from=build /src/OdbDesign/PyOdbDesignLib /OdbDesign/PyOdbDesignLib
# copy PyOdbDesignLib files
COPY --from=build /src/OdbDesign/out/build/python-linux-release/OdbDesignLib/_PyOdbDesignLib.so /PyOdbDesignServer/PyOdbDesignLib/
COPY --from=build /src/OdbDesign/PyOdbDesignLib/PyOdbDesignLib.py /PyOdbDesignServer/PyOdbDesignLib/
RUN touch /PyOdbDesignServer/PyOdbDesignLib/__init__.py

# install Python3 dev
RUN apt-get update && \
apt-get install -y --no-install-recommends \
python3-dev
#python3-pip \
#python3-virtualenv
apt-get install -y --no-install-recommends \
python3-dev \
python3-pip

# # install package into Python environment
# RUN python3 -m pip install --break-system-packages \
# --no-index \
# --find-links PyOdbDesignLib/dist/*.whl \
# PyOdbDesignLib
RUN python3 -m pip install -r /PyOdbDesignServer/requirements.txt --break-system-packages

# run
ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/OdbDesign/bin
ENTRYPOINT [ "bin/OdbDesignApp" ]
WORKDIR /PyOdbDesignServer
EXPOSE 8000
CMD ["gunicorn", "--bind", ":8000", "--workers", "3", "PyOdbDesignServer.wsgi:application"]
Loading