Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

minor fix for template instantiation #1

Open
wants to merge 25 commits into
base: inst_func
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
2076af0
Remove deprecated use of set-env
kunaltyagi Oct 24, 2020
8a7d89f
Merge pull request #9 from kunaltyagi/set-env.action
kunaltyagi Oct 24, 2020
24dc99f
Remove unneeded file
kunaltyagi Oct 25, 2020
98495f0
Changes to make it more intuitive to use
kunaltyagi Oct 25, 2020
387e9bf
Add a sample (simple?) CMake project
kunaltyagi Oct 25, 2020
b1bd3a6
All hail Black
kunaltyagi Oct 25, 2020
29276b9
Merge pull request #10 from kunaltyagi/remove_unwanted
kunaltyagi Oct 30, 2020
821a662
Add minimal files required for a nice interface and conan integration
kunaltyagi Nov 1, 2020
555b8f5
Initial README.md (#16)
felximenes Nov 5, 2020
410503c
Merge pull request #11 from kunaltyagi/cmake-integration
kunaltyagi Nov 5, 2020
957433b
Merge pull request #14 from kunaltyagi/conan.integrate
kunaltyagi Nov 5, 2020
4343a2f
[interface.py] CMakeLists.txt.in filepath minor fix
diivm Mar 28, 2021
5e07663
Merge pull request #17 from divmadan/fix/interface_filepath
kunaltyagi Mar 29, 2021
3e1d46f
[clang_utils.py] Introduce clang_utils.py to get checks from cindex.py
diivm Oct 23, 2020
8f07af2
changes coming from clang_utils.py
diivm Feb 27, 2021
f0edede
[pytest.yml] bump clang to v12 and ubuntu to 20.04
diivm Feb 27, 2021
838ac87
[compilation_database.py] add compilation_database.py
diivm Jun 3, 2021
6f4c13e
update compilation db logic
diivm Jun 3, 2021
cc1ffd7
Merge pull request #6 from divmadan/polish-parse
kunaltyagi Jun 12, 2021
de695ad
Add class explicitly to the tests
kunaltyagi Oct 24, 2020
721bbe0
Add tests for instantiation parsing
kunaltyagi Oct 24, 2020
647b0e5
Satisfy black
kunaltyagi Oct 24, 2020
b990c30
Update just in case the ill-formed nature of previous example was the…
kunaltyagi Oct 25, 2020
dcac8f9
[test_parse.py] changes coming from clang_utils.py
diivm Jun 16, 2021
36350bf
[clang_utils.py] add objc_type_encoding to ignore list
diivm Jun 16, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ on: [push, pull_request]
jobs:
Pytest:
# The type of runner that the job will run on
runs-on: ubuntu-latest
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: [3.6, 3.7, 3.8, 3.9]
Expand Down Expand Up @@ -34,16 +34,16 @@ jobs:
- name: Add llvm keys
run: |
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
echo 'deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-11 main' | sudo tee -a /etc/apt/sources.list
echo 'deb-src http://apt.llvm.org/bionic/ llvm-toolchain-bionic-11 main' | sudo tee -a /etc/apt/sources.list
echo 'deb http://apt.llvm.org/focal/ llvm-toolchain-focal-12 main' | sudo tee -a /etc/apt/sources.list
echo 'deb-src http://apt.llvm.org/focal/ llvm-toolchain-focal-12 main' | sudo tee -a /etc/apt/sources.list
- name: Install libclang and its python bindings
run: |
sudo apt-get update
sudo apt-get install -y libclang-11-dev python3-clang-11
sudo apt-get install -y libclang-12-dev python3-clang-12

# Add dist-package to path to enable apt installed python3-clang import
- name: Add dist-packages to PYTHONPATH
run: echo "::set-env name=PYTHONPATH::${PYTHON_PATH}:/usr/lib/python3/dist-packages"
run: echo "PYTHONPATH=${PYTHON_PATH}:/usr/lib/python3/dist-packages" >> $GITHUB_ENV
- name: Display PYTHONPATH
run: python -c "import sys; print('\n'.join(sys.path))"

Expand Down
33 changes: 33 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Description

The clang-bind is a project to generate python bindings for C++ code using clang python bindings and pybind11.

# Dependencies

**C++**

*libclang*

```
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
echo 'deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-11 main' | sudo tee -a /etc/apt/sources.list
echo 'deb-src http://apt.llvm.org/bionic/ llvm-toolchain-bionic-11 main' | sudo tee -a /etc/apt/sources.list
sudo apt-get update
sudo apt-get install -y libclang-11-dev python3-clang-11
```

**Python**

`pip install -r requirements.txt`

# Demonstration

1. Go to `clang-bind/bindings/python/tests/test_project/` folder
2. Create a build folder
3. Run `cmake ..`
4. Run `make -j$(nproc)`
5. Run `python ../../scripts/parse.py --com ./ ../src/simple.cpp`
6. Run `python ../../scripts/generate.py --com json/src/simple.json`

The binding code will be available in `pybind11-gen/src` folder.

52 changes: 52 additions & 0 deletions bindings/python/scripts/CMakeLists.txt.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
cmake_minimum_required(VERSION 3.10)
project (${project.name}_pybind11)

% if not cmake.no_conan:
include (conan.cmake)

conan_cmake_run (CONANFILE conanfile.txt
BASIC_SETUP
CMAKE_TARGETS
BUILD missing
% if conan.keep_rpaths:
KEEP_RPATHS
% endif
% if conan.no_output_dirs:
NO_OUTPUT_DIRS
% endif
% if conan.arch:
ARCH ${conan.arch}
% endif
% if conan.build_type:
BUILD_TYPE ${conan.build_type}
% endif
% if conan.profile:
PROFILE ${conan.profile}
% endif
% if conan.profile_auto:
PROFILE_AUTO ${conan.profile_auto}
% endif
)
% endif

# `find_package` can be replaced with `add_subdirectory` by a power user
# https://pybind11.readthedocs.io/en/stable/compiling.html#find-package-vs-add-subdirectory
find_package(pybind11 REQUIRED)

set (CMAKE_CXX_STANDARD ${cmake.cpp_standard})

pybind11_add_module (${project.name}
% if pybind11.lib_type:
# for MODULE or SHARED
${pybind11.lib_type}
% endif
% if pybind11.thin_lto:
THIN_LTO
% endif
% if pybind11.optimise_for_size:
OPT_SIZE
% endif
% for file in files:
file
% endfor
)
121 changes: 121 additions & 0 deletions bindings/python/scripts/clang_utils.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
import inspect
import clang.cindex as clang


def getmembers_static(object, predicate=None):
"""
Return all members of an object as (name, value) pairs sorted by name via `getattr_static`.
Optionally, only return members that satisfy a given predicate.


- A static version of `get_members` function at:
https://github.com/python/cpython/blob/3.9/Lib/inspect.py#L326-L368
https://github.com/python/cpython/blob/14ba761078b5ae83519e34d66ab883743912c45b/Lib/inspect.py#L444-L486
- `getmembers` function (from the inspect module) triggers execution instead of doing static analysis.
- This leads to errors, particularly on properties of classes in cindex.py, which causes segmentation errors or raises an Exception if a particular condition is not satisfied.
- To curb this, we fetch the members statically. We define a custom function based on the one in the inspect module.
"""

results = []
names = dir(object)
# :dd any DynamicClassAttributes to the list of names if object is a class;
# this may result in duplicate entries if, for example, a virtual
# attribute with the same name as a DynamicClassAttribute exists
try:
base_members = filter(
lambda k, v: isinstance(v, types.DynamicClassAttribute),
object.__bases__.__dict__.items(),
)
names.extend(base_members)
except AttributeError:
pass
for key in names:
value = inspect.getattr_static(object, key)
if not predicate or predicate(value):
results.append((key, value))
results.sort(key=lambda pair: pair[0])
return results


class ClangUtils:
"""
Clang's cindex class utilities.

Supports the following objects:
CursorKind:
https://github.com/llvm/llvm-project/blob/release/12.x/clang/bindings/python/clang/cindex.py#L657
https://github.com/llvm/llvm-project/blob/1acd9a1a29ac30044ecefb6613485d5d168f66ca/clang/bindings/python/clang/cindex.py#L657
- A CursorKind describes the kind of entity that a cursor points to.
Cursor:
https://github.com/llvm/llvm-project/blob/release/12.x/clang/bindings/python/clang/cindex.py#L1415
https://github.com/llvm/llvm-project/blob/1acd9a1a29ac30044ecefb6613485d5d168f66ca/clang/bindings/python/clang/cindex.py#L1415
- The Cursor class represents a reference to an element within the AST. It acts as a kind of iterator.
Type:
https://github.com/llvm/llvm-project/blob/release/12.x/clang/bindings/python/clang/cindex.py#L2180
https://github.com/llvm/llvm-project/blob/1acd9a1a29ac30044ecefb6613485d5d168f66ca/clang/bindings/python/clang/cindex.py#L2180
- The Type class represents the type of an element in the abstract syntax tree.
"""

def __init__(self, object):
if not (
isinstance(object, clang.CursorKind)
or isinstance(object, clang.Cursor)
or isinstance(object, clang.Type)
):
raise NotImplementedError(f"Not implemented for {object}")

self.check_functions_dict = {}
self.get_functions_dict = {}
self.properties_dict = {}

# A list to ignore the functions/properties that causes segmentation errors.
ignore_list = [
"mangled_name",
"get_address_space",
"get_typedef_name",
"tls_kind",
"objc_type_encoding"
]

# populate dicts
valid_entries = filter(
lambda entry: entry[0] not in ignore_list, getmembers_static(object)
)
for name, func in valid_entries:
if inspect.isfunction(func): # if function
try: # cindex.py's functions raise exceptions internally
if name.startswith("is_"):
self.check_functions_dict[name] = func(object)
if name.startswith("get_"):
self.get_functions_dict[name] = func(object)
except:
continue
elif isinstance(func, property): # else, property
try: # cindex.py's property functions raise exceptions internally
self.properties_dict[name] = getattr(object, name)
except:
continue

def get_check_functions_dict(self):
"""
Returns: `check_functions_dict`:
- functions that begin with "is_" i.e., checking functions
- {function_name, function_result}
"""
return self.check_functions_dict

def get_get_functions_dict(self):
"""
Returns: `get_functions_dict`:
- functions that begin with "get_" i.e., getter functions
- {function_name, function_result}
"""
return self.get_functions_dict

def get_properties_dict(self):
"""
Returns: properties_dict
- Properties
- {property_name, property}
"""
return self.properties_dict
40 changes: 40 additions & 0 deletions bindings/python/scripts/compilation_database.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import clang.cindex as clang


class CompilationDatabase:
"""
Build a compilation database from a given directory
"""

def __init__(self, compilation_database_path):
self.compilation_database = clang.CompilationDatabase.fromDirectory(
buildDir=compilation_database_path
)

def get_compilation_arguments(self, filename=None):
"""
Returns the compilation commands extracted from the compilation database

Parameters:
- compilation_database_path: The path to `compile_commands.json`
- filename (optional): To get compilaton commands of a file

Returns:
- compilation_arguments (dict): {filename: compiler arguments}
"""

if filename:
# Get compilation commands from the compilation database for the given file
compilation_commands = self.compilation_database.getCompileCommands(
filename=filename
)
else:
# Get all compilation commands from the compilation database
compilation_commands = self.compilation_database.getAllCompileCommands()

# {file: compiler arguments}
compilation_arguments = {
command.filename: list(command.arguments)[1:-1]
for command in compilation_commands
}
return compilation_arguments
5 changes: 5 additions & 0 deletions bindings/python/scripts/conanfile.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[requires]
pybind11[>2.2.2]

[generators]
cmake
Loading