Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add option to not use references in yaml output #70

Open
wants to merge 371 commits into
base: master
Choose a base branch
from
Open
Changes from 37 commits
Commits
Show all changes
371 commits
Select commit Hold shift + click to select a range
000ecf1
consolidate settings into an object that can be passed around
AndrewPickford Aug 8, 2017
dec210a
add inventory query options
AndrewPickford Aug 9, 2017
6a00c56
make interpolation error messages more informative
AndrewPickford Aug 9, 2017
06352f0
fix test broken by output change
AndrewPickford Aug 9, 2017
1e2fd7f
bring ClassNotFound and BadReference errors into unified Interpolatio…
AndrewPickford Aug 10, 2017
3c7e567
add tests for invquery ignore errors option
AndrewPickford Aug 10, 2017
ce4d860
rename ret to node in _nodeinfo
AndrewPickford Aug 10, 2017
2689522
Merge pull request #1 from epcim/pr/fix/fix_raise_UndefinedVariableError
epcim Aug 16, 2017
b5c4c93
Merge pull request #2 from bbinet/optionnal-classes
epcim Aug 16, 2017
ff7e9c1
Fix reclass parser to include OPT_IGNORE_CLASS_NOTFOUND
chnyda Aug 17, 2017
1c2b748
Merge pull request #4 from chnyda/master
epcim Aug 17, 2017
a3af54a
Modify parser and spread ignore_class_not_found from cmd line
chnyda Aug 17, 2017
58f5981
Pass ingore_class_notfound defaults properly
epcim Aug 18, 2017
f36c449
Merge pull request #5 from salt-formulas/pass-defaults
epcim Aug 18, 2017
34a9e5d
add option to ignore missing classes
AndrewPickford Sep 4, 2017
8fe8468
rename class_not_found to class_notfound
AndrewPickford Sep 4, 2017
8a05d71
command line option for ignore class not found
AndrewPickford Sep 4, 2017
49ef7db
improve readability
AndrewPickford Sep 4, 2017
fbb1306
pass command line options into settings
AndrewPickford Sep 4, 2017
8258dad
add OPT_ prefix to allow and inventory options
AndrewPickford Sep 4, 2017
27f2fa7
bump version to 1.5.1
AndrewPickford Sep 5, 2017
e415d16
fix typo
AndrewPickford Sep 5, 2017
8099f08
correct option passing for reclass-salt top
AndrewPickford Sep 5, 2017
6a77d07
account for ancester and non existant paths
AndrewPickford Sep 5, 2017
d7045b0
deep copying parser objects fails so just shallow copy Settings objec…
AndrewPickford Sep 5, 2017
d590976
use better algorithm for individual export evaluation
AndrewPickford Sep 6, 2017
62f2ef0
fix ancestor key dereferencing in exports
AndrewPickford Sep 6, 2017
d29a684
remove unused function deepest_match_in
AndrewPickford Sep 6, 2017
e6fba88
extend the current path rather than create a new subpath object
AndrewPickford Sep 6, 2017
3fece05
remove unused line
AndrewPickford Sep 6, 2017
2a28327
sync version number to 1.5.2
AndrewPickford Sep 7, 2017
bb0a619
initial documentation for extensions
AndrewPickford Sep 8, 2017
d1fb0b4
ignore missing class by regexp
epcim Sep 11, 2017
31a7e03
Merge pull request #6 from salt-formulas/ignore_class_regexp
chnyda Sep 12, 2017
6c025e5
fix typo in ingore_class_regexp
epcim Sep 12, 2017
fa7cb98
Merge pull request #7 from salt-formulas/fix-ignore_class_regexp
epcim Sep 12, 2017
2f7d4f0
fix syntax overlooked
epcim Sep 12, 2017
116a062
rename README.ententions to README-extentions.rst
AndrewPickford Sep 12, 2017
94b3e08
log the skipped class notfound
epcim Sep 12, 2017
ffdfb39
also prune new test dir reclass/test
AndrewPickford Sep 13, 2017
f4b93c0
add inventory query example
AndrewPickford Sep 13, 2017
fd6a3a3
documentation updates from salt-formulas/reclass
AndrewPickford Sep 13, 2017
bdf4147
update copyright and maintainer email
AndrewPickford Sep 13, 2017
2c48d60
add ignore class not found regexp from salt-formulas/reclass
AndrewPickford Sep 13, 2017
38cd76b
Use isinstance() insted of type()
pjediny Sep 13, 2017
5474e2b
Merge pull request #10 from pjediny/fix-isinstance
epcim Sep 13, 2017
897f8b6
fix ignore_class_regexp default pattern
epcim Sep 14, 2017
5a35e87
Merge remote-tracking branch 'salt-formula/master' into sf-merge-test
AndrewPickford Sep 18, 2017
ad44117
Documentation updates
epcim Sep 18, 2017
4ab4ed9
Merge branch 'sf-merge-test' into updates-for-andrew
epcim Sep 18, 2017
1932474
Fix typo in README
epcim Sep 18, 2017
ab9cac3
README update, fix rst syntax
epcim Sep 18, 2017
e6743ef
Merge pull request #2 from salt-formulas/updates-for-andrew
AndrewPickford Sep 19, 2017
3469eb5
Merge pull request #12 from AndrewPickford/master
epcim Sep 20, 2017
8dadd45
check for existance of required export when doing an if inv query
AndrewPickford Oct 12, 2017
1039b21
update requirements
epcim Nov 15, 2017
67ad77c
update requirements
epcim Nov 15, 2017
411a551
update requirements
epcim Nov 15, 2017
e2e3a21
Update reclass system dependencies
epcim Nov 15, 2017
4a61aad
AndrewPickford: check for existance of required export when doing an …
epcim Dec 7, 2017
a0e800f
fix merging of escaped strings by allowing a parameter object to
AndrewPickford Feb 5, 2018
edacfdf
YAML: Use C bindings if available
alexandruavadanii Feb 21, 2018
70f010d
Merge pull request #3 from alexandruavadanii/yaml-c-bindings
AndrewPickford Feb 23, 2018
51c5280
fix bug introduced when fixing merging of escaped strings, when a par…
AndrewPickford Mar 13, 2018
8e41525
when merging values add option to ignore missing references that are …
AndrewPickford Mar 15, 2018
37846a4
set default for ignore_overwritten_missing_references to true as that
AndrewPickford Mar 15, 2018
e0eb7b6
add documentation for ignoring overwritten references
AndrewPickford Mar 16, 2018
ffd77b4
group resolve errors together and report them at the end of the inter…
AndrewPickford Mar 16, 2018
2786bc0
when merging values add option to ignore missing references that are …
AndrewPickford Mar 15, 2018
94dda32
set default for ignore_overwritten_missing_references to true as that
AndrewPickford Mar 15, 2018
9880037
add documentation for ignoring overwritten references
AndrewPickford Mar 16, 2018
e1dc1a8
group resolve errors together and report them at the end of the inter…
AndrewPickford Mar 16, 2018
56bf553
Merge pull request #18 from salt-formulas/andrew-missed-ref2
epcim Mar 17, 2018
445db72
work around to retry connection to git server several times before ra…
AndrewPickford Mar 19, 2018
c8e9902
fix merging of escaped strings by allowing a parameter object to
AndrewPickford Feb 5, 2018
ac4ab03
fix bug introduced when fixing merging of escaped strings, when a par…
AndrewPickford Mar 13, 2018
7d0101a
fix escaped strings when the escaped string is in the second dict to …
AndrewPickford Mar 19, 2018
bba7631
fix escaped strings when the escaped string is in the second dict to …
AndrewPickford Mar 19, 2018
f08f744
Merge pull request #19 from salt-formulas/andrew-fix-escapes
epcim Mar 21, 2018
f83a6c7
Update dependencies
epcim Mar 28, 2018
a610eb5
Merge pull request #20 from salt-formulas/update-dependencies
epcim Mar 28, 2018
bd08875
update dependencies on setup.py
epcim Mar 28, 2018
0a3a368
Allow override Scalar (assumed for dict,list) by None
epcim Mar 28, 2018
684e0a6
Merge pull request #21 from salt-formulas/option-to-allow-none-over-s…
epcim Mar 28, 2018
017b1bc
Revert misunderstood add of dependences + partially py3 compatibility…
epcim Mar 28, 2018
2a4beb8
Python3 rationals from #16
epcim Mar 28, 2018
0c6e898
Python3 rationals from #16
epcim Mar 28, 2018
7115849
Update allow_none_override as agreed under PR #21
epcim Mar 29, 2018
5c5d0bc
Interpolate references in class names
epcim Apr 12, 2018
cb18ed4
Merge pull request #15 from salt-formulas/develop
jakubjosef Apr 17, 2018
d04d8e9
Merge pull request #22 from salt-formulas/update-pr21
epcim Apr 17, 2018
5fbab1d
Merge pull request #26 from salt-formulas/ref-in-classes
epcim Apr 17, 2018
3048be2
develop rebase. six.iteritems usage. Other python3 stuff. Tests OK
adrianchifor Apr 5, 2018
412740a
Merge pull request #25 from salt-formulas/python3-new-adrian
epcim Apr 17, 2018
36ecd9d
Removed install prefix
adrianchifor Apr 18, 2018
13a2472
Merge pull request #28 from adrianchifor/install_prefix_remove
epcim Apr 19, 2018
f6775cd
Revert "Interpolate references in class names"
epcim Apr 19, 2018
df55039
Merge pull request #29 from salt-formulas/revert-26-ref-in-classes
epcim Apr 19, 2018
8812e17
Fix failure of parameter overwrites in some cases
AndrewPickford Apr 20, 2018
4b6e914
develop rebase. six.iteritems usage. Other python3 stuff. Tests OK
adrianchifor Apr 5, 2018
bea4512
Allow override Scalar (assumed for dict,list) by None
epcim Mar 28, 2018
6f20a1a
Fix failure of parameter overwrites in some cases
AndrewPickford Apr 20, 2018
03192c8
Removed install prefix
adrianchifor Apr 18, 2018
9711b12
Fix typo entry_points not entry_point
AndrewPickford Apr 23, 2018
b333921
Revert prefix removal
AndrewPickford Apr 23, 2018
2eff4c4
Fix typo entry_points not entry_point
AndrewPickford Apr 23, 2018
9d61ea8
Merge pull request #32 from salt-formulas/andrew-fix-setup-typo
epcim Apr 23, 2018
8e7eb57
Merge pull request #31 from salt-formulas/andrew-fix-overwrites
epcim Apr 25, 2018
44e8331
Implement merge_over method for CompItem class
AndrewPickford Apr 25, 2018
53a570e
Implement a better fix for escaped strings in referenced dictionaries
AndrewPickford Apr 25, 2018
94a2a62
Add tests for handling overwrites of escaped strings
AndrewPickford Apr 26, 2018
7904774
Add travis ci (#35)
epcim May 9, 2018
04cb20a
Allow use of references in classes to load
AndrewPickford Apr 24, 2018
67e737c
Minor fix in doc/readme
epcim Apr 25, 2018
46942b3
Implement merge_over method for CompItem class
AndrewPickford Apr 25, 2018
f1aaff5
Implement a better fix for escaped strings in referenced dictionaries
AndrewPickford Apr 25, 2018
55f1825
Add tests for handling overwrites of escaped strings
AndrewPickford Apr 26, 2018
4493d8a
Merge pull request #42 from salt-formulas/pr/37
epcim May 9, 2018
2853cb1
Merge pull request #43 from salt-formulas/pr/38
epcim May 9, 2018
77d80a8
Bump version to 1.5.3
epcim May 14, 2018
4867a6c
Add travis ci (#35)
epcim May 9, 2018
c350e22
Allow use of references in classes to load
AndrewPickford Apr 24, 2018
6d4a57e
Minor fix in doc/readme
epcim Apr 25, 2018
8125b09
Update allow_none_override as agreed under PR #21
epcim Mar 29, 2018
2d12f7a
YAML: Use C bindings if available
alexandruavadanii Feb 21, 2018
c647dc9
Merge pull request #46 from salt-formulas/andrew-use-c-bindings
epcim May 17, 2018
37fe9e5
Merge pull request #45 from salt-formulas/bump-version-1.5.3
epcim May 17, 2018
d5ec259
Bump version to 1.5.3
epcim May 14, 2018
2652216
Merge pull request #47 from salt-formulas/develop
epcim May 25, 2018
3d313c1
Update README, reclass related tools
epcim Jun 7, 2018
2e01223
Fix: salt 2018.3 + py2.7/3.x compatibility
epcim Jun 18, 2018
520c14f
Merge pull request #49 from salt-formulas/maxarnold-salt20183-fix
epcim Jun 18, 2018
89c31ba
Bump version 1.5.4
epcim Jun 18, 2018
44b576c
Merge pull request #50 from salt-formulas/develop
epcim Jun 19, 2018
4cc8adb
Add uri() method to valuelist class
AndrewPickford Jun 20, 2018
8129ab1
Fix: salt 2018.3 + py2.7/3.x compatibility
epcim Jun 18, 2018
9258cf1
Bump version 1.5.4
epcim Jun 18, 2018
9ac0247
Update README, reclass related tools
epcim Jun 7, 2018
8ef5327
Sync with salt-formulas: remove [install] prefix setting
AndrewPickford Jun 20, 2018
f1889b4
Add uri() method to valuelist class
AndrewPickford Jun 20, 2018
a3f207e
Merge pull request #51 from salt-formulas/andrew-valuelist-uri
epcim Jun 21, 2018
fbe61ba
Move merging logic into one place in ValueList class, improve error r…
AndrewPickford Jun 25, 2018
061e4c6
Replace a nested if chain in DictPath with a single if/elif chain
AndrewPickford Jun 26, 2018
16b7b24
Add new key modifier fixed, default symbol '='
AndrewPickford Jun 26, 2018
a80aa07
Add documentation for fixed/immutable parameter prefix
AndrewPickford Jun 27, 2018
902c6b6
Move merging logic into one place in ValueList class, improve error r…
AndrewPickford Jun 25, 2018
eca7d09
Replace a nested if chain in DictPath with a single if/elif chain
AndrewPickford Jun 26, 2018
fa991d9
Add new key modifier fixed, default symbol '='
AndrewPickford Jun 26, 2018
e3b114e
Add documentation for fixed/immutable parameter prefix
AndrewPickford Jun 27, 2018
67451e4
Fix more stringent tests that failed for python3
AndrewPickford Jun 28, 2018
7784041
Return correct uri for TypeMergeError exceptions with dictionaries
AndrewPickford Jun 28, 2018
a628ca6
Fix more stringent tests that failed for python3
AndrewPickford Jun 28, 2018
5b31850
Fix more stringent tests that failed for python3
AndrewPickford Jun 28, 2018
9229823
Return correct uri for TypeMergeError exceptions with dictionaries
AndrewPickford Jun 28, 2018
4eeb692
Fix: use returned parameter from _render_simple_dict/list in exports …
AndrewPickford Jul 6, 2018
581860f
Merge pull request #52 from salt-formulas/andrewp-overwrite-bug-fix
epcim Jul 9, 2018
e3a0df1
Rename fixed/immutable parameters to constant parameters
AndrewPickford Jul 18, 2018
34d1445
Rename fixed/immutable parameters to constant parameters
AndrewPickford Jul 18, 2018
1c0d8a7
Merge pull request #54 from salt-formulas/andrewp-fixed-parameters
epcim Jul 19, 2018
a92cc93
Fix: use returned parameter from _render_simple_dict/list in exports …
AndrewPickford Jul 6, 2018
1c332a0
Merge pull request #55 from salt-formulas/andrewp-fix-exports
epcim Jul 19, 2018
21d6238
Refactoring pt. 1
a-ovchinnikov Jul 19, 2018
aeea688
Refactoring pt. 2
a-ovchinnikov Jul 19, 2018
286ed46
Bump version 1.5.5
Martin819 Jul 19, 2018
f3f752c
Merge pull request #56 from salt-formulas/develop
Martin819 Jul 19, 2018
d5ba465
Merge pull request #53 from a-ovchinnikov/develop
epcim Jul 20, 2018
c04e1b0
Add option add_subdir_to_node
gburiola Jul 17, 2018
294cc43
Update third.yml
epcim Jul 23, 2018
4ae9ef7
Fix numeric keys at key prefix check
epcim Jul 23, 2018
23949f4
Merge pull request #58 from salt-formulas/pr/57
epcim Jul 23, 2018
8de37a3
update changelog
epcim Jul 23, 2018
96d7ace
Rename option add_subdir_to_node to compose_node_name
gburiola Jul 22, 2018
a7bfd74
Ignore path starting with _ on compose_node_name
gburiola Jul 23, 2018
8a869a6
Merge pull request #48 from gburiola/develop
epcim Jul 30, 2018
7102e87
bump 1.5.6 version
epcim Aug 1, 2018
7fb396d
Merge pull request #61 from salt-formulas/bump-156
epcim Aug 1, 2018
18fd5b2
Merge pull request #62 from salt-formulas/develop
epcim Aug 1, 2018
d15bffc
Fix, class name references in 2nd level
epcim Aug 4, 2018
3627638
Merge pull request #64 from salt-formulas/fix-class-ref-descend
epcim Aug 7, 2018
e768c1a
Merge pull request #65 from salt-formulas/develop
epcim Aug 9, 2018
ae7c3b9
Refactoring pt. 1
a-ovchinnikov Jul 19, 2018
a53b55c
Refactoring pt. 2
a-ovchinnikov Jul 19, 2018
2ad16d6
Bump version 1.5.5
Martin819 Jul 19, 2018
f7e37b0
Add option add_subdir_to_node
gburiola Jul 17, 2018
516dc55
Update third.yml
epcim Jul 23, 2018
46dbd15
Fix numeric keys at key prefix check
epcim Jul 23, 2018
f2f6e03
update changelog
epcim Jul 23, 2018
6dfb57a
Rename option add_subdir_to_node to compose_node_name
gburiola Jul 22, 2018
1f28186
Ignore path starting with _ on compose_node_name
gburiola Jul 23, 2018
6e04545
bump 1.5.6 version
epcim Aug 1, 2018
e53ba05
Fix, class name references in 2nd level
epcim Aug 4, 2018
7ce295c
Fix references bug introduced in refactoring
AndrewPickford Aug 24, 2018
379c8f9
Merge pull request #66 from salt-formulas/andrewp-invitem-refactoring…
epcim Aug 27, 2018
ae337c3
Fix references bug introduced in refactoring
AndrewPickford Aug 24, 2018
97b87cb
Update mixed and yaml_git for node name changes introduced with compo…
AndrewPickford Aug 28, 2018
30abae2
Fix references bug introduced in refactoring
AndrewPickford Aug 24, 2018
d2762b0
Feature, classes, use relative reference
epcim Sep 5, 2018
234ce10
Merge pull request #67 from salt-formulas/andrewp-compose-node-name-f…
epcim Sep 7, 2018
3fb65d8
Merge pull request #68 from salt-formulas/feature-relative-class-name
epcim Sep 7, 2018
23fcc23
Update README.rst
epcim Sep 7, 2018
26b20f7
Update README.rst
epcim Sep 7, 2018
0344dcc
rename README-extensions.rst
epcim Sep 7, 2018
cd8abe9
Update README.rst
epcim Sep 7, 2018
871c61d
Update README.rst
epcim Sep 7, 2018
eb62d35
Update README.rst
epcim Sep 7, 2018
36bc995
Update README.rst
epcim Sep 7, 2018
bf3597a
Update README.rst
epcim Sep 7, 2018
45a675a
Fix merging inv query results
AndrewPickford Sep 11, 2018
2c16887
Update mixed and yaml_git for node name changes introduced with compo…
AndrewPickford Aug 28, 2018
26bc3e5
Feature, classes, use relative reference
epcim Sep 5, 2018
b44ebe0
Merge changes from salt-formulas/develop
AndrewPickford Sep 11, 2018
059726d
Merge pull request #69 from salt-formulas/andrewp-fix-inv-query-merging
epcim Sep 19, 2018
773c05c
Merge remote-tracking branch 'salt-formulas/develop'
AndrewPickford Sep 20, 2018
168a034
Fix relative class name functionality
AndrewPickford Sep 21, 2018
0206c9f
Add tests for compose node name functionality
AndrewPickford Sep 21, 2018
7b03872
Merge pull request #70 from salt-formulas/andrewp-fix-relative-classes
epcim Sep 24, 2018
fa2785f
Add a locking mechanism to yaml_git storage
AndrewPickford Sep 25, 2018
424769f
Add documentation for yaml_git and mixed storage types
AndrewPickford Sep 25, 2018
27dea66
Allow to use '..' as a reference to higher level in class structure
Martin819 Oct 5, 2018
c394ea5
Fix tests
Martin819 Oct 8, 2018
767b772
Add test case for '..'
Martin819 Oct 8, 2018
3471d5d
Edit tests
Martin819 Oct 8, 2018
5a2f9da
Split ifs to separated functions
Martin819 Oct 11, 2018
7794b68
Simplify the absolute name parser
Martin819 Oct 11, 2018
d159be1
Fix error reporting for inventory queries
AndrewPickford Oct 11, 2018
720ad9b
Merge pull request #72 from Martin819/develop
epcim Oct 12, 2018
d1a099a
Merge pull request #71 from salt-formulas/andrewp-yaml-git
epcim Oct 18, 2018
3af1e0d
More refactoring
a-ovchinnikov Oct 19, 2018
c193191
Settings cleanup
a-ovchinnikov Oct 19, 2018
2294aef
Minor changes
a-ovchinnikov Oct 19, 2018
bb76a54
Fix for reference check in core
a-ovchinnikov Oct 19, 2018
ece09c6
Fix for parser reinstantiation bug
a-ovchinnikov Oct 26, 2018
acad24c
Merge pull request #60 from a-ovchinnikov/develop
epcim Nov 5, 2018
8cdce39
Tests for parsers are added
a-ovchinnikov Nov 5, 2018
637a909
bump version 1.7.0
epcim Nov 6, 2018
a83680c
Merge pull request #73 from a-ovchinnikov/develop
epcim Nov 6, 2018
4ba6925
fix version name to 1.6.x
epcim Nov 6, 2018
8c1979d
Fix a typo: Canot -> Cannot
maljub01 Nov 8, 2018
3f0171d
Fix ignore_overwritten_missing_reference default
alexandruavadanii Nov 20, 2018
d00e3ba
Fixed typo in settings definitions.
horakmar Nov 21, 2018
2d93d69
Merge pull request #79 from horakmar/develop
cznewt Nov 21, 2018
bd78071
Merge pull request #76 from maljub01/fix-typo
fpytloun Dec 27, 2018
d9ce787
Merge pull request #78 from alexandruavadanii/fix-default-ignore_over…
epcim Mar 8, 2019
bba9562
added support for .yaml along with .yml
pranavgupta1234 May 2, 2019
8d17363
removed fnmatch import
pranavgupta1234 May 2, 2019
8283bfb
extension support added for yaml_git storage as well
pranavgupta1234 May 2, 2019
961aa7f
remove import fnmatch from yaml_git as well
pranavgupta1234 May 2, 2019
cc7d59d
removed unnecessary typecasting
pranavgupta1234 May 3, 2019
b8de2fb
Merge pull request #85 from pranavgupta1234/extend_file_extension
epcim Jun 19, 2019
e1cec6e
Merge in changes from salt-formulas/develop
AndrewPickford Jun 10, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
126 changes: 124 additions & 2 deletions README-extensions.rst
Original file line number Diff line number Diff line change
@@ -340,17 +340,19 @@ Load classes with relative names
--------------------------------

Load referenced class from a relative location to the current class.
To load class from relative location start the class uri with "." char.
To load class from relative location start the class uri with "." or ".." char.
The only supported reference is to nested tree structure below the current class.

You are allowed to use syntax for relative uri to required class on any place on your model (first class loaded, init.yml, regular class .yml).

The feature is expected to improve flexibility while sharing classes between your models.

Please mpte that you can't use '..' without any calss following. If you want simply up in the sctructure, type in '..init'.

It's a new feature use it with care and mind that using "relative syntax" lower traceability of
your pillar composition.

Example usage of relative class name:
Example usage of relative class name using '.' and '..':

.. code-block:: yaml

@@ -366,6 +368,12 @@ Example usage of relative class name:
classes:
- .defaults

.. code-block:: yaml

#/etc/reclass/classes/component/configuration/init.yml
classes:
- ..defaults


Inventory Queries
-----------------
@@ -569,3 +577,117 @@ This adds the subfolder to the node name and the structure above can then be use
...

If the subfolder path starts with the underscore character ``_``, then the subfolder path is NOT added to the node name.


Git storage type
----------------

Reclass node and class yaml files can be read from a remote git repository with the yaml_git storage type. Use nodes_uri and
classes_uri to define the git repos to use for nodes and classes. These can be the same repo.

For salt masters using ssh connections the private and public keys must be readable by the salt daemon, which requires the
private key NOT be password protected. For stand alone reclass using ssh connections if the privkey and pubkey options
are not defined then any in memory key (from ssh-add) will be used.

Salt master reclass config example:

.. code-block:: yaml

storage_type:yaml_git
nodes_uri:
# branch to use
branch: master

# cache directory (default: ~/.reclass/git/cache)
cache_dir: /var/cache/reclass/git

# lock directory (default: ~/.reclass/git/lock)
lock_dir: /var/cache/reclass/lock

# private key for ssh connections (no default, but will used keys stored
# by ssh-add in memory if privkey and pubkey are not set)
privkey: /root/salt_rsa
# public key for ssh connections
pubkey: /root/salt_rsa.pub

repo: git+ssh://[email protected]:salt/nodes.git

classes_uri:
# branch to use or __env__ to use the branch matching the node
# environment name
branch: __env__

# cache directory (default: ~/.reclass/git/cache)
cache_dir: /var/cache/reclass/git

# lock directory (default: ~/.reclass/git/lock)
lock_dir: /var/cache/reclass/lock

# private key for ssh connections (no default, but will used keys stored
# by ssh-add in memory if privkey and pubkey are not set)
privkey: /root/salt_rsa
# public key for ssh connections
pubkey: /root/salt_rsa.pub

# branch/env overrides for specific branches
env_overrides:
# prod env uses master branch
- prod:
branch: master
# use master branch for nodes with no environment defined
- none:
branch: master

repo: git+ssh://[email protected]:salt/site.git

# root directory of the class hierarcy in git repo
# defaults to root directory of git repo if not given
root: classes


Mixed storage type
------------------

Use a mixture of storage types.

Salt master reclass config example, which by default uses yaml_git storage but overrides the location for
classes for the pre-prod environment to use a directory on the local disc:

.. code-block:: yaml

storage_type: mixed
nodes_uri:
# storage type to use
storage_type: yaml_git

# yaml_git storage options
branch: master
cache_dir: /var/cache/reclass/git
lock_dir: /var/cache/reclass/lock
privkey: /root/salt_rsa
pubkey: /root/salt_rsa.pub
repo: git+ssh://[email protected]:salt/nodes.git

classes_uri:
# storage type to use
storage_type: yaml_git

# yaml_git storage options
branch: __env__
cache_dir: /var/cache/reclass/git
lock_dir: /var/cache/reclass/lock
privkey: /root/salt_rsa
pubkey: /root/salt_rsa.pub
repo: git+ssh://[email protected]:salt/site.git
root: classes

env_overrides:
- prod:
branch: master
- none:
branch: master
- pre-prod:
# override storage type for this environment
storage_type: yaml_fs
# options for yaml_fs storage type
uri: /srv/salt/env/pre-prod/classes
4 changes: 4 additions & 0 deletions doc/source/changelog.rst
Original file line number Diff line number Diff line change
@@ -5,6 +5,10 @@ ChangeLog
========= ========== ========================================================
Version Date Changes
========= ========== ========================================================
1.6.0 2018-11-06 * Python code and parser refactoring by a-ovchinnikov
* Improvements in yaml_git and mixed setup by Andrew Pickford
* Relative paths in class names by Petr Michalec, Martin Polreich and Andrew Pickford
* Bug Fixes for recently added features
1.5.6 2018-07-30 * Fix, usage of integers as pillar keys
* Refactoring python codebase by @a-ovchinkonv
* New feature, "compose node name" from node subdirectory structure (by @gburiola)
2 changes: 1 addition & 1 deletion reclass/__init__.py
Original file line number Diff line number Diff line change
@@ -19,7 +19,7 @@ def get_storage(storage_type, nodes_uri, classes_uri, compose_node_name, **kwarg
storage_class = StorageBackendLoader(storage_type).load()
return MemcacheProxy(storage_class(nodes_uri, classes_uri, compose_node_name, **kwargs))

def get_path_mangler(storage_type,**kwargs):
def get_path_mangler(storage_type, **kwargs):
return StorageBackendLoader(storage_type).path_mangler()

def output(data, fmt, pretty_print=False, no_refs=False):
34 changes: 20 additions & 14 deletions reclass/core.py
Original file line number Diff line number Diff line change
@@ -23,7 +23,6 @@
from six import iteritems

from reclass.settings import Settings
from reclass.output.yaml_outputter import ExplicitDumper
from reclass.datatypes import Entity, Classes, Parameters, Exports
from reclass.errors import MappingFormatError, ClassNameResolveError, ClassNotFound, InvQueryClassNameResolveError, InvQueryClassNotFound, InvQueryError, InterpolationError, ResolveError
from reclass.values.parser import Parser
@@ -39,7 +38,8 @@ def __init__(self, storage, class_mappings, settings, input_data=None):
self._settings = settings
self._input_data = input_data
if self._settings.ignore_class_notfound:
self._cnf_r = re.compile('|'.join([x for x in self._settings.ignore_class_notfound_regexp]))
self._cnf_r = re.compile(
'|'.join(self._settings.ignore_class_notfound_regexp))

@staticmethod
def _get_timestamp():
@@ -114,7 +114,9 @@ def _recurse_entity(self, entity, merge_base=None, context=None, seen=None, node

for klass in entity.classes.as_list():
# class name contain reference
if klass.count('$') > 0:
num_references = klass.count(self._settings.reference_sentinels[0]) +\
klass.count(self._settings.export_sentinels[0])
if num_references > 0:
try:
klass = str(self._parser.parse(klass, self._settings).render(merge_base.parameters.as_dict(), {}))
except ResolveError as e:
@@ -123,10 +125,6 @@ def _recurse_entity(self, entity, merge_base=None, context=None, seen=None, node
except ResolveError as e:
raise ClassNameResolveError(klass, nodename, entity.uri)

# for convenience, first level classes_uri/class.yml can have un-interpolated "."
if klass.startswith('.'):
klass = klass[1:]

if klass not in seen:
try:
class_entity = self._storage.get_class(klass, environment, self._settings)
@@ -156,8 +154,16 @@ def _recurse_entity(self, entity, merge_base=None, context=None, seen=None, node

def _get_automatic_parameters(self, nodename, environment):
if self._settings.automatic_parameters:
return Parameters({ '_reclass_': { 'name': { 'full': nodename, 'short': nodename.split('.')[0] },
'environment': environment } }, self._settings, '__auto__')
pars = {
'_reclass_': {
'name': {
'full': nodename,
'short': nodename.split('.')[0]
},
'environment': environment
}
}
return Parameters(pars, self._settings, '__auto__')
else:
return Parameters({}, self._settings, '')

@@ -166,13 +172,12 @@ def _get_inventory(self, all_envs, environment, queries):
for nodename in self._storage.enumerate_nodes():
try:
node_base = self._storage.get_node(nodename, self._settings)
if node_base.environment == None:
if node_base.environment is None:
node_base.environment = self._settings.default_environment
except yaml.scanner.ScannerError as e:
if self._settings.inventory_ignore_failed_node:
continue
else:
raise
raise

if all_envs or node_base.environment == environment:
try:
@@ -193,7 +198,7 @@ def _get_inventory(self, all_envs, environment, queries):
node.interpolate_single_export(q)
except InterpolationError as e:
e.nodename = nodename
raise InvQueryError(q.contents(), e, context=p, uri=q.uri)
raise InvQueryError(q.contents, e, context=p, uri=q.uri)
inventory[nodename] = node.exports.as_dict()
return inventory

@@ -224,7 +229,8 @@ def _nodeinfo(self, nodename, inventory):
raise

def _nodeinfo_as_dict(self, nodename, entity):
ret = {'__reclass__' : {'node': entity.name, 'name': nodename,
ret = {'__reclass__' : {'node': entity.name,
'name': nodename,
'uri': entity.uri,
'environment': entity.environment,
'timestamp': Core._get_timestamp()
12 changes: 4 additions & 8 deletions reclass/datatypes/applications.py
Original file line number Diff line number Diff line change
@@ -28,18 +28,14 @@ class Applications(Classes):

def __init__(self, iterable=None,
negation_prefix=DEFAULT_NEGATION_PREFIX):
self._negation_prefix = negation_prefix
self.negation_prefix = negation_prefix
self._offset = len(negation_prefix)
self._negations = []
super(Applications, self).__init__(iterable)

@property
def negation_prefix(self):
return self._negation_prefix

def append_if_new(self, item):
self._assert_is_string(item)
if item.startswith(self._negation_prefix):
if item.startswith(self.negation_prefix):
item = item[self._offset:]
self._negations.append(item)
try:
@@ -64,6 +60,6 @@ def merge_unique(self, iterable):

def __repr__(self):
contents = self._items + \
['%s%s' % (self._negation_prefix, i) for i in self._negations]
['%s%s' % (self.negation_prefix, i) for i in self._negations]
return "%s(%r, %r)" % (self.__class__.__name__, contents,
str(self._negation_prefix))
str(self.negation_prefix))
2 changes: 1 addition & 1 deletion reclass/datatypes/classes.py
Original file line number Diff line number Diff line change
@@ -57,7 +57,7 @@ def merge_unique(self, iterable):

def _assert_is_string(self, item):
if not isinstance(item, six.string_types):
raise TypeError('%s instances can only contain strings, '\
raise TypeError('%s instances can only contain strings, '
'not %s' % (self.__class__.__name__, type(item)))

def _assert_valid_characters(self, item):
21 changes: 10 additions & 11 deletions reclass/datatypes/entity.py
Original file line number Diff line number Diff line change
@@ -35,7 +35,6 @@ def __init__(self, settings, classes=None, applications=None,
self._environment = environment

name = property(lambda s: s._name)
short_name = property(lambda s: s._short_name)
uri = property(lambda s: s._uri)
classes = property(lambda s: s._classes)
applications = property(lambda s: s._applications)
@@ -61,10 +60,10 @@ def _set_field(self, received_value, expected_type, parameters=None):
return received_value

def merge(self, other):
self._classes.merge_unique(other._classes)
self._applications.merge_unique(other._applications)
self._parameters.merge(other._parameters)
self._exports.merge(other._exports)
self._classes.merge_unique(other.classes)
self._applications.merge_unique(other.applications)
self._parameters.merge(other.parameters)
self._exports.merge(other.exports)
self._name = other.name
self._uri = other.uri
self._parameters._uri = other.uri
@@ -91,12 +90,12 @@ def interpolate_single_export(self, references):

def __eq__(self, other):
return isinstance(other, type(self)) \
and self._applications == other._applications \
and self._classes == other._classes \
and self._parameters == other._parameters \
and self._exports == other._exports \
and self._name == other._name \
and self._uri == other._uri
and self._applications == other.applications \
and self._classes == other.classes \
and self._parameters == other.parameters \
and self._exports == other.exports \
and self._name == other.name \
and self._uri == other.uri

def __ne__(self, other):
return not self.__eq__(other)
3 changes: 0 additions & 3 deletions reclass/datatypes/exports.py
Original file line number Diff line number Diff line change
@@ -23,9 +23,6 @@ class Exports(Parameters):
def __init__(self, mapping, settings, uri):
super(Exports, self).__init__(mapping, settings, uri)

def __repr__(self):
return '%s(%r)' % (self.__class__.__name__, self._base)

def delete_key(self, key):
self._base.pop(key, None)
self._unrendered.pop(key, None)
19 changes: 9 additions & 10 deletions reclass/datatypes/parameters.py
Original file line number Diff line number Diff line change
@@ -105,24 +105,23 @@ def _wrap_value(self, value):
e.context = DictPath(self._settings.delimiter)
raise

def _get_wrapped(self, position, value):
try:
return self._wrap_value(value)
except InterpolationError as e:
e.context.add_ancestor(str(position))
raise

def _wrap_list(self, source):
l = ParameterList(uri=self._uri)
for (k, v) in enumerate(source):
try:
l.append(self._wrap_value(v))
except InterpolationError as e:
e.context.add_ancestor(str(k))
raise
l.append(self._get_wrapped(k, v))
return l

def _wrap_dict(self, source):
d = ParameterDict(uri=self._uri)
for (k, v) in iteritems(source):
try:
d[k] = self._wrap_value(v)
except InterpolationError as e:
e.context.add_ancestor(str(k))
raise
d[k] = self._get_wrapped(k, v)
return d

def _update_value(self, cur, new):
Loading