-
Notifications
You must be signed in to change notification settings - Fork 36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Invalid input metadata format doesn't interrupt pipeline processing #61
Comments
|
Having an option to express that behavior would be fine for us. |
Closing this issue to re-open as feature request. |
cf #62 |
I am running the latest pyff code (head from github).
We had an issue while running a pyff pipeline that merges two metadata file, one from eduGAIN and one from our federation registry. Our input metadata file was syntactically incorrect. Pyff detected that but however generated the output metadata with only eduGAIN SAML entity in it.
I would expect that pyff would interrupt the pipeline if one of the input source is missing, thus preventing the output metadata from being updated with incomplete data.
This issue is somehow related to #47
I had a look at the code but I am not a python developer and I'm not familiar with pyff internals. Shouldn't that MetadataException be handled in pipes::_process() subroutine?
Here is a pipeline that can be used to reproduce the problem:
Running this pipeline with pyff should fail because one of the input metadata file has an incorrect format
Name: https://metadata.renater.fr/
cacheDuration: PT1H
validUntil: P6D
Here is the pyff output while running this pipeline:
DEBUG:root:Processing
DEBUG:root:load parsing '/tmp/saml-metadata-format-issue.xml'
DEBUG:root:file /tmp/saml-metadata-format-issue.xml verify None as None via None
DEBUG:root:load parsing '/var/www/html/edugain/sps-edugain-metadata.xml'
DEBUG:root:file /var/www/html/edugain/sps-edugain-metadata.xml verify None as None via None
DEBUG:root:fetching (caching: True) 'file:///tmp/saml-metadata-format-issue.xml'
ERROR:root:Extra content at the end of the document, line 87, column 1
DEBUG:root:fetching (caching: True) 'file:///var/www/html/edugain/sps-edugain-metadata.xml'
ERROR:root:'file:///tmp/saml-metadata-format-issue.xml' generated an exception: no valid metadata found at 'file:///tmp/saml-metadata-format-issue.xml'
DEBUG:iso8601.iso8601:Parsed 2015-05-04T10:30:31Z into {'tz_sign': None, 'second_fraction': None, 'hour': '10', 'daydash': '04', 'tz_hour': None, 'month': None, 'timezone': 'Z', 'second': '31', 'tz_minute': None, 'year': '2015', 'separator': 'T', 'monthdash': '05', 'day': None, 'minute': '30'} with default timezone <iso8601.iso8601.Utc object at 0x7f614dd449d0>
DEBUG:iso8601.iso8601:Got '2015' for 'year' with default None
DEBUG:iso8601.iso8601:Got '05' for 'monthdash' with default 1
DEBUG:iso8601.iso8601:Got 5 for 'month' with default 5
DEBUG:iso8601.iso8601:Got '04' for 'daydash' with default 1
DEBUG:iso8601.iso8601:Got 4 for 'day' with default 4
DEBUG:iso8601.iso8601:Got '10' for 'hour' with default None
DEBUG:iso8601.iso8601:Got '30' for 'minute' with default None
DEBUG:iso8601.iso8601:Got '31' for 'second' with default None
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/ws-federation.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' oasis-200401-wss-wssecurity-secext-1.0.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' oasis-200401-wss-wssecurity-utility-1.0.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' xml.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' xmldsig-core-schema.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' ws-addr.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' MetadataExchange.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' saml-schema-metadata-2.0.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' xenc-schema.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' sstc-saml-schema-assertion-2.0.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' ws-securitypolicy-1.2.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' ws-authorization.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/cs-sstc-schema-assertion-1.1.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/sstc-saml-metadata-ui-v1.0.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/privacy.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/shibboleth-metadata-1.0.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/shibboleth-trust-1.0.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/shibboleth.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/sstc-metadata-attr.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/sstc-saml-metadata-algsupport.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/sstc-saml-idp-discovery.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/atom.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/xrd-1.0-os.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/saml-metadata-rpi-v1.0.xsd' for 'None'
DEBUG:root:returning 916 valid entities
DEBUG:root:{'Status': 'success', 'Time Spent': '0.0 seconds', 'Last-Modified': '2015-04-28 12:31:27.419980', 'URL': None, 'Bytes': '6350212', 'Expiration Time': '2015-05-04 12:30:31.343367', 'Cached': 'False', 'Tries': '1', 'Size': '916', 'Date': '2015-04-28 12:50:01.060455', 'Description': 'Remote metadata', 'Cache TTL': '517230.0', 'Validation Errors': {}}
DEBUG:root:retrying []
DEBUG:root:selecting using args: None
DEBUG:root:calling store lookup https://federation.renater.fr/edugain/
DEBUG:root:selecting 916 entities before validation
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/ws-federation.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' oasis-200401-wss-wssecurity-secext-1.0.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' oasis-200401-wss-wssecurity-utility-1.0.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' xml.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' xmldsig-core-schema.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' ws-addr.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' MetadataExchange.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' saml-schema-metadata-2.0.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' xenc-schema.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' sstc-saml-schema-assertion-2.0.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' ws-securitypolicy-1.2.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' ws-authorization.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/cs-sstc-schema-assertion-1.1.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/sstc-saml-metadata-ui-v1.0.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/privacy.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/shibboleth-metadata-1.0.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/shibboleth-trust-1.0.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/shibboleth.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/sstc-metadata-attr.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/sstc-saml-metadata-algsupport.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/sstc-saml-idp-discovery.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/atom.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/xrd-1.0-os.xsd' for 'None'
DEBUG:root:resolve SYSTEM URL' /opt/pyff-latest/lib/python2.7/site-packages/pyFF-0.10.0.dev0-py2.7.egg/pyff/schema/saml-metadata-rpi-v1.0.xsd' for 'None'
DEBUG:root:publish /tmp/output-metadata.xml
DEBUG:root:output_file=/tmp/output-metadata.xml, resource_name=/tmp/output-metadata.xml
total size: 916
selected: 916
idps: 4
sps: 916
The text was updated successfully, but these errors were encountered: