Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1161/fixes summary missing required field info #1192

Closed
Closed
Changes from 1 commit
Commits
Show all changes
545 commits
Select commit Hold shift + click to select a range
cdf7f79
Fixed some linting errors
roll Jul 2, 2022
0468cee
Renamed create_resource -> detect_resource
roll Jul 2, 2022
d0fbbff
Fixed resource.__create__
roll Jul 2, 2022
bb85b02
Rebased on resource.type and profiles
roll Jul 2, 2022
661ad0f
Removed unused profiles
roll Jul 2, 2022
c20e7ea
Fixed stats in infer
roll Jul 2, 2022
279299c
Recovered resource tests
roll Jul 2, 2022
53fb04e
Improved package.__create__
roll Jul 2, 2022
d960c6e
Fixed some tests
roll Jul 2, 2022
cd7e2b0
Recovered more tests
roll Jul 2, 2022
c6a7e36
Recovered package tests
roll Jul 2, 2022
8be80ab
Merge pull request #1153 from frictionlessdata/944/simplify-metadata
roll Jul 2, 2022
9bbfdf8
Merge pull request #1154 from frictionlessdata/944/simplify-metadata
roll Jul 2, 2022
694721f
Added hashing inheritance package -> resource
roll Jul 2, 2022
610a145
Added dialect package -> resource inheritance
roll Jul 2, 2022
a31c245
Rebased Pipeline on dataclass
roll Jul 2, 2022
b19c03d
Rebased checklist on dataclass
roll Jul 2, 2022
0d36fb3
Synced connect style
roll Jul 2, 2022
74c3615
Fixed metadata.__repr__
roll Jul 2, 2022
4a4e361
Removed not neded package/resource inheritance
roll Jul 3, 2022
7f3b7c0
Split resource class into multiple files
roll Jul 3, 2022
539b30e
Split package class into multiple files
roll Jul 3, 2022
6854533
Fixed sources/contributors validation
roll Jul 3, 2022
7f28565
Recovered checks
roll Jul 3, 2022
ddb3444
Recovered detector tests
roll Jul 3, 2022
583daee
Recovered validate general
roll Jul 4, 2022
fed1693
Recovered validate resource checklist
roll Jul 4, 2022
2c6b0fc
Recovered resource compression validate
roll Jul 4, 2022
4ea1fcc
Recovered resource validate dialect
roll Jul 4, 2022
c935834
Recovered resource validate encoding
roll Jul 4, 2022
0110316
Recovered resource validate format
roll Jul 4, 2022
6463f41
Recovered resource validate schema
roll Jul 4, 2022
0cb1c03
Recovered resource validate scheme
roll Jul 4, 2022
80e1a37
Recovered resource validate stats
roll Jul 4, 2022
f0ea4bf
Recovered resource describe skip
roll Jul 4, 2022
e0e7aea
Recovered resource extract tests
roll Jul 4, 2022
5cea167
Recovered resource transform tests
roll Jul 4, 2022
af75158
Renamed Problems to Bugs -- test comment
roll Jul 4, 2022
6a14b66
Recovered system tests
roll Jul 4, 2022
1d3ce75
Recovered metadata tests
roll Jul 4, 2022
47ed787
Recovered more tests
roll Jul 4, 2022
50ad70d
Recovered table tests
roll Jul 4, 2022
98e2748
Recovered schema tests
roll Jul 4, 2022
b6df3e6
Recovered cell/field steps
roll Jul 4, 2022
69a8127
Recovered resource step tests
roll Jul 4, 2022
2144399
Recovered row step tests
roll Jul 4, 2022
aff3eaa
Recovered report tests
roll Jul 4, 2022
fcd1ac5
Recovered step tests
roll Jul 4, 2022
782f4b2
Fixed package.validate
roll Jul 4, 2022
b79541c
Recovered Inqiury tests
roll Jul 4, 2022
7f3e123
Recovered formats tests
roll Jul 4, 2022
a8d970e
Fixed csv header writing
roll Jul 4, 2022
5a68a14
Recovered csv write tests
roll Jul 4, 2022
2981e3b
Improved csv parser
roll Jul 4, 2022
dd6f1b2
Recovered excel write
roll Jul 4, 2022
be2cefe
Recovered json tests
roll Jul 4, 2022
efb2213
Recovered html tests
roll Jul 4, 2022
d5f19f4
Fixed gsheets writing
roll Jul 4, 2022
f99fe15
Recovered ods writing
roll Jul 4, 2022
d065297
Recovered inline writing
roll Jul 4, 2022
e245ce9
Updated sqlite tests
roll Jul 4, 2022
8e26fac
Improved metadata.metadata_defaults
roll Jul 5, 2022
48489f1
Recovered detector tests
roll Jul 5, 2022
df94bf3
Added internal detect methods to resource
roll Jul 5, 2022
37b0636
Maked plugins responsible for resource.type
roll Jul 5, 2022
621b699
Recovered dialect tests
roll Jul 5, 2022
84234e1
Recovered xls tests
roll Jul 5, 2022
10a21d7
Recovered xlsx tests
roll Jul 5, 2022
ff5cd90
Fixed sync_schema logic
roll Jul 5, 2022
8022645
Fixed csv tests
roll Jul 5, 2022
d8fc7fc
Added support for mediatype
roll Jul 5, 2022
c806894
Recovered ckan tests
roll Jul 5, 2022
30745bc
Recovered bigquery tests
roll Jul 5, 2022
5a5244e
Recovered spss tests
roll Jul 5, 2022
e5f0cd4
Improved helpers
roll Jul 5, 2022
bf97952
Recovered pandas tests
roll Jul 5, 2022
ca2d6ab
Fixed mediatype in profile
roll Jul 5, 2022
32c2525
Recovered fields tests
roll Jul 5, 2022
2d7ff14
Recovered sqlite tests
roll Jul 5, 2022
6377a21
Recovered postgresql tests
roll Jul 5, 2022
09b5d2f
Recovered mysql tests
roll Jul 5, 2022
651fe15
Recovered schemes tests
roll Jul 5, 2022
bddbb41
Recovered multipart
roll Jul 6, 2022
0d2139c
Recovered s3 tests
roll Jul 6, 2022
c61db32
Renamed s3 plugin to aws
roll Jul 6, 2022
9902ede
Recovered inquiry tests
roll Jul 6, 2022
44fe120
Recovered table tests
roll Jul 6, 2022
4ff5801
Recovered report tests
roll Jul 6, 2022
0e634fd
Simplified InquiryTask
roll Jul 6, 2022
db2d0ac
Added system.standards_version
roll Jul 6, 2022
d4f6c27
Recovered unzip_descriptor
roll Jul 6, 2022
e40154b
Added v1-compat level to Package/Resource/Schema
roll Jul 6, 2022
201441c
Recovered general package tests
roll Jul 6, 2022
b2b2569
Recovered package convert tests
roll Jul 6, 2022
0b0df24
Uncommented package tests
roll Jul 6, 2022
1f32b6f
Merge branch 'main' into v5
roll Jul 6, 2022
d8455f5
Recovered schema convert
roll Jul 6, 2022
0c7f423
Recovered schema tests
roll Jul 6, 2022
1534506
Fixed formats/inquiry tests
roll Jul 6, 2022
9833040
Recovered describe actions
roll Jul 7, 2022
cff74bf
Recovered extract actions
roll Jul 7, 2022
79bfdda
Recovered validate actions
roll Jul 7, 2022
0efdfaf
Recovered transform actions
roll Jul 7, 2022
040c8f0
Fixed last actions tests
roll Jul 7, 2022
57b8e4a
Xfailed last actions
roll Jul 7, 2022
150c470
Recovered package tests
roll Jul 7, 2022
e104b38
Added support for custom metadata
roll Jul 7, 2022
0e3ff4b
Fixed error.to_descriptor
roll Jul 7, 2022
20a9035
Recovered package tests
roll Jul 7, 2022
1f22043
Recovered program.describe
roll Jul 7, 2022
727990f
Fixed metadata
roll Jul 7, 2022
b4f07d3
Added sample argument to infer
roll Jul 8, 2022
96931ba
Recovered program.extract
roll Jul 8, 2022
238a702
Recovered program.extract tests
roll Jul 8, 2022
979540a
Fixed resource's lookup
roll Jul 8, 2022
b3816ff
Recovered program.validate
roll Jul 8, 2022
8755573
Recovered program.validate tests
roll Jul 8, 2022
760d4cd
Recovered program.summary tests
roll Jul 8, 2022
c32497a
Recovered program tests
roll Jul 8, 2022
35d9a46
Merge branch 'main' into v5
roll Jul 8, 2022
4e289fa
Fixed report tests
roll Jul 8, 2022
4f0e18c
Xfailed failing tests
roll Jul 8, 2022
d93118c
Recovered resource.compression tests
roll Jul 8, 2022
82cf8e7
Recovered resource.convert tests
roll Jul 8, 2022
9481353
Recovered resource.detector tests
roll Jul 8, 2022
a2c981e
Recovered resource.dialect tests
roll Jul 8, 2022
c2214c4
Recovered resource.format tests
roll Jul 8, 2022
0e59628
Recover resource.general tests
roll Jul 8, 2022
866ac41
Recovered resource.infer tests
roll Jul 8, 2022
64bb7b3
Recovered resource.onerror tests
roll Jul 8, 2022
07191ed
Recovered resource.open tests
roll Jul 8, 2022
dde85fc
Recovered resource.read tests
roll Jul 8, 2022
6f76892
Recovered resource.schema tests
roll Jul 8, 2022
4a29330
Recovered resource.write tests
roll Jul 8, 2022
31669d9
Recovered resoruce methods tests
roll Jul 8, 2022
346e0e7
Recovered resource.validate tests
roll Jul 8, 2022
6b5b6dd
Recovered resource tests
roll Jul 8, 2022
42ef78a
Recovered package test
roll Jul 8, 2022
63342d0
Rebased steps tests on xfail
roll Jul 8, 2022
c731938
Recovered linting
roll Jul 8, 2022
4cee4c3
Skipped failing type checks for now
roll Jul 8, 2022
9cd1c20
Recovered CI tests
roll Jul 8, 2022
519b2d7
Fixed linting
roll Jul 8, 2022
6456275
Fixed tests
roll Jul 8, 2022
4b485b8
Disable windown testing
roll Jul 8, 2022
f7e89ed
Fixed CI
roll Jul 8, 2022
59b827b
Returned windows to CI
roll Jul 9, 2022
ea92096
Added TODO
roll Jul 9, 2022
2f27531
Fixed table dimensions check
roll Jul 9, 2022
64c6f2f
Added csv/excel/json to the core deps
roll Jul 9, 2022
cb2b257
Rebased on helpers.import_from_extras
roll Jul 9, 2022
c1a299d
Added todos
roll Jul 9, 2022
95b7c12
Added infer to package.to_er_diagram
roll Jul 9, 2022
5a43dbf
Returned metadata_export/import
roll Jul 9, 2022
ad430b7
Removed plugin.code for now
roll Jul 9, 2022
7d3f51e
Removed limitMemory
roll Jul 9, 2022
1dfbb39
Reworked limit_row/errors
roll Jul 9, 2022
c5e8c8c
Renamed "original" to "strict"
roll Jul 9, 2022
dc0468a
Added multipart support to resource.place
roll Jul 9, 2022
4f58080
Rebased on reportTask.type
roll Jul 9, 2022
7defb8d
Added inquiryTask.type
roll Jul 9, 2022
94e9611
Added debug to CLI
roll Jul 9, 2022
53c825d
Added standards option to program.describe
roll Jul 9, 2022
7aab9f0
Added a TODO
roll Jul 9, 2022
282650a
Fixed resource.metadata_export
roll Jul 10, 2022
59cae85
Improved parser.write_row_stream API
roll Jul 10, 2022
359a445
Renamed list_stream to cell_stream
roll Jul 10, 2022
8b92336
Removed version from report
roll Jul 10, 2022
337330f
Improved program.validate
roll Jul 10, 2022
02cffba
Added package/schema.flatten()
roll Jul 10, 2022
344caa7
Rebased on metadata.metadata_Types
roll Jul 10, 2022
c3f8d25
Removed metadata_properties
roll Jul 11, 2022
b1d98ce
Rebased row reading on cell readers
roll Jul 11, 2022
3652f67
Added markfown option to frictionless describe
roll Jul 11, 2022
da65e23
Implemented checks option for program.validate
roll Jul 11, 2022
4324964
Added steps options to program.transform
roll Jul 11, 2022
85cf582
Reworked dialect descriptor
roll Jul 12, 2022
79f6dfe
Reworked dialect
roll Jul 12, 2022
278035a
Improved tests
roll Jul 12, 2022
4c467de
Added support for v1 dialect.[csv] props
roll Jul 12, 2022
778d9e7
Added support for resource.layout (v1.5)
roll Jul 12, 2022
ad18c7c
Fixed linting
roll Jul 12, 2022
d345ebd
Supported dialect to v1 export
roll Jul 12, 2022
8a6e4e4
Apply comment char only to strings
roll Jul 12, 2022
616faca
Renamed check/error.code -> type
roll Jul 12, 2022
3472330
Renamed control.code -> type
roll Jul 12, 2022
9fb4f33
Renamed step.code -> type
roll Jul 12, 2022
afe70f9
Added dialect/control.name
roll Jul 12, 2022
6421249
Started adding name to Checklist/Check
roll Jul 12, 2022
0745159
Added name/title/descriptor for Checklist/Check
roll Jul 13, 2022
27ac9ec
Added name/title/description to Detector
roll Jul 13, 2022
4aebc3c
Added name/title/description to Dialect
roll Jul 13, 2022
a0fb78b
Added name/title/description to Inquiry
roll Jul 13, 2022
8743457
Added name/title/description to Schema
roll Jul 13, 2022
156956d
Added name/title/description to Report
roll Jul 13, 2022
02173d6
Added name/title/description to Pipeline/Step
roll Jul 13, 2022
385b209
Renamed error.name -> title
roll Jul 13, 2022
961f92f
Fixed tests
roll Jul 13, 2022
2be636a
Added jsonmerge
roll Jul 13, 2022
003234a
Migrated checks/controls to metadata_merge
roll Jul 13, 2022
fa4d06c
Rebased on metadata_profile_patch
roll Jul 13, 2022
f5ea880
Rebased errors on metadata_profile_patch
roll Jul 13, 2022
70cb48d
Rebased steps on metadata_profile_patch
roll Jul 13, 2022
2c411ff
Improved Metadata
roll Jul 13, 2022
07dd677
Rebase resource validation on metadata_profile
roll Jul 13, 2022
16e35c9
Rebased package validateion on metadata_profile
roll Jul 13, 2022
67df936
Rebased schema validation on metadata_profile
roll Jul 13, 2022
c07d29f
Rebased field validation on metadata_profile
roll Jul 13, 2022
5a6e90b
Rebased fields validation on metadata_profile
roll Jul 14, 2022
11e7977
Improved required in metadata_profiles
roll Jul 14, 2022
66bc4e3
Fixed additional args in actions
roll Jul 14, 2022
4a2d851
Added minimal resource example
roll Jul 14, 2022
126d0d5
Improved validation error
roll Jul 14, 2022
3e69355
Recovered strict mode validation
roll Jul 14, 2022
540c2bd
Improved InquiryTask error message
roll Jul 14, 2022
a0e7761
Added required value check (#1178)
shashigharti Jul 14, 2022
7e83132
Updated to metadata_profile_patch
roll Jul 14, 2022
38391b9
Enable metadata validation on resource.open
roll Jul 14, 2022
899243b
Improved resource.write
roll Jul 14, 2022
4a374af
Support legacy pipelines
roll Jul 14, 2022
cfdd5ea
Added support for legacy inquiry
roll Jul 14, 2022
cc5e2a2
Added frictionless$@4 deprecation warnings
roll Jul 14, 2022
8cb467b
Added name/type patterns to profiles
roll Jul 14, 2022
5d20bb9
Required unique field names
roll Jul 14, 2022
58bb372
Rebased on has_dialect/schema
roll Jul 15, 2022
80887ff
Improved internal resource logic
roll Jul 15, 2022
45591e4
Fixed tests
roll Jul 15, 2022
f71a400
Enabled strict mode for formats.csv
roll Jul 15, 2022
4cc0ac6
Removed file-level type ignores
roll Jul 15, 2022
4760990
Include tests for pyright (the checker is broken currently)
roll Jul 15, 2022
22f8e55
Migrated json parser from create_parser
roll Jul 16, 2022
b2fc79d
Migrated jsonl parser from create_parser
roll Jul 16, 2022
7d6916a
Rebased from fullpath to normpath
roll Jul 16, 2022
07da81f
Index package.analyze/extract by resource.name
roll Jul 16, 2022
c801739
Updated dev deps
roll Jul 16, 2022
d974bfe
Implemented resource.normdata
roll Jul 16, 2022
9547873
Bootstrapped Catalog class
roll Jul 16, 2022
cdb54cc
Bootstrapped Manager API
roll Jul 16, 2022
618f070
Bootstrapped portals.ckan
roll Jul 16, 2022
8af3b73
First version of ckanManager.read_package
roll Jul 16, 2022
e8336ac
Support opening package from a manager
roll Jul 16, 2022
1b612cb
Fixed version on catalog from ckan
roll Jul 16, 2022
25b9630
Support datapackage.json/yaml on ckan
roll Jul 16, 2022
9afe085
Replaced old ckan implementation
roll Jul 16, 2022
e76bd1c
First version of Github manager
roll Jul 16, 2022
d28c89e
Bootstrapped portals.zenodo
roll Jul 16, 2022
786747c
Added Metadata.metadata_type
roll Jul 17, 2022
774d91c
First zenodo implementation
roll Jul 17, 2022
f3d5192
Fixed linting
roll Jul 17, 2022
f9dddac
Fixed missing required info in resource summary
shashigharti Jul 18, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Migrated jsonl parser from create_parser
roll committed Jul 16, 2022

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit b2fc79d7c05558297381a545e19a2516ebb9c5b1
16 changes: 5 additions & 11 deletions frictionless/formats/json/parsers/jsonl.py
Original file line number Diff line number Diff line change
@@ -4,7 +4,6 @@
from ...inline import InlineControl
from ....resource import Resource
from ..control import JsonControl
from ....dialect import Dialect
from ....resource import Parser
from ....system import system

@@ -30,17 +29,12 @@ def read_cell_stream_create(self):
control = JsonControl.from_dialect(self.resource.dialect)
source = iter(jsonlines.Reader(self.loader.text_stream))
inline_control = InlineControl(keys=control.keys)
resource = Resource(
data=source,
format="inline",
dialect=Dialect(controls=[inline_control]),
)
with system.create_parser(resource) as parser:
yield next(parser.cell_stream) # type: ignore
parser_control = InlineControl.from_dialect(parser.resource.dialect)
if parser_control.keyed:
with Resource(data=source, format="inline", control=inline_control) as resource:
yield next(resource.cell_stream) # type: ignore
inline_control = InlineControl.from_dialect(resource.dialect)
if inline_control.keyed:
control.keyed = True
yield from parser.cell_stream
yield from resource.cell_stream

# Write