Skip to content

Commit

Permalink
rewrite interface{} to any
Browse files Browse the repository at this point in the history
  • Loading branch information
itchyny committed Feb 3, 2023
1 parent e696a30 commit 4c776c0
Show file tree
Hide file tree
Showing 39 changed files with 653 additions and 657 deletions.
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ docker run -i --rm ghcr.io/itchyny/gojq
- gojq supports arbitrary-precision integer calculation while jq does not; jq loses the precision of large integers when calculation is involved. Note that even with gojq, all mathematical functions, including `floor` and `round`, convert integers to floating-point numbers; only addition, subtraction, multiplication, modulo, and division operators (when divisible) keep the integer precision. To calculate floor division of integers without losing the precision, use `def idivide($n): (. - . % $n) / $n;`. To round down floating-point numbers to integers, use `def ifloor: floor | tostring | tonumber;`, but note that this function does not work with large floating-point numbers and also loses the precision of large integers.
- gojq fixes various bugs of jq. gojq correctly deletes elements of arrays by `|= empty` ([jq#2051](https://github.com/stedolan/jq/issues/2051)). gojq fixes `try`/`catch` handling ([jq#1859](https://github.com/stedolan/jq/issues/1859), [jq#1885](https://github.com/stedolan/jq/issues/1885), [jq#2140](https://github.com/stedolan/jq/issues/2140)). gojq fixes `nth/2` to output nothing when the count is equal to or larger than the stream size ([jq#1867](https://github.com/stedolan/jq/issues/1867)). gojq consistently counts by characters (not by bytes) in `index`, `rindex`, and `indices` functions; `"12345" | .[index("3"):]` results in `"345"` ([jq#1430](https://github.com/stedolan/jq/issues/1430), [jq#1624](https://github.com/stedolan/jq/issues/1624)). gojq handles overlapping occurrence differently in `rindex` and `indices`; `"ababa" | [rindex("aba"), indices("aba")]` results in `[2,[0,2]]` ([jq#2433](https://github.com/stedolan/jq/issues/2433)). gojq supports string indexing; `"abcde"[2]` ([jq#1520](https://github.com/stedolan/jq/issues/1520)). gojq accepts indexing query `.e0` ([jq#1526](https://github.com/stedolan/jq/issues/1526), [jq#1651](https://github.com/stedolan/jq/issues/1651)), and allows `gsub` to handle patterns including `"^"` ([jq#2148](https://github.com/stedolan/jq/issues/2148)). gojq improves variable lexer to allow using keywords for variable names, especially in binding patterns, also disallows spaces after `$` ([jq#526](https://github.com/stedolan/jq/issues/526)). gojq fixes handling files with no newline characters at the end ([jq#2374](https://github.com/stedolan/jq/issues/2374)).
- gojq truncates down floating-point numbers on indexing (`[0] | .[0.5]` results in `0` not `null`), and slicing (`[0,1,2] | .[0.5:1.5]` results in `[0]` not `[0,1]`). gojq parses unary operators with higher precedence than variable binding (`[-1 as $x | 1,$x]` results in `[1,-1]` not `[-1,-1]`). gojq implements `@uri` to escape all the reserved characters defined in RFC 3986, Sec. 2.2 ([jq#1506](https://github.com/stedolan/jq/issues/1506)), and fixes `@base64d` to allow binary string as the decoded string ([jq#1931](https://github.com/stedolan/jq/issues/1931)). gojq improves time formatting and parsing; deals with `%f` in `strftime` and `strptime` ([jq#1409](https://github.com/stedolan/jq/issues/1409)), parses timezone offsets with `fromdate` and `fromdateiso8601` ([jq#1053](https://github.com/stedolan/jq/issues/1053)), supports timezone name/offset with `%Z`/`%z` in `strptime` ([jq#929](https://github.com/stedolan/jq/issues/929), [jq#2195](https://github.com/stedolan/jq/issues/2195)), and looks up correct timezone during daylight saving time on formatting with `%Z` ([jq#1912](https://github.com/stedolan/jq/issues/1912)). gojq supports nanoseconds in date and time functions.
- gojq does not support some functions intentionally; `get_jq_origin`, `get_prog_origin`, `get_search_list` (unstable, not listed in jq document), `input_line_number`, `$__loc__` (performance issue), `recurse_down` (deprecated in jq). gojq does not support some flags; `--ascii-output, -a` (performance issue), `--seq` (not used commonly), `--sort-keys, -S` (sorts by default because `map[string]interface{}` does not keep the order), `--unbuffered` (unbuffered by default). gojq does not parse JSON extensions supported by jq; `NaN`, `Infinity`, and `[000]`. gojq normalizes floating-point numbers to fit to double-precision floating-point numbers. gojq does not support or behaves differently with some regular expression metacharacters and flags (regular expression engine differences). gojq does not support BOM (`encoding/json` does not support this). gojq disallows using keywords for function names (`def true: .; true` is a confusing query), and module name prefixes in function declarations (using module prefixes like `def m::f: .;` is undocumented).
- gojq does not support some functions intentionally; `get_jq_origin`, `get_prog_origin`, `get_search_list` (unstable, not listed in jq document), `input_line_number`, `$__loc__` (performance issue), `recurse_down` (deprecated in jq). gojq does not support some flags; `--ascii-output, -a` (performance issue), `--seq` (not used commonly), `--sort-keys, -S` (sorts by default because `map[string]any` does not keep the order), `--unbuffered` (unbuffered by default). gojq does not parse JSON extensions supported by jq; `NaN`, `Infinity`, and `[000]`. gojq normalizes floating-point numbers to fit to double-precision floating-point numbers. gojq does not support or behaves differently with some regular expression metacharacters and flags (regular expression engine differences). gojq does not support BOM (`encoding/json` does not support this). gojq disallows using keywords for function names (`def true: .; true` is a confusing query), and module name prefixes in function declarations (using module prefixes like `def m::f: .;` is undocumented).
- gojq supports reading from YAML input (`--yaml-input`) while jq does not. gojq also supports YAML output (`--yaml-output`).
### Color configuration
Expand Down Expand Up @@ -109,7 +109,7 @@ func main() {
if err != nil {
log.Fatalln(err)
}
input := map[string]interface{}{"foo": []interface{}{1, 2, 3}}
input := map[string]any{"foo": []any{1, 2, 3}}
iter := query.Run(input) // or query.RunWithContext
for {
v, ok := iter.Next()
Expand All @@ -128,9 +128,9 @@ func main() {
- Secondly, get the result iterator
- using [`query.Run`](https://pkg.go.dev/github.com/itchyny/gojq#Query.Run) or [`query.RunWithContext`](https://pkg.go.dev/github.com/itchyny/gojq#Query.RunWithContext)
- or alternatively, compile the query using [`gojq.Compile`](https://pkg.go.dev/github.com/itchyny/gojq#Compile) and then [`code.Run`](https://pkg.go.dev/github.com/itchyny/gojq#Code.Run) or [`code.RunWithContext`](https://pkg.go.dev/github.com/itchyny/gojq#Code.RunWithContext). You can reuse the `*Code` against multiple inputs to avoid compilation of the same query. But for arguments of `code.Run`, do not give values sharing same data between multiple calls.
- In either case, you cannot use custom type values as the query input. The type should be `[]interface{}` for an array and `map[string]interface{}` for a map (just like decoded to an `interface{}` using the [encoding/json](https://golang.org/pkg/encoding/json/) package). You can't use `[]int` or `map[string]string`, for example. If you want to query your custom struct, marshal to JSON, unmarshal to `interface{}` and use it as the query input.
- Thirdly, iterate through the results using [`iter.Next() (interface{}, bool)`](https://pkg.go.dev/github.com/itchyny/gojq#Iter). The iterator can emit an error so make sure to handle it. The method returns `true` with results, and `false` when the iterator terminates.
- The return type is not `(interface{}, error)` because iterators can emit multiple errors and you can continue after an error. It is difficult for the iterator to tell the termination in this situation.
- In either case, you cannot use custom type values as the query input. The type should be `[]any` for an array and `map[string]any` for a map (just like decoded to an `any` using the [encoding/json](https://golang.org/pkg/encoding/json/) package). You can't use `[]int` or `map[string]string`, for example. If you want to query your custom struct, marshal to JSON, unmarshal to `any` and use it as the query input.
- Thirdly, iterate through the results using [`iter.Next() (any, bool)`](https://pkg.go.dev/github.com/itchyny/gojq#Iter). The iterator can emit an error so make sure to handle it. The method returns `true` with results, and `false` when the iterator terminates.
- The return type is not `(any, error)` because iterators can emit multiple errors and you can continue after an error. It is difficult for the iterator to tell the termination in this situation.
- Note that the result iterator may emit infinite number of values; `repeat(0)` and `range(infinite)`. It may stuck with no output value; `def f: f; f`. Use `RunWithContext` when you want to limit the execution time.
[`gojq.Compile`](https://pkg.go.dev/github.com/itchyny/gojq#Compile) allows to configure the following compiler options.
Expand Down
22 changes: 11 additions & 11 deletions cli/cli.go
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ type cli struct {
inputYAML bool

argnames []string
argvalues []interface{}
argvalues []any

outputYAMLSeparator bool
exitCodeError error
Expand All @@ -75,8 +75,8 @@ type flagopts struct {
ArgJSON map[string]string `long:"argjson" description:"set variable to JSON value"`
SlurpFile map[string]string `long:"slurpfile" description:"set variable to the JSON contents of the file"`
RawFile map[string]string `long:"rawfile" description:"set variable to the contents of the file"`
Args []interface{} `long:"args" positional:"" description:"consume remaining arguments as positional string values"`
JSONArgs []interface{} `long:"jsonargs" positional:"" description:"consume remaining arguments as positional JSON values"`
Args []any `long:"args" positional:"" description:"consume remaining arguments as positional string values"`
JSONArgs []any `long:"jsonargs" positional:"" description:"consume remaining arguments as positional JSON values"`
ExitStatus bool `short:"e" long:"exit-status" description:"exit 1 when the last value is false or null"`
Version bool `short:"v" long:"version" description:"print version"`
Help bool `short:"h" long:"help" description:"print this help"`
Expand Down Expand Up @@ -181,7 +181,7 @@ Usage:
cli.argnames = append(cli.argnames, "$"+k)
cli.argvalues = append(cli.argvalues, string(val))
}
named := make(map[string]interface{}, len(cli.argnames))
named := make(map[string]any, len(cli.argnames))
for i, name := range cli.argnames {
named[name[1:]] = cli.argvalues[i]
}
Expand All @@ -200,7 +200,7 @@ Usage:
}
}
cli.argnames = append(cli.argnames, "$ARGS")
cli.argvalues = append(cli.argvalues, map[string]interface{}{
cli.argvalues = append(cli.argvalues, map[string]any{
"named": named,
"positional": positional,
})
Expand Down Expand Up @@ -242,8 +242,8 @@ Usage:
gojq.WithFunction("debug", 0, 0, cli.funcDebug),
gojq.WithFunction("stderr", 0, 0, cli.funcStderr),
gojq.WithFunction("input_filename", 0, 0,
func(iter inputIter) func(interface{}, []interface{}) interface{} {
return func(interface{}, []interface{}) interface{} {
func(iter inputIter) func(any, []any) any {
return func(any, []any) any {
if fname := iter.Name(); fname != "" && (len(args) > 0 || !opts.InputNull) {
return fname
}
Expand Down Expand Up @@ -291,7 +291,7 @@ func listDefaultModulePaths() []string {
return modulePaths
}

func slurpFile(name string) (interface{}, error) {
func slurpFile(name string) (any, error) {
iter := newSlurpInputIter(
newFilesInputIter(newJSONInputIter, []string{name}, nil),
)
Expand Down Expand Up @@ -408,13 +408,13 @@ func (cli *cli) createMarshaler() marshaler {
return f
}

func (cli *cli) funcDebug(v interface{}, _ []interface{}) interface{} {
newEncoder(false, 0).marshal([]interface{}{"DEBUG:", v}, cli.errStream)
func (cli *cli) funcDebug(v any, _ []any) any {
newEncoder(false, 0).marshal([]any{"DEBUG:", v}, cli.errStream)
cli.errStream.Write([]byte{'\n'})
return v
}

func (cli *cli) funcStderr(v interface{}, _ []interface{}) interface{} {
func (cli *cli) funcStderr(v any, _ []any) any {
newEncoder(false, 0).marshal(v, cli.errStream)
return v
}
Expand Down
14 changes: 7 additions & 7 deletions cli/encoder.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ func (e *encoder) flush() error {
return err
}

func (e *encoder) marshal(v interface{}, w io.Writer) error {
func (e *encoder) marshal(v any, w io.Writer) error {
e.out = w
err := e.encode(v)
if ferr := e.flush(); ferr != nil && err == nil {
Expand All @@ -40,7 +40,7 @@ func (e *encoder) marshal(v interface{}, w io.Writer) error {
return err
}

func (e *encoder) encode(v interface{}) error {
func (e *encoder) encode(v any) error {
switch v := v.(type) {
case nil:
e.write([]byte("null"), nullColor)
Expand All @@ -58,11 +58,11 @@ func (e *encoder) encode(v interface{}) error {
e.write(v.Append(e.buf[:0], 10), numberColor)
case string:
e.encodeString(v, stringColor)
case []interface{}:
case []any:
if err := e.encodeArray(v); err != nil {
return err
}
case map[string]interface{}:
case map[string]any:
if err := e.encodeMap(v); err != nil {
return err
}
Expand Down Expand Up @@ -163,7 +163,7 @@ func (e *encoder) encodeString(s string, color []byte) {
}
}

func (e *encoder) encodeArray(vs []interface{}) error {
func (e *encoder) encodeArray(vs []any) error {
e.writeByte('[', arrayColor)
e.depth += e.indent
for i, v := range vs {
Expand All @@ -185,12 +185,12 @@ func (e *encoder) encodeArray(vs []interface{}) error {
return nil
}

func (e *encoder) encodeMap(vs map[string]interface{}) error {
func (e *encoder) encodeMap(vs map[string]any) error {
e.writeByte('{', objectColor)
e.depth += e.indent
type keyVal struct {
key string
val interface{}
val any
}
kvs := make([]keyVal, len(vs))
var i int
Expand Down
4 changes: 2 additions & 2 deletions cli/flags.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import (
"strings"
)

func parseFlags(args []string, opts interface{}) ([]string, error) {
func parseFlags(args []string, opts any) ([]string, error) {
rest := make([]string, 0, len(args))
val := reflect.ValueOf(opts).Elem()
typ := val.Type()
Expand Down Expand Up @@ -158,7 +158,7 @@ func parseFlags(args []string, opts interface{}) ([]string, error) {
return rest, nil
}

func formatFlags(opts interface{}) string {
func formatFlags(opts any) string {
val := reflect.ValueOf(opts).Elem()
typ := val.Type()
var sb strings.Builder
Expand Down
28 changes: 14 additions & 14 deletions cli/inputs.go
Original file line number Diff line number Diff line change
Expand Up @@ -83,11 +83,11 @@ func newJSONInputIter(r io.Reader, fname string) inputIter {
return &jsonInputIter{dec: dec, ir: ir, fname: fname}
}

func (i *jsonInputIter) Next() (interface{}, bool) {
func (i *jsonInputIter) Next() (any, bool) {
if i.err != nil {
return nil, false
}
var v interface{}
var v any
if err := i.dec.Decode(&v); err != nil {
if err == io.EOF {
i.err = err
Expand Down Expand Up @@ -127,7 +127,7 @@ func newNullInputIter() inputIter {
return &nullInputIter{}
}

func (i *nullInputIter) Next() (interface{}, bool) {
func (i *nullInputIter) Next() (any, bool) {
if i.err != nil {
return nil, false
}
Expand Down Expand Up @@ -159,7 +159,7 @@ func newFilesInputIter(
return &filesInputIter{newIter: newIter, fnames: fnames, stdin: stdin}
}

func (i *filesInputIter) Next() (interface{}, bool) {
func (i *filesInputIter) Next() (any, bool) {
if i.err != nil {
return nil, false
}
Expand Down Expand Up @@ -227,7 +227,7 @@ func newRawInputIter(r io.Reader, fname string) inputIter {
return &rawInputIter{r: bufio.NewReader(r), fname: fname}
}

func (i *rawInputIter) Next() (interface{}, bool) {
func (i *rawInputIter) Next() (any, bool) {
if i.err != nil {
return nil, false
}
Expand Down Expand Up @@ -269,7 +269,7 @@ func newStreamInputIter(r io.Reader, fname string) inputIter {
return &streamInputIter{stream: newJSONStream(dec), ir: ir, fname: fname}
}

func (i *streamInputIter) Next() (interface{}, bool) {
func (i *streamInputIter) Next() (any, bool) {
if i.err != nil {
return nil, false
}
Expand Down Expand Up @@ -318,11 +318,11 @@ func newYAMLInputIter(r io.Reader, fname string) inputIter {
return &yamlInputIter{dec: dec, ir: ir, fname: fname}
}

func (i *yamlInputIter) Next() (interface{}, bool) {
func (i *yamlInputIter) Next() (any, bool) {
if i.err != nil {
return nil, false
}
var v interface{}
var v any
if err := i.dec.Decode(&v); err != nil {
if err == io.EOF {
i.err = err
Expand Down Expand Up @@ -352,12 +352,12 @@ func newSlurpInputIter(iter inputIter) inputIter {
return &slurpInputIter{iter: iter}
}

func (i *slurpInputIter) Next() (interface{}, bool) {
func (i *slurpInputIter) Next() (any, bool) {
if i.err != nil {
return nil, false
}
var vs []interface{}
var v interface{}
var vs []any
var v any
var ok bool
for {
v, ok = i.iter.Next()
Expand Down Expand Up @@ -395,7 +395,7 @@ func newReadAllIter(r io.Reader, fname string) inputIter {
return &readAllIter{r: r, fname: fname}
}

func (i *readAllIter) Next() (interface{}, bool) {
func (i *readAllIter) Next() (any, bool) {
if i.err != nil {
return nil, false
}
Expand Down Expand Up @@ -425,12 +425,12 @@ func newSlurpRawInputIter(iter inputIter) inputIter {
return &slurpRawInputIter{iter: iter}
}

func (i *slurpRawInputIter) Next() (interface{}, bool) {
func (i *slurpRawInputIter) Next() (any, bool) {
if i.err != nil {
return nil, false
}
var vs []string
var v interface{}
var v any
var ok bool
for {
v, ok = i.iter.Next()
Expand Down
6 changes: 3 additions & 3 deletions cli/marshaler.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@ import (
)

type marshaler interface {
marshal(interface{}, io.Writer) error
marshal(any, io.Writer) error
}

type rawMarshaler struct {
m marshaler
}

func (m *rawMarshaler) marshal(v interface{}, w io.Writer) error {
func (m *rawMarshaler) marshal(v any, w io.Writer) error {
if s, ok := v.(string); ok {
_, err := w.Write([]byte(s))
return err
Expand All @@ -30,7 +30,7 @@ type yamlMarshaler struct {
indent *int
}

func (m *yamlMarshaler) marshal(v interface{}, w io.Writer) error {
func (m *yamlMarshaler) marshal(v any, w io.Writer) error {
enc := yaml.NewEncoder(w)
if i := m.indent; i != nil {
enc.SetIndent(*i)
Expand Down
Loading

0 comments on commit 4c776c0

Please sign in to comment.