pax_global_header00006660000000000000000000000064145633051140014514gustar00rootroot0000000000000052 comment=bbf40bb0e4f47860677c7115b2eaa5b3413ce23e gjson-1.17.1/000077500000000000000000000000001456330511400127235ustar00rootroot00000000000000gjson-1.17.1/.github/000077500000000000000000000000001456330511400142635ustar00rootroot00000000000000gjson-1.17.1/.github/workflows/000077500000000000000000000000001456330511400163205ustar00rootroot00000000000000gjson-1.17.1/.github/workflows/go.yml000066400000000000000000000012221456330511400174450ustar00rootroot00000000000000name: Go on: push: branches: [ master ] pull_request: branches: [ master ] jobs: build: name: Build runs-on: ubuntu-latest steps: - name: Set up Go 1.x uses: actions/setup-go@v2 with: go-version: ^1.13 - name: Check out code into the Go module directory uses: actions/checkout@v2 - name: Get dependencies run: | go get -v -t -d ./... if [ -f Gopkg.toml ]; then curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh dep ensure fi - name: Build run: go build -v . - name: Test run: go test -v . gjson-1.17.1/LICENSE000066400000000000000000000020651456330511400137330ustar00rootroot00000000000000The MIT License (MIT) Copyright (c) 2016 Josh Baker Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. gjson-1.17.1/README.md000066400000000000000000000347151456330511400142140ustar00rootroot00000000000000

GJSON
GoDoc GJSON Playground GJSON Syntax

get json values quickly

GJSON is a Go package that provides a [fast](#performance) and [simple](#get-a-value) way to get values from a json document. It has features such as [one line retrieval](#get-a-value), [dot notation paths](#path-syntax), [iteration](#iterate-through-an-object-or-array), and [parsing json lines](#json-lines). Also check out [SJSON](https://github.com/tidwall/sjson) for modifying json, and the [JJ](https://github.com/tidwall/jj) command line tool. This README is a quick overview of how to use GJSON, for more information check out [GJSON Syntax](SYNTAX.md). GJSON is also available for [Python](https://github.com/volans-/gjson-py) and [Rust](https://github.com/tidwall/gjson.rs) Getting Started =============== ## Installing To start using GJSON, install Go and run `go get`: ```sh $ go get -u github.com/tidwall/gjson ``` This will retrieve the library. ## Get a value Get searches json for the specified path. A path is in dot syntax, such as "name.last" or "age". When the value is found it's returned immediately. ```go package main import "github.com/tidwall/gjson" const json = `{"name":{"first":"Janet","last":"Prichard"},"age":47}` func main() { value := gjson.Get(json, "name.last") println(value.String()) } ``` This will print: ``` Prichard ``` *There's also the [GetMany](#get-multiple-values-at-once) function to get multiple values at once, and [GetBytes](#working-with-bytes) for working with JSON byte slices.* ## Path Syntax Below is a quick overview of the path syntax, for more complete information please check out [GJSON Syntax](SYNTAX.md). A path is a series of keys separated by a dot. A key may contain special wildcard characters '\*' and '?'. To access an array value use the index as the key. To get the number of elements in an array or to access a child path, use the '#' character. The dot and wildcard characters can be escaped with '\\'. ```json { "name": {"first": "Tom", "last": "Anderson"}, "age":37, "children": ["Sara","Alex","Jack"], "fav.movie": "Deer Hunter", "friends": [ {"first": "Dale", "last": "Murphy", "age": 44, "nets": ["ig", "fb", "tw"]}, {"first": "Roger", "last": "Craig", "age": 68, "nets": ["fb", "tw"]}, {"first": "Jane", "last": "Murphy", "age": 47, "nets": ["ig", "tw"]} ] } ``` ``` "name.last" >> "Anderson" "age" >> 37 "children" >> ["Sara","Alex","Jack"] "children.#" >> 3 "children.1" >> "Alex" "child*.2" >> "Jack" "c?ildren.0" >> "Sara" "fav\.movie" >> "Deer Hunter" "friends.#.first" >> ["Dale","Roger","Jane"] "friends.1.last" >> "Craig" ``` You can also query an array for the first match by using `#(...)`, or find all matches with `#(...)#`. Queries support the `==`, `!=`, `<`, `<=`, `>`, `>=` comparison operators and the simple pattern matching `%` (like) and `!%` (not like) operators. ``` friends.#(last=="Murphy").first >> "Dale" friends.#(last=="Murphy")#.first >> ["Dale","Jane"] friends.#(age>45)#.last >> ["Craig","Murphy"] friends.#(first%"D*").last >> "Murphy" friends.#(first!%"D*").last >> "Craig" friends.#(nets.#(=="fb"))#.first >> ["Dale","Roger"] ``` *Please note that prior to v1.3.0, queries used the `#[...]` brackets. This was changed in v1.3.0 as to avoid confusion with the new [multipath](SYNTAX.md#multipaths) syntax. For backwards compatibility, `#[...]` will continue to work until the next major release.* ## Result Type GJSON supports the json types `string`, `number`, `bool`, and `null`. Arrays and Objects are returned as their raw json types. The `Result` type holds one of these: ``` bool, for JSON booleans float64, for JSON numbers string, for JSON string literals nil, for JSON null ``` To directly access the value: ```go result.Type // can be String, Number, True, False, Null, or JSON result.Str // holds the string result.Num // holds the float64 number result.Raw // holds the raw json result.Index // index of raw value in original json, zero means index unknown result.Indexes // indexes of all the elements that match on a path containing the '#' query character. ``` There are a variety of handy functions that work on a result: ```go result.Exists() bool result.Value() interface{} result.Int() int64 result.Uint() uint64 result.Float() float64 result.String() string result.Bool() bool result.Time() time.Time result.Array() []gjson.Result result.Map() map[string]gjson.Result result.Get(path string) Result result.ForEach(iterator func(key, value Result) bool) result.Less(token Result, caseSensitive bool) bool ``` The `result.Value()` function returns an `interface{}` which requires type assertion and is one of the following Go types: ```go boolean >> bool number >> float64 string >> string null >> nil array >> []interface{} object >> map[string]interface{} ``` The `result.Array()` function returns back an array of values. If the result represents a non-existent value, then an empty array will be returned. If the result is not a JSON array, the return value will be an array containing one result. ### 64-bit integers The `result.Int()` and `result.Uint()` calls are capable of reading all 64 bits, allowing for large JSON integers. ```go result.Int() int64 // -9223372036854775808 to 9223372036854775807 result.Uint() uint64 // 0 to 18446744073709551615 ``` ## Modifiers and path chaining New in version 1.2 is support for modifier functions and path chaining. A modifier is a path component that performs custom processing on the json. Multiple paths can be "chained" together using the pipe character. This is useful for getting results from a modified query. For example, using the built-in `@reverse` modifier on the above json document, we'll get `children` array and reverse the order: ``` "children|@reverse" >> ["Jack","Alex","Sara"] "children|@reverse|0" >> "Jack" ``` There are currently the following built-in modifiers: - `@reverse`: Reverse an array or the members of an object. - `@ugly`: Remove all whitespace from a json document. - `@pretty`: Make the json document more human readable. - `@this`: Returns the current element. It can be used to retrieve the root element. - `@valid`: Ensure the json document is valid. - `@flatten`: Flattens an array. - `@join`: Joins multiple objects into a single object. - `@keys`: Returns an array of keys for an object. - `@values`: Returns an array of values for an object. - `@tostr`: Converts json to a string. Wraps a json string. - `@fromstr`: Converts a string from json. Unwraps a json string. - `@group`: Groups arrays of objects. See [e4fc67c](https://github.com/tidwall/gjson/commit/e4fc67c92aeebf2089fabc7872f010e340d105db). - `@dig`: Search for a value without providing its entire path. See [e8e87f2](https://github.com/tidwall/gjson/commit/e8e87f2a00dc41f3aba5631094e21f59a8cf8cbf). ### Modifier arguments A modifier may accept an optional argument. The argument can be a valid JSON document or just characters. For example, the `@pretty` modifier takes a json object as its argument. ``` @pretty:{"sortKeys":true} ``` Which makes the json pretty and orders all of its keys. ```json { "age":37, "children": ["Sara","Alex","Jack"], "fav.movie": "Deer Hunter", "friends": [ {"age": 44, "first": "Dale", "last": "Murphy"}, {"age": 68, "first": "Roger", "last": "Craig"}, {"age": 47, "first": "Jane", "last": "Murphy"} ], "name": {"first": "Tom", "last": "Anderson"} } ``` *The full list of `@pretty` options are `sortKeys`, `indent`, `prefix`, and `width`. Please see [Pretty Options](https://github.com/tidwall/pretty#customized-output) for more information.* ### Custom modifiers You can also add custom modifiers. For example, here we create a modifier that makes the entire json document upper or lower case. ```go gjson.AddModifier("case", func(json, arg string) string { if arg == "upper" { return strings.ToUpper(json) } if arg == "lower" { return strings.ToLower(json) } return json }) ``` ``` "children|@case:upper" >> ["SARA","ALEX","JACK"] "children|@case:lower|@reverse" >> ["jack","alex","sara"] ``` ## JSON Lines There's support for [JSON Lines](http://jsonlines.org/) using the `..` prefix, which treats a multilined document as an array. For example: ``` {"name": "Gilbert", "age": 61} {"name": "Alexa", "age": 34} {"name": "May", "age": 57} {"name": "Deloise", "age": 44} ``` ``` ..# >> 4 ..1 >> {"name": "Alexa", "age": 34} ..3 >> {"name": "Deloise", "age": 44} ..#.name >> ["Gilbert","Alexa","May","Deloise"] ..#(name="May").age >> 57 ``` The `ForEachLines` function will iterate through JSON lines. ```go gjson.ForEachLine(json, func(line gjson.Result) bool{ println(line.String()) return true }) ``` ## Get nested array values Suppose you want all the last names from the following json: ```json { "programmers": [ { "firstName": "Janet", "lastName": "McLaughlin", }, { "firstName": "Elliotte", "lastName": "Hunter", }, { "firstName": "Jason", "lastName": "Harold", } ] } ``` You would use the path "programmers.#.lastName" like such: ```go result := gjson.Get(json, "programmers.#.lastName") for _, name := range result.Array() { println(name.String()) } ``` You can also query an object inside an array: ```go name := gjson.Get(json, `programmers.#(lastName="Hunter").firstName`) println(name.String()) // prints "Elliotte" ``` ## Iterate through an object or array The `ForEach` function allows for quickly iterating through an object or array. The key and value are passed to the iterator function for objects. Only the value is passed for arrays. Returning `false` from an iterator will stop iteration. ```go result := gjson.Get(json, "programmers") result.ForEach(func(key, value gjson.Result) bool { println(value.String()) return true // keep iterating }) ``` ## Simple Parse and Get There's a `Parse(json)` function that will do a simple parse, and `result.Get(path)` that will search a result. For example, all of these will return the same result: ```go gjson.Parse(json).Get("name").Get("last") gjson.Get(json, "name").Get("last") gjson.Get(json, "name.last") ``` ## Check for the existence of a value Sometimes you just want to know if a value exists. ```go value := gjson.Get(json, "name.last") if !value.Exists() { println("no last name") } else { println(value.String()) } // Or as one step if gjson.Get(json, "name.last").Exists() { println("has a last name") } ``` ## Validate JSON The `Get*` and `Parse*` functions expects that the json is well-formed. Bad json will not panic, but it may return back unexpected results. If you are consuming JSON from an unpredictable source then you may want to validate prior to using GJSON. ```go if !gjson.Valid(json) { return errors.New("invalid json") } value := gjson.Get(json, "name.last") ``` ## Unmarshal to a map To unmarshal to a `map[string]interface{}`: ```go m, ok := gjson.Parse(json).Value().(map[string]interface{}) if !ok { // not a map } ``` ## Working with Bytes If your JSON is contained in a `[]byte` slice, there's the [GetBytes](https://godoc.org/github.com/tidwall/gjson#GetBytes) function. This is preferred over `Get(string(data), path)`. ```go var json []byte = ... result := gjson.GetBytes(json, path) ``` If you are using the `gjson.GetBytes(json, path)` function and you want to avoid converting `result.Raw` to a `[]byte`, then you can use this pattern: ```go var json []byte = ... result := gjson.GetBytes(json, path) var raw []byte if result.Index > 0 { raw = json[result.Index:result.Index+len(result.Raw)] } else { raw = []byte(result.Raw) } ``` This is a best-effort no allocation sub slice of the original json. This method utilizes the `result.Index` field, which is the position of the raw data in the original json. It's possible that the value of `result.Index` equals zero, in which case the `result.Raw` is converted to a `[]byte`. ## Performance Benchmarks of GJSON alongside [encoding/json](https://golang.org/pkg/encoding/json/), [ffjson](https://github.com/pquerna/ffjson), [EasyJSON](https://github.com/mailru/easyjson), [jsonparser](https://github.com/buger/jsonparser), and [json-iterator](https://github.com/json-iterator/go) ``` BenchmarkGJSONGet-16 11644512 311 ns/op 0 B/op 0 allocs/op BenchmarkGJSONUnmarshalMap-16 1122678 3094 ns/op 1920 B/op 26 allocs/op BenchmarkJSONUnmarshalMap-16 516681 6810 ns/op 2944 B/op 69 allocs/op BenchmarkJSONUnmarshalStruct-16 697053 5400 ns/op 928 B/op 13 allocs/op BenchmarkJSONDecoder-16 330450 10217 ns/op 3845 B/op 160 allocs/op BenchmarkFFJSONLexer-16 1424979 2585 ns/op 880 B/op 8 allocs/op BenchmarkEasyJSONLexer-16 3000000 729 ns/op 501 B/op 5 allocs/op BenchmarkJSONParserGet-16 3000000 366 ns/op 21 B/op 0 allocs/op BenchmarkJSONIterator-16 3000000 869 ns/op 693 B/op 14 allocs/op ``` JSON document used: ```json { "widget": { "debug": "on", "window": { "title": "Sample Konfabulator Widget", "name": "main_window", "width": 500, "height": 500 }, "image": { "src": "Images/Sun.png", "hOffset": 250, "vOffset": 250, "alignment": "center" }, "text": { "data": "Click Here", "size": 36, "style": "bold", "vOffset": 100, "alignment": "center", "onMouseUp": "sun1.opacity = (sun1.opacity / 100) * 90;" } } } ``` Each operation was rotated through one of the following search paths: ``` widget.window.name widget.image.hOffset widget.text.onMouseUp ``` *These benchmarks were run on a MacBook Pro 16" 2.4 GHz Intel Core i9 using Go 1.17 and can be found [here](https://github.com/tidwall/gjson-benchmarks).* gjson-1.17.1/SYNTAX.md000066400000000000000000000250231456330511400142750ustar00rootroot00000000000000# GJSON Path Syntax A GJSON Path is a text string syntax that describes a search pattern for quickly retreiving values from a JSON payload. This document is designed to explain the structure of a GJSON Path through examples. - [Path structure](#path-structure) - [Basic](#basic) - [Wildcards](#wildcards) - [Escape Character](#escape-character) - [Arrays](#arrays) - [Queries](#queries) - [Dot vs Pipe](#dot-vs-pipe) - [Modifiers](#modifiers) - [Multipaths](#multipaths) - [Literals](#literals) The definitive implemenation is [github.com/tidwall/gjson](https://github.com/tidwall/gjson). Use the [GJSON Playground](https://gjson.dev) to experiment with the syntax online. ## Path structure A GJSON Path is intended to be easily expressed as a series of components seperated by a `.` character. Along with `.` character, there are a few more that have special meaning, including `|`, `#`, `@`, `\`, `*`, `!`, and `?`. ## Example Given this JSON ```json { "name": {"first": "Tom", "last": "Anderson"}, "age":37, "children": ["Sara","Alex","Jack"], "fav.movie": "Deer Hunter", "friends": [ {"first": "Dale", "last": "Murphy", "age": 44, "nets": ["ig", "fb", "tw"]}, {"first": "Roger", "last": "Craig", "age": 68, "nets": ["fb", "tw"]}, {"first": "Jane", "last": "Murphy", "age": 47, "nets": ["ig", "tw"]} ] } ``` The following GJSON Paths evaluate to the accompanying values. ### Basic In many cases you'll just want to retreive values by object name or array index. ```go name.last "Anderson" name.first "Tom" age 37 children ["Sara","Alex","Jack"] children.0 "Sara" children.1 "Alex" friends.1 {"first": "Roger", "last": "Craig", "age": 68} friends.1.first "Roger" ``` ### Wildcards A key may contain the special wildcard characters `*` and `?`. The `*` will match on any zero+ characters, and `?` matches on any one character. ```go child*.2 "Jack" c?ildren.0 "Sara" ``` ### Escape character Special purpose characters, such as `.`, `*`, and `?` can be escaped with `\`. ```go fav\.movie "Deer Hunter" ``` You'll also need to make sure that the `\` character is correctly escaped when hardcoding a path in your source code. ```go // Go val := gjson.Get(json, "fav\\.movie") // must escape the slash val := gjson.Get(json, `fav\.movie`) // no need to escape the slash ``` ```rust // Rust let val = gjson::get(json, "fav\\.movie") // must escape the slash let val = gjson::get(json, r#"fav\.movie"#) // no need to escape the slash ``` ### Arrays The `#` character allows for digging into JSON Arrays. To get the length of an array you'll just use the `#` all by itself. ```go friends.# 3 friends.#.age [44,68,47] ``` ### Queries You can also query an array for the first match by using `#(...)`, or find all matches with `#(...)#`. Queries support the `==`, `!=`, `<`, `<=`, `>`, `>=` comparison operators, and the simple pattern matching `%` (like) and `!%` (not like) operators. ```go friends.#(last=="Murphy").first "Dale" friends.#(last=="Murphy")#.first ["Dale","Jane"] friends.#(age>45)#.last ["Craig","Murphy"] friends.#(first%"D*").last "Murphy" friends.#(first!%"D*").last "Craig" ``` To query for a non-object value in an array, you can forgo the string to the right of the operator. ```go children.#(!%"*a*") "Alex" children.#(%"*a*")# ["Sara","Jack"] ``` Nested queries are allowed. ```go friends.#(nets.#(=="fb"))#.first >> ["Dale","Roger"] ``` *Please note that prior to v1.3.0, queries used the `#[...]` brackets. This was changed in v1.3.0 as to avoid confusion with the new [multipath](#multipaths) syntax. For backwards compatibility, `#[...]` will continue to work until the next major release.* The `~` (tilde) operator will convert a value to a boolean before comparison. Supported tilde comparison type are: ``` ~true Converts true-ish values to true ~false Converts false-ish and non-existent values to true ~null Converts null and non-existent values to true ~* Converts any existing value to true ``` For example, using the following JSON: ```json { "vals": [ { "a": 1, "b": "data" }, { "a": 2, "b": true }, { "a": 3, "b": false }, { "a": 4, "b": "0" }, { "a": 5, "b": 0 }, { "a": 6, "b": "1" }, { "a": 7, "b": 1 }, { "a": 8, "b": "true" }, { "a": 9, "b": false }, { "a": 10, "b": null }, { "a": 11 } ] } ``` To query for all true-ish or false-ish values: ``` vals.#(b==~true)#.a >> [2,6,7,8] vals.#(b==~false)#.a >> [3,4,5,9,10,11] ``` The last value which was non-existent is treated as `false` To query for null and explicit value existence: ``` vals.#(b==~null)#.a >> [10,11] vals.#(b==~*)#.a >> [1,2,3,4,5,6,7,8,9,10] vals.#(b!=~*)#.a >> [11] ``` ### Dot vs Pipe The `.` is standard separator, but it's also possible to use a `|`. In most cases they both end up returning the same results. The cases where`|` differs from `.` is when it's used after the `#` for [Arrays](#arrays) and [Queries](#queries). Here are some examples ```go friends.0.first "Dale" friends|0.first "Dale" friends.0|first "Dale" friends|0|first "Dale" friends|# 3 friends.# 3 friends.#(last="Murphy")# [{"first": "Dale", "last": "Murphy", "age": 44},{"first": "Jane", "last": "Murphy", "age": 47}] friends.#(last="Murphy")#.first ["Dale","Jane"] friends.#(last="Murphy")#|first friends.#(last="Murphy")#.0 [] friends.#(last="Murphy")#|0 {"first": "Dale", "last": "Murphy", "age": 44} friends.#(last="Murphy")#.# [] friends.#(last="Murphy")#|# 2 ``` Let's break down a few of these. The path `friends.#(last="Murphy")#` all by itself results in ```json [{"first": "Dale", "last": "Murphy", "age": 44},{"first": "Jane", "last": "Murphy", "age": 47}] ``` The `.first` suffix will process the `first` path on each array element *before* returning the results. Which becomes ```json ["Dale","Jane"] ``` But the `|first` suffix actually processes the `first` path *after* the previous result. Since the previous result is an array, not an object, it's not possible to process because `first` does not exist. Yet, `|0` suffix returns ```json {"first": "Dale", "last": "Murphy", "age": 44} ``` Because `0` is the first index of the previous result. ### Modifiers A modifier is a path component that performs custom processing on the JSON. For example, using the built-in `@reverse` modifier on the above JSON payload will reverse the `children` array: ```go children.@reverse ["Jack","Alex","Sara"] children.@reverse.0 "Jack" ``` There are currently the following built-in modifiers: - `@reverse`: Reverse an array or the members of an object. - `@ugly`: Remove all whitespace from JSON. - `@pretty`: Make the JSON more human readable. - `@this`: Returns the current element. It can be used to retrieve the root element. - `@valid`: Ensure the json document is valid. - `@flatten`: Flattens an array. - `@join`: Joins multiple objects into a single object. - `@keys`: Returns an array of keys for an object. - `@values`: Returns an array of values for an object. - `@tostr`: Converts json to a string. Wraps a json string. - `@fromstr`: Converts a string from json. Unwraps a json string. - `@group`: Groups arrays of objects. See [e4fc67c](https://github.com/tidwall/gjson/commit/e4fc67c92aeebf2089fabc7872f010e340d105db). - `@dig`: Search for a value without providing its entire path. See [e8e87f2](https://github.com/tidwall/gjson/commit/e8e87f2a00dc41f3aba5631094e21f59a8cf8cbf). #### Modifier arguments A modifier may accept an optional argument. The argument can be a valid JSON payload or just characters. For example, the `@pretty` modifier takes a json object as its argument. ``` @pretty:{"sortKeys":true} ``` Which makes the json pretty and orders all of its keys. ```json { "age":37, "children": ["Sara","Alex","Jack"], "fav.movie": "Deer Hunter", "friends": [ {"age": 44, "first": "Dale", "last": "Murphy"}, {"age": 68, "first": "Roger", "last": "Craig"}, {"age": 47, "first": "Jane", "last": "Murphy"} ], "name": {"first": "Tom", "last": "Anderson"} } ``` *The full list of `@pretty` options are `sortKeys`, `indent`, `prefix`, and `width`. Please see [Pretty Options](https://github.com/tidwall/pretty#customized-output) for more information.* #### Custom modifiers You can also add custom modifiers. For example, here we create a modifier which makes the entire JSON payload upper or lower case. ```go gjson.AddModifier("case", func(json, arg string) string { if arg == "upper" { return strings.ToUpper(json) } if arg == "lower" { return strings.ToLower(json) } return json }) "children.@case:upper" ["SARA","ALEX","JACK"] "children.@case:lower.@reverse" ["jack","alex","sara"] ``` *Note: Custom modifiers are not yet available in the Rust version* ### Multipaths Starting with v1.3.0, GJSON added the ability to join multiple paths together to form new documents. Wrapping comma-separated paths between `[...]` or `{...}` will result in a new array or object, respectively. For example, using the given multipath: ``` {name.first,age,"the_murphys":friends.#(last="Murphy")#.first} ``` Here we selected the first name, age, and the first name for friends with the last name "Murphy". You'll notice that an optional key can be provided, in this case "the_murphys", to force assign a key to a value. Otherwise, the name of the actual field will be used, in this case "first". If a name cannot be determined, then "_" is used. This results in ```json {"first":"Tom","age":37,"the_murphys":["Dale","Jane"]} ``` ### Literals Starting with v1.12.0, GJSON added support of json literals, which provides a way for constructing static blocks of json. This is can be particularly useful when constructing a new json document using [multipaths](#multipaths). A json literal begins with the '!' declaration character. For example, using the given multipath: ``` {name.first,age,"company":!"Happysoft","employed":!true} ``` Here we selected the first name and age. Then add two new fields, "company" and "employed". This results in ```json {"first":"Tom","age":37,"company":"Happysoft","employed":true} ``` *See issue [#249](https://github.com/tidwall/gjson/issues/249) for additional context on JSON Literals.* gjson-1.17.1/gjson.go000066400000000000000000002200361456330511400143750ustar00rootroot00000000000000// Package gjson provides searching for json strings. package gjson import ( "strconv" "strings" "time" "unicode/utf16" "unicode/utf8" "unsafe" "github.com/tidwall/match" "github.com/tidwall/pretty" ) // Type is Result type type Type int const ( // Null is a null json value Null Type = iota // False is a json false boolean False // Number is json number Number // String is a json string String // True is a json true boolean True // JSON is a raw block of JSON JSON ) // String returns a string representation of the type. func (t Type) String() string { switch t { default: return "" case Null: return "Null" case False: return "False" case Number: return "Number" case String: return "String" case True: return "True" case JSON: return "JSON" } } // Result represents a json value that is returned from Get(). type Result struct { // Type is the json type Type Type // Raw is the raw json Raw string // Str is the json string Str string // Num is the json number Num float64 // Index of raw value in original json, zero means index unknown Index int // Indexes of all the elements that match on a path containing the '#' // query character. Indexes []int } // String returns a string representation of the value. func (t Result) String() string { switch t.Type { default: return "" case False: return "false" case Number: if len(t.Raw) == 0 { // calculated result return strconv.FormatFloat(t.Num, 'f', -1, 64) } var i int if t.Raw[0] == '-' { i++ } for ; i < len(t.Raw); i++ { if t.Raw[i] < '0' || t.Raw[i] > '9' { return strconv.FormatFloat(t.Num, 'f', -1, 64) } } return t.Raw case String: return t.Str case JSON: return t.Raw case True: return "true" } } // Bool returns an boolean representation. func (t Result) Bool() bool { switch t.Type { default: return false case True: return true case String: b, _ := strconv.ParseBool(strings.ToLower(t.Str)) return b case Number: return t.Num != 0 } } // Int returns an integer representation. func (t Result) Int() int64 { switch t.Type { default: return 0 case True: return 1 case String: n, _ := parseInt(t.Str) return n case Number: // try to directly convert the float64 to int64 i, ok := safeInt(t.Num) if ok { return i } // now try to parse the raw string i, ok = parseInt(t.Raw) if ok { return i } // fallback to a standard conversion return int64(t.Num) } } // Uint returns an unsigned integer representation. func (t Result) Uint() uint64 { switch t.Type { default: return 0 case True: return 1 case String: n, _ := parseUint(t.Str) return n case Number: // try to directly convert the float64 to uint64 i, ok := safeInt(t.Num) if ok && i >= 0 { return uint64(i) } // now try to parse the raw string u, ok := parseUint(t.Raw) if ok { return u } // fallback to a standard conversion return uint64(t.Num) } } // Float returns an float64 representation. func (t Result) Float() float64 { switch t.Type { default: return 0 case True: return 1 case String: n, _ := strconv.ParseFloat(t.Str, 64) return n case Number: return t.Num } } // Time returns a time.Time representation. func (t Result) Time() time.Time { res, _ := time.Parse(time.RFC3339, t.String()) return res } // Array returns back an array of values. // If the result represents a null value or is non-existent, then an empty // array will be returned. // If the result is not a JSON array, the return value will be an // array containing one result. func (t Result) Array() []Result { if t.Type == Null { return []Result{} } if !t.IsArray() { return []Result{t} } r := t.arrayOrMap('[', false) return r.a } // IsObject returns true if the result value is a JSON object. func (t Result) IsObject() bool { return t.Type == JSON && len(t.Raw) > 0 && t.Raw[0] == '{' } // IsArray returns true if the result value is a JSON array. func (t Result) IsArray() bool { return t.Type == JSON && len(t.Raw) > 0 && t.Raw[0] == '[' } // IsBool returns true if the result value is a JSON boolean. func (t Result) IsBool() bool { return t.Type == True || t.Type == False } // ForEach iterates through values. // If the result represents a non-existent value, then no values will be // iterated. If the result is an Object, the iterator will pass the key and // value of each item. If the result is an Array, the iterator will only pass // the value of each item. If the result is not a JSON array or object, the // iterator will pass back one value equal to the result. func (t Result) ForEach(iterator func(key, value Result) bool) { if !t.Exists() { return } if t.Type != JSON { iterator(Result{}, t) return } json := t.Raw var obj bool var i int var key, value Result for ; i < len(json); i++ { if json[i] == '{' { i++ key.Type = String obj = true break } else if json[i] == '[' { i++ key.Type = Number key.Num = -1 break } if json[i] > ' ' { return } } var str string var vesc bool var ok bool var idx int for ; i < len(json); i++ { if obj { if json[i] != '"' { continue } s := i i, str, vesc, ok = parseString(json, i+1) if !ok { return } if vesc { key.Str = unescape(str[1 : len(str)-1]) } else { key.Str = str[1 : len(str)-1] } key.Raw = str key.Index = s + t.Index } else { key.Num += 1 } for ; i < len(json); i++ { if json[i] <= ' ' || json[i] == ',' || json[i] == ':' { continue } break } s := i i, value, ok = parseAny(json, i, true) if !ok { return } if t.Indexes != nil { if idx < len(t.Indexes) { value.Index = t.Indexes[idx] } } else { value.Index = s + t.Index } if !iterator(key, value) { return } idx++ } } // Map returns back a map of values. The result should be a JSON object. // If the result is not a JSON object, the return value will be an empty map. func (t Result) Map() map[string]Result { if t.Type != JSON { return map[string]Result{} } r := t.arrayOrMap('{', false) return r.o } // Get searches result for the specified path. // The result should be a JSON array or object. func (t Result) Get(path string) Result { r := Get(t.Raw, path) if r.Indexes != nil { for i := 0; i < len(r.Indexes); i++ { r.Indexes[i] += t.Index } } else { r.Index += t.Index } return r } type arrayOrMapResult struct { a []Result ai []interface{} o map[string]Result oi map[string]interface{} vc byte } func (t Result) arrayOrMap(vc byte, valueize bool) (r arrayOrMapResult) { var json = t.Raw var i int var value Result var count int var key Result if vc == 0 { for ; i < len(json); i++ { if json[i] == '{' || json[i] == '[' { r.vc = json[i] i++ break } if json[i] > ' ' { goto end } } } else { for ; i < len(json); i++ { if json[i] == vc { i++ break } if json[i] > ' ' { goto end } } r.vc = vc } if r.vc == '{' { if valueize { r.oi = make(map[string]interface{}) } else { r.o = make(map[string]Result) } } else { if valueize { r.ai = make([]interface{}, 0) } else { r.a = make([]Result, 0) } } for ; i < len(json); i++ { if json[i] <= ' ' { continue } // get next value if json[i] == ']' || json[i] == '}' { break } switch json[i] { default: if (json[i] >= '0' && json[i] <= '9') || json[i] == '-' { value.Type = Number value.Raw, value.Num = tonum(json[i:]) value.Str = "" } else { continue } case '{', '[': value.Type = JSON value.Raw = squash(json[i:]) value.Str, value.Num = "", 0 case 'n': value.Type = Null value.Raw = tolit(json[i:]) value.Str, value.Num = "", 0 case 't': value.Type = True value.Raw = tolit(json[i:]) value.Str, value.Num = "", 0 case 'f': value.Type = False value.Raw = tolit(json[i:]) value.Str, value.Num = "", 0 case '"': value.Type = String value.Raw, value.Str = tostr(json[i:]) value.Num = 0 } value.Index = i + t.Index i += len(value.Raw) - 1 if r.vc == '{' { if count%2 == 0 { key = value } else { if valueize { if _, ok := r.oi[key.Str]; !ok { r.oi[key.Str] = value.Value() } } else { if _, ok := r.o[key.Str]; !ok { r.o[key.Str] = value } } } count++ } else { if valueize { r.ai = append(r.ai, value.Value()) } else { r.a = append(r.a, value) } } } end: if t.Indexes != nil { if len(t.Indexes) != len(r.a) { for i := 0; i < len(r.a); i++ { r.a[i].Index = 0 } } else { for i := 0; i < len(r.a); i++ { r.a[i].Index = t.Indexes[i] } } } return } // Parse parses the json and returns a result. // // This function expects that the json is well-formed, and does not validate. // Invalid json will not panic, but it may return back unexpected results. // If you are consuming JSON from an unpredictable source then you may want to // use the Valid function first. func Parse(json string) Result { var value Result i := 0 for ; i < len(json); i++ { if json[i] == '{' || json[i] == '[' { value.Type = JSON value.Raw = json[i:] // just take the entire raw break } if json[i] <= ' ' { continue } switch json[i] { case '+', '-', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'i', 'I', 'N': value.Type = Number value.Raw, value.Num = tonum(json[i:]) case 'n': if i+1 < len(json) && json[i+1] != 'u' { // nan value.Type = Number value.Raw, value.Num = tonum(json[i:]) } else { // null value.Type = Null value.Raw = tolit(json[i:]) } case 't': value.Type = True value.Raw = tolit(json[i:]) case 'f': value.Type = False value.Raw = tolit(json[i:]) case '"': value.Type = String value.Raw, value.Str = tostr(json[i:]) default: return Result{} } break } if value.Exists() { value.Index = i } return value } // ParseBytes parses the json and returns a result. // If working with bytes, this method preferred over Parse(string(data)) func ParseBytes(json []byte) Result { return Parse(string(json)) } func squash(json string) string { // expects that the lead character is a '[' or '{' or '(' or '"' // squash the value, ignoring all nested arrays and objects. var i, depth int if json[0] != '"' { i, depth = 1, 1 } for ; i < len(json); i++ { if json[i] >= '"' && json[i] <= '}' { switch json[i] { case '"': i++ s2 := i for ; i < len(json); i++ { if json[i] > '\\' { continue } if json[i] == '"' { // look for an escaped slash if json[i-1] == '\\' { n := 0 for j := i - 2; j > s2-1; j-- { if json[j] != '\\' { break } n++ } if n%2 == 0 { continue } } break } } if depth == 0 { if i >= len(json) { return json } return json[:i+1] } case '{', '[', '(': depth++ case '}', ']', ')': depth-- if depth == 0 { return json[:i+1] } } } } return json } func tonum(json string) (raw string, num float64) { for i := 1; i < len(json); i++ { // less than dash might have valid characters if json[i] <= '-' { if json[i] <= ' ' || json[i] == ',' { // break on whitespace and comma raw = json[:i] num, _ = strconv.ParseFloat(raw, 64) return } // could be a '+' or '-'. let's assume so. } else if json[i] == ']' || json[i] == '}' { // break on ']' or '}' raw = json[:i] num, _ = strconv.ParseFloat(raw, 64) return } } raw = json num, _ = strconv.ParseFloat(raw, 64) return } func tolit(json string) (raw string) { for i := 1; i < len(json); i++ { if json[i] < 'a' || json[i] > 'z' { return json[:i] } } return json } func tostr(json string) (raw string, str string) { // expects that the lead character is a '"' for i := 1; i < len(json); i++ { if json[i] > '\\' { continue } if json[i] == '"' { return json[:i+1], json[1:i] } if json[i] == '\\' { i++ for ; i < len(json); i++ { if json[i] > '\\' { continue } if json[i] == '"' { // look for an escaped slash if json[i-1] == '\\' { n := 0 for j := i - 2; j > 0; j-- { if json[j] != '\\' { break } n++ } if n%2 == 0 { continue } } return json[:i+1], unescape(json[1:i]) } } var ret string if i+1 < len(json) { ret = json[:i+1] } else { ret = json[:i] } return ret, unescape(json[1:i]) } } return json, json[1:] } // Exists returns true if value exists. // // if gjson.Get(json, "name.last").Exists(){ // println("value exists") // } func (t Result) Exists() bool { return t.Type != Null || len(t.Raw) != 0 } // Value returns one of these types: // // bool, for JSON booleans // float64, for JSON numbers // Number, for JSON numbers // string, for JSON string literals // nil, for JSON null // map[string]interface{}, for JSON objects // []interface{}, for JSON arrays func (t Result) Value() interface{} { if t.Type == String { return t.Str } switch t.Type { default: return nil case False: return false case Number: return t.Num case JSON: r := t.arrayOrMap(0, true) if r.vc == '{' { return r.oi } else if r.vc == '[' { return r.ai } return nil case True: return true } } func parseString(json string, i int) (int, string, bool, bool) { var s = i for ; i < len(json); i++ { if json[i] > '\\' { continue } if json[i] == '"' { return i + 1, json[s-1 : i+1], false, true } if json[i] == '\\' { i++ for ; i < len(json); i++ { if json[i] > '\\' { continue } if json[i] == '"' { // look for an escaped slash if json[i-1] == '\\' { n := 0 for j := i - 2; j > 0; j-- { if json[j] != '\\' { break } n++ } if n%2 == 0 { continue } } return i + 1, json[s-1 : i+1], true, true } } break } } return i, json[s-1:], false, false } func parseNumber(json string, i int) (int, string) { var s = i i++ for ; i < len(json); i++ { if json[i] <= ' ' || json[i] == ',' || json[i] == ']' || json[i] == '}' { return i, json[s:i] } } return i, json[s:] } func parseLiteral(json string, i int) (int, string) { var s = i i++ for ; i < len(json); i++ { if json[i] < 'a' || json[i] > 'z' { return i, json[s:i] } } return i, json[s:] } type arrayPathResult struct { part string path string pipe string piped bool more bool alogok bool arrch bool alogkey string query struct { on bool all bool path string op string value string } } func parseArrayPath(path string) (r arrayPathResult) { for i := 0; i < len(path); i++ { if path[i] == '|' { r.part = path[:i] r.pipe = path[i+1:] r.piped = true return } if path[i] == '.' { r.part = path[:i] if !r.arrch && i < len(path)-1 && isDotPiperChar(path[i+1:]) { r.pipe = path[i+1:] r.piped = true } else { r.path = path[i+1:] r.more = true } return } if path[i] == '#' { r.arrch = true if i == 0 && len(path) > 1 { if path[1] == '.' { r.alogok = true r.alogkey = path[2:] r.path = path[:1] } else if path[1] == '[' || path[1] == '(' { // query r.query.on = true qpath, op, value, _, fi, vesc, ok := parseQuery(path[i:]) if !ok { // bad query, end now break } if len(value) >= 2 && value[0] == '"' && value[len(value)-1] == '"' { value = value[1 : len(value)-1] if vesc { value = unescape(value) } } r.query.path = qpath r.query.op = op r.query.value = value i = fi - 1 if i+1 < len(path) && path[i+1] == '#' { r.query.all = true } } } continue } } r.part = path r.path = "" return } // splitQuery takes a query and splits it into three parts: // // path, op, middle, and right. // // So for this query: // // #(first_name=="Murphy").last // // Becomes // // first_name # path // =="Murphy" # middle // .last # right // // Or, // // #(service_roles.#(=="one")).cap // // Becomes // // service_roles.#(=="one") # path // # middle // .cap # right func parseQuery(query string) ( path, op, value, remain string, i int, vesc, ok bool, ) { if len(query) < 2 || query[0] != '#' || (query[1] != '(' && query[1] != '[') { return "", "", "", "", i, false, false } i = 2 j := 0 // start of value part depth := 1 for ; i < len(query); i++ { if depth == 1 && j == 0 { switch query[i] { case '!', '=', '<', '>', '%': // start of the value part j = i continue } } if query[i] == '\\' { i++ } else if query[i] == '[' || query[i] == '(' { depth++ } else if query[i] == ']' || query[i] == ')' { depth-- if depth == 0 { break } } else if query[i] == '"' { // inside selector string, balance quotes i++ for ; i < len(query); i++ { if query[i] == '\\' { vesc = true i++ } else if query[i] == '"' { break } } } } if depth > 0 { return "", "", "", "", i, false, false } if j > 0 { path = trim(query[2:j]) value = trim(query[j:i]) remain = query[i+1:] // parse the compare op from the value var opsz int switch { case len(value) == 1: opsz = 1 case value[0] == '!' && value[1] == '=': opsz = 2 case value[0] == '!' && value[1] == '%': opsz = 2 case value[0] == '<' && value[1] == '=': opsz = 2 case value[0] == '>' && value[1] == '=': opsz = 2 case value[0] == '=' && value[1] == '=': value = value[1:] opsz = 1 case value[0] == '<': opsz = 1 case value[0] == '>': opsz = 1 case value[0] == '=': opsz = 1 case value[0] == '%': opsz = 1 } op = value[:opsz] value = trim(value[opsz:]) } else { path = trim(query[2:i]) remain = query[i+1:] } return path, op, value, remain, i + 1, vesc, true } func trim(s string) string { left: if len(s) > 0 && s[0] <= ' ' { s = s[1:] goto left } right: if len(s) > 0 && s[len(s)-1] <= ' ' { s = s[:len(s)-1] goto right } return s } // peek at the next byte and see if it's a '@', '[', or '{'. func isDotPiperChar(s string) bool { if DisableModifiers { return false } c := s[0] if c == '@' { // check that the next component is *not* a modifier. i := 1 for ; i < len(s); i++ { if s[i] == '.' || s[i] == '|' || s[i] == ':' { break } } _, ok := modifiers[s[1:i]] return ok } return c == '[' || c == '{' } type objectPathResult struct { part string path string pipe string piped bool wild bool more bool } func parseObjectPath(path string) (r objectPathResult) { for i := 0; i < len(path); i++ { if path[i] == '|' { r.part = path[:i] r.pipe = path[i+1:] r.piped = true return } if path[i] == '.' { r.part = path[:i] if i < len(path)-1 && isDotPiperChar(path[i+1:]) { r.pipe = path[i+1:] r.piped = true } else { r.path = path[i+1:] r.more = true } return } if path[i] == '*' || path[i] == '?' { r.wild = true continue } if path[i] == '\\' { // go into escape mode. this is a slower path that // strips off the escape character from the part. epart := []byte(path[:i]) i++ if i < len(path) { epart = append(epart, path[i]) i++ for ; i < len(path); i++ { if path[i] == '\\' { i++ if i < len(path) { epart = append(epart, path[i]) } continue } else if path[i] == '.' { r.part = string(epart) if i < len(path)-1 && isDotPiperChar(path[i+1:]) { r.pipe = path[i+1:] r.piped = true } else { r.path = path[i+1:] r.more = true } return } else if path[i] == '|' { r.part = string(epart) r.pipe = path[i+1:] r.piped = true return } else if path[i] == '*' || path[i] == '?' { r.wild = true } epart = append(epart, path[i]) } } // append the last part r.part = string(epart) return } } r.part = path return } func parseSquash(json string, i int) (int, string) { // expects that the lead character is a '[' or '{' or '(' // squash the value, ignoring all nested arrays and objects. // the first '[' or '{' or '(' has already been read s := i i++ depth := 1 for ; i < len(json); i++ { if json[i] >= '"' && json[i] <= '}' { switch json[i] { case '"': i++ s2 := i for ; i < len(json); i++ { if json[i] > '\\' { continue } if json[i] == '"' { // look for an escaped slash if json[i-1] == '\\' { n := 0 for j := i - 2; j > s2-1; j-- { if json[j] != '\\' { break } n++ } if n%2 == 0 { continue } } break } } case '{', '[', '(': depth++ case '}', ']', ')': depth-- if depth == 0 { i++ return i, json[s:i] } } } } return i, json[s:] } func parseObject(c *parseContext, i int, path string) (int, bool) { var pmatch, kesc, vesc, ok, hit bool var key, val string rp := parseObjectPath(path) if !rp.more && rp.piped { c.pipe = rp.pipe c.piped = true } for i < len(c.json) { for ; i < len(c.json); i++ { if c.json[i] == '"' { // parse_key_string // this is slightly different from getting s string value // because we don't need the outer quotes. i++ var s = i for ; i < len(c.json); i++ { if c.json[i] > '\\' { continue } if c.json[i] == '"' { i, key, kesc, ok = i+1, c.json[s:i], false, true goto parse_key_string_done } if c.json[i] == '\\' { i++ for ; i < len(c.json); i++ { if c.json[i] > '\\' { continue } if c.json[i] == '"' { // look for an escaped slash if c.json[i-1] == '\\' { n := 0 for j := i - 2; j > 0; j-- { if c.json[j] != '\\' { break } n++ } if n%2 == 0 { continue } } i, key, kesc, ok = i+1, c.json[s:i], true, true goto parse_key_string_done } } break } } key, kesc, ok = c.json[s:], false, false parse_key_string_done: break } if c.json[i] == '}' { return i + 1, false } } if !ok { return i, false } if rp.wild { if kesc { pmatch = matchLimit(unescape(key), rp.part) } else { pmatch = matchLimit(key, rp.part) } } else { if kesc { pmatch = rp.part == unescape(key) } else { pmatch = rp.part == key } } hit = pmatch && !rp.more for ; i < len(c.json); i++ { var num bool switch c.json[i] { default: continue case '"': i++ i, val, vesc, ok = parseString(c.json, i) if !ok { return i, false } if hit { if vesc { c.value.Str = unescape(val[1 : len(val)-1]) } else { c.value.Str = val[1 : len(val)-1] } c.value.Raw = val c.value.Type = String return i, true } case '{': if pmatch && !hit { i, hit = parseObject(c, i+1, rp.path) if hit { return i, true } } else { i, val = parseSquash(c.json, i) if hit { c.value.Raw = val c.value.Type = JSON return i, true } } case '[': if pmatch && !hit { i, hit = parseArray(c, i+1, rp.path) if hit { return i, true } } else { i, val = parseSquash(c.json, i) if hit { c.value.Raw = val c.value.Type = JSON return i, true } } case 'n': if i+1 < len(c.json) && c.json[i+1] != 'u' { num = true break } fallthrough case 't', 'f': vc := c.json[i] i, val = parseLiteral(c.json, i) if hit { c.value.Raw = val switch vc { case 't': c.value.Type = True case 'f': c.value.Type = False } return i, true } case '+', '-', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'i', 'I', 'N': num = true } if num { i, val = parseNumber(c.json, i) if hit { c.value.Raw = val c.value.Type = Number c.value.Num, _ = strconv.ParseFloat(val, 64) return i, true } } break } } return i, false } // matchLimit will limit the complexity of the match operation to avoid ReDos // attacks from arbritary inputs. // See the github.com/tidwall/match.MatchLimit function for more information. func matchLimit(str, pattern string) bool { matched, _ := match.MatchLimit(str, pattern, 10000) return matched } func falseish(t Result) bool { switch t.Type { case Null: return true case False: return true case String: b, err := strconv.ParseBool(strings.ToLower(t.Str)) if err != nil { return false } return !b case Number: return t.Num == 0 default: return false } } func trueish(t Result) bool { switch t.Type { case True: return true case String: b, err := strconv.ParseBool(strings.ToLower(t.Str)) if err != nil { return false } return b case Number: return t.Num != 0 default: return false } } func nullish(t Result) bool { return t.Type == Null } func queryMatches(rp *arrayPathResult, value Result) bool { rpv := rp.query.value if len(rpv) > 0 { if rpv[0] == '~' { // convert to bool rpv = rpv[1:] var ish, ok bool switch rpv { case "*": ish, ok = value.Exists(), true case "null": ish, ok = nullish(value), true case "true": ish, ok = trueish(value), true case "false": ish, ok = falseish(value), true } if ok { rpv = "true" if ish { value = Result{Type: True} } else { value = Result{Type: False} } } else { rpv = "" value = Result{} } } } if !value.Exists() { return false } if rp.query.op == "" { // the query is only looking for existence, such as: // friends.#(name) // which makes sure that the array "friends" has an element of // "name" that exists return true } switch value.Type { case String: switch rp.query.op { case "=": return value.Str == rpv case "!=": return value.Str != rpv case "<": return value.Str < rpv case "<=": return value.Str <= rpv case ">": return value.Str > rpv case ">=": return value.Str >= rpv case "%": return matchLimit(value.Str, rpv) case "!%": return !matchLimit(value.Str, rpv) } case Number: rpvn, _ := strconv.ParseFloat(rpv, 64) switch rp.query.op { case "=": return value.Num == rpvn case "!=": return value.Num != rpvn case "<": return value.Num < rpvn case "<=": return value.Num <= rpvn case ">": return value.Num > rpvn case ">=": return value.Num >= rpvn } case True: switch rp.query.op { case "=": return rpv == "true" case "!=": return rpv != "true" case ">": return rpv == "false" case ">=": return true } case False: switch rp.query.op { case "=": return rpv == "false" case "!=": return rpv != "false" case "<": return rpv == "true" case "<=": return true } } return false } func parseArray(c *parseContext, i int, path string) (int, bool) { var pmatch, vesc, ok, hit bool var val string var h int var alog []int var partidx int var multires []byte var queryIndexes []int rp := parseArrayPath(path) if !rp.arrch { n, ok := parseUint(rp.part) if !ok { partidx = -1 } else { partidx = int(n) } } if !rp.more && rp.piped { c.pipe = rp.pipe c.piped = true } procQuery := func(qval Result) bool { if rp.query.all { if len(multires) == 0 { multires = append(multires, '[') } } var tmp parseContext tmp.value = qval fillIndex(c.json, &tmp) parentIndex := tmp.value.Index var res Result if qval.Type == JSON { res = qval.Get(rp.query.path) } else { if rp.query.path != "" { return false } res = qval } if queryMatches(&rp, res) { if rp.more { left, right, ok := splitPossiblePipe(rp.path) if ok { rp.path = left c.pipe = right c.piped = true } res = qval.Get(rp.path) } else { res = qval } if rp.query.all { raw := res.Raw if len(raw) == 0 { raw = res.String() } if raw != "" { if len(multires) > 1 { multires = append(multires, ',') } multires = append(multires, raw...) queryIndexes = append(queryIndexes, res.Index+parentIndex) } } else { c.value = res return true } } return false } for i < len(c.json)+1 { if !rp.arrch { pmatch = partidx == h hit = pmatch && !rp.more } h++ if rp.alogok { alog = append(alog, i) } for ; ; i++ { var ch byte if i > len(c.json) { break } else if i == len(c.json) { ch = ']' } else { ch = c.json[i] } var num bool switch ch { default: continue case '"': i++ i, val, vesc, ok = parseString(c.json, i) if !ok { return i, false } if rp.query.on { var qval Result if vesc { qval.Str = unescape(val[1 : len(val)-1]) } else { qval.Str = val[1 : len(val)-1] } qval.Raw = val qval.Type = String if procQuery(qval) { return i, true } } else if hit { if rp.alogok { break } if vesc { c.value.Str = unescape(val[1 : len(val)-1]) } else { c.value.Str = val[1 : len(val)-1] } c.value.Raw = val c.value.Type = String return i, true } case '{': if pmatch && !hit { i, hit = parseObject(c, i+1, rp.path) if hit { if rp.alogok { break } return i, true } } else { i, val = parseSquash(c.json, i) if rp.query.on { if procQuery(Result{Raw: val, Type: JSON}) { return i, true } } else if hit { if rp.alogok { break } c.value.Raw = val c.value.Type = JSON return i, true } } case '[': if pmatch && !hit { i, hit = parseArray(c, i+1, rp.path) if hit { if rp.alogok { break } return i, true } } else { i, val = parseSquash(c.json, i) if rp.query.on { if procQuery(Result{Raw: val, Type: JSON}) { return i, true } } else if hit { if rp.alogok { break } c.value.Raw = val c.value.Type = JSON return i, true } } case 'n': if i+1 < len(c.json) && c.json[i+1] != 'u' { num = true break } fallthrough case 't', 'f': vc := c.json[i] i, val = parseLiteral(c.json, i) if rp.query.on { var qval Result qval.Raw = val switch vc { case 't': qval.Type = True case 'f': qval.Type = False } if procQuery(qval) { return i, true } } else if hit { if rp.alogok { break } c.value.Raw = val switch vc { case 't': c.value.Type = True case 'f': c.value.Type = False } return i, true } case '+', '-', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'i', 'I', 'N': num = true case ']': if rp.arrch && rp.part == "#" { if rp.alogok { left, right, ok := splitPossiblePipe(rp.alogkey) if ok { rp.alogkey = left c.pipe = right c.piped = true } var indexes = make([]int, 0, 64) var jsons = make([]byte, 0, 64) jsons = append(jsons, '[') for j, k := 0, 0; j < len(alog); j++ { idx := alog[j] for idx < len(c.json) { switch c.json[idx] { case ' ', '\t', '\r', '\n': idx++ continue } break } if idx < len(c.json) && c.json[idx] != ']' { _, res, ok := parseAny(c.json, idx, true) if ok { res := res.Get(rp.alogkey) if res.Exists() { if k > 0 { jsons = append(jsons, ',') } raw := res.Raw if len(raw) == 0 { raw = res.String() } jsons = append(jsons, []byte(raw)...) indexes = append(indexes, res.Index) k++ } } } } jsons = append(jsons, ']') c.value.Type = JSON c.value.Raw = string(jsons) c.value.Indexes = indexes return i + 1, true } if rp.alogok { break } c.value.Type = Number c.value.Num = float64(h - 1) c.value.Raw = strconv.Itoa(h - 1) c.calcd = true return i + 1, true } if !c.value.Exists() { if len(multires) > 0 { c.value = Result{ Raw: string(append(multires, ']')), Type: JSON, Indexes: queryIndexes, } } else if rp.query.all { c.value = Result{ Raw: "[]", Type: JSON, } } } return i + 1, false } if num { i, val = parseNumber(c.json, i) if rp.query.on { var qval Result qval.Raw = val qval.Type = Number qval.Num, _ = strconv.ParseFloat(val, 64) if procQuery(qval) { return i, true } } else if hit { if rp.alogok { break } c.value.Raw = val c.value.Type = Number c.value.Num, _ = strconv.ParseFloat(val, 64) return i, true } } break } } return i, false } func splitPossiblePipe(path string) (left, right string, ok bool) { // take a quick peek for the pipe character. If found we'll split the piped // part of the path into the c.pipe field and shorten the rp. var possible bool for i := 0; i < len(path); i++ { if path[i] == '|' { possible = true break } } if !possible { return } if len(path) > 0 && path[0] == '{' { squashed := squash(path[1:]) if len(squashed) < len(path)-1 { squashed = path[:len(squashed)+1] remain := path[len(squashed):] if remain[0] == '|' { return squashed, remain[1:], true } } return } // split the left and right side of the path with the pipe character as // the delimiter. This is a little tricky because we'll need to basically // parse the entire path. for i := 0; i < len(path); i++ { if path[i] == '\\' { i++ } else if path[i] == '.' { if i == len(path)-1 { return } if path[i+1] == '#' { i += 2 if i == len(path) { return } if path[i] == '[' || path[i] == '(' { var start, end byte if path[i] == '[' { start, end = '[', ']' } else { start, end = '(', ')' } // inside selector, balance brackets i++ depth := 1 for ; i < len(path); i++ { if path[i] == '\\' { i++ } else if path[i] == start { depth++ } else if path[i] == end { depth-- if depth == 0 { break } } else if path[i] == '"' { // inside selector string, balance quotes i++ for ; i < len(path); i++ { if path[i] == '\\' { i++ } else if path[i] == '"' { break } } } } } } } else if path[i] == '|' { return path[:i], path[i+1:], true } } return } // ForEachLine iterates through lines of JSON as specified by the JSON Lines // format (http://jsonlines.org/). // Each line is returned as a GJSON Result. func ForEachLine(json string, iterator func(line Result) bool) { var res Result var i int for { i, res, _ = parseAny(json, i, true) if !res.Exists() { break } if !iterator(res) { return } } } type subSelector struct { name string path string } // parseSubSelectors returns the subselectors belonging to a '[path1,path2]' or // '{"field1":path1,"field2":path2}' type subSelection. It's expected that the // first character in path is either '[' or '{', and has already been checked // prior to calling this function. func parseSubSelectors(path string) (sels []subSelector, out string, ok bool) { modifier := 0 depth := 1 colon := 0 start := 1 i := 1 pushSel := func() { var sel subSelector if colon == 0 { sel.path = path[start:i] } else { sel.name = path[start:colon] sel.path = path[colon+1 : i] } sels = append(sels, sel) colon = 0 modifier = 0 start = i + 1 } for ; i < len(path); i++ { switch path[i] { case '\\': i++ case '@': if modifier == 0 && i > 0 && (path[i-1] == '.' || path[i-1] == '|') { modifier = i } case ':': if modifier == 0 && colon == 0 && depth == 1 { colon = i } case ',': if depth == 1 { pushSel() } case '"': i++ loop: for ; i < len(path); i++ { switch path[i] { case '\\': i++ case '"': break loop } } case '[', '(', '{': depth++ case ']', ')', '}': depth-- if depth == 0 { pushSel() path = path[i+1:] return sels, path, true } } } return } // nameOfLast returns the name of the last component func nameOfLast(path string) string { for i := len(path) - 1; i >= 0; i-- { if path[i] == '|' || path[i] == '.' { if i > 0 { if path[i-1] == '\\' { continue } } return path[i+1:] } } return path } func isSimpleName(component string) bool { for i := 0; i < len(component); i++ { if component[i] < ' ' { return false } switch component[i] { case '[', ']', '{', '}', '(', ')', '#', '|', '!': return false } } return true } var hexchars = [...]byte{ '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'b', 'c', 'd', 'e', 'f', } func appendHex16(dst []byte, x uint16) []byte { return append(dst, hexchars[x>>12&0xF], hexchars[x>>8&0xF], hexchars[x>>4&0xF], hexchars[x>>0&0xF], ) } // AppendJSONString is a convenience function that converts the provided string // to a valid JSON string and appends it to dst. func AppendJSONString(dst []byte, s string) []byte { dst = append(dst, make([]byte, len(s)+2)...) dst = append(dst[:len(dst)-len(s)-2], '"') for i := 0; i < len(s); i++ { if s[i] < ' ' { dst = append(dst, '\\') switch s[i] { case '\b': dst = append(dst, 'b') case '\f': dst = append(dst, 'f') case '\n': dst = append(dst, 'n') case '\r': dst = append(dst, 'r') case '\t': dst = append(dst, 't') default: dst = append(dst, 'u') dst = appendHex16(dst, uint16(s[i])) } } else if s[i] == '>' || s[i] == '<' || s[i] == '&' { dst = append(dst, '\\', 'u') dst = appendHex16(dst, uint16(s[i])) } else if s[i] == '\\' { dst = append(dst, '\\', '\\') } else if s[i] == '"' { dst = append(dst, '\\', '"') } else if s[i] > 127 { // read utf8 character r, n := utf8.DecodeRuneInString(s[i:]) if n == 0 { break } if r == utf8.RuneError && n == 1 { dst = append(dst, `\ufffd`...) } else if r == '\u2028' || r == '\u2029' { dst = append(dst, `\u202`...) dst = append(dst, hexchars[r&0xF]) } else { dst = append(dst, s[i:i+n]...) } i = i + n - 1 } else { dst = append(dst, s[i]) } } return append(dst, '"') } type parseContext struct { json string value Result pipe string piped bool calcd bool lines bool } // Get searches json for the specified path. // A path is in dot syntax, such as "name.last" or "age". // When the value is found it's returned immediately. // // A path is a series of keys separated by a dot. // A key may contain special wildcard characters '*' and '?'. // To access an array value use the index as the key. // To get the number of elements in an array or to access a child path, use // the '#' character. // The dot and wildcard character can be escaped with '\'. // // { // "name": {"first": "Tom", "last": "Anderson"}, // "age":37, // "children": ["Sara","Alex","Jack"], // "friends": [ // {"first": "James", "last": "Murphy"}, // {"first": "Roger", "last": "Craig"} // ] // } // "name.last" >> "Anderson" // "age" >> 37 // "children" >> ["Sara","Alex","Jack"] // "children.#" >> 3 // "children.1" >> "Alex" // "child*.2" >> "Jack" // "c?ildren.0" >> "Sara" // "friends.#.first" >> ["James","Roger"] // // This function expects that the json is well-formed, and does not validate. // Invalid json will not panic, but it may return back unexpected results. // If you are consuming JSON from an unpredictable source then you may want to // use the Valid function first. func Get(json, path string) Result { if len(path) > 1 { if (path[0] == '@' && !DisableModifiers) || path[0] == '!' { // possible modifier var ok bool var npath string var rjson string if path[0] == '@' && !DisableModifiers { npath, rjson, ok = execModifier(json, path) } else if path[0] == '!' { npath, rjson, ok = execStatic(json, path) } if ok { path = npath if len(path) > 0 && (path[0] == '|' || path[0] == '.') { res := Get(rjson, path[1:]) res.Index = 0 res.Indexes = nil return res } return Parse(rjson) } } if path[0] == '[' || path[0] == '{' { // using a subselector path kind := path[0] var ok bool var subs []subSelector subs, path, ok = parseSubSelectors(path) if ok { if len(path) == 0 || (path[0] == '|' || path[0] == '.') { var b []byte b = append(b, kind) var i int for _, sub := range subs { res := Get(json, sub.path) if res.Exists() { if i > 0 { b = append(b, ',') } if kind == '{' { if len(sub.name) > 0 { if sub.name[0] == '"' && Valid(sub.name) { b = append(b, sub.name...) } else { b = AppendJSONString(b, sub.name) } } else { last := nameOfLast(sub.path) if isSimpleName(last) { b = AppendJSONString(b, last) } else { b = AppendJSONString(b, "_") } } b = append(b, ':') } var raw string if len(res.Raw) == 0 { raw = res.String() if len(raw) == 0 { raw = "null" } } else { raw = res.Raw } b = append(b, raw...) i++ } } b = append(b, kind+2) var res Result res.Raw = string(b) res.Type = JSON if len(path) > 0 { res = res.Get(path[1:]) } res.Index = 0 return res } } } } var i int var c = &parseContext{json: json} if len(path) >= 2 && path[0] == '.' && path[1] == '.' { c.lines = true parseArray(c, 0, path[2:]) } else { for ; i < len(c.json); i++ { if c.json[i] == '{' { i++ parseObject(c, i, path) break } if c.json[i] == '[' { i++ parseArray(c, i, path) break } } } if c.piped { res := c.value.Get(c.pipe) res.Index = 0 return res } fillIndex(json, c) return c.value } // GetBytes searches json for the specified path. // If working with bytes, this method preferred over Get(string(data), path) func GetBytes(json []byte, path string) Result { return getBytes(json, path) } // runeit returns the rune from the the \uXXXX func runeit(json string) rune { n, _ := strconv.ParseUint(json[:4], 16, 64) return rune(n) } // unescape unescapes a string func unescape(json string) string { var str = make([]byte, 0, len(json)) for i := 0; i < len(json); i++ { switch { default: str = append(str, json[i]) case json[i] < ' ': return string(str) case json[i] == '\\': i++ if i >= len(json) { return string(str) } switch json[i] { default: return string(str) case '\\': str = append(str, '\\') case '/': str = append(str, '/') case 'b': str = append(str, '\b') case 'f': str = append(str, '\f') case 'n': str = append(str, '\n') case 'r': str = append(str, '\r') case 't': str = append(str, '\t') case '"': str = append(str, '"') case 'u': if i+5 > len(json) { return string(str) } r := runeit(json[i+1:]) i += 5 if utf16.IsSurrogate(r) { // need another code if len(json[i:]) >= 6 && json[i] == '\\' && json[i+1] == 'u' { // we expect it to be correct so just consume it r = utf16.DecodeRune(r, runeit(json[i+2:])) i += 6 } } // provide enough space to encode the largest utf8 possible str = append(str, 0, 0, 0, 0, 0, 0, 0, 0) n := utf8.EncodeRune(str[len(str)-8:], r) str = str[:len(str)-8+n] i-- // backtrack index by one } } } return string(str) } // Less return true if a token is less than another token. // The caseSensitive paramater is used when the tokens are Strings. // The order when comparing two different type is: // // Null < False < Number < String < True < JSON func (t Result) Less(token Result, caseSensitive bool) bool { if t.Type < token.Type { return true } if t.Type > token.Type { return false } if t.Type == String { if caseSensitive { return t.Str < token.Str } return stringLessInsensitive(t.Str, token.Str) } if t.Type == Number { return t.Num < token.Num } return t.Raw < token.Raw } func stringLessInsensitive(a, b string) bool { for i := 0; i < len(a) && i < len(b); i++ { if a[i] >= 'A' && a[i] <= 'Z' { if b[i] >= 'A' && b[i] <= 'Z' { // both are uppercase, do nothing if a[i] < b[i] { return true } else if a[i] > b[i] { return false } } else { // a is uppercase, convert a to lowercase if a[i]+32 < b[i] { return true } else if a[i]+32 > b[i] { return false } } } else if b[i] >= 'A' && b[i] <= 'Z' { // b is uppercase, convert b to lowercase if a[i] < b[i]+32 { return true } else if a[i] > b[i]+32 { return false } } else { // neither are uppercase if a[i] < b[i] { return true } else if a[i] > b[i] { return false } } } return len(a) < len(b) } // parseAny parses the next value from a json string. // A Result is returned when the hit param is set. // The return values are (i int, res Result, ok bool) func parseAny(json string, i int, hit bool) (int, Result, bool) { var res Result var val string for ; i < len(json); i++ { if json[i] == '{' || json[i] == '[' { i, val = parseSquash(json, i) if hit { res.Raw = val res.Type = JSON } var tmp parseContext tmp.value = res fillIndex(json, &tmp) return i, tmp.value, true } if json[i] <= ' ' { continue } var num bool switch json[i] { case '"': i++ var vesc bool var ok bool i, val, vesc, ok = parseString(json, i) if !ok { return i, res, false } if hit { res.Type = String res.Raw = val if vesc { res.Str = unescape(val[1 : len(val)-1]) } else { res.Str = val[1 : len(val)-1] } } return i, res, true case 'n': if i+1 < len(json) && json[i+1] != 'u' { num = true break } fallthrough case 't', 'f': vc := json[i] i, val = parseLiteral(json, i) if hit { res.Raw = val switch vc { case 't': res.Type = True case 'f': res.Type = False } return i, res, true } case '+', '-', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'i', 'I', 'N': num = true } if num { i, val = parseNumber(json, i) if hit { res.Raw = val res.Type = Number res.Num, _ = strconv.ParseFloat(val, 64) } return i, res, true } } return i, res, false } // GetMany searches json for the multiple paths. // The return value is a Result array where the number of items // will be equal to the number of input paths. func GetMany(json string, path ...string) []Result { res := make([]Result, len(path)) for i, path := range path { res[i] = Get(json, path) } return res } // GetManyBytes searches json for the multiple paths. // The return value is a Result array where the number of items // will be equal to the number of input paths. func GetManyBytes(json []byte, path ...string) []Result { res := make([]Result, len(path)) for i, path := range path { res[i] = GetBytes(json, path) } return res } func validpayload(data []byte, i int) (outi int, ok bool) { for ; i < len(data); i++ { switch data[i] { default: i, ok = validany(data, i) if !ok { return i, false } for ; i < len(data); i++ { switch data[i] { default: return i, false case ' ', '\t', '\n', '\r': continue } } return i, true case ' ', '\t', '\n', '\r': continue } } return i, false } func validany(data []byte, i int) (outi int, ok bool) { for ; i < len(data); i++ { switch data[i] { default: return i, false case ' ', '\t', '\n', '\r': continue case '{': return validobject(data, i+1) case '[': return validarray(data, i+1) case '"': return validstring(data, i+1) case '-', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9': return validnumber(data, i+1) case 't': return validtrue(data, i+1) case 'f': return validfalse(data, i+1) case 'n': return validnull(data, i+1) } } return i, false } func validobject(data []byte, i int) (outi int, ok bool) { for ; i < len(data); i++ { switch data[i] { default: return i, false case ' ', '\t', '\n', '\r': continue case '}': return i + 1, true case '"': key: if i, ok = validstring(data, i+1); !ok { return i, false } if i, ok = validcolon(data, i); !ok { return i, false } if i, ok = validany(data, i); !ok { return i, false } if i, ok = validcomma(data, i, '}'); !ok { return i, false } if data[i] == '}' { return i + 1, true } i++ for ; i < len(data); i++ { switch data[i] { default: return i, false case ' ', '\t', '\n', '\r': continue case '"': goto key } } return i, false } } return i, false } func validcolon(data []byte, i int) (outi int, ok bool) { for ; i < len(data); i++ { switch data[i] { default: return i, false case ' ', '\t', '\n', '\r': continue case ':': return i + 1, true } } return i, false } func validcomma(data []byte, i int, end byte) (outi int, ok bool) { for ; i < len(data); i++ { switch data[i] { default: return i, false case ' ', '\t', '\n', '\r': continue case ',': return i, true case end: return i, true } } return i, false } func validarray(data []byte, i int) (outi int, ok bool) { for ; i < len(data); i++ { switch data[i] { default: for ; i < len(data); i++ { if i, ok = validany(data, i); !ok { return i, false } if i, ok = validcomma(data, i, ']'); !ok { return i, false } if data[i] == ']' { return i + 1, true } } case ' ', '\t', '\n', '\r': continue case ']': return i + 1, true } } return i, false } func validstring(data []byte, i int) (outi int, ok bool) { for ; i < len(data); i++ { if data[i] < ' ' { return i, false } else if data[i] == '\\' { i++ if i == len(data) { return i, false } switch data[i] { default: return i, false case '"', '\\', '/', 'b', 'f', 'n', 'r', 't': case 'u': for j := 0; j < 4; j++ { i++ if i >= len(data) { return i, false } if !((data[i] >= '0' && data[i] <= '9') || (data[i] >= 'a' && data[i] <= 'f') || (data[i] >= 'A' && data[i] <= 'F')) { return i, false } } } } else if data[i] == '"' { return i + 1, true } } return i, false } func validnumber(data []byte, i int) (outi int, ok bool) { i-- // sign if data[i] == '-' { i++ if i == len(data) { return i, false } if data[i] < '0' || data[i] > '9' { return i, false } } // int if i == len(data) { return i, false } if data[i] == '0' { i++ } else { for ; i < len(data); i++ { if data[i] >= '0' && data[i] <= '9' { continue } break } } // frac if i == len(data) { return i, true } if data[i] == '.' { i++ if i == len(data) { return i, false } if data[i] < '0' || data[i] > '9' { return i, false } i++ for ; i < len(data); i++ { if data[i] >= '0' && data[i] <= '9' { continue } break } } // exp if i == len(data) { return i, true } if data[i] == 'e' || data[i] == 'E' { i++ if i == len(data) { return i, false } if data[i] == '+' || data[i] == '-' { i++ } if i == len(data) { return i, false } if data[i] < '0' || data[i] > '9' { return i, false } i++ for ; i < len(data); i++ { if data[i] >= '0' && data[i] <= '9' { continue } break } } return i, true } func validtrue(data []byte, i int) (outi int, ok bool) { if i+3 <= len(data) && data[i] == 'r' && data[i+1] == 'u' && data[i+2] == 'e' { return i + 3, true } return i, false } func validfalse(data []byte, i int) (outi int, ok bool) { if i+4 <= len(data) && data[i] == 'a' && data[i+1] == 'l' && data[i+2] == 's' && data[i+3] == 'e' { return i + 4, true } return i, false } func validnull(data []byte, i int) (outi int, ok bool) { if i+3 <= len(data) && data[i] == 'u' && data[i+1] == 'l' && data[i+2] == 'l' { return i + 3, true } return i, false } // Valid returns true if the input is valid json. // // if !gjson.Valid(json) { // return errors.New("invalid json") // } // value := gjson.Get(json, "name.last") func Valid(json string) bool { _, ok := validpayload(stringBytes(json), 0) return ok } // ValidBytes returns true if the input is valid json. // // if !gjson.Valid(json) { // return errors.New("invalid json") // } // value := gjson.Get(json, "name.last") // // If working with bytes, this method preferred over ValidBytes(string(data)) func ValidBytes(json []byte) bool { _, ok := validpayload(json, 0) return ok } func parseUint(s string) (n uint64, ok bool) { var i int if i == len(s) { return 0, false } for ; i < len(s); i++ { if s[i] >= '0' && s[i] <= '9' { n = n*10 + uint64(s[i]-'0') } else { return 0, false } } return n, true } func parseInt(s string) (n int64, ok bool) { var i int var sign bool if len(s) > 0 && s[0] == '-' { sign = true i++ } if i == len(s) { return 0, false } for ; i < len(s); i++ { if s[i] >= '0' && s[i] <= '9' { n = n*10 + int64(s[i]-'0') } else { return 0, false } } if sign { return n * -1, true } return n, true } // safeInt validates a given JSON number // ensures it lies within the minimum and maximum representable JSON numbers func safeInt(f float64) (n int64, ok bool) { // https://tc39.es/ecma262/#sec-number.min_safe_integer // https://tc39.es/ecma262/#sec-number.max_safe_integer if f < -9007199254740991 || f > 9007199254740991 { return 0, false } return int64(f), true } // execStatic parses the path to find a static value. // The input expects that the path already starts with a '!' func execStatic(json, path string) (pathOut, res string, ok bool) { name := path[1:] if len(name) > 0 { switch name[0] { case '{', '[', '"', '+', '-', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9': _, res = parseSquash(name, 0) pathOut = name[len(res):] return pathOut, res, true } } for i := 1; i < len(path); i++ { if path[i] == '|' { pathOut = path[i:] name = path[1:i] break } if path[i] == '.' { pathOut = path[i:] name = path[1:i] break } } switch strings.ToLower(name) { case "true", "false", "null", "nan", "inf": return pathOut, name, true } return pathOut, res, false } // execModifier parses the path to find a matching modifier function. // The input expects that the path already starts with a '@' func execModifier(json, path string) (pathOut, res string, ok bool) { name := path[1:] var hasArgs bool for i := 1; i < len(path); i++ { if path[i] == ':' { pathOut = path[i+1:] name = path[1:i] hasArgs = len(pathOut) > 0 break } if path[i] == '|' { pathOut = path[i:] name = path[1:i] break } if path[i] == '.' { pathOut = path[i:] name = path[1:i] break } } if fn, ok := modifiers[name]; ok { var args string if hasArgs { var parsedArgs bool switch pathOut[0] { case '{', '[', '"': // json arg res := Parse(pathOut) if res.Exists() { args = squash(pathOut) pathOut = pathOut[len(args):] parsedArgs = true } } if !parsedArgs { // simple arg i := 0 for ; i < len(pathOut); i++ { if pathOut[i] == '|' { break } switch pathOut[i] { case '{', '[', '"', '(': s := squash(pathOut[i:]) i += len(s) - 1 } } args = pathOut[:i] pathOut = pathOut[i:] } } return pathOut, fn(json, args), true } return pathOut, res, false } // unwrap removes the '[]' or '{}' characters around json func unwrap(json string) string { json = trim(json) if len(json) >= 2 && (json[0] == '[' || json[0] == '{') { json = json[1 : len(json)-1] } return json } // DisableModifiers will disable the modifier syntax var DisableModifiers = false var modifiers map[string]func(json, arg string) string func init() { modifiers = map[string]func(json, arg string) string{ "pretty": modPretty, "ugly": modUgly, "reverse": modReverse, "this": modThis, "flatten": modFlatten, "join": modJoin, "valid": modValid, "keys": modKeys, "values": modValues, "tostr": modToStr, "fromstr": modFromStr, "group": modGroup, "dig": modDig, } } // AddModifier binds a custom modifier command to the GJSON syntax. // This operation is not thread safe and should be executed prior to // using all other gjson function. func AddModifier(name string, fn func(json, arg string) string) { modifiers[name] = fn } // ModifierExists returns true when the specified modifier exists. func ModifierExists(name string, fn func(json, arg string) string) bool { _, ok := modifiers[name] return ok } // cleanWS remove any non-whitespace from string func cleanWS(s string) string { for i := 0; i < len(s); i++ { switch s[i] { case ' ', '\t', '\n', '\r': continue default: var s2 []byte for i := 0; i < len(s); i++ { switch s[i] { case ' ', '\t', '\n', '\r': s2 = append(s2, s[i]) } } return string(s2) } } return s } // @pretty modifier makes the json look nice. func modPretty(json, arg string) string { if len(arg) > 0 { opts := *pretty.DefaultOptions Parse(arg).ForEach(func(key, value Result) bool { switch key.String() { case "sortKeys": opts.SortKeys = value.Bool() case "indent": opts.Indent = cleanWS(value.String()) case "prefix": opts.Prefix = cleanWS(value.String()) case "width": opts.Width = int(value.Int()) } return true }) return bytesString(pretty.PrettyOptions(stringBytes(json), &opts)) } return bytesString(pretty.Pretty(stringBytes(json))) } // @this returns the current element. Can be used to retrieve the root element. func modThis(json, arg string) string { return json } // @ugly modifier removes all whitespace. func modUgly(json, arg string) string { return bytesString(pretty.Ugly(stringBytes(json))) } // @reverse reverses array elements or root object members. func modReverse(json, arg string) string { res := Parse(json) if res.IsArray() { var values []Result res.ForEach(func(_, value Result) bool { values = append(values, value) return true }) out := make([]byte, 0, len(json)) out = append(out, '[') for i, j := len(values)-1, 0; i >= 0; i, j = i-1, j+1 { if j > 0 { out = append(out, ',') } out = append(out, values[i].Raw...) } out = append(out, ']') return bytesString(out) } if res.IsObject() { var keyValues []Result res.ForEach(func(key, value Result) bool { keyValues = append(keyValues, key, value) return true }) out := make([]byte, 0, len(json)) out = append(out, '{') for i, j := len(keyValues)-2, 0; i >= 0; i, j = i-2, j+1 { if j > 0 { out = append(out, ',') } out = append(out, keyValues[i+0].Raw...) out = append(out, ':') out = append(out, keyValues[i+1].Raw...) } out = append(out, '}') return bytesString(out) } return json } // @flatten an array with child arrays. // // [1,[2],[3,4],[5,[6,7]]] -> [1,2,3,4,5,[6,7]] // // The {"deep":true} arg can be provide for deep flattening. // // [1,[2],[3,4],[5,[6,7]]] -> [1,2,3,4,5,6,7] // // The original json is returned when the json is not an array. func modFlatten(json, arg string) string { res := Parse(json) if !res.IsArray() { return json } var deep bool if arg != "" { Parse(arg).ForEach(func(key, value Result) bool { if key.String() == "deep" { deep = value.Bool() } return true }) } var out []byte out = append(out, '[') var idx int res.ForEach(func(_, value Result) bool { var raw string if value.IsArray() { if deep { raw = unwrap(modFlatten(value.Raw, arg)) } else { raw = unwrap(value.Raw) } } else { raw = value.Raw } raw = strings.TrimSpace(raw) if len(raw) > 0 { if idx > 0 { out = append(out, ',') } out = append(out, raw...) idx++ } return true }) out = append(out, ']') return bytesString(out) } // @keys extracts the keys from an object. // // {"first":"Tom","last":"Smith"} -> ["first","last"] func modKeys(json, arg string) string { v := Parse(json) if !v.Exists() { return "[]" } obj := v.IsObject() var out strings.Builder out.WriteByte('[') var i int v.ForEach(func(key, _ Result) bool { if i > 0 { out.WriteByte(',') } if obj { out.WriteString(key.Raw) } else { out.WriteString("null") } i++ return true }) out.WriteByte(']') return out.String() } // @values extracts the values from an object. // // {"first":"Tom","last":"Smith"} -> ["Tom","Smith"] func modValues(json, arg string) string { v := Parse(json) if !v.Exists() { return "[]" } if v.IsArray() { return json } var out strings.Builder out.WriteByte('[') var i int v.ForEach(func(_, value Result) bool { if i > 0 { out.WriteByte(',') } out.WriteString(value.Raw) i++ return true }) out.WriteByte(']') return out.String() } // @join multiple objects into a single object. // // [{"first":"Tom"},{"last":"Smith"}] -> {"first","Tom","last":"Smith"} // // The arg can be "true" to specify that duplicate keys should be preserved. // // [{"first":"Tom","age":37},{"age":41}] -> {"first","Tom","age":37,"age":41} // // Without preserved keys: // // [{"first":"Tom","age":37},{"age":41}] -> {"first","Tom","age":41} // // The original json is returned when the json is not an object. func modJoin(json, arg string) string { res := Parse(json) if !res.IsArray() { return json } var preserve bool if arg != "" { Parse(arg).ForEach(func(key, value Result) bool { if key.String() == "preserve" { preserve = value.Bool() } return true }) } var out []byte out = append(out, '{') if preserve { // Preserve duplicate keys. var idx int res.ForEach(func(_, value Result) bool { if !value.IsObject() { return true } if idx > 0 { out = append(out, ',') } out = append(out, unwrap(value.Raw)...) idx++ return true }) } else { // Deduplicate keys and generate an object with stable ordering. var keys []Result kvals := make(map[string]Result) res.ForEach(func(_, value Result) bool { if !value.IsObject() { return true } value.ForEach(func(key, value Result) bool { k := key.String() if _, ok := kvals[k]; !ok { keys = append(keys, key) } kvals[k] = value return true }) return true }) for i := 0; i < len(keys); i++ { if i > 0 { out = append(out, ',') } out = append(out, keys[i].Raw...) out = append(out, ':') out = append(out, kvals[keys[i].String()].Raw...) } } out = append(out, '}') return bytesString(out) } // @valid ensures that the json is valid before moving on. An empty string is // returned when the json is not valid, otherwise it returns the original json. func modValid(json, arg string) string { if !Valid(json) { return "" } return json } // @fromstr converts a string to json // // "{\"id\":1023,\"name\":\"alert\"}" -> {"id":1023,"name":"alert"} func modFromStr(json, arg string) string { if !Valid(json) { return "" } return Parse(json).String() } // @tostr converts a string to json // // {"id":1023,"name":"alert"} -> "{\"id\":1023,\"name\":\"alert\"}" func modToStr(str, arg string) string { return string(AppendJSONString(nil, str)) } func modGroup(json, arg string) string { res := Parse(json) if !res.IsObject() { return "" } var all [][]byte res.ForEach(func(key, value Result) bool { if !value.IsArray() { return true } var idx int value.ForEach(func(_, value Result) bool { if idx == len(all) { all = append(all, []byte{}) } all[idx] = append(all[idx], ("," + key.Raw + ":" + value.Raw)...) idx++ return true }) return true }) var data []byte data = append(data, '[') for i, item := range all { if i > 0 { data = append(data, ',') } data = append(data, '{') data = append(data, item[1:]...) data = append(data, '}') } data = append(data, ']') return string(data) } // stringHeader instead of reflect.StringHeader type stringHeader struct { data unsafe.Pointer len int } // sliceHeader instead of reflect.SliceHeader type sliceHeader struct { data unsafe.Pointer len int cap int } // getBytes casts the input json bytes to a string and safely returns the // results as uniquely allocated data. This operation is intended to minimize // copies and allocations for the large json string->[]byte. func getBytes(json []byte, path string) Result { var result Result if json != nil { // unsafe cast to string result = Get(*(*string)(unsafe.Pointer(&json)), path) // safely get the string headers rawhi := *(*stringHeader)(unsafe.Pointer(&result.Raw)) strhi := *(*stringHeader)(unsafe.Pointer(&result.Str)) // create byte slice headers rawh := sliceHeader{data: rawhi.data, len: rawhi.len, cap: rawhi.len} strh := sliceHeader{data: strhi.data, len: strhi.len, cap: rawhi.len} if strh.data == nil { // str is nil if rawh.data == nil { // raw is nil result.Raw = "" } else { // raw has data, safely copy the slice header to a string result.Raw = string(*(*[]byte)(unsafe.Pointer(&rawh))) } result.Str = "" } else if rawh.data == nil { // raw is nil result.Raw = "" // str has data, safely copy the slice header to a string result.Str = string(*(*[]byte)(unsafe.Pointer(&strh))) } else if uintptr(strh.data) >= uintptr(rawh.data) && uintptr(strh.data)+uintptr(strh.len) <= uintptr(rawh.data)+uintptr(rawh.len) { // Str is a substring of Raw. start := uintptr(strh.data) - uintptr(rawh.data) // safely copy the raw slice header result.Raw = string(*(*[]byte)(unsafe.Pointer(&rawh))) // substring the raw result.Str = result.Raw[start : start+uintptr(strh.len)] } else { // safely copy both the raw and str slice headers to strings result.Raw = string(*(*[]byte)(unsafe.Pointer(&rawh))) result.Str = string(*(*[]byte)(unsafe.Pointer(&strh))) } } return result } // fillIndex finds the position of Raw data and assigns it to the Index field // of the resulting value. If the position cannot be found then Index zero is // used instead. func fillIndex(json string, c *parseContext) { if len(c.value.Raw) > 0 && !c.calcd { jhdr := *(*stringHeader)(unsafe.Pointer(&json)) rhdr := *(*stringHeader)(unsafe.Pointer(&(c.value.Raw))) c.value.Index = int(uintptr(rhdr.data) - uintptr(jhdr.data)) if c.value.Index < 0 || c.value.Index >= len(json) { c.value.Index = 0 } } } func stringBytes(s string) []byte { return *(*[]byte)(unsafe.Pointer(&sliceHeader{ data: (*stringHeader)(unsafe.Pointer(&s)).data, len: len(s), cap: len(s), })) } func bytesString(b []byte) string { return *(*string)(unsafe.Pointer(&b)) } func revSquash(json string) string { // reverse squash // expects that the tail character is a ']' or '}' or ')' or '"' // squash the value, ignoring all nested arrays and objects. i := len(json) - 1 var depth int if json[i] != '"' { depth++ } if json[i] == '}' || json[i] == ']' || json[i] == ')' { i-- } for ; i >= 0; i-- { switch json[i] { case '"': i-- for ; i >= 0; i-- { if json[i] == '"' { esc := 0 for i > 0 && json[i-1] == '\\' { i-- esc++ } if esc%2 == 1 { continue } i += esc break } } if depth == 0 { if i < 0 { i = 0 } return json[i:] } case '}', ']', ')': depth++ case '{', '[', '(': depth-- if depth == 0 { return json[i:] } } } return json } // Paths returns the original GJSON paths for a Result where the Result came // from a simple query path that returns an array, like: // // gjson.Get(json, "friends.#.first") // // The returned value will be in the form of a JSON array: // // ["friends.0.first","friends.1.first","friends.2.first"] // // The param 'json' must be the original JSON used when calling Get. // // Returns an empty string if the paths cannot be determined, which can happen // when the Result came from a path that contained a multipath, modifier, // or a nested query. func (t Result) Paths(json string) []string { if t.Indexes == nil { return nil } paths := make([]string, 0, len(t.Indexes)) t.ForEach(func(_, value Result) bool { paths = append(paths, value.Path(json)) return true }) if len(paths) != len(t.Indexes) { return nil } return paths } // Path returns the original GJSON path for a Result where the Result came // from a simple path that returns a single value, like: // // gjson.Get(json, "friends.#(last=Murphy)") // // The returned value will be in the form of a JSON string: // // "friends.0" // // The param 'json' must be the original JSON used when calling Get. // // Returns an empty string if the paths cannot be determined, which can happen // when the Result came from a path that contained a multipath, modifier, // or a nested query. func (t Result) Path(json string) string { var path []byte var comps []string // raw components i := t.Index - 1 if t.Index+len(t.Raw) > len(json) { // JSON cannot safely contain Result. goto fail } if !strings.HasPrefix(json[t.Index:], t.Raw) { // Result is not at the JSON index as exepcted. goto fail } for ; i >= 0; i-- { if json[i] <= ' ' { continue } if json[i] == ':' { // inside of object, get the key for ; i >= 0; i-- { if json[i] != '"' { continue } break } raw := revSquash(json[:i+1]) i = i - len(raw) comps = append(comps, raw) // key gotten, now squash the rest raw = revSquash(json[:i+1]) i = i - len(raw) i++ // increment the index for next loop step } else if json[i] == '{' { // Encountered an open object. The original result was probably an // object key. goto fail } else if json[i] == ',' || json[i] == '[' { // inside of an array, count the position var arrIdx int if json[i] == ',' { arrIdx++ i-- } for ; i >= 0; i-- { if json[i] == ':' { // Encountered an unexpected colon. The original result was // probably an object key. goto fail } else if json[i] == ',' { arrIdx++ } else if json[i] == '[' { comps = append(comps, strconv.Itoa(arrIdx)) break } else if json[i] == ']' || json[i] == '}' || json[i] == '"' { raw := revSquash(json[:i+1]) i = i - len(raw) + 1 } } } } if len(comps) == 0 { if DisableModifiers { goto fail } return "@this" } for i := len(comps) - 1; i >= 0; i-- { rcomp := Parse(comps[i]) if !rcomp.Exists() { goto fail } comp := Escape(rcomp.String()) path = append(path, '.') path = append(path, comp...) } if len(path) > 0 { path = path[1:] } return string(path) fail: return "" } // isSafePathKeyChar returns true if the input character is safe for not // needing escaping. func isSafePathKeyChar(c byte) bool { return (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c <= ' ' || c > '~' || c == '_' || c == '-' || c == ':' } // Escape returns an escaped path component. // // json := `{ // "user":{ // "first.name": "Janet", // "last.name": "Prichard" // } // }` // user := gjson.Get(json, "user") // println(user.Get(gjson.Escape("first.name")) // println(user.Get(gjson.Escape("last.name")) // // Output: // // Janet // // Prichard func Escape(comp string) string { for i := 0; i < len(comp); i++ { if !isSafePathKeyChar(comp[i]) { ncomp := make([]byte, len(comp)+1) copy(ncomp, comp[:i]) ncomp = ncomp[:i] for ; i < len(comp); i++ { if !isSafePathKeyChar(comp[i]) { ncomp = append(ncomp, '\\') } ncomp = append(ncomp, comp[i]) } return string(ncomp) } } return comp } func parseRecursiveDescent(all []Result, parent Result, path string) []Result { if res := parent.Get(path); res.Exists() { all = append(all, res) } if parent.IsArray() || parent.IsObject() { parent.ForEach(func(_, val Result) bool { all = parseRecursiveDescent(all, val, path) return true }) } return all } func modDig(json, arg string) string { all := parseRecursiveDescent(nil, Parse(json), arg) var out []byte out = append(out, '[') for i, res := range all { if i > 0 { out = append(out, ',') } out = append(out, res.Raw...) } out = append(out, ']') return string(out) } gjson-1.17.1/gjson_test.go000066400000000000000000002264461456330511400154470ustar00rootroot00000000000000package gjson import ( "bytes" "encoding/hex" "encoding/json" "fmt" "math" "math/rand" "strconv" "strings" "testing" "time" "github.com/tidwall/pretty" ) // TestRandomData is a fuzzing test that throws random data at the Parse // function looking for panics. func TestRandomData(t *testing.T) { var lstr string defer func() { if v := recover(); v != nil { println("'" + hex.EncodeToString([]byte(lstr)) + "'") println("'" + lstr + "'") panic(v) } }() rand.Seed(time.Now().UnixNano()) b := make([]byte, 200) for i := 0; i < 2000000; i++ { n, err := rand.Read(b[:rand.Int()%len(b)]) if err != nil { t.Fatal(err) } lstr = string(b[:n]) GetBytes([]byte(lstr), "zzzz") Parse(lstr) } } func TestRandomValidStrings(t *testing.T) { rand.Seed(time.Now().UnixNano()) b := make([]byte, 200) for i := 0; i < 100000; i++ { n, err := rand.Read(b[:rand.Int()%len(b)]) if err != nil { t.Fatal(err) } sm, err := json.Marshal(string(b[:n])) if err != nil { t.Fatal(err) } var su string if err := json.Unmarshal([]byte(sm), &su); err != nil { t.Fatal(err) } token := Get(`{"str":`+string(sm)+`}`, "str") if token.Type != String || token.Str != su { println("["+token.Raw+"]", "["+token.Str+"]", "["+su+"]", "["+string(sm)+"]") t.Fatal("string mismatch") } } } func TestEmoji(t *testing.T) { const input = `{"utf8":"Example emoji, KO: \ud83d\udd13, \ud83c\udfc3 ` + `OK: \u2764\ufe0f "}` value := Get(input, "utf8") var s string json.Unmarshal([]byte(value.Raw), &s) if value.String() != s { t.Fatalf("expected '%v', got '%v'", s, value.String()) } } func testEscapePath(t *testing.T, json, path, expect string) { if Get(json, path).String() != expect { t.Fatalf("expected '%v', got '%v'", expect, Get(json, path).String()) } } func TestEscapePath(t *testing.T) { json := `{ "test":{ "*":"valZ", "*v":"val0", "keyv*":"val1", "key*v":"val2", "keyv?":"val3", "key?v":"val4", "keyv.":"val5", "key.v":"val6", "keyk*":{"key?":"val7"} } }` testEscapePath(t, json, "test.\\*", "valZ") testEscapePath(t, json, "test.\\*v", "val0") testEscapePath(t, json, "test.keyv\\*", "val1") testEscapePath(t, json, "test.key\\*v", "val2") testEscapePath(t, json, "test.keyv\\?", "val3") testEscapePath(t, json, "test.key\\?v", "val4") testEscapePath(t, json, "test.keyv\\.", "val5") testEscapePath(t, json, "test.key\\.v", "val6") testEscapePath(t, json, "test.keyk\\*.key\\?", "val7") } // this json block is poorly formed on purpose. var basicJSON = ` {"age":100, "name":{"here":"B\\\"R"}, "noop":{"what is a wren?":"a bird"}, "happy":true,"immortal":false, "items":[1,2,3,{"tags":[1,2,3],"points":[[1,2],[3,4]]},4,5,6,7], "arr":["1",2,"3",{"hello":"world"},"4",5], "vals":[1,2,3,{"sadf":sdf"asdf"}],"name":{"first":"tom","last":null}, "created":"2014-05-16T08:28:06.989Z", "loggy":{ "programmers": [ { "firstName": "Brett", "lastName": "McLaughlin", "email": "aaaa", "tag": "good" }, { "firstName": "Jason", "lastName": "Hunter", "email": "bbbb", "tag": "bad" }, { "firstName": "Elliotte", "lastName": "Harold", "email": "cccc", "tag":, "good" }, { "firstName": 1002.3, "age": 101 } ] }, "lastly":{"end...ing":"soon","yay":"final"} }` func TestPath(t *testing.T) { json := basicJSON r := Get(json, "@this") path := r.Path(json) if path != "@this" { t.FailNow() } r = Parse(json) path = r.Path(json) if path != "@this" { t.FailNow() } obj := Parse(json) obj.ForEach(func(key, val Result) bool { kp := key.Path(json) assert(t, kp == "") vp := val.Path(json) if vp == "name" { // there are two "name" keys return true } val2 := obj.Get(vp) assert(t, val2.Raw == val.Raw) return true }) arr := obj.Get("loggy.programmers") arr.ForEach(func(_, val Result) bool { vp := val.Path(json) val2 := Get(json, vp) assert(t, val2.Raw == val.Raw) return true }) get := func(path string) { r1 := Get(json, path) path2 := r1.Path(json) r2 := Get(json, path2) assert(t, r1.Raw == r2.Raw) } get("age") get("name") get("name.here") get("noop") get("noop.what is a wren?") get("arr.0") get("arr.1") get("arr.2") get("arr.3") get("arr.3.hello") get("arr.4") get("arr.5") get("loggy.programmers.2.email") get("lastly.end\\.\\.\\.ing") get("lastly.yay") } func TestTimeResult(t *testing.T) { assert(t, Get(basicJSON, "created").String() == Get(basicJSON, "created").Time().Format(time.RFC3339Nano)) } func TestParseAny(t *testing.T) { assert(t, Parse("100").Float() == 100) assert(t, Parse("true").Bool()) assert(t, Parse("false").Bool() == false) assert(t, Parse("yikes").Exists() == false) } func TestManyVariousPathCounts(t *testing.T) { json := `{"a":"a","b":"b","c":"c"}` counts := []int{3, 4, 7, 8, 9, 15, 16, 17, 31, 32, 33, 63, 64, 65, 127, 128, 129, 255, 256, 257, 511, 512, 513} paths := []string{"a", "b", "c"} expects := []string{"a", "b", "c"} for _, count := range counts { var gpaths []string for i := 0; i < count; i++ { if i < len(paths) { gpaths = append(gpaths, paths[i]) } else { gpaths = append(gpaths, fmt.Sprintf("not%d", i)) } } results := GetMany(json, gpaths...) for i := 0; i < len(paths); i++ { if results[i].String() != expects[i] { t.Fatalf("expected '%v', got '%v'", expects[i], results[i].String()) } } } } func TestManyRecursion(t *testing.T) { var json string var path string for i := 0; i < 100; i++ { json += `{"a":` path += ".a" } json += `"b"` for i := 0; i < 100; i++ { json += `}` } path = path[1:] assert(t, GetMany(json, path)[0].String() == "b") } func TestByteSafety(t *testing.T) { jsonb := []byte(`{"name":"Janet","age":38}`) mtok := GetBytes(jsonb, "name") if mtok.String() != "Janet" { t.Fatalf("expected %v, got %v", "Jason", mtok.String()) } mtok2 := GetBytes(jsonb, "age") if mtok2.Raw != "38" { t.Fatalf("expected %v, got %v", "Jason", mtok2.Raw) } jsonb[9] = 'T' jsonb[12] = 'd' jsonb[13] = 'y' if mtok.String() != "Janet" { t.Fatalf("expected %v, got %v", "Jason", mtok.String()) } } func get(json, path string) Result { return GetBytes([]byte(json), path) } func TestBasic(t *testing.T) { var mtok Result mtok = get(basicJSON, `loggy.programmers.#[tag="good"].firstName`) if mtok.String() != "Brett" { t.Fatalf("expected %v, got %v", "Brett", mtok.String()) } mtok = get(basicJSON, `loggy.programmers.#[tag="good"]#.firstName`) if mtok.String() != `["Brett","Elliotte"]` { t.Fatalf("expected %v, got %v", `["Brett","Elliotte"]`, mtok.String()) } } func TestIsArrayIsObject(t *testing.T) { mtok := get(basicJSON, "loggy") assert(t, mtok.IsObject()) assert(t, !mtok.IsArray()) mtok = get(basicJSON, "loggy.programmers") assert(t, !mtok.IsObject()) assert(t, mtok.IsArray()) mtok = get(basicJSON, `loggy.programmers.#[tag="good"]#.firstName`) assert(t, mtok.IsArray()) mtok = get(basicJSON, `loggy.programmers.0.firstName`) assert(t, !mtok.IsObject()) assert(t, !mtok.IsArray()) } func TestPlus53BitInts(t *testing.T) { json := `{"IdentityData":{"GameInstanceId":634866135153775564}}` value := Get(json, "IdentityData.GameInstanceId") assert(t, value.Uint() == 634866135153775564) assert(t, value.Int() == 634866135153775564) assert(t, value.Float() == 634866135153775616) json = `{"IdentityData":{"GameInstanceId":634866135153775564.88172}}` value = Get(json, "IdentityData.GameInstanceId") assert(t, value.Uint() == 634866135153775616) assert(t, value.Int() == 634866135153775616) assert(t, value.Float() == 634866135153775616.88172) json = `{ "min_uint64": 0, "max_uint64": 18446744073709551615, "overflow_uint64": 18446744073709551616, "min_int64": -9223372036854775808, "max_int64": 9223372036854775807, "overflow_int64": 9223372036854775808, "min_uint53": 0, "max_uint53": 4503599627370495, "overflow_uint53": 4503599627370496, "min_int53": -2251799813685248, "max_int53": 2251799813685247, "overflow_int53": 2251799813685248 }` assert(t, Get(json, "min_uint53").Uint() == 0) assert(t, Get(json, "max_uint53").Uint() == 4503599627370495) assert(t, Get(json, "overflow_uint53").Int() == 4503599627370496) assert(t, Get(json, "min_int53").Int() == -2251799813685248) assert(t, Get(json, "max_int53").Int() == 2251799813685247) assert(t, Get(json, "overflow_int53").Int() == 2251799813685248) assert(t, Get(json, "min_uint64").Uint() == 0) assert(t, Get(json, "max_uint64").Uint() == 18446744073709551615) // this next value overflows the max uint64 by one which will just // flip the number to zero assert(t, Get(json, "overflow_uint64").Int() == 0) assert(t, Get(json, "min_int64").Int() == -9223372036854775808) assert(t, Get(json, "max_int64").Int() == 9223372036854775807) // this next value overflows the max int64 by one which will just // flip the number to the negative sign. assert(t, Get(json, "overflow_int64").Int() == -9223372036854775808) } func TestIssue38(t *testing.T) { // These should not fail, even though the unicode is invalid. Get(`["S3O PEDRO DO BUTI\udf93"]`, "0") Get(`["S3O PEDRO DO BUTI\udf93asdf"]`, "0") Get(`["S3O PEDRO DO BUTI\udf93\u"]`, "0") Get(`["S3O PEDRO DO BUTI\udf93\u1"]`, "0") Get(`["S3O PEDRO DO BUTI\udf93\u13"]`, "0") Get(`["S3O PEDRO DO BUTI\udf93\u134"]`, "0") Get(`["S3O PEDRO DO BUTI\udf93\u1345"]`, "0") Get(`["S3O PEDRO DO BUTI\udf93\u1345asd"]`, "0") } func TestTypes(t *testing.T) { assert(t, (Result{Type: String}).Type.String() == "String") assert(t, (Result{Type: Number}).Type.String() == "Number") assert(t, (Result{Type: Null}).Type.String() == "Null") assert(t, (Result{Type: False}).Type.String() == "False") assert(t, (Result{Type: True}).Type.String() == "True") assert(t, (Result{Type: JSON}).Type.String() == "JSON") assert(t, (Result{Type: 100}).Type.String() == "") // bool assert(t, (Result{Type: True}).Bool() == true) assert(t, (Result{Type: False}).Bool() == false) assert(t, (Result{Type: Number, Num: 1}).Bool() == true) assert(t, (Result{Type: Number, Num: 0}).Bool() == false) assert(t, (Result{Type: String, Str: "1"}).Bool() == true) assert(t, (Result{Type: String, Str: "T"}).Bool() == true) assert(t, (Result{Type: String, Str: "t"}).Bool() == true) assert(t, (Result{Type: String, Str: "true"}).Bool() == true) assert(t, (Result{Type: String, Str: "True"}).Bool() == true) assert(t, (Result{Type: String, Str: "TRUE"}).Bool() == true) assert(t, (Result{Type: String, Str: "tRuE"}).Bool() == true) assert(t, (Result{Type: String, Str: "0"}).Bool() == false) assert(t, (Result{Type: String, Str: "f"}).Bool() == false) assert(t, (Result{Type: String, Str: "F"}).Bool() == false) assert(t, (Result{Type: String, Str: "false"}).Bool() == false) assert(t, (Result{Type: String, Str: "False"}).Bool() == false) assert(t, (Result{Type: String, Str: "FALSE"}).Bool() == false) assert(t, (Result{Type: String, Str: "fAlSe"}).Bool() == false) assert(t, (Result{Type: String, Str: "random"}).Bool() == false) // int assert(t, (Result{Type: String, Str: "1"}).Int() == 1) assert(t, (Result{Type: True}).Int() == 1) assert(t, (Result{Type: False}).Int() == 0) assert(t, (Result{Type: Number, Num: 1}).Int() == 1) // uint assert(t, (Result{Type: String, Str: "1"}).Uint() == 1) assert(t, (Result{Type: True}).Uint() == 1) assert(t, (Result{Type: False}).Uint() == 0) assert(t, (Result{Type: Number, Num: 1}).Uint() == 1) // float assert(t, (Result{Type: String, Str: "1"}).Float() == 1) assert(t, (Result{Type: True}).Float() == 1) assert(t, (Result{Type: False}).Float() == 0) assert(t, (Result{Type: Number, Num: 1}).Float() == 1) } func TestForEach(t *testing.T) { Result{}.ForEach(nil) Result{Type: String, Str: "Hello"}.ForEach(func(_, value Result) bool { assert(t, value.String() == "Hello") return false }) Result{Type: JSON, Raw: "*invalid*"}.ForEach(nil) json := ` {"name": {"first": "Janet","last": "Prichard"}, "asd\nf":"\ud83d\udd13","age": 47}` var count int ParseBytes([]byte(json)).ForEach(func(key, value Result) bool { count++ return true }) assert(t, count == 3) ParseBytes([]byte(`{"bad`)).ForEach(nil) ParseBytes([]byte(`{"ok":"bad`)).ForEach(nil) } func TestMap(t *testing.T) { assert(t, len(ParseBytes([]byte(`"asdf"`)).Map()) == 0) assert(t, ParseBytes([]byte(`{"asdf":"ghjk"`)).Map()["asdf"].String() == "ghjk") assert(t, len(Result{Type: JSON, Raw: "**invalid**"}.Map()) == 0) assert(t, Result{Type: JSON, Raw: "**invalid**"}.Value() == nil) assert(t, Result{Type: JSON, Raw: "{"}.Map() != nil) } func TestBasic1(t *testing.T) { mtok := get(basicJSON, `loggy.programmers`) var count int mtok.ForEach(func(key, value Result) bool { assert(t, key.Exists()) assert(t, key.String() == fmt.Sprint(count)) assert(t, key.Int() == int64(count)) count++ if count == 3 { return false } if count == 1 { i := 0 value.ForEach(func(key, value Result) bool { switch i { case 0: if key.String() != "firstName" || value.String() != "Brett" { t.Fatalf("expected %v/%v got %v/%v", "firstName", "Brett", key.String(), value.String()) } case 1: if key.String() != "lastName" || value.String() != "McLaughlin" { t.Fatalf("expected %v/%v got %v/%v", "lastName", "McLaughlin", key.String(), value.String()) } case 2: if key.String() != "email" || value.String() != "aaaa" { t.Fatalf("expected %v/%v got %v/%v", "email", "aaaa", key.String(), value.String()) } } i++ return true }) } return true }) if count != 3 { t.Fatalf("expected %v, got %v", 3, count) } } func TestBasic2(t *testing.T) { mtok := get(basicJSON, `loggy.programmers.#[age=101].firstName`) if mtok.String() != "1002.3" { t.Fatalf("expected %v, got %v", "1002.3", mtok.String()) } mtok = get(basicJSON, `loggy.programmers.#[firstName != "Brett"].firstName`) if mtok.String() != "Jason" { t.Fatalf("expected %v, got %v", "Jason", mtok.String()) } mtok = get(basicJSON, `loggy.programmers.#[firstName % "Bre*"].email`) if mtok.String() != "aaaa" { t.Fatalf("expected %v, got %v", "aaaa", mtok.String()) } mtok = get(basicJSON, `loggy.programmers.#[firstName !% "Bre*"].email`) if mtok.String() != "bbbb" { t.Fatalf("expected %v, got %v", "bbbb", mtok.String()) } mtok = get(basicJSON, `loggy.programmers.#[firstName == "Brett"].email`) if mtok.String() != "aaaa" { t.Fatalf("expected %v, got %v", "aaaa", mtok.String()) } mtok = get(basicJSON, "loggy") if mtok.Type != JSON { t.Fatalf("expected %v, got %v", JSON, mtok.Type) } if len(mtok.Map()) != 1 { t.Fatalf("expected %v, got %v", 1, len(mtok.Map())) } programmers := mtok.Map()["programmers"] if programmers.Array()[1].Map()["firstName"].Str != "Jason" { t.Fatalf("expected %v, got %v", "Jason", mtok.Map()["programmers"].Array()[1].Map()["firstName"].Str) } } func TestBasic3(t *testing.T) { var mtok Result if Parse(basicJSON).Get("loggy.programmers").Get("1"). Get("firstName").Str != "Jason" { t.Fatalf("expected %v, got %v", "Jason", Parse(basicJSON). Get("loggy.programmers").Get("1").Get("firstName").Str) } var token Result if token = Parse("-102"); token.Num != -102 { t.Fatalf("expected %v, got %v", -102, token.Num) } if token = Parse("102"); token.Num != 102 { t.Fatalf("expected %v, got %v", 102, token.Num) } if token = Parse("102.2"); token.Num != 102.2 { t.Fatalf("expected %v, got %v", 102.2, token.Num) } if token = Parse(`"hello"`); token.Str != "hello" { t.Fatalf("expected %v, got %v", "hello", token.Str) } if token = Parse(`"\"he\nllo\""`); token.Str != "\"he\nllo\"" { t.Fatalf("expected %v, got %v", "\"he\nllo\"", token.Str) } mtok = get(basicJSON, "loggy.programmers.#.firstName") if len(mtok.Array()) != 4 { t.Fatalf("expected 4, got %v", len(mtok.Array())) } for i, ex := range []string{"Brett", "Jason", "Elliotte", "1002.3"} { if mtok.Array()[i].String() != ex { t.Fatalf("expected '%v', got '%v'", ex, mtok.Array()[i].String()) } } mtok = get(basicJSON, "loggy.programmers.#.asd") if mtok.Type != JSON { t.Fatalf("expected %v, got %v", JSON, mtok.Type) } if len(mtok.Array()) != 0 { t.Fatalf("expected 0, got %v", len(mtok.Array())) } } func TestBasic4(t *testing.T) { if get(basicJSON, "items.3.tags.#").Num != 3 { t.Fatalf("expected 3, got %v", get(basicJSON, "items.3.tags.#").Num) } if get(basicJSON, "items.3.points.1.#").Num != 2 { t.Fatalf("expected 2, got %v", get(basicJSON, "items.3.points.1.#").Num) } if get(basicJSON, "items.#").Num != 8 { t.Fatalf("expected 6, got %v", get(basicJSON, "items.#").Num) } if get(basicJSON, "vals.#").Num != 4 { t.Fatalf("expected 4, got %v", get(basicJSON, "vals.#").Num) } if !get(basicJSON, "name.last").Exists() { t.Fatal("expected true, got false") } token := get(basicJSON, "name.here") if token.String() != "B\\\"R" { t.Fatal("expecting 'B\\\"R'", "got", token.String()) } token = get(basicJSON, "arr.#") if token.String() != "6" { fmt.Printf("%#v\n", token) t.Fatal("expecting 6", "got", token.String()) } token = get(basicJSON, "arr.3.hello") if token.String() != "world" { t.Fatal("expecting 'world'", "got", token.String()) } _ = token.Value().(string) token = get(basicJSON, "name.first") if token.String() != "tom" { t.Fatal("expecting 'tom'", "got", token.String()) } _ = token.Value().(string) token = get(basicJSON, "name.last") if token.String() != "" { t.Fatal("expecting ''", "got", token.String()) } if token.Value() != nil { t.Fatal("should be nil") } } func TestBasic5(t *testing.T) { token := get(basicJSON, "age") if token.String() != "100" { t.Fatal("expecting '100'", "got", token.String()) } _ = token.Value().(float64) token = get(basicJSON, "happy") if token.String() != "true" { t.Fatal("expecting 'true'", "got", token.String()) } _ = token.Value().(bool) token = get(basicJSON, "immortal") if token.String() != "false" { t.Fatal("expecting 'false'", "got", token.String()) } _ = token.Value().(bool) token = get(basicJSON, "noop") if token.String() != `{"what is a wren?":"a bird"}` { t.Fatal("expecting '"+`{"what is a wren?":"a bird"}`+"'", "got", token.String()) } _ = token.Value().(map[string]interface{}) if get(basicJSON, "").Value() != nil { t.Fatal("should be nil") } get(basicJSON, "vals.hello") type msi = map[string]interface{} type fi = []interface{} mm := Parse(basicJSON).Value().(msi) fn := mm["loggy"].(msi)["programmers"].(fi)[1].(msi)["firstName"].(string) if fn != "Jason" { t.Fatalf("expecting %v, got %v", "Jason", fn) } } func TestUnicode(t *testing.T) { var json = `{"key":0,"的情况下解":{"key":1,"的情况":2}}` if Get(json, "的情况下解.key").Num != 1 { t.Fatal("fail") } if Get(json, "的情况下解.的情况").Num != 2 { t.Fatal("fail") } if Get(json, "的情况下解.的?况").Num != 2 { t.Fatal("fail") } if Get(json, "的情况下解.的?*").Num != 2 { t.Fatal("fail") } if Get(json, "的情况下解.*?况").Num != 2 { t.Fatal("fail") } if Get(json, "的情?下解.*?况").Num != 2 { t.Fatal("fail") } if Get(json, "的情下解.*?况").Num != 0 { t.Fatal("fail") } } func TestUnescape(t *testing.T) { unescape(string([]byte{'\\', '\\', 0})) unescape(string([]byte{'\\', '/', '\\', 'b', '\\', 'f'})) } func assert(t testing.TB, cond bool) { if !cond { panic("assert failed") } } func TestLess(t *testing.T) { assert(t, !Result{Type: Null}.Less(Result{Type: Null}, true)) assert(t, Result{Type: Null}.Less(Result{Type: False}, true)) assert(t, Result{Type: Null}.Less(Result{Type: True}, true)) assert(t, Result{Type: Null}.Less(Result{Type: JSON}, true)) assert(t, Result{Type: Null}.Less(Result{Type: Number}, true)) assert(t, Result{Type: Null}.Less(Result{Type: String}, true)) assert(t, !Result{Type: False}.Less(Result{Type: Null}, true)) assert(t, Result{Type: False}.Less(Result{Type: True}, true)) assert(t, Result{Type: String, Str: "abc"}.Less(Result{Type: String, Str: "bcd"}, true)) assert(t, Result{Type: String, Str: "ABC"}.Less(Result{Type: String, Str: "abc"}, true)) assert(t, !Result{Type: String, Str: "ABC"}.Less(Result{Type: String, Str: "abc"}, false)) assert(t, Result{Type: Number, Num: 123}.Less(Result{Type: Number, Num: 456}, true)) assert(t, !Result{Type: Number, Num: 456}.Less(Result{Type: Number, Num: 123}, true)) assert(t, !Result{Type: Number, Num: 456}.Less(Result{Type: Number, Num: 456}, true)) assert(t, stringLessInsensitive("abcde", "BBCDE")) assert(t, stringLessInsensitive("abcde", "bBCDE")) assert(t, stringLessInsensitive("Abcde", "BBCDE")) assert(t, stringLessInsensitive("Abcde", "bBCDE")) assert(t, !stringLessInsensitive("bbcde", "aBCDE")) assert(t, !stringLessInsensitive("bbcde", "ABCDE")) assert(t, !stringLessInsensitive("Bbcde", "aBCDE")) assert(t, !stringLessInsensitive("Bbcde", "ABCDE")) assert(t, !stringLessInsensitive("abcde", "ABCDE")) assert(t, !stringLessInsensitive("Abcde", "ABCDE")) assert(t, !stringLessInsensitive("abcde", "ABCDE")) assert(t, !stringLessInsensitive("ABCDE", "ABCDE")) assert(t, !stringLessInsensitive("abcde", "abcde")) assert(t, !stringLessInsensitive("123abcde", "123Abcde")) assert(t, !stringLessInsensitive("123Abcde", "123Abcde")) assert(t, !stringLessInsensitive("123Abcde", "123abcde")) assert(t, !stringLessInsensitive("123abcde", "123abcde")) assert(t, !stringLessInsensitive("124abcde", "123abcde")) assert(t, !stringLessInsensitive("124Abcde", "123Abcde")) assert(t, !stringLessInsensitive("124Abcde", "123abcde")) assert(t, !stringLessInsensitive("124abcde", "123abcde")) assert(t, stringLessInsensitive("124abcde", "125abcde")) assert(t, stringLessInsensitive("124Abcde", "125Abcde")) assert(t, stringLessInsensitive("124Abcde", "125abcde")) assert(t, stringLessInsensitive("124abcde", "125abcde")) } func TestIssue6(t *testing.T) { data := `{ "code": 0, "msg": "", "data": { "sz002024": { "qfqday": [ [ "2014-01-02", "8.93", "9.03", "9.17", "8.88", "621143.00" ], [ "2014-01-03", "9.03", "9.30", "9.47", "8.98", "1624438.00" ] ] } } }` var num []string for _, v := range Get(data, "data.sz002024.qfqday.0").Array() { num = append(num, v.String()) } if fmt.Sprintf("%v", num) != "[2014-01-02 8.93 9.03 9.17 8.88 621143.00]" { t.Fatalf("invalid result") } } var exampleJSON = `{ "widget": { "debug": "on", "window": { "title": "Sample Konfabulator Widget", "name": "main_window", "width": 500, "height": 500 }, "image": { "src": "Images/Sun.png", "hOffset": 250, "vOffset": 250, "alignment": "center" }, "text": { "data": "Click Here", "size": 36, "style": "bold", "vOffset": 100, "alignment": "center", "onMouseUp": "sun1.opacity = (sun1.opacity / 100) * 90;" } } }` func TestUnmarshalMap(t *testing.T) { var m1 = Parse(exampleJSON).Value().(map[string]interface{}) var m2 map[string]interface{} if err := json.Unmarshal([]byte(exampleJSON), &m2); err != nil { t.Fatal(err) } b1, err := json.Marshal(m1) if err != nil { t.Fatal(err) } b2, err := json.Marshal(m2) if err != nil { t.Fatal(err) } if !bytes.Equal(b1, b2) { t.Fatal("b1 != b2") } } func TestSingleArrayValue(t *testing.T) { var json = `{"key": "value","key2":[1,2,3,4,"A"]}` var result = Get(json, "key") var array = result.Array() if len(array) != 1 { t.Fatal("array is empty") } if array[0].String() != "value" { t.Fatalf("got %s, should be %s", array[0].String(), "value") } array = Get(json, "key2.#").Array() if len(array) != 1 { t.Fatalf("got '%v', expected '%v'", len(array), 1) } array = Get(json, "key3").Array() if len(array) != 0 { t.Fatalf("got '%v', expected '%v'", len(array), 0) } } var manyJSON = ` { "a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{ "a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{ "a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{ "a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{ "a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{ "a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{ "a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":{"hello":"world" }}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}} "position":{"type":"Point","coordinates":[-115.24,33.09]}, "loves":["world peace"], "name":{"last":"Anderson","first":"Nancy"}, "age":31 "":{"a":"emptya","b":"emptyb"}, "name.last":"Yellow", "name.first":"Cat", }` var testWatchForFallback bool func TestManyBasic(t *testing.T) { testWatchForFallback = true defer func() { testWatchForFallback = false }() testMany := func(shouldFallback bool, expect string, paths ...string) { results := GetManyBytes( []byte(manyJSON), paths..., ) if len(results) != len(paths) { t.Fatalf("expected %v, got %v", len(paths), len(results)) } if fmt.Sprintf("%v", results) != expect { fmt.Printf("%v\n", paths) t.Fatalf("expected %v, got %v", expect, results) } } testMany(false, "[Point]", "position.type") testMany(false, `[emptya ["world peace"] 31]`, ".a", "loves", "age") testMany(false, `[["world peace"]]`, "loves") testMany(false, `[{"last":"Anderson","first":"Nancy"} Nancy]`, "name", "name.first") testMany(true, `[]`, strings.Repeat("a.", 40)+"hello") res := Get(manyJSON, strings.Repeat("a.", 48)+"a") testMany(true, `[`+res.String()+`]`, strings.Repeat("a.", 48)+"a") // these should fallback testMany(true, `[Cat Nancy]`, "name\\.first", "name.first") testMany(true, `[world]`, strings.Repeat("a.", 70)+"hello") } func testMany(t *testing.T, json string, paths, expected []string) { testManyAny(t, json, paths, expected, true) testManyAny(t, json, paths, expected, false) } func testManyAny(t *testing.T, json string, paths, expected []string, bytes bool) { var result []Result for i := 0; i < 2; i++ { var which string if i == 0 { which = "Get" result = nil for j := 0; j < len(expected); j++ { if bytes { result = append(result, GetBytes([]byte(json), paths[j])) } else { result = append(result, Get(json, paths[j])) } } } else if i == 1 { which = "GetMany" if bytes { result = GetManyBytes([]byte(json), paths...) } else { result = GetMany(json, paths...) } } for j := 0; j < len(expected); j++ { if result[j].String() != expected[j] { t.Fatalf("Using key '%s' for '%s'\nexpected '%v', got '%v'", paths[j], which, expected[j], result[j].String()) } } } } func TestIssue20(t *testing.T) { json := `{ "name": "FirstName", "name1": "FirstName1", ` + `"address": "address1", "addressDetails": "address2", }` paths := []string{"name", "name1", "address", "addressDetails"} expected := []string{"FirstName", "FirstName1", "address1", "address2"} t.Run("SingleMany", func(t *testing.T) { testMany(t, json, paths, expected) }) } func TestIssue21(t *testing.T) { json := `{ "Level1Field1":3, "Level1Field4":4, "Level1Field2":{ "Level2Field1":[ "value1", "value2" ], "Level2Field2":{ "Level3Field1":[ { "key1":"value1" } ] } } }` paths := []string{"Level1Field1", "Level1Field2.Level2Field1", "Level1Field2.Level2Field2.Level3Field1", "Level1Field4"} expected := []string{"3", `[ "value1", "value2" ]`, `[ { "key1":"value1" } ]`, "4"} t.Run("SingleMany", func(t *testing.T) { testMany(t, json, paths, expected) }) } func TestRandomMany(t *testing.T) { var lstr string defer func() { if v := recover(); v != nil { println("'" + hex.EncodeToString([]byte(lstr)) + "'") println("'" + lstr + "'") panic(v) } }() rand.Seed(time.Now().UnixNano()) b := make([]byte, 512) for i := 0; i < 50000; i++ { n, err := rand.Read(b[:rand.Int()%len(b)]) if err != nil { t.Fatal(err) } lstr = string(b[:n]) paths := make([]string, rand.Int()%64) for i := range paths { var b []byte n := rand.Int() % 5 for j := 0; j < n; j++ { if j > 0 { b = append(b, '.') } nn := rand.Int() % 10 for k := 0; k < nn; k++ { b = append(b, 'a'+byte(rand.Int()%26)) } } paths[i] = string(b) } GetMany(lstr, paths...) } } var complicatedJSON = ` { "tagged": "OK", "Tagged": "KO", "NotTagged": true, "unsettable": 101, "Nested": { "Yellow": "Green", "yellow": "yellow" }, "nestedTagged": { "Green": "Green", "Map": { "this": "that", "and": "the other thing" }, "Ints": { "Uint": 99, "Uint16": 16, "Uint32": 32, "Uint64": 65 }, "Uints": { "int": -99, "Int": -98, "Int16": -16, "Int32": -32, "int64": -64, "Int64": -65 }, "Uints": { "Float32": 32.32, "Float64": 64.64 }, "Byte": 254, "Bool": true }, "LeftOut": "you shouldn't be here", "SelfPtr": {"tagged":"OK","nestedTagged":{"Ints":{"Uint32":32}}}, "SelfSlice": [{"tagged":"OK","nestedTagged":{"Ints":{"Uint32":32}}}], "SelfSlicePtr": [{"tagged":"OK","nestedTagged":{"Ints":{"Uint32":32}}}], "SelfPtrSlice": [{"tagged":"OK","nestedTagged":{"Ints":{"Uint32":32}}}], "interface": "Tile38 Rocks!", "Interface": "Please Download", "Array": [0,2,3,4,5], "time": "2017-05-07T13:24:43-07:00", "Binary": "R0lGODlhPQBEAPeo", "NonBinary": [9,3,100,115] } ` func testvalid(t *testing.T, json string, expect bool) { t.Helper() _, ok := validpayload([]byte(json), 0) if ok != expect { t.Fatal("mismatch") } } func TestValidBasic(t *testing.T) { testvalid(t, "0", true) testvalid(t, "00", false) testvalid(t, "-00", false) testvalid(t, "-.", false) testvalid(t, "-.123", false) testvalid(t, "0.0", true) testvalid(t, "10.0", true) testvalid(t, "10e1", true) testvalid(t, "10EE", false) testvalid(t, "10E-", false) testvalid(t, "10E+", false) testvalid(t, "10E123", true) testvalid(t, "10E-123", true) testvalid(t, "10E-0123", true) testvalid(t, "", false) testvalid(t, " ", false) testvalid(t, "{}", true) testvalid(t, "{", false) testvalid(t, "-", false) testvalid(t, "-1", true) testvalid(t, "-1.", false) testvalid(t, "-1.0", true) testvalid(t, " -1.0", true) testvalid(t, " -1.0 ", true) testvalid(t, "-1.0 ", true) testvalid(t, "-1.0 i", false) testvalid(t, "-1.0 i", false) testvalid(t, "true", true) testvalid(t, " true", true) testvalid(t, " true ", true) testvalid(t, " True ", false) testvalid(t, " tru", false) testvalid(t, "false", true) testvalid(t, " false", true) testvalid(t, " false ", true) testvalid(t, " False ", false) testvalid(t, " fals", false) testvalid(t, "null", true) testvalid(t, " null", true) testvalid(t, " null ", true) testvalid(t, " Null ", false) testvalid(t, " nul", false) testvalid(t, " []", true) testvalid(t, " [true]", true) testvalid(t, " [ true, null ]", true) testvalid(t, " [ true,]", false) testvalid(t, `{"hello":"world"}`, true) testvalid(t, `{ "hello": "world" }`, true) testvalid(t, `{ "hello": "world", }`, false) testvalid(t, `{"a":"b",}`, false) testvalid(t, `{"a":"b","a"}`, false) testvalid(t, `{"a":"b","a":}`, false) testvalid(t, `{"a":"b","a":1}`, true) testvalid(t, `{"a":"b",2"1":2}`, false) testvalid(t, `{"a":"b","a": 1, "c":{"hi":"there"} }`, true) testvalid(t, `{"a":"b","a": 1, "c":{"hi":"there", "easy":["going",`+ `{"mixed":"bag"}]} }`, true) testvalid(t, `""`, true) testvalid(t, `"`, false) testvalid(t, `"\n"`, true) testvalid(t, `"\"`, false) testvalid(t, `"\\"`, true) testvalid(t, `"a\\b"`, true) testvalid(t, `"a\\b\\\"a"`, true) testvalid(t, `"a\\b\\\uFFAAa"`, true) testvalid(t, `"a\\b\\\uFFAZa"`, false) testvalid(t, `"a\\b\\\uFFA"`, false) testvalid(t, string(complicatedJSON), true) testvalid(t, string(exampleJSON), true) testvalid(t, "[-]", false) testvalid(t, "[-.123]", false) } var jsonchars = []string{"{", "[", ",", ":", "}", "]", "1", "0", "true", "false", "null", `""`, `"\""`, `"a"`} func makeRandomJSONChars(b []byte) { var bb []byte for len(bb) < len(b) { bb = append(bb, jsonchars[rand.Int()%len(jsonchars)]...) } copy(b, bb[:len(b)]) } func TestValidRandom(t *testing.T) { rand.Seed(time.Now().UnixNano()) b := make([]byte, 100000) start := time.Now() for time.Since(start) < time.Second*3 { n := rand.Int() % len(b) rand.Read(b[:n]) validpayload(b[:n], 0) } start = time.Now() for time.Since(start) < time.Second*3 { n := rand.Int() % len(b) makeRandomJSONChars(b[:n]) validpayload(b[:n], 0) } } func TestGetMany47(t *testing.T) { json := `{"bar": {"id": 99, "mybar": "my mybar" }, "foo": ` + `{"myfoo": [605]}}` paths := []string{"foo.myfoo", "bar.id", "bar.mybar", "bar.mybarx"} expected := []string{"[605]", "99", "my mybar", ""} results := GetMany(json, paths...) if len(expected) != len(results) { t.Fatalf("expected %v, got %v", len(expected), len(results)) } for i, path := range paths { if results[i].String() != expected[i] { t.Fatalf("expected '%v', got '%v' for path '%v'", expected[i], results[i].String(), path) } } } func TestGetMany48(t *testing.T) { json := `{"bar": {"id": 99, "xyz": "my xyz"}, "foo": {"myfoo": [605]}}` paths := []string{"foo.myfoo", "bar.id", "bar.xyz", "bar.abc"} expected := []string{"[605]", "99", "my xyz", ""} results := GetMany(json, paths...) if len(expected) != len(results) { t.Fatalf("expected %v, got %v", len(expected), len(results)) } for i, path := range paths { if results[i].String() != expected[i] { t.Fatalf("expected '%v', got '%v' for path '%v'", expected[i], results[i].String(), path) } } } func TestResultRawForLiteral(t *testing.T) { for _, lit := range []string{"null", "true", "false"} { result := Parse(lit) if result.Raw != lit { t.Fatalf("expected '%v', got '%v'", lit, result.Raw) } } } func TestNullArray(t *testing.T) { n := len(Get(`{"data":null}`, "data").Array()) if n != 0 { t.Fatalf("expected '%v', got '%v'", 0, n) } n = len(Get(`{}`, "data").Array()) if n != 0 { t.Fatalf("expected '%v', got '%v'", 0, n) } n = len(Get(`{"data":[]}`, "data").Array()) if n != 0 { t.Fatalf("expected '%v', got '%v'", 0, n) } n = len(Get(`{"data":[null]}`, "data").Array()) if n != 1 { t.Fatalf("expected '%v', got '%v'", 1, n) } } func TestIssue54(t *testing.T) { var r []Result json := `{"MarketName":null,"Nounce":6115}` r = GetMany(json, "Nounce", "Buys", "Sells", "Fills") if strings.Replace(fmt.Sprintf("%v", r), " ", "", -1) != "[6115]" { t.Fatalf("expected '%v', got '%v'", "[6115]", strings.Replace(fmt.Sprintf("%v", r), " ", "", -1)) } r = GetMany(json, "Nounce", "Buys", "Sells") if strings.Replace(fmt.Sprintf("%v", r), " ", "", -1) != "[6115]" { t.Fatalf("expected '%v', got '%v'", "[6115]", strings.Replace(fmt.Sprintf("%v", r), " ", "", -1)) } r = GetMany(json, "Nounce") if strings.Replace(fmt.Sprintf("%v", r), " ", "", -1) != "[6115]" { t.Fatalf("expected '%v', got '%v'", "[6115]", strings.Replace(fmt.Sprintf("%v", r), " ", "", -1)) } } func TestIssue55(t *testing.T) { json := `{"one": {"two": 2, "three": 3}, "four": 4, "five": 5}` results := GetMany(json, "four", "five", "one.two", "one.six") expected := []string{"4", "5", "2", ""} for i, r := range results { if r.String() != expected[i] { t.Fatalf("expected %v, got %v", expected[i], r.String()) } } } func TestIssue58(t *testing.T) { json := `{"data":[{"uid": 1},{"uid": 2}]}` res := Get(json, `data.#[uid!=1]`).Raw if res != `{"uid": 2}` { t.Fatalf("expected '%v', got '%v'", `{"uid": 1}`, res) } } func TestObjectGrouping(t *testing.T) { json := ` [ true, {"name":"tom"}, false, {"name":"janet"}, null ] ` res := Get(json, "#.name") if res.String() != `["tom","janet"]` { t.Fatalf("expected '%v', got '%v'", `["tom","janet"]`, res.String()) } } func TestJSONLines(t *testing.T) { json := ` true false {"name":"tom"} [1,2,3,4,5] {"name":"janet"} null 12930.1203 ` paths := []string{"..#", "..0", "..2.name", "..#.name", "..6", "..7"} ress := []string{"7", "true", "tom", `["tom","janet"]`, "12930.1203", ""} for i, path := range paths { res := Get(json, path) if res.String() != ress[i] { t.Fatalf("expected '%v', got '%v'", ress[i], res.String()) } } json = ` {"name": "Gilbert", "wins": [["straight", "7♣"], ["one pair", "10♥"]]} {"name": "Alexa", "wins": [["two pair", "4♠"], ["two pair", "9♠"]]} {"name": "May", "wins": []} {"name": "Deloise", "wins": [["three of a kind", "5♣"]]} ` var i int lines := strings.Split(strings.TrimSpace(json), "\n") ForEachLine(json, func(line Result) bool { if line.Raw != lines[i] { t.Fatalf("expected '%v', got '%v'", lines[i], line.Raw) } i++ return true }) if i != 4 { t.Fatalf("expected '%v', got '%v'", 4, i) } } func TestNumUint64String(t *testing.T) { var i int64 = 9007199254740993 //2^53 + 1 j := fmt.Sprintf(`{"data": [ %d, "hello" ] }`, i) res := Get(j, "data.0") if res.String() != "9007199254740993" { t.Fatalf("expected '%v', got '%v'", "9007199254740993", res.String()) } } func TestNumInt64String(t *testing.T) { var i int64 = -9007199254740993 j := fmt.Sprintf(`{"data":[ "hello", %d ]}`, i) res := Get(j, "data.1") if res.String() != "-9007199254740993" { t.Fatalf("expected '%v', got '%v'", "-9007199254740993", res.String()) } } func TestNumBigString(t *testing.T) { i := "900719925474099301239109123101" // very big j := fmt.Sprintf(`{"data":[ "hello", "%s" ]}`, i) res := Get(j, "data.1") if res.String() != "900719925474099301239109123101" { t.Fatalf("expected '%v', got '%v'", "900719925474099301239109123101", res.String()) } } func TestNumFloatString(t *testing.T) { var i int64 = -9007199254740993 j := fmt.Sprintf(`{"data":[ "hello", %d ]}`, i) //No quotes around value!! res := Get(j, "data.1") if res.String() != "-9007199254740993" { t.Fatalf("expected '%v', got '%v'", "-9007199254740993", res.String()) } } func TestDuplicateKeys(t *testing.T) { // this is valid json according to the JSON spec var json = `{"name": "Alex","name": "Peter"}` if Parse(json).Get("name").String() != Parse(json).Map()["name"].String() { t.Fatalf("expected '%v', got '%v'", Parse(json).Get("name").String(), Parse(json).Map()["name"].String(), ) } if !Valid(json) { t.Fatal("should be valid") } } func TestArrayValues(t *testing.T) { var json = `{"array": ["PERSON1","PERSON2",0],}` values := Get(json, "array").Array() var output string for i, val := range values { if i > 0 { output += "\n" } output += fmt.Sprintf("%#v", val) } expect := strings.Join([]string{ `gjson.Result{Type:3, Raw:"\"PERSON1\"", Str:"PERSON1", Num:0, ` + `Index:11, Indexes:[]int(nil)}`, `gjson.Result{Type:3, Raw:"\"PERSON2\"", Str:"PERSON2", Num:0, ` + `Index:21, Indexes:[]int(nil)}`, `gjson.Result{Type:2, Raw:"0", Str:"", Num:0, Index:31, Indexes:[]int(nil)}`, }, "\n") if output != expect { t.Fatalf("expected '%v', got '%v'", expect, output) } } func BenchmarkValid(b *testing.B) { for i := 0; i < b.N; i++ { Valid(complicatedJSON) } } func BenchmarkValidBytes(b *testing.B) { complicatedJSON := []byte(complicatedJSON) for i := 0; i < b.N; i++ { ValidBytes(complicatedJSON) } } func BenchmarkGoStdlibValidBytes(b *testing.B) { complicatedJSON := []byte(complicatedJSON) for i := 0; i < b.N; i++ { json.Valid(complicatedJSON) } } func TestModifier(t *testing.T) { json := `{"other":{"hello":"world"},"arr":[1,2,3,4,5,6]}` opts := *pretty.DefaultOptions opts.SortKeys = true exp := string(pretty.PrettyOptions([]byte(json), &opts)) res := Get(json, `@pretty:{"sortKeys":true}`).String() if res != exp { t.Fatalf("expected '%v', got '%v'", exp, res) } res = Get(res, "@pretty|@reverse|@ugly").String() if res != json { t.Fatalf("expected '%v', got '%v'", json, res) } if res := Get(res, "@this").String(); res != json { t.Fatalf("expected '%v', got '%v'", json, res) } if res := Get(res, "other.@this").String(); res != `{"hello":"world"}` { t.Fatalf("expected '%v', got '%v'", json, res) } res = Get(res, "@pretty|@reverse|arr|@reverse|2").String() if res != "4" { t.Fatalf("expected '%v', got '%v'", "4", res) } AddModifier("case", func(json, arg string) string { if arg == "upper" { return strings.ToUpper(json) } if arg == "lower" { return strings.ToLower(json) } return json }) res = Get(json, "other|@case:upper").String() if res != `{"HELLO":"WORLD"}` { t.Fatalf("expected '%v', got '%v'", `{"HELLO":"WORLD"}`, res) } } func TestChaining(t *testing.T) { json := `{ "info": { "friends": [ {"first": "Dale", "last": "Murphy", "age": 44}, {"first": "Roger", "last": "Craig", "age": 68}, {"first": "Jane", "last": "Murphy", "age": 47} ] } }` res := Get(json, "info.friends|0|first").String() if res != "Dale" { t.Fatalf("expected '%v', got '%v'", "Dale", res) } res = Get(json, "info.friends|@reverse|0|age").String() if res != "47" { t.Fatalf("expected '%v', got '%v'", "47", res) } res = Get(json, "@ugly|i\\nfo|friends.0.first").String() if res != "Dale" { t.Fatalf("expected '%v', got '%v'", "Dale", res) } } func TestSplitPipe(t *testing.T) { split := func(t *testing.T, path, el, er string, eo bool) { t.Helper() left, right, ok := splitPossiblePipe(path) // fmt.Printf("%-40s [%v] [%v] [%v]\n", path, left, right, ok) if left != el || right != er || ok != eo { t.Fatalf("expected '%v/%v/%v', got '%v/%v/%v", el, er, eo, left, right, ok) } } split(t, "hello", "", "", false) split(t, "hello.world", "", "", false) split(t, "hello|world", "hello", "world", true) split(t, "hello\\|world", "", "", false) split(t, "hello.#", "", "", false) split(t, `hello.#[a|1="asdf\"|1324"]#\|that`, "", "", false) split(t, `hello.#[a|1="asdf\"|1324"]#|that.more|yikes`, `hello.#[a|1="asdf\"|1324"]#`, "that.more|yikes", true) split(t, `a.#[]#\|b`, "", "", false) } func TestArrayEx(t *testing.T) { json := ` [ { "c":[ {"a":10.11} ] }, { "c":[ {"a":11.11} ] } ]` res := Get(json, "@ugly|#.c.#[a=10.11]").String() if res != `[{"a":10.11}]` { t.Fatalf("expected '%v', got '%v'", `[{"a":10.11}]`, res) } res = Get(json, "@ugly|#.c.#").String() if res != `[1,1]` { t.Fatalf("expected '%v', got '%v'", `[1,1]`, res) } res = Get(json, "@reverse|0|c|0|a").String() if res != "11.11" { t.Fatalf("expected '%v', got '%v'", "11.11", res) } res = Get(json, "#.c|#").String() if res != "2" { t.Fatalf("expected '%v', got '%v'", "2", res) } } func TestPipeDotMixing(t *testing.T) { json := `{ "info": { "friends": [ {"first": "Dale", "last": "Murphy", "age": 44}, {"first": "Roger", "last": "Craig", "age": 68}, {"first": "Jane", "last": "Murphy", "age": 47} ] } }` var res string res = Get(json, `info.friends.#[first="Dale"].last`).String() if res != "Murphy" { t.Fatalf("expected '%v', got '%v'", "Murphy", res) } res = Get(json, `info|friends.#[first="Dale"].last`).String() if res != "Murphy" { t.Fatalf("expected '%v', got '%v'", "Murphy", res) } res = Get(json, `info|friends.#[first="Dale"]|last`).String() if res != "Murphy" { t.Fatalf("expected '%v', got '%v'", "Murphy", res) } res = Get(json, `info|friends|#[first="Dale"]|last`).String() if res != "Murphy" { t.Fatalf("expected '%v', got '%v'", "Murphy", res) } res = Get(json, `@ugly|info|friends|#[first="Dale"]|last`).String() if res != "Murphy" { t.Fatalf("expected '%v', got '%v'", "Murphy", res) } res = Get(json, `@ugly|info.@ugly|friends|#[first="Dale"]|last`).String() if res != "Murphy" { t.Fatalf("expected '%v', got '%v'", "Murphy", res) } res = Get(json, `@ugly.info|@ugly.friends|#[first="Dale"]|last`).String() if res != "Murphy" { t.Fatalf("expected '%v', got '%v'", "Murphy", res) } } func TestDeepSelectors(t *testing.T) { json := `{ "info": { "friends": [ { "first": "Dale", "last": "Murphy", "extra": [10,20,30], "details": { "city": "Tempe", "state": "Arizona" } }, { "first": "Roger", "last": "Craig", "extra": [40,50,60], "details": { "city": "Phoenix", "state": "Arizona" } } ] } }` var res string res = Get(json, `info.friends.#[first="Dale"].extra.0`).String() if res != "10" { t.Fatalf("expected '%v', got '%v'", "10", res) } res = Get(json, `info.friends.#[first="Dale"].extra|0`).String() if res != "10" { t.Fatalf("expected '%v', got '%v'", "10", res) } res = Get(json, `info.friends.#[first="Dale"]|extra|0`).String() if res != "10" { t.Fatalf("expected '%v', got '%v'", "10", res) } res = Get(json, `info.friends.#[details.city="Tempe"].last`).String() if res != "Murphy" { t.Fatalf("expected '%v', got '%v'", "Murphy", res) } res = Get(json, `info.friends.#[details.city="Phoenix"].last`).String() if res != "Craig" { t.Fatalf("expected '%v', got '%v'", "Craig", res) } res = Get(json, `info.friends.#[details.state="Arizona"].last`).String() if res != "Murphy" { t.Fatalf("expected '%v', got '%v'", "Murphy", res) } } func TestMultiArrayEx(t *testing.T) { json := `{ "info": { "friends": [ { "first": "Dale", "last": "Murphy", "kind": "Person", "cust1": true, "extra": [10,20,30], "details": { "city": "Tempe", "state": "Arizona" } }, { "first": "Roger", "last": "Craig", "kind": "Person", "cust2": false, "extra": [40,50,60], "details": { "city": "Phoenix", "state": "Arizona" } } ] } }` var res string res = Get(json, `info.friends.#[kind="Person"]#.kind|0`).String() if res != "Person" { t.Fatalf("expected '%v', got '%v'", "Person", res) } res = Get(json, `info.friends.#.kind|0`).String() if res != "Person" { t.Fatalf("expected '%v', got '%v'", "Person", res) } res = Get(json, `info.friends.#[kind="Person"]#.kind`).String() if res != `["Person","Person"]` { t.Fatalf("expected '%v', got '%v'", `["Person","Person"]`, res) } res = Get(json, `info.friends.#.kind`).String() if res != `["Person","Person"]` { t.Fatalf("expected '%v', got '%v'", `["Person","Person"]`, res) } res = Get(json, `info.friends.#[kind="Person"]#|kind`).String() if res != `` { t.Fatalf("expected '%v', got '%v'", ``, res) } res = Get(json, `info.friends.#|kind`).String() if res != `` { t.Fatalf("expected '%v', got '%v'", ``, res) } res = Get(json, `i*.f*.#[kind="Other"]#`).String() if res != `[]` { t.Fatalf("expected '%v', got '%v'", `[]`, res) } } func TestQueries(t *testing.T) { json := `{ "info": { "friends": [ { "first": "Dale", "last": "Murphy", "kind": "Person", "cust1": true, "extra": [10,20,30], "details": { "city": "Tempe", "state": "Arizona" } }, { "first": "Roger", "last": "Craig", "kind": "Person", "cust2": false, "extra": [40,50,60], "details": { "city": "Phoenix", "state": "Arizona" } } ] } }` // numbers assert(t, Get(json, "i*.f*.#[extra.0<11].first").Exists()) assert(t, Get(json, "i*.f*.#[extra.0<=11].first").Exists()) assert(t, !Get(json, "i*.f*.#[extra.0<10].first").Exists()) assert(t, Get(json, "i*.f*.#[extra.0<=10].first").Exists()) assert(t, Get(json, "i*.f*.#[extra.0=10].first").Exists()) assert(t, !Get(json, "i*.f*.#[extra.0=11].first").Exists()) assert(t, Get(json, "i*.f*.#[extra.0!=10].first").String() == "Roger") assert(t, Get(json, "i*.f*.#[extra.0>10].first").String() == "Roger") assert(t, Get(json, "i*.f*.#[extra.0>=10].first").String() == "Dale") // strings assert(t, Get(json, `i*.f*.#[extra.0<"11"].first`).Exists()) assert(t, Get(json, `i*.f*.#[first>"Dale"].last`).String() == "Craig") assert(t, Get(json, `i*.f*.#[first>="Dale"].last`).String() == "Murphy") assert(t, Get(json, `i*.f*.#[first="Dale"].last`).String() == "Murphy") assert(t, Get(json, `i*.f*.#[first!="Dale"].last`).String() == "Craig") assert(t, !Get(json, `i*.f*.#[first<"Dale"].last`).Exists()) assert(t, Get(json, `i*.f*.#[first<="Dale"].last`).Exists()) assert(t, Get(json, `i*.f*.#[first%"Da*"].last`).Exists()) assert(t, Get(json, `i*.f*.#[first%"Dale"].last`).Exists()) assert(t, Get(json, `i*.f*.#[first%"*a*"]#|#`).String() == "1") assert(t, Get(json, `i*.f*.#[first%"*e*"]#|#`).String() == "2") assert(t, Get(json, `i*.f*.#[first!%"*e*"]#|#`).String() == "0") // trues assert(t, Get(json, `i*.f*.#[cust1=true].first`).String() == "Dale") assert(t, Get(json, `i*.f*.#[cust2=false].first`).String() == "Roger") assert(t, Get(json, `i*.f*.#[cust1!=false].first`).String() == "Dale") assert(t, Get(json, `i*.f*.#[cust2!=true].first`).String() == "Roger") assert(t, !Get(json, `i*.f*.#[cust1>true].first`).Exists()) assert(t, Get(json, `i*.f*.#[cust1>=true].first`).Exists()) assert(t, !Get(json, `i*.f*.#[cust29)#|#").Int() == 4) assert(t, Get(json, "friends.#(a>10)#|#").Int() == 3) assert(t, Get(json, "friends.#(a>40)#|#").Int() == 0) } func TestSubSelectors(t *testing.T) { json := `{ "info": { "friends": [ { "first": "Dale", "last": "Murphy", "kind": "Person", "cust1": true, "extra": [10,20,30], "details": { "city": "Tempe", "state": "Arizona" } }, { "first": "Roger", "last": "Craig", "kind": "Person", "cust2": false, "extra": [40,50,60], "details": { "city": "Phoenix", "state": "Arizona" } } ] } }` assert(t, Get(json, "[]").String() == "[]") assert(t, Get(json, "{}").String() == "{}") res := Get(json, `{`+ `abc:info.friends.0.first,`+ `info.friends.1.last,`+ `"a`+"\r"+`a":info.friends.0.kind,`+ `"abc":info.friends.1.kind,`+ `{123:info.friends.1.cust2},`+ `[info.friends.#[details.city="Phoenix"]#|#]`+ `}.@pretty.@ugly`).String() // println(res) // {"abc":"Dale","last":"Craig","\"a\ra\"":"Person","_":{"123":false},"_":[1]} assert(t, Get(res, "abc").String() == "Dale") assert(t, Get(res, "last").String() == "Craig") assert(t, Get(res, "\"a\ra\"").String() == "Person") assert(t, Get(res, "@reverse.abc").String() == "Person") assert(t, Get(res, "_.123").String() == "false") assert(t, Get(res, "@reverse._.0").String() == "1") assert(t, Get(json, "info.friends.[0.first,1.extra.0]").String() == `["Dale",40]`) assert(t, Get(json, "info.friends.#.[first,extra.0]").String() == `[["Dale",10],["Roger",40]]`) } func TestArrayCountRawOutput(t *testing.T) { assert(t, Get(`[1,2,3,4]`, "#").Raw == "4") } func TestParseQuery(t *testing.T) { var path, op, value, remain string var ok bool path, op, value, remain, _, _, ok = parseQuery(`#(service_roles.#(=="one").()==asdf).cap`) assert(t, ok && path == `service_roles.#(=="one").()` && op == "=" && value == `asdf` && remain == `.cap`) path, op, value, remain, _, _, ok = parseQuery(`#(first_name%"Murphy").last`) assert(t, ok && path == `first_name` && op == `%` && value == `"Murphy"` && remain == `.last`) path, op, value, remain, _, _, ok = parseQuery(`#( first_name !% "Murphy" ).last`) assert(t, ok && path == `first_name` && op == `!%` && value == `"Murphy"` && remain == `.last`) path, op, value, remain, _, _, ok = parseQuery(`#(service_roles.#(=="one"))`) assert(t, ok && path == `service_roles.#(=="one")` && op == `` && value == `` && remain == ``) path, op, value, remain, _, _, ok = parseQuery(`#(a\("\"(".#(=="o\"(ne")%"ab\")").remain`) assert(t, ok && path == `a\("\"(".#(=="o\"(ne")` && op == "%" && value == `"ab\")"` && remain == `.remain`) } func TestParentSubQuery(t *testing.T) { var json = `{ "topology": { "instances": [ { "service_version": "1.2.3", "service_locale": {"lang": "en"}, "service_roles": ["one", "two"] }, { "service_version": "1.2.4", "service_locale": {"lang": "th"}, "service_roles": ["three", "four"] }, { "service_version": "1.2.2", "service_locale": {"lang": "en"}, "service_roles": ["one"] } ] } }` res := Get(json, `topology.instances.#( service_roles.#(=="one"))#.service_version`) // should return two instances assert(t, res.String() == `["1.2.3","1.2.2"]`) } func TestSingleModifier(t *testing.T) { var data = `{"@key": "value"}` assert(t, Get(data, "@key").String() == "value") assert(t, Get(data, "\\@key").String() == "value") } func TestModifiersInMultipaths(t *testing.T) { AddModifier("case", func(json, arg string) string { if arg == "upper" { return strings.ToUpper(json) } if arg == "lower" { return strings.ToLower(json) } return json }) json := `{"friends": [ {"age": 44, "first": "Dale", "last": "Murphy"}, {"age": 68, "first": "Roger", "last": "Craig"}, {"age": 47, "first": "Jane", "last": "Murphy"} ]}` res := Get(json, `friends.#.{age,first|@case:upper}|@ugly`) exp := `[{"age":44,"@case:upper":"DALE"},{"age":68,"@case:upper":"ROGER"},{"age":47,"@case:upper":"JANE"}]` assert(t, res.Raw == exp) res = Get(json, `{friends.#.{age,first:first|@case:upper}|0.first}`) exp = `{"first":"DALE"}` assert(t, res.Raw == exp) res = Get(readmeJSON, `{"children":children|@case:upper,"name":name.first,"age":age}`) exp = `{"children":["SARA","ALEX","JACK"],"name":"Tom","age":37}` assert(t, res.Raw == exp) } func TestIssue141(t *testing.T) { json := `{"data": [{"q": 11, "w": 12}, {"q": 21, "w": 22}, {"q": 31, "w": 32} ], "sql": "some stuff here"}` assert(t, Get(json, "data.#").Int() == 3) assert(t, Get(json, "data.#.{q}|@ugly").Raw == `[{"q":11},{"q":21},{"q":31}]`) assert(t, Get(json, "data.#.q|@ugly").Raw == `[11,21,31]`) } func TestChainedModifierStringArgs(t *testing.T) { // issue #143 AddModifier("push", func(json, arg string) string { json = strings.TrimSpace(json) if len(json) < 2 || !Parse(json).IsArray() { return json } json = strings.TrimSpace(json[1 : len(json)-1]) if len(json) == 0 { return "[" + arg + "]" } return "[" + json + "," + arg + "]" }) res := Get("[]", `@push:"2"|@push:"3"|@push:{"a":"b","c":["e","f"]}|@push:true|@push:10.23`) assert(t, res.String() == `["2","3",{"a":"b","c":["e","f"]},true,10.23]`) } func TestFlatten(t *testing.T) { json := `[1,[2],[3,4],[5,[6,[7]]],{"hi":"there"},8,[9]]` assert(t, Get(json, "@flatten").String() == `[1,2,3,4,5,[6,[7]],{"hi":"there"},8,9]`) assert(t, Get(json, `@flatten:{"deep":true}`).String() == `[1,2,3,4,5,6,7,{"hi":"there"},8,9]`) assert(t, Get(`{"9999":1234}`, "@flatten").String() == `{"9999":1234}`) } func TestJoin(t *testing.T) { assert(t, Get(`[{},{}]`, "@join").String() == `{}`) assert(t, Get(`[{"a":1},{"b":2}]`, "@join").String() == `{"a":1,"b":2}`) assert(t, Get(`[{"a":1,"b":1},{"b":2}]`, "@join").String() == `{"a":1,"b":2}`) assert(t, Get(`[{"a":1,"b":1},{"b":2},5,{"c":3}]`, "@join").String() == `{"a":1,"b":2,"c":3}`) assert(t, Get(`[{"a":1,"b":1},{"b":2},5,{"c":3}]`, `@join:{"preserve":true}`).String() == `{"a":1,"b":1,"b":2,"c":3}`) assert(t, Get(`[{"a":1,"b":1},{"b":2},5,{"c":3}]`, `@join:{"preserve":true}.b`).String() == `1`) assert(t, Get(`{"9999":1234}`, "@join").String() == `{"9999":1234}`) } func TestValid(t *testing.T) { assert(t, Get("[{}", "@valid").Exists() == false) assert(t, Get("[{}]", "@valid").Exists() == true) } // https://github.com/tidwall/gjson/issues/152 func TestJoin152(t *testing.T) { var json = `{ "distance": 1374.0, "validFrom": "2005-11-14", "historical": { "type": "Day", "name": "last25Hours", "summary": { "units": { "temperature": "C", "wind": "m/s", "snow": "cm", "precipitation": "mm" }, "days": [ { "time": "2020-02-08", "hours": [ { "temperature": { "min": -2.0, "max": -1.6, "value": -1.6 }, "wind": {}, "precipitation": {}, "humidity": { "value": 92.0 }, "snow": { "depth": 49.0 }, "time": "2020-02-08T16:00:00+01:00" }, { "temperature": { "min": -1.7, "max": -1.3, "value": -1.3 }, "wind": {}, "precipitation": {}, "humidity": { "value": 92.0 }, "snow": { "depth": 49.0 }, "time": "2020-02-08T17:00:00+01:00" }, { "temperature": { "min": -1.3, "max": -0.9, "value": -1.2 }, "wind": {}, "precipitation": {}, "humidity": { "value": 91.0 }, "snow": { "depth": 49.0 }, "time": "2020-02-08T18:00:00+01:00" } ] }, { "time": "2020-02-09", "hours": [ { "temperature": { "min": -1.7, "max": -0.9, "value": -1.5 }, "wind": {}, "precipitation": {}, "humidity": { "value": 91.0 }, "snow": { "depth": 49.0 }, "time": "2020-02-09T00:00:00+01:00" }, { "temperature": { "min": -1.5, "max": 0.9, "value": 0.2 }, "wind": {}, "precipitation": {}, "humidity": { "value": 67.0 }, "snow": { "depth": 49.0 }, "time": "2020-02-09T01:00:00+01:00" } ] } ] } } }` res := Get(json, "historical.summary.days.#.hours|@flatten|#.humidity.value") assert(t, res.Raw == `[92.0,92.0,91.0,91.0,67.0]`) } func TestVariousFuzz(t *testing.T) { // Issue #192 assert(t, squash(`"000"hello`) == `"000"`) assert(t, squash(`"000"`) == `"000"`) assert(t, squash(`"000`) == `"000`) assert(t, squash(`"`) == `"`) assert(t, squash(`[000]hello`) == `[000]`) assert(t, squash(`[000]`) == `[000]`) assert(t, squash(`[000`) == `[000`) assert(t, squash(`[`) == `[`) assert(t, squash(`]`) == `]`) testJSON := `0.#[[{}]].@valid:"000` Get(testJSON, testJSON) // Issue #195 testJSON = `\************************************` + `**********{**",**,,**,**,**,**,"",**,**,**,**,**,**,**,**,**,**]` Get(testJSON, testJSON) // Issue #196 testJSON = `[#.@pretty.@join:{""[]""preserve"3,"][{]]]` Get(testJSON, testJSON) // Issue #237 testJSON1 := `["*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,,,,,,"]` testJSON2 := `#[%"*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,,,,,,""*,*"]` Get(testJSON1, testJSON2) } func TestSubpathsWithMultipaths(t *testing.T) { const json = ` [ {"a": 1}, {"a": 2, "values": ["a", "b", "c", "d", "e"]}, true, ["a", "b", "c", "d", "e"], 4 ] ` assert(t, Get(json, `1.values.@ugly`).Raw == `["a","b","c","d","e"]`) assert(t, Get(json, `1.values.[0,3]`).Raw == `["a","d"]`) assert(t, Get(json, `3.@ugly`).Raw == `["a","b","c","d","e"]`) assert(t, Get(json, `3.[0,3]`).Raw == `["a","d"]`) assert(t, Get(json, `#.@ugly`).Raw == `[{"a":1},{"a":2,"values":["a","b","c","d","e"]},true,["a","b","c","d","e"],4]`) assert(t, Get(json, `#.[0,3]`).Raw == `[[],[],[],["a","d"],[]]`) } func TestFlattenRemoveNonExist(t *testing.T) { raw := Get("[[1],[2,[[],[3]],[4,[5],[],[[[6]]]]]]", `@flatten:{"deep":true}`).Raw assert(t, raw == "[1,2,3,4,5,6]") } func TestPipeEmptyArray(t *testing.T) { raw := Get("[]", `#(hello)#`).Raw assert(t, raw == "[]") } func TestEncodedQueryString(t *testing.T) { json := `{ "friends": [ {"first": "Dale", "last": "Mur\nphy", "age": 44}, {"first": "Roger", "last": "Craig", "age": 68}, {"first": "Jane", "last": "Murphy", "age": 47} ] }` assert(t, Get(json, `friends.#(last=="Mur\nphy").age`).Int() == 44) assert(t, Get(json, `friends.#(last=="Murphy").age`).Int() == 47) } func TestTildeQueries(t *testing.T) { json := `{ "vals": [ { "a": 1, "b": "data" }, { "a": 2, "b": true }, { "a": 3, "b": false }, { "a": 4, "b": "0" }, { "a": 5, "b": 0 }, { "a": 6, "b": "1" }, { "a": 7, "b": 1 }, { "a": 8, "b": "true" }, { "a": 9, "b": false }, { "a": 10, "b": null }, { "a": 11 } ] }` trues := Get(json, `vals.#(b==~true)#.a`).Raw truesNOT := Get(json, `vals.#(b!=~true)#.a`).Raw falses := Get(json, `vals.#(b==~false)#.a`).Raw falsesNOT := Get(json, `vals.#(b!=~false)#.a`).Raw nulls := Get(json, `vals.#(b==~null)#.a`).Raw nullsNOT := Get(json, `vals.#(b!=~null)#.a`).Raw exists := Get(json, `vals.#(b==~*)#.a`).Raw existsNOT := Get(json, `vals.#(b!=~*)#.a`).Raw assert(t, trues == "[2,6,7,8]") assert(t, truesNOT == "[1,3,4,5,9,10,11]") assert(t, falses == "[3,4,5,9,10,11]") assert(t, falsesNOT == "[1,2,6,7,8]") assert(t, nulls == "[10,11]") assert(t, nullsNOT == "[1,2,3,4,5,6,7,8,9]") assert(t, exists == "[1,2,3,4,5,6,7,8,9,10]") assert(t, existsNOT == "[11]") json = `{ "vals": [ { "a": 1, "b": "something" }, { "a": 2, "b": "else" }, { "a": 3, "b": false }, { "a": 4, "b": "0" }, { "a": 5, "b": 0 }, { "a": 6, "b": "1" }, { "a": 7, "b": 1 }, { "a": 8, "b": "true" }, { "a": 9, "b": false }, { "a": 10, "b": null }, { "a": 11 } ], "anything": "else" }` trues = Get(json, `vals.#(b==~true)#.a`).Raw truesNOT = Get(json, `vals.#(b!=~true)#.a`).Raw falses = Get(json, `vals.#(b==~false)#.a`).Raw falsesNOT = Get(json, `vals.#(b!=~false)#.a`).Raw nulls = Get(json, `vals.#(b==~null)#.a`).Raw nullsNOT = Get(json, `vals.#(b!=~null)#.a`).Raw exists = Get(json, `vals.#(b==~*)#.a`).Raw existsNOT = Get(json, `vals.#(b!=~*)#.a`).Raw assert(t, trues == "[6,7,8]") assert(t, truesNOT == "[1,2,3,4,5,9,10,11]") assert(t, falses == "[3,4,5,9,10,11]") assert(t, falsesNOT == "[1,2,6,7,8]") assert(t, nulls == "[10,11]") assert(t, nullsNOT == "[1,2,3,4,5,6,7,8,9]") assert(t, exists == "[1,2,3,4,5,6,7,8,9,10]") assert(t, existsNOT == "[11]") } func TestModifierDoubleQuotes(t *testing.T) { josn := `{ "data": [ { "name": "Product P4", "productId": "1bb3", "vendorId": "10de" }, { "name": "Product P4", "productId": "1cc3", "vendorId": "20de" }, { "name": "Product P4", "productId": "1dd3", "vendorId": "30de" } ] }` AddModifier("string", func(josn, arg string) string { return strconv.Quote(josn) }) res := Get(josn, "data.#.{name,value:{productId,vendorId}.@string.@ugly}") assert(t, res.Raw == `[`+ `{"name":"Product P4","value":"{\"productId\":\"1bb3\",\"vendorId\":\"10de\"}"},`+ `{"name":"Product P4","value":"{\"productId\":\"1cc3\",\"vendorId\":\"20de\"}"},`+ `{"name":"Product P4","value":"{\"productId\":\"1dd3\",\"vendorId\":\"30de\"}"}`+ `]`) } func TestIndexes(t *testing.T) { var exampleJSON = `{ "vals": [ [1,66,{test: 3}], [4,5,[6]] ], "objectArray":[ {"first": "Dale", "age": 44}, {"first": "Roger", "age": 68}, ] }` testCases := []struct { path string expected []string }{ { `vals.#.1`, []string{`6`, "5"}, }, { `vals.#.2`, []string{"{", "["}, }, { `objectArray.#(age>43)#.first`, []string{`"`, `"`}, }, { `objectArray.@reverse.#.first`, nil, }, } for _, tc := range testCases { r := Get(exampleJSON, tc.path) assert(t, len(r.Indexes) == len(tc.expected)) for i, a := range r.Indexes { assert(t, string(exampleJSON[a]) == tc.expected[i]) } } } func TestIndexesMatchesRaw(t *testing.T) { var exampleJSON = `{ "objectArray":[ {"first": "Jason", "age": 41}, {"first": "Dale", "age": 44}, {"first": "Roger", "age": 68}, {"first": "Mandy", "age": 32} ] }` r := Get(exampleJSON, `objectArray.#(age>43)#.first`) assert(t, len(r.Indexes) == 2) assert(t, Parse(exampleJSON[r.Indexes[0]:]).String() == "Dale") assert(t, Parse(exampleJSON[r.Indexes[1]:]).String() == "Roger") r = Get(exampleJSON, `objectArray.#(age>43)#`) assert(t, Parse(exampleJSON[r.Indexes[0]:]).Get("first").String() == "Dale") assert(t, Parse(exampleJSON[r.Indexes[1]:]).Get("first").String() == "Roger") } func TestIssue240(t *testing.T) { nonArrayData := `{"jsonrpc":"2.0","method":"subscription","params": {"channel":"funny_channel","data": {"name":"Jason","company":"good_company","number":12345} } }` parsed := Parse(nonArrayData) assert(t, len(parsed.Get("params.data").Array()) == 1) arrayData := `{"jsonrpc":"2.0","method":"subscription","params": {"channel":"funny_channel","data":[ {"name":"Jason","company":"good_company","number":12345} ]} }` parsed = Parse(arrayData) assert(t, len(parsed.Get("params.data").Array()) == 1) } func TestKeysValuesModifier(t *testing.T) { var json = `{ "1300014": { "code": "1300014", "price": 59.18, "symbol": "300014", "update": "2020/04/15 15:59:54", }, "1300015": { "code": "1300015", "price": 43.31, "symbol": "300015", "update": "2020/04/15 15:59:54", } }` assert(t, Get(json, `@keys`).String() == `["1300014","1300015"]`) assert(t, Get(``, `@keys`).String() == `[]`) assert(t, Get(`"hello"`, `@keys`).String() == `[null]`) assert(t, Get(`[]`, `@keys`).String() == `[]`) assert(t, Get(`[1,2,3]`, `@keys`).String() == `[null,null,null]`) assert(t, Get(json, `@values.#.code`).String() == `["1300014","1300015"]`) assert(t, Get(``, `@values`).String() == `[]`) assert(t, Get(`"hello"`, `@values`).String() == `["hello"]`) assert(t, Get(`[]`, `@values`).String() == `[]`) assert(t, Get(`[1,2,3]`, `@values`).String() == `[1,2,3]`) } func TestNaNInf(t *testing.T) { json := `[+Inf,-Inf,Inf,iNF,-iNF,+iNF,NaN,nan,nAn,-0,+0]` raws := []string{"+Inf", "-Inf", "Inf", "iNF", "-iNF", "+iNF", "NaN", "nan", "nAn", "-0", "+0"} nums := []float64{math.Inf(+1), math.Inf(-1), math.Inf(0), math.Inf(0), math.Inf(-1), math.Inf(+1), math.NaN(), math.NaN(), math.NaN(), math.Copysign(0, -1), 0} assert(t, int(Get(json, `#`).Int()) == len(raws)) for i := 0; i < len(raws); i++ { r := Get(json, fmt.Sprintf("%d", i)) assert(t, r.Raw == raws[i]) assert(t, r.Num == nums[i] || (math.IsNaN(r.Num) && math.IsNaN(nums[i]))) assert(t, r.Type == Number) } var i int Parse(json).ForEach(func(_, r Result) bool { assert(t, r.Raw == raws[i]) assert(t, r.Num == nums[i] || (math.IsNaN(r.Num) && math.IsNaN(nums[i]))) assert(t, r.Type == Number) i++ return true }) // Parse should also return valid numbers assert(t, math.IsNaN(Parse("nan").Float())) assert(t, math.IsNaN(Parse("NaN").Float())) assert(t, math.IsNaN(Parse(" NaN").Float())) assert(t, math.IsInf(Parse("+inf").Float(), +1)) assert(t, math.IsInf(Parse("-inf").Float(), -1)) assert(t, math.IsInf(Parse("+INF").Float(), +1)) assert(t, math.IsInf(Parse("-INF").Float(), -1)) assert(t, math.IsInf(Parse(" +INF").Float(), +1)) assert(t, math.IsInf(Parse(" -INF").Float(), -1)) } func TestEmptyValueQuery(t *testing.T) { // issue: https://github.com/tidwall/gjson/issues/246 assert(t, Get( `["ig","","tw","fb","tw","ig","tw"]`, `#(!="")#`).Raw == `["ig","tw","fb","tw","ig","tw"]`) assert(t, Get( `["ig","","tw","fb","tw","ig","tw"]`, `#(!=)#`).Raw == `["ig","tw","fb","tw","ig","tw"]`) } func TestParseIndex(t *testing.T) { assert(t, Parse(`{}`).Index == 0) assert(t, Parse(` {}`).Index == 1) assert(t, Parse(` []`).Index == 1) assert(t, Parse(` true`).Index == 1) assert(t, Parse(` false`).Index == 1) assert(t, Parse(` null`).Index == 1) assert(t, Parse(` +inf`).Index == 1) assert(t, Parse(` -inf`).Index == 1) } func TestRevSquash(t *testing.T) { assert(t, revSquash(` {}`) == `{}`) assert(t, revSquash(` }`) == ` }`) assert(t, revSquash(` [123]`) == `[123]`) assert(t, revSquash(` ,123,123]`) == ` ,123,123]`) assert(t, revSquash(` hello,[[true,false],[0,1,2,3,5],[123]]`) == `[[true,false],[0,1,2,3,5],[123]]`) assert(t, revSquash(` "hello"`) == `"hello"`) assert(t, revSquash(` "hel\\lo"`) == `"hel\\lo"`) assert(t, revSquash(` "hel\\"lo"`) == `"lo"`) assert(t, revSquash(` "hel\\\"lo"`) == `"hel\\\"lo"`) assert(t, revSquash(`hel\\\"lo"`) == `hel\\\"lo"`) assert(t, revSquash(`\"hel\\\"lo"`) == `\"hel\\\"lo"`) assert(t, revSquash(`\\\"hel\\\"lo"`) == `\\\"hel\\\"lo"`) assert(t, revSquash(`\\\\"hel\\\"lo"`) == `"hel\\\"lo"`) assert(t, revSquash(`hello"`) == `hello"`) json := `true,[0,1,"sadf\"asdf",{"hi":["hello","t\"\"u",{"a":"b"}]},9]` assert(t, revSquash(json) == json[5:]) assert(t, revSquash(json[:len(json)-3]) == `{"hi":["hello","t\"\"u",{"a":"b"}]}`) assert(t, revSquash(json[:len(json)-4]) == `["hello","t\"\"u",{"a":"b"}]`) assert(t, revSquash(json[:len(json)-5]) == `{"a":"b"}`) assert(t, revSquash(json[:len(json)-6]) == `"b"`) assert(t, revSquash(json[:len(json)-10]) == `"a"`) assert(t, revSquash(json[:len(json)-15]) == `"t\"\"u"`) assert(t, revSquash(json[:len(json)-24]) == `"hello"`) assert(t, revSquash(json[:len(json)-33]) == `"hi"`) assert(t, revSquash(json[:len(json)-39]) == `"sadf\"asdf"`) } const readmeJSON = ` { "name": {"first": "Tom", "last": "Anderson"}, "age":37, "children": ["Sara","Alex","Jack"], "fav.movie": "Deer Hunter", "friends": [ {"first": "Dale", "last": "Murphy", "age": 44, "nets": ["ig", "fb", "tw"]}, {"first": "Roger", "last": "Craig", "age": 68, "nets": ["fb", "tw"]}, {"first": "Jane", "last": "Murphy", "age": 47, "nets": ["ig", "tw"]} ] } ` func TestQueryGetPath(t *testing.T) { assert(t, strings.Join( Get(readmeJSON, "friends.#.first").Paths(readmeJSON), " ") == "friends.0.first friends.1.first friends.2.first") assert(t, strings.Join( Get(readmeJSON, "friends.#(last=Murphy)").Paths(readmeJSON), " ") == "") assert(t, Get(readmeJSON, "friends.#(last=Murphy)").Path(readmeJSON) == "friends.0") assert(t, strings.Join( Get(readmeJSON, "friends.#(last=Murphy)#").Paths(readmeJSON), " ") == "friends.0 friends.2") arr := Get(readmeJSON, "friends.#.first").Array() for i := 0; i < len(arr); i++ { assert(t, arr[i].Path(readmeJSON) == fmt.Sprintf("friends.%d.first", i)) } } func TestStaticJSON(t *testing.T) { json := `{ "name": {"first": "Tom", "last": "Anderson"} }` assert(t, Get(json, `"bar"`).Raw == ``) assert(t, Get(json, `!"bar"`).Raw == `"bar"`) assert(t, Get(json, `!{"name":{"first":"Tom"}}.{name.first}.first`).Raw == `"Tom"`) assert(t, Get(json, `{name.last,"foo":!"bar"}`).Raw == `{"last":"Anderson","foo":"bar"}`) assert(t, Get(json, `{name.last,"foo":!{"a":"b"},"that"}`).Raw == `{"last":"Anderson","foo":{"a":"b"}}`) assert(t, Get(json, `{name.last,"foo":!{"c":"d"},!"that"}`).Raw == `{"last":"Anderson","foo":{"c":"d"},"_":"that"}`) assert(t, Get(json, `[!true,!false,!null,!inf,!nan,!hello,{"name":!"andy",name.last},+inf,!["any","thing"]]`).Raw == `[true,false,null,inf,nan,{"name":"andy","last":"Anderson"},["any","thing"]]`, ) } func TestArrayKeys(t *testing.T) { N := 100 json := "[" for i := 0; i < N; i++ { if i > 0 { json += "," } json += fmt.Sprint(i) } json += "]" var i int Parse(json).ForEach(func(key, value Result) bool { assert(t, key.String() == fmt.Sprint(i)) assert(t, key.Int() == int64(i)) i++ return true }) assert(t, i == N) } func TestToFromStr(t *testing.T) { json := `{"Message":"{\"Records\":[{\"eventVersion\":\"2.1\"}]"}` res := Get(json, "Message.@fromstr.Records.#.eventVersion.@tostr").Raw assert(t, res == `["\"2.1\""]`) } func TestGroup(t *testing.T) { json := `{"id":["123","456","789"],"val":[2,1]}` res := Get(json, "@group").Raw assert(t, res == `[{"id":"123","val":2},{"id":"456","val":1},{"id":"789"}]`) json = ` { "issues": [ { "fields": { "labels": [ "milestone_1", "group:foo", "plan:a", "plan:b" ] }, "id": "123" },{ "fields": { "labels": [ "milestone_1", "group:foo", "plan:a", "plan" ] }, "id": "456" } ] } ` res = Get(json, `{"id":issues.#.id,"plans":issues.#.fields.labels.#(%"plan:*")#|#.#}|@group|#(plans>=2)#.id`).Raw assert(t, res == `["123"]`) } func testJSONString(t *testing.T, str string) { gjsonString := string(AppendJSONString(nil, str)) data, err := json.Marshal(str) if err != nil { panic(123) } goString := string(data) if gjsonString != goString { t.Fatal(strconv.Quote(str) + "\n\t" + gjsonString + "\n\t" + goString + "\n\t<<< MISMATCH >>>") } } func TestJSONString(t *testing.T) { testJSONString(t, "hello") testJSONString(t, "he\"llo") testJSONString(t, "he\"l\\lo") const input = `{"utf8":"Example emoji, KO: \ud83d\udd13, \ud83c\udfc3 ` + `OK: \u2764\ufe0f "}` value := Get(input, "utf8") var s string json.Unmarshal([]byte(value.Raw), &s) if value.String() != s { t.Fatalf("expected '%v', got '%v'", s, value.String()) } testJSONString(t, s) testJSONString(t, "R\xfd\xfc\a!\x82eO\x16?_\x0f\x9ab\x1dr") testJSONString(t, "_\xb9\v\xad\xb3|X!\xb6\xd9U&\xa4\x1a\x95\x04") data, _ := json.Marshal("\b\f") if (string(data) == "\"\\b\\f\"") { // Go version 1.22+ encodes "\b" and "\f" correctly. testJSONString(t, "\b\f") rng := rand.New(rand.NewSource(time.Now().UnixNano())) start := time.Now() var buf [16]byte for time.Since(start) < time.Second*2 { if _, err := rng.Read(buf[:]); err != nil { t.Fatal(err) } testJSONString(t, string(buf[:])) } } } func TestIndexAtSymbol(t *testing.T) { json := `{ "@context": { "rdfs": "http://www.w3.org/2000/01/rdf-schema#", "@vocab": "http://schema.org/", "sh": "http://www.w3.org/ns/shacl#" } }` assert(t, Get(json, "@context.@vocab").Index == 85) } func TestDeepModifierWithOptions(t *testing.T) { rawJson := `{"x":[{"y":[{"z":{"b":1, "c": 2, "a": 3}}]}]}` jsonPathExpr := `x.#.y.#.z.@pretty:{"sortKeys":true}` results := GetManyBytes([]byte(rawJson), jsonPathExpr) assert(t, len(results) == 1) actual := results[0].Raw expected := `[[{ "a": 3, "b": 1, "c": 2 } ]]` if expected != actual { t.Fatal(strconv.Quote(rawJson) + "\n\t" + expected + "\n\t" + actual + "\n\t<<< MISMATCH >>>") } } func TestIssue301(t *testing.T) { json := `{ "children": ["Sara","Alex","Jack"], "fav.movie": ["Deer Hunter"] }` assert(t, Get(json, `children.0`).String() == "Sara") assert(t, Get(json, `children.[0]`).String() == `["Sara"]`) assert(t, Get(json, `children.1`).String() == "Alex") assert(t, Get(json, `children.[1]`).String() == `["Alex"]`) assert(t, Get(json, `children.[10]`).String() == `[]`) assert(t, Get(json, `fav\.movie.0`).String() == "Deer Hunter") assert(t, Get(json, `fav\.movie.[0]`).String() == `["Deer Hunter"]`) assert(t, Get(json, `fav\.movie.1`).String() == "") assert(t, Get(json, `fav\.movie.[1]`).String() == "[]") } func TestModDig(t *testing.T) { json := ` { "group": { "issues": [ { "fields": { "labels": [ "milestone_1", "group:foo", "plan:a", "plan:b" ] }, "refid": "123" },{ "fields": { "labels": [ "milestone_2", "group:foo", "plan:a", "plan" ] }, "refid": "456" },[ {"extra_deep":[{ "fields": { "labels": [ "milestone_3", "group:foo", "plan:a", "plan" ] }, "refid": "789" }] }] ] } } ` assert(t, Get(json, "group.@dig:#(refid=123)|0.fields.labels.0").String() == "milestone_1") assert(t, Get(json, "group.@dig:#(refid=456)|0.fields.labels.0").String() == "milestone_2") assert(t, Get(json, "group.@dig:#(refid=789)|0.fields.labels.0").String() == "milestone_3") json = ` { "something": { "anything": { "abcdefg": { "finally": { "important": { "secret": "password", "name": "jake" } }, "name": "melinda" } } } }` assert(t, Get(json, "@dig:name").String() == `["melinda","jake"]`) assert(t, Get(json, "@dig:secret").String() == `["password"]`) } func TestEscape(t *testing.T) { json := `{ "user":{ "first.name": "Janet", "last.name": "Prichard" } }` user := Get(json, "user") assert(t, user.Get(Escape("first.name")).String() == "Janet") assert(t, user.Get(Escape("last.name")).String() == "Prichard") assert(t, user.Get("first.name").String() == "") } gjson-1.17.1/go.mod000066400000000000000000000001711456330511400140300ustar00rootroot00000000000000module github.com/tidwall/gjson go 1.12 require ( github.com/tidwall/match v1.1.1 github.com/tidwall/pretty v1.2.0 ) gjson-1.17.1/go.sum000066400000000000000000000005201456330511400140530ustar00rootroot00000000000000github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA= github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM= github.com/tidwall/pretty v1.2.0 h1:RWIZEg2iJ8/g6fDDYzMpobmaoGh5OLl4AXtGUGPcqCs= github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU= gjson-1.17.1/logo.png000066400000000000000000000371001456330511400143720ustar00rootroot00000000000000PNG  IHDR S?ytEXtSoftwareAdobe ImageReadyqe<iTXtXML:com.adobe.xmp Kz:TIDATx_H]gaF*5Ti4d zK9{ EQ apnwwP( *S@`AJC3615Q8\u֟MNΙ3k=k糾ϟ}`߾xxxxxxxxxxx._wWQEy9HZ~|8]hN q/6i/dˇx x`a]¢$/b@z!Rr(\QAB{eH]@.C vH;P) k1kˉ2O@ȑ x${dt$؉$''0יx`t*MK*?;#ekh%w7mu4H$)yRZK@^ H< ؐ $ ~4(̿x9x`boxܶ+"! R͓kSO07H2xskw'lCi\r•Sj_9N%2(\C͆hH/tܙl 1/M IK+ -~;SYg08!pXv:u-7ܴ:o4}9/NJcpTئEGmgnoxs@h'9x?ZY\FNߗИ́FS!H Q=pQ?|]AS{GoŊk;Io]rHޠKnl)>I6HH޶uWpm  2ȣZQ\4p~L$Jqk7yXv2c3;lޥn9z5 ܵ1o ßçOO~ dNz}Hv<ޑ8VV_Wrir5tޤ{lcA`&1=W2@& Bw޷")=&rx( xDW#1᣻_d~tϧ1YFb!#UtBߟ=8aFpK8J*CJ?:x3z6]X\}TztOF``ygWb;Ԕ< ?-ăL/ Lj#tx*b;0@\g&0/NT0(N)t$$ 0jr4L,UR]13bzHݶ?zoWئACg_i+7i#߂yIvۋVqmA}IBMG=l[W)5]Ylu=#]Pk+ r{AǟEy9WNtʿZ>K+}}Q%`cWؠlS)O?{~A!H׬l !MJk3׻hu`7*6{09E_hzX#m$)_j;)G_"16{:;շ޳3~Po4,Uޏ1 iaK7Vw0ݰgP,ă =*Hb"$?!L8MiWm,Ee$CBWs]P?N7?CV!9テ)01 ǃĴZSg$K[m0j^u!@\0+;zHߤ|DĘLt9nURgƚQ@᣿ΔK[y0بcOEC6F{ڗqq,7lЂE9\)hw7ж( x7hviŸ]Q75gʍEHO'2W ꁤ%F6}+Up_g+diHn11!P@4gҿxmULj6e NY{{xW5t l}kFz2iGh\JmU 4 j,? kjgyOCQJ*^?LGePY2:7s2BxmCh>梀Avz;0(;\=a:'ij#e _Yv}ǜAh}aԛ@<'>~fJlt h~GhRNBس&sx¢fM/CEx`Oؽd?6T?i?4-fq(uK]uF_qƔ>3FCMxK+lSoaLda{Wv۔1Vۗn¢@<؃)j^|asBBaBu[;U]]ԔQs/g&-W|{JϸcljV. 86jKssm΍5#0;7fl A{"/+-y5YdI̓{ySm I]@<01||cuwݟm Wůh.߇$)]5vg-?wd)[Lj)ךc&U=jI,`F2`VMtD4XN Aiԏ,j;4227것u0"#Y\@pFx`CZ"J@zrsTlN4<܈6Ÿ !g٪Q@ aךEf$i)WjkuSZ=`]mTϜ =#@nhEy9,Xj\-ޢyey\p,~^=Ʀ# {@/J.r4w WwF)|bk+ S@ 5(Ku-7͹DB_ 4Wj*ˡyѿcva;\Ǽd-YH0*[g'o{pk / zo͠U#'Α<8B'gIU8@CăWѶL_T]S`_<d}Ǚ|EE<YއGAM pJzںoŊk;{F 3p.tDx)wH 7Y0h# k f;jYV&[mKeIh>}t'ZYQ@ xk9ONuJʐ0No!@x{mKM;BBǎR;7|m<0R '޺)aMO>5@@mr_Ȣ1\}IK9$0 ֠$EԎa?waӊBV.H&)>5vHR*;amLף@<oŊkY( `Λ5d 9!iF=yB LcR~gJ;Sف$ gG#U=#Ci4H,&Þ؁ @lɡHùo]mkngSj x+"jA^n1fV8[W_SuL-@+i3x@< cz&fNuu06͡FE<48'] ,5U@< @05=ݷby\T`l[ x `GJBh:Ph4pC\D+ݷY\.rp<)F?L! y_r(4pvTVRFH<%! s(#XY@$$W:oSu`s<`0w*!MYW! Ʀ30DyYʚBn7 P+!!i0$(3L2M $ N~Lrv061ÖrBBBݻ7u+,N<păTr8$PL;S W;Ky1ăt侠wI ?laut=Bʱm0 "x`}3¦@X8G6( 7G+.0*Z(KY/I }L'!“V ,ʹN!dzqăO4hO@k!aɑ[1^j)/ ^~6K<ج6@]^5`/bҤQ\t)ڄ x9@ȭ]8HgVj 'UfbxȢT5=hLP&W~2t>ڎKl}}ʹ Jxe_/FE]uO[lm[@}z9B)\@x@~>6B~AsSG}|t_9U=Tn)$xp7h9J;|}+v⭫ K=8${7p4W(Iadn $4w ,?~ Tq$h.0O&3െΛe}!fǠ .,ZEGY\3ԾrȈI3iL0' F G"JAйKw g&f:zFCtJmlxYJA@<0A?O/aE&]S ^1@lMMWEY 3D YHt`rez;gGv}7<. !${Zƈ1$,+ (_U TC\461}kלҁ;JV0e e?E@<0@nvVqcgݟ1Gp EHy\qmgxDP>; ǦLiM_ adQktx`Ԡo)HuȐ%B&+[fQAM]vDn?E]@`@#ZұASVJ}]#.+߰ҁ @;9?@CZ_̷>\+#\] V4ҜyқHP5慱dRȭSWExAx]nvV}Ӕ)Џx 㺛?W\ịyC7Lzg5`*FOV⾶3X/v =4fu %tMIuW)*găt\9unh|tOuU[gqpl:gć軜wW?/6wvE7W##Z;뙪&+fg>~gG'NgQzhVni}}tcVFyE3h!Eƍr֟x&&%kExeF:Lq4~5ˉJN1/$#:h\npfc'!lBd}L)Y3 mL'~?<2e OXqmg{H!AzP<$8ce_&.\찡j35 M?XT90sh^Y;(o5r6|t2B?3THhPۣڰw^V u-7Җ&aJ?_Mt:f++b=S.US!~`ݑG}Auvý|,?+x+g06-< r.4|%3e[WzF2F=#ȅIxZ|,!ϋc5;%.VzeݭE*VxGmHt~_u-7nz$ϕƌކysHu(_v@HM"lc~Rۙ%gV>qgLy+] Ր|2DMYv0j]tGO~wɮnnjŷm $Jˑcr\ć@:Vfv2|I6."D) 4 ͉.V7H> HaKO-j~uT+Kx@Lgؽ0-@zNƽ~|Y"ҋ3^8S{##=}MwdAakgДkP΢CFl8EKWuJgK#%/GK^(w}&&,NTtS_`7wr@: M]^-w6]8ZP@ș$ EndaL%5G 7Vv 73a+v8懲r#i!L23,W'oV ><\`୶f յܰ/C Q:Hqc:ÊQqXE!x旧M {bdQ ܥL[ϟқP[|ċ *&@58E8#ʎW +-4Fč5*dFe.Q@ (V1bdQOtW&J6` k)BB) )V1¶YWJl>NXJfg *ƶSBzݲu022/eqlEx`|9V,JOKwo0EAjc##GR^;V_yu,Juݷbc3Ik \};i;,rF?q[")yLN8! z3q -=|&f[֤"%eg6NpB&c.,Mozy9MhWi*lL<0yV/ rc:H=]vF -'gA66|\ӔP&)#p\ĭ G4eТ@<0>UyZYr>z[.Hs/ܩifmJ JizcølCBFqAȵ]$6LtGí`gLYe;2JHpX- T<&dxp:EKx`g_fO#n#vs&u=$0ܕ)'0n-\Ld~ cAjOģ;SK+>m=A-W%!{D1aMVp䑠6{P'\4o3AGNL$j@bd $~$$ ~ hˑ $yI!yއ-WùN2XHBPl鴙(kabxsO|z OEyM6C {3 qjË,eRrB>#xml'xcɟ$TI$( jP;^-W=xh3Pw'g|ތD')rT:yɇsr0ȝA IxGCM~TZX?ͭ*'Riby/OL_\{G S?9J(?zPvL*5*HO$fElh^XUD&Wq+z:*.C2C,QXPX[ Yy/6d(PI%ҥ+r>yO@nNϵ[ T08YlOpq$] ? Bw_3;Eyؿ<CJ8?_WaWqZi<#-l9T$[SG0)< M?xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx:_+؄tIENDB`