Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dasel V3 #432

Draft
wants to merge 58 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
c9f0b5c
Initial start on selector parser rewrite
TomWright Sep 24, 2024
20f73ab
Add spread selectors
TomWright Sep 24, 2024
e09907d
Start getting v3 core functionality working
TomWright Sep 30, 2024
f4427b6
Fix issues
TomWright Sep 30, 2024
daef53f
Merge branch 'master' into v3
TomWright Sep 30, 2024
1657e5d
Get initial binary expressions working
TomWright Oct 1, 2024
5058ae6
Get binary binding powers working
TomWright Oct 1, 2024
b792aaf
Add a null token
TomWright Oct 1, 2024
28cd36a
Implement grouping
TomWright Oct 1, 2024
ff73190
Add string concat test
TomWright Oct 1, 2024
174ec83
Add a test for setting evaluated fields
TomWright Oct 1, 2024
bf9f18f
More progress
TomWright Oct 1, 2024
482c6e0
More setup
TomWright Oct 2, 2024
d92a503
General fixes and improvements
TomWright Oct 3, 2024
1f26dbd
Add if/elseif/else statements
TomWright Oct 5, 2024
dbf367f
Small cleanup
TomWright Oct 5, 2024
2f36b1f
More improvements
TomWright Oct 6, 2024
25b2604
Update container.yaml for new selector format
TomWright Oct 6, 2024
bb3b2b4
Fix build-test.yaml
TomWright Oct 6, 2024
5d3379f
Update tests
TomWright Oct 6, 2024
8f1ea53
Delete the old dasel implementation
TomWright Oct 6, 2024
f8a5e27
Add initial branch support
TomWright Oct 7, 2024
0a7aee3
Add array support
TomWright Oct 7, 2024
1a1e6c9
More progress
TomWright Oct 10, 2024
9010bd5
Add filter support
TomWright Oct 10, 2024
797f001
Simplify map expr
TomWright Oct 10, 2024
605ec48
Add regex support
TomWright Oct 10, 2024
d857cb9
Add missing error handling
TomWright Oct 10, 2024
9999c1b
Improve spreading into object defs and rework functions
TomWright Oct 18, 2024
8bbae1e
Ensure variables are propagated and update root command
TomWright Oct 19, 2024
7a9a924
Fix tests
TomWright Oct 19, 2024
15bffc6
Fix toString and remove test file
TomWright Oct 19, 2024
5d6576e
Add reverse()
TomWright Oct 19, 2024
1b27839
Update tests and fix range
TomWright Oct 20, 2024
cfa37ca
Implement sortBy func
TomWright Oct 20, 2024
bfb0bd8
Rework JSON parser
TomWright Oct 22, 2024
e081e49
Add unary expr support and tests
TomWright Oct 23, 2024
9dbe990
Migrate to kong, make parsers extensible, add tests
TomWright Oct 24, 2024
48ec95d
Add version command
TomWright Oct 24, 2024
662076f
Default to query command
TomWright Oct 24, 2024
9f8b20e
Merge branch 'master' into v3
TomWright Oct 24, 2024
dd5953c
Remove legacy dencoding package
TomWright Oct 24, 2024
28d9744
Fix default command
TomWright Oct 24, 2024
31c8096
Remove unused code
TomWright Oct 24, 2024
99a9e8b
Fix CLI output and improve branching capability
TomWright Oct 24, 2024
0ffe7cc
Fix --root flag
TomWright Oct 24, 2024
ba0aa78
Add mix, max and fix IsNull issue
TomWright Oct 24, 2024
3ac9c8b
Reimplement yaml parser
TomWright Oct 27, 2024
4ef342c
Start implementing CSV parser
TomWright Oct 27, 2024
bbc873f
Add coalesce operator and unstable flag
TomWright Oct 28, 2024
de29bd2
Add interactive mode, fixes and improvements
TomWright Oct 29, 2024
f6f2be3
Pass stdout option
TomWright Oct 29, 2024
d67cb15
Fix bug with parsing unfinished quoted strings
TomWright Oct 29, 2024
11c723e
General fixes and improvements
TomWright Oct 30, 2024
a6074e8
Start implementing HCL parser
TomWright Oct 31, 2024
b5aa01d
HCL improvements
TomWright Oct 31, 2024
38ec57f
Deregister unimplemented XML writer
TomWright Oct 31, 2024
7e48c95
Implement first draft of HCL writer
TomWright Nov 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**

- OS: [e.g. iOS]
- Version [e.g. 22] (`dasel --version`)
- Version [e.g. 22] (`dasel version`)

**Additional context**
Add any other context about the problem here.
6 changes: 3 additions & 3 deletions .github/workflows/build-dev.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -65,11 +65,11 @@ jobs:
- uses: actions/checkout@v3
- uses: actions/setup-go@v4
with:
go-version: '^1.21.0' # The Go version to download (if necessary) and use.
go-version: '^1.23.0' # The Go version to download (if necessary) and use.
- name: Set env
run: echo RELEASE_VERSION=development >> $GITHUB_ENV
- name: Build
run: GOOS=${{ matrix.os }} GOARCH=${{ matrix.arch }} CGO_ENABLED=0 go build -o target/release/${{ matrix.artifact_name }} -ldflags="-X 'github.com/tomwright/dasel/v2/internal.Version=${{ env.RELEASE_VERSION }}'" ./cmd/dasel
run: GOOS=${{ matrix.os }} GOARCH=${{ matrix.arch }} CGO_ENABLED=0 go build -o target/release/${{ matrix.artifact_name }} -ldflags="-X 'github.com/tomwright/dasel/v3/internal.Version=${{ env.RELEASE_VERSION }}'" ./cmd/dasel
- name: Test version
if: matrix.test_version == true
run: ./target/release/${{ matrix.artifact_name }} --version
run: ./target/release/${{ matrix.artifact_name }} version
8 changes: 4 additions & 4 deletions .github/workflows/build-test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -77,15 +77,15 @@ jobs:
- uses: actions/checkout@v3
- uses: actions/setup-go@v4
with:
go-version: '^1.21.0' # The Go version to download (if necessary) and use.
go-version: '^1.23.0' # The Go version to download (if necessary) and use.
- name: Set env
run: echo RELEASE_VERSION=development >> $GITHUB_ENV
- name: Build
run: GOOS=${{ matrix.os }} GOARCH=${{ matrix.arch }} CGO_ENABLED=0 go build -o target/release/${{ matrix.artifact_name }} -ldflags="-X 'github.com/tomwright/dasel/v2/internal.Version=${{ env.RELEASE_VERSION }}'" ./cmd/dasel
run: GOOS=${{ matrix.os }} GOARCH=${{ matrix.arch }} CGO_ENABLED=0 go build -o target/release/${{ matrix.artifact_name }} -ldflags="-X 'github.com/tomwright/dasel/v3/internal.Version=${{ env.RELEASE_VERSION }}'" ./cmd/dasel
- name: Test version
if: matrix.test_version == true
run: ./target/release/${{ matrix.artifact_name }} --version
run: ./target/release/${{ matrix.artifact_name }} version
- name: Test execution
if: matrix.test_execution == true
run: |
echo '{"hello": "World"}' | ./target/release/${{ matrix.artifact_name }} -r json 'hello'
echo '{"hello": "World"}' | ./target/release/${{ matrix.artifact_name }} -i json 'hello'
6 changes: 3 additions & 3 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -73,14 +73,14 @@ jobs:
- uses: actions/checkout@v3
- uses: actions/setup-go@v4
with:
go-version: '^1.21.0'
go-version: '^1.23.0'
- name: Set env
run: echo RELEASE_VERSION=${GITHUB_REF:10} >> $GITHUB_ENV
- name: Build
run: GOOS=${{ matrix.os }} GOARCH=${{ matrix.arch }} CGO_ENABLED=0 go build -o target/release/${{ matrix.artifact_name }} -ldflags="-X 'github.com/tomwright/dasel/v2/internal.Version=${{ env.RELEASE_VERSION }}'" ./cmd/dasel
run: GOOS=${{ matrix.os }} GOARCH=${{ matrix.arch }} CGO_ENABLED=0 go build -o target/release/${{ matrix.artifact_name }} -ldflags="-X 'github.com/tomwright/dasel/v3/internal.Version=${{ env.RELEASE_VERSION }}'" ./cmd/dasel
- name: Test version
if: matrix.test_version == true
run: ./target/release/${{ matrix.artifact_name }} --version
run: ./target/release/${{ matrix.artifact_name }} version
- name: Gzip binaries
run: gzip -c ./target/release/${{ matrix.artifact_name }} > ./target/release/${{ matrix.artifact_name }}.gz
- name: Upload binaries to release
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/container.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ jobs:
tags: dasel:test
- name: Test
run: |
echo '{"hello": "World"}' | docker run -i --rm dasel:test -r json 'hello'
echo '{"hello": "World"}' | docker run -i --rm dasel:test -i json 'hello'
- name: Set version tag variables
if: ${{ steps.version.outputs.is_valid == 'true' }}
run: |
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ jobs:
- name: Install Go
uses: actions/setup-go@v4
with:
go-version: '^1.21.0'
go-version: '^1.23.0'
- name: Checkout code
uses: actions/checkout@v3
- uses: actions/cache@v3
Expand Down
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Fixed

- Changed go module to `github.com/tomwright/dasel/v2` to ensure it works correctly with go modules.
- Changed go module to `github.com/tomwright/dasel/v3` to ensure it works correctly with go modules.

## [v2.1.0] - 2023-01-11

Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# dasel

[![Gitbook](https://badges.aleen42.com/src/gitbook_1.svg)](https://daseldocs.tomwright.me)
[![Go Report Card](https://goreportcard.com/badge/github.com/TomWright/dasel/v2)](https://goreportcard.com/report/github.com/TomWright/dasel/v2)
[![PkgGoDev](https://pkg.go.dev/badge/github.com/tomwright/dasel)](https://pkg.go.dev/github.com/tomwright/dasel/v2)
[![Go Report Card](https://goreportcard.com/badge/github.com/tomwright/dasel/v3)](https://goreportcard.com/report/github.com/tomwright/dasel/v3)
[![PkgGoDev](https://pkg.go.dev/badge/github.com/tomwright/dasel)](https://pkg.go.dev/github.com/tomwright/dasel/v3)
![Test](https://github.com/TomWright/dasel/workflows/Test/badge.svg)
![Build](https://github.com/TomWright/dasel/workflows/Build/badge.svg)
[![codecov](https://codecov.io/gh/TomWright/dasel/branch/master/graph/badge.svg)](https://codecov.io/gh/TomWright/dasel)
Expand Down Expand Up @@ -75,7 +75,7 @@ brew install dasel
You can also install a [development version](https://daseldocs.tomwright.me/installation#development-version) with:

```bash
go install github.com/tomwright/dasel/v2/cmd/dasel@master
go install github.com/tomwright/dasel/v3/cmd/dasel@master
```

For more information see the [installation documentation](https://daseldocs.tomwright.me/installation).
Expand Down Expand Up @@ -181,7 +181,7 @@ Please [open a discussion](https://github.com/TomWright/dasel/discussions) if:
- Uses a [standard query/selector syntax](https://daseldocs.tomwright.me/functions/selector-overview) across all data formats.
- Zero runtime dependencies.
- [Available on Linux, Mac and Windows](https://daseldocs.tomwright.me/installation).
- Available to [import and use in your own projects](https://pkg.go.dev/github.com/tomwright/dasel/v2).
- Available to [import and use in your own projects](https://pkg.go.dev/github.com/tomwright/dasel/v3).
- [Run via Docker](https://daseldocs.tomwright.me/installation#docker).
- [Faster than jq/yq](#benchmarks).
- [Pre-commit hooks](#pre-commit).
Expand Down
55 changes: 55 additions & 0 deletions api.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
// Package dasel contains everything you'll need to use dasel from a go application.
package dasel

import (
"github.com/tomwright/dasel/v3/execution"
"github.com/tomwright/dasel/v3/model"
)

// Query queries the data using the selector and returns the results.
func Query(data any, selector string, opts ...execution.ExecuteOptionFn) ([]*model.Value, int, error) {
options := execution.NewOptions(opts...)
val := model.NewValue(data)
out, err := execution.ExecuteSelector(selector, val, options)
if err != nil {
return nil, 0, err
}

if out.IsBranch() {
res := make([]*model.Value, 0)
if err := out.RangeSlice(func(i int, v *model.Value) error {
res = append(res, v)
return nil
}); err != nil {
return nil, 0, err
}
return res, len(res), nil
}

return []*model.Value{out}, 1, nil
}

func Select(data any, selector string, opts ...execution.ExecuteOptionFn) (any, int, error) {
res, count, err := Query(data, selector, opts...)
if err != nil {
return nil, 0, err
}
out := make([]any, 0)
for _, v := range res {
out = append(out, v.Interface())
}
return out, count, err
}

func Modify(data any, selector string, newValue any, opts ...execution.ExecuteOptionFn) (int, error) {
res, count, err := Query(data, selector, opts...)
if err != nil {
return 0, err
}
for _, v := range res {
if err := v.Set(model.NewValue(newValue)); err != nil {
return 0, err
}
}
return count, nil
}
48 changes: 48 additions & 0 deletions api_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
package dasel_test

import (
"testing"

"github.com/google/go-cmp/cmp"
"github.com/tomwright/dasel/v3"
)

type modifyTestCase struct {
selector string
in any
value any
exp any
count int
}

func (tc modifyTestCase) run(t *testing.T) {
count, err := dasel.Modify(&tc.in, tc.selector, tc.value)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if count != tc.count {
t.Errorf("unexpected count: %d", count)
}
if !cmp.Equal(tc.exp, tc.in) {
t.Errorf("unexpected result: %s", cmp.Diff(tc.exp, tc.in))
}
}

func TestModify(t *testing.T) {
t.Run("index", func(t *testing.T) {
t.Run("int over int", modifyTestCase{
selector: "$this[1]",
in: []int{1, 2, 3},
value: 4,
exp: []int{1, 4, 3},
count: 1,
}.run)
t.Run("string over int", modifyTestCase{
selector: "$this[1]",
in: []any{1, 2, 3},
value: "4",
exp: []any{1, "4", 3},
count: 1,
}.run)
})
}
113 changes: 0 additions & 113 deletions benchmark/README.md

This file was deleted.

25 changes: 0 additions & 25 deletions benchmark/data.json

This file was deleted.

18 changes: 0 additions & 18 deletions benchmark/data.yaml

This file was deleted.

Loading
Loading