Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Guess we doin V now (a new validation framework) #459

Open
wants to merge 33 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
41687f2
Guess we doin V now
Tinche Nov 22, 2023
d554ef8
Conditional import
Tinche Nov 23, 2023
bf166d6
Fix import
Tinche Nov 23, 2023
0d3fe85
Why have nice things when we can not have them?
Tinche Nov 23, 2023
3e722a4
Test with no detailed validation
Tinche Nov 23, 2023
bb1cd92
A small benchmark, as a treat
Tinche Nov 23, 2023
dacd24b
Validator coverage
Tinche Nov 23, 2023
a016e78
Exclude protocols from coverage
Tinche Nov 23, 2023
ef206d2
Error handling coverage
Tinche Nov 23, 2023
6edb9eb
Improve coverage tweak
Tinche Nov 23, 2023
2fab457
Ignore assert_nevers for coverage
Tinche Nov 23, 2023
eada8ad
Set up mypy tests
Tinche Nov 24, 2023
e5f337b
Renaming also validates
Tinche Nov 24, 2023
0d6e695
Maybe fix tests?
Tinche Nov 24, 2023
5c5f9e6
`is_unique` validator
Tinche Nov 24, 2023
a7bffce
Fix type annotation
Tinche Nov 24, 2023
cb6118d
Clean up import
Tinche Nov 24, 2023
86fdd00
ignoring_none
Tinche Nov 25, 2023
dd61ea3
Add import for 3.8
Tinche Nov 25, 2023
fd057ad
More coverage
Tinche Nov 25, 2023
6e9ce20
Tests for all_elements_must
Tinche Nov 27, 2023
be9a406
Rename, more tests
Tinche Nov 27, 2023
0a85e4d
Remove unused import
Tinche Nov 27, 2023
ab5eb98
Relock
Tinche Jan 12, 2024
a23589b
Update lockfile
Tinche Jan 12, 2024
9a039cf
Introduce VAnnotation
Tinche Jan 23, 2024
07218d5
General purpose `ensure`
Tinche Feb 3, 2024
7405d20
Properly import Annotated
Tinche Feb 4, 2024
430d0e7
More fixes
Tinche Feb 4, 2024
0daf1c3
Fix tests
Tinche Feb 4, 2024
059d949
Work on dicts
Tinche Feb 6, 2024
44f1118
Fix merge
Tinche Feb 16, 2024
91b0367
Initial dataclass support, start of docs
Tinche Mar 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions HISTORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ can now be used as decorators and have gained new features.
([#472](https://github.com/python-attrs/cattrs/pull/472))
- The default union handler now also handles dataclasses.
([#426](https://github.com/python-attrs/cattrs/issues/426) [#477](https://github.com/python-attrs/cattrs/pull/477))
- **Potentially breaking**: `IterableValidationError`s now require their subexceptions to have appropriate notes attached.
This was always the case internally in _cattrs_, but is now required of errors produced outside too.
- Add support for [PEP 695](https://peps.python.org/pep-0695/) type aliases.
([#452](https://github.com/python-attrs/cattrs/pull/452))
- Add support for [PEP 696](https://peps.python.org/pep-0696/) `TypeVar`s with defaults.
Expand Down
5 changes: 2 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,6 @@ lint: ## check style with ruff and black
test: ## run tests quickly with the default Python
pdm run pytest -x --ff -n auto tests


test-all: ## run tests on every Python version with tox
tox

Expand All @@ -78,7 +77,7 @@ servedocs: docs ## compile the docs watching for changes
watchmedo shell-command -p '*.rst' -c '$(MAKE) -C docs html' -R -D .

bench-cmp:
pytest bench --benchmark-compare
pytest --benchmark-sort=fullname --benchmark-warmup=true --benchmark-warmup-iterations=5 --benchmark-group-by=fullname bench --benchmark-compare

bench:
pytest bench --benchmark-save base
pytest --benchmark-sort=fullname --benchmark-warmup=true --benchmark-warmup-iterations=5 --benchmark-group-by=fullname bench --benchmark-save base
57 changes: 27 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,31 +13,21 @@

---

**cattrs** is an open source Python library for structuring and unstructuring
data. _cattrs_ works best with _attrs_ classes, dataclasses and the usual
Python collections, but other kinds of classes are supported by manually
registering converters.

Python has a rich set of powerful, easy to use, built-in data types like
dictionaries, lists and tuples. These data types are also the lingua franca
of most data serialization libraries, for formats like json, msgpack, cbor,
yaml or toml.

Data types like this, and mappings like `dict` s in particular, represent
unstructured data. Your data is, in all likelihood, structured: not all
combinations of field names or values are valid inputs to your programs. In
Python, structured data is better represented with classes and enumerations.
_attrs_ is an excellent library for declaratively describing the structure of
your data, and validating it.

When you're handed unstructured data (by your network, file system, database...),
_cattrs_ helps to convert this data into structured data. When you have to
convert your structured data into data types other libraries can handle,
_cattrs_ turns your classes and enumerations into dictionaries, integers and
strings.

Here's a simple taste. The list containing a float, an int and a string
gets converted into a tuple of three ints.
**cattrs** is an open source Python library for structuring and unstructuring data.
_cattrs_ works best with _attrs_ classes, dataclasses and the usual Python collections, but other kinds of classes are supported by manually registering converters.

Python has a rich set of powerful, easy to use, built-in data types like dictionaries, lists and tuples.
These data types are also the lingua franca of most data serialization libraries, for formats like json, msgpack, cbor, yaml or toml.

Data types like this, and mappings like `dict` s in particular, represent unstructured data.
Your data is, in all likelihood, structured: not all combinations of field names or values are valid inputs to your programs.
In Python, structured data is better represented with classes and enumerations.
_attrs_ is an excellent library for declaratively describing the structure of your data and validating it.

When you're handed unstructured data (by your network, file system, database...), _cattrs_ helps to convert this data into structured data.
When you have to convert your structured data into data types other libraries can handle, _cattrs_ turns your classes and enumerations into dictionaries, integers and strings.

Here's a simple taste. The list containing a float, an int and a string gets converted into a tuple of three ints.

```python
>>> import cattrs
Expand Down Expand Up @@ -68,7 +58,7 @@ Here's a much more complex example, involving _attrs_ classes with type metadata

```python
>>> from enum import unique, Enum
>>> from typing import Optional, Sequence, Union
>>> from typing import Sequence
>>> from cattrs import structure, unstructure
>>> from attrs import define, field

Expand All @@ -91,14 +81,18 @@ Here's a much more complex example, involving _attrs_ classes with type metadata
>>> @define
... class Dog:
... cuteness: int
... chip: Optional[DogMicrochip] = None
... chip: DogMicrochip | None = None

>>> p = unstructure([Dog(cuteness=1, chip=DogMicrochip(chip_id=1, time_chipped=10.0)),
... Cat(breed=CatBreed.MAINE_COON, names=('Fluffly', 'Fluffer'))])
>>> p = unstructure(
... [
... Dog(cuteness=1, chip=DogMicrochip(chip_id=1, time_chipped=10.0)),
... Cat(CatBreed.MAINE_COON, names=('Fluffly', 'Fluffer'))
... ]
... )

>>> print(p)
[{'cuteness': 1, 'chip': {'chip_id': 1, 'time_chipped': 10.0}}, {'breed': 'maine_coon', 'names': ('Fluffly', 'Fluffer')}]
>>> print(structure(p, list[Union[Dog, Cat]]))
>>> print(structure(p, list[Dog | Cat]))
[Dog(cuteness=1, chip=DogMicrochip(chip_id=1, time_chipped=10.0)), Cat(breed=<CatBreed.MAINE_COON: 'maine_coon'>, names=['Fluffly', 'Fluffer'])]
```

Expand Down Expand Up @@ -151,6 +145,9 @@ _cattrs_ is based on a few fundamental design decisions.
- Un/structuring rules are separate from the models.
This allows models to have a one-to-many relationship with un/structuring rules, and to create un/structuring rules for models which you do not own and you cannot change.
(_cattrs_ can be configured to use un/structuring rules from models using the [`use_class_methods` strategy](https://catt.rs/en/latest/strategies.html#using-class-specific-structure-and-unstructure-methods).)
- Strongly lean on function composition.
Almost all problems in _cattrs_ can be solved by writing and composing functions (called _hooks_), instead of writing classes and subclassing.
This makes _cattrs_ code elegant, concise, powerful and amenable to all the rich Python ways of working with functions.
- Invent as little as possible; reuse existing ordinary Python instead.
For example, _cattrs_ did not have a custom exception type to group exceptions until the sanctioned Python [`exceptiongroups`](https://docs.python.org/3/library/exceptions.html#ExceptionGroup).
A side-effect of this design decision is that, in a lot of cases, when you're solving _cattrs_ problems you're actually learning Python instead of learning _cattrs_.
Expand Down
26 changes: 13 additions & 13 deletions bench/test_attrs_collections.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
from enum import IntEnum
from typing import Dict, List, Mapping, MutableMapping

import attr
import pytest
from attrs import define, frozen

from cattr import BaseConverter, Converter, UnstructureStrategy
from cattrs import BaseConverter, Converter, UnstructureStrategy


@pytest.mark.parametrize("converter_cls", [BaseConverter, Converter])
Expand All @@ -21,7 +21,7 @@ class E(IntEnum):
ONE = 1
TWO = 2

@attr.define
@define
class C:
a: List[int]
b: List[float]
Expand Down Expand Up @@ -62,32 +62,32 @@ class C:
[1] * 3,
[1.0] * 3,
["a small string"] * 3,
["test".encode()] * 3,
[b"test"] * 3,
[E.ONE] * 3,
[2] * 3,
[2.0] * 3,
["a small string"] * 3,
["test".encode()] * 3,
[b"test"] * 3,
[E.TWO] * 3,
[3] * 3,
[3.0] * 3,
["a small string"] * 3,
["test".encode()] * 3,
[b"test"] * 3,
[E.ONE] * 3,
[4] * 3,
[4.0] * 3,
["a small string"] * 3,
["test".encode()] * 3,
[b"test"] * 3,
[E.TWO] * 3,
[5] * 3,
[5.0] * 3,
["a small string"] * 3,
["test".encode()] * 3,
[b"test"] * 3,
[E.ONE] * 3,
[6] * 3,
[6.0] * 3,
["a small string"] * 3,
["test".encode()] * 3,
[b"test"] * 3,
[E.TWO] * 3,
),
)
Expand All @@ -102,11 +102,11 @@ def test_unstructure_attrs_mappings(benchmark, converter_cls, unstructure_strat)
Benchmark an attrs class containing mappings.
"""

@attr.frozen
@frozen
class FrozenCls:
a: int

@attr.define
@define
class C:
a: Mapping[int, str]
b: Dict[float, bytes]
Expand All @@ -130,11 +130,11 @@ def test_structure_attrs_mappings(benchmark, converter_cls):
Benchmark an attrs class containing mappings.
"""

@attr.frozen
@frozen
class FrozenCls:
a: int

@attr.define
@define
class C:
a: Mapping[int, str]
b: Dict[float, bytes]
Expand Down
40 changes: 20 additions & 20 deletions bench/test_attrs_nested.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
"""Benchmark attrs containing other attrs classes."""
import attr
import pytest
from attrs import define

from cattr import BaseConverter, Converter, UnstructureStrategy
from cattrs import BaseConverter, Converter, UnstructureStrategy


@pytest.mark.parametrize("converter_cls", [BaseConverter, Converter])
Expand All @@ -12,42 +12,42 @@
def test_unstructure_attrs_nested(benchmark, converter_cls, unstructure_strat):
c = converter_cls(unstruct_strat=unstructure_strat)

@attr.define
@define
class InnerA:
a: int
b: float
c: str
d: bytes

@attr.define
@define
class InnerB:
a: int
b: float
c: str
d: bytes

@attr.define
@define
class InnerC:
a: int
b: float
c: str
d: bytes

@attr.define
@define
class InnerD:
a: int
b: float
c: str
d: bytes

@attr.define
@define
class InnerE:
a: int
b: float
c: str
d: bytes

@attr.define
@define
class Outer:
a: InnerA
b: InnerB
Expand All @@ -56,11 +56,11 @@ class Outer:
e: InnerE

inst = Outer(
InnerA(1, 1.0, "one", "one".encode()),
InnerB(2, 2.0, "two", "two".encode()),
InnerC(3, 3.0, "three", "three".encode()),
InnerD(4, 4.0, "four", "four".encode()),
InnerE(5, 5.0, "five", "five".encode()),
InnerA(1, 1.0, "one", b"one"),
InnerB(2, 2.0, "two", b"two"),
InnerC(3, 3.0, "three", b"three"),
InnerD(4, 4.0, "four", b"four"),
InnerE(5, 5.0, "five", b"five"),
)

benchmark(c.unstructure, inst)
Expand All @@ -73,49 +73,49 @@ class Outer:
def test_unstruct_attrs_deep_nest(benchmark, converter_cls, unstructure_strat):
c = converter_cls(unstruct_strat=unstructure_strat)

@attr.define
@define
class InnerA:
a: int
b: float
c: str
d: bytes

@attr.define
@define
class InnerB:
a: InnerA
b: InnerA
c: InnerA
d: InnerA

@attr.define
@define
class InnerC:
a: InnerB
b: InnerB
c: InnerB
d: InnerB

@attr.define
@define
class InnerD:
a: InnerC
b: InnerC
c: InnerC
d: InnerC

@attr.define
@define
class InnerE:
a: InnerD
b: InnerD
c: InnerD
d: InnerD

@attr.define
@define
class Outer:
a: InnerE
b: InnerE
c: InnerE
d: InnerE

make_inner_a = lambda: InnerA(1, 1.0, "one", "one".encode())
make_inner_a = lambda: InnerA(1, 1.0, "one", b"one")
make_inner_b = lambda: InnerB(*[make_inner_a() for _ in range(4)])
make_inner_c = lambda: InnerC(*[make_inner_b() for _ in range(4)])
make_inner_d = lambda: InnerD(*[make_inner_c() for _ in range(4)])
Expand Down
Loading
Loading