PEP 764: Inlined typed dictionaries #4082

Viicos · 2024-10-24T21:04:45Z

Basic requirements (all PEP Types)

Standards Track requirements

📚 Documentation preview 📚: https://pep-previews--4082.org.readthedocs.build/pep-0764/

AA-Turner · 2024-10-24T21:34:23Z

Thank you for opening this PR. PEPs require a core developer or PEP editor who is willing to sponsor it, which your PR currently lacks. I would advise you to start a thread on the https://discuss.python.org/ forum to see if there is support for this proposal and if someone is willing to sponsor it.

A

erictraut

Thanks for writing this. From my perspective, it's looking pretty good.

I added some comments and questions for your consideration.

erictraut · 2024-10-24T21:22:32Z

peps/pep-9995.rst

+don't have a name. For this reason, their :attr:`~type.__name__` attribute
+will be set to an empty string.
+
+It is not possible to specify any class arguments such as ``total``.


I think that Required, NotRequired and ReadOnly should be permitted here. Absent any mention of this, the typing spec could be interpreted as disallowing this. It's better to make it clear.

I presume that all inlined TypedDicts are implicitly "total" (i.e. all items are assumed to be Required unless explicitly NotRequired). This makes sense because it's consistent with precedent, but it would be good to spell it out in the spec.

I think it's also worth considering making all inlined TypedDicts implicitly "closed" (as defined in draft PEP 728). This would deviate from precedent, but I think one could make a good argument for why inlined TypedDicts should be closed.

I think that Required, NotRequired and ReadOnly should be permitted here. Absent any mention of this, the typing spec could be interpreted as disallowing this. It's better to make it clear.

I presume that all inlined TypedDicts are implicitly "total" (i.e. all items are assumed to be Required unless explicitly NotRequired). This makes sense because it's consistent with precedent, but it would be good to spell it out in the spec.

Updated.

I think it's also worth considering making all inlined TypedDicts implicitly "closed" (as defined in draft PEP 728). This would deviate from precedent, but I think one could make a good argument for why inlined TypedDicts should be closed.

Sounds reasonable. I have to say I'm still a bit confused with PEP 728 and closed=True. If we disallow subclassing inlined typed dictionaries, it means that every inlined typed dictionary is implicitly @final. What's the purpose of having closed=True then? I guess I should ask my question in the PEP 728 discussion thread instead.

peps/pep-9995.rst

erictraut · 2024-10-24T21:34:08Z

peps/pep-9995.rst

+don't have a name. For this reason, their :attr:`~type.__name__` attribute
+will be set to an empty string.
+
+It is not possible to specify any class arguments such as ``total``.


Am I correct in assuming that there is no way for an inlined TypedDict to "extend" another (named) TypedDict? One could imagine supporting this using dictionary expansion syntax, but that adds complexity that may not be justifiable. It would also impose additional runtime requirements on TypedDict classes (they'd need to be iterable).

A good way to support inheritance could be to allow more than one argument to the subscript: TypedDict[Base, {"a": int}].

peps/pep-9995.rst

erictraut · 2024-10-24T21:38:32Z

peps/pep-9995.rst

+  parametrization overloads. On ther other hand, :class:`~typing.TypedDict` is
+  already a :term:`special form <typing:special form>`.
+
+* If fufure work extends what inlined typed dictionaries can do, we don't have


Did you consider reusing typing.Dict here? I can see some potential advantages. It's less verbose, doesn't have the baggage of builtins.dict, and the fact that this is a "typed" dict is already implicit given that it is imported from typing and involves type annotations.

Showing my TypeScript bias here 😅 but I'm wondering how sacrilegious it would be to avoid the subscription all together and simply use the dictionary itself?

def get_movie() -> {'name': str, 'year': int}: return { 'name': 'Blade Runner', 'year': 1982, }

This causes problems with the | operator at least. I've been thinking of writing up something like "Why can't we ...?" discussing why certain "obvious" ways to make typing more ergonomic may not work well in practice.

I think that such writeup would be very useful!

I added an extra section in the rejected ideas regarding the plain dict syntax.

On using typing.Dict, I kind of like the idea for the reasons you mentioned @erictraut. Dict[{...}] implicitly spells out as something like:

Dict[{...}] | |------> ...with some typed fields |--> a dict...

However, I still think this is debatable for a couple reasons:

typing.Dict is currently marked as deprecated in the Python documentation (although not scheduled for removal). It might be confusing to undeprecate it.

The same subscripting syntax — with the only difference being the number of type arguments — means two different things. Can be confusing as well.

I'll add an entry to the open issues so that we can further discuss this option.

peps/pep-9995.rst

erictraut · 2024-10-24T21:45:06Z

peps/pep-9995.rst

+Subclassing an inlined typed dictionary
+---------------------------------------
+
+Should we allow the following?::


No, this shouldn't be allowed by a static type checker. In this example, InlinedTD is a type alias.

If you change this to InlinedTD = TypedDict('InlinedTD', {'a': int}), then it should be allowed (as it is today) because InlinedTD is now a class, which can be used as a base class.

Yes, I think we should also discuss whether an inlined typed dictonary is a proper class or an instance of some new internal typing._InlinedTypedDict class. As specified currently in this PEP, an inlined typed dictionary is a also a class, with an empty string for __name__ (which isn't ideal to be fair). Initially, I wanted inlined typed dictionaries to be an instance of some _InlinedTypedDict class, but this complicates the runtime behavior. For example, should we allow accessing the introspection attributes?

InlinedTD = TypedDict[{'a': int}] InlinedTD.__required_keys__ InlinedTD.__total__ # etc

erictraut · 2024-10-24T21:51:36Z

@Viicos, as a member of the typing council, I'm willing to sponsor this PEP.

JelleZijlstra · 2024-10-24T23:49:01Z

Reopened as Eric offered to sponsor (thanks Eric!).

Viicos · 2024-10-25T06:50:28Z

I just realized I opened this PR on the original repository, but meant to open it on my fork as part of this discussion.

But I'm happy to keep going with this PEP. Regardless of the chosen implementation for partial, pick, etc, I can see how inlined typed dictionaries can be useful (Partial[SomeTD] needs to return a new typed dictionary, and this raises the question of which __name__ to choose — inlined typed dictionaries fulfill this requirement).

Thanks for the feedback. Should I open a separate discussion on the forum? Should it be in the PEPs/Typing category?

hugovk

I just realized I opened this PR on the original repository, but meant to open it on my fork as part of this discussion.

Whichever you prefer :) Probably depends on how much more work you think it needs before you're ready to submit this to be merged as a draft.

Should I open a separate discussion on the forum? Should it be in the PEPs/Typing category?

Wait until this draft is merged, then open a new discussion in the Typing category so people can discuss the merged and proposed draft (see https://peps.python.org/pep-0001/#discussing-a-pep).

Also please review PEP 1 and PEP 12 for the PEP process.

I've updated the OP to include the new PEP checklist, please work through it and check off things as appropriate. Thanks!

peps/pep-9995.rst

erictraut · 2024-10-29T16:15:19Z

peps/pep-9995.rst

+
+The new inlined syntax can be used to resolve these problems::
+
+    def get_movie() -> TypedDict[{'name': str, year: int, 'production': {'name': str, 'location': str}}]:


I don't think we should support nested inlined TypedDict definitions as shown above. We should require that the type annotation for each item be a valid type expression, and a dictionary expression not enclosed by a TypedDict is an invalid type expression. The above example should therefore be changed to:

def get_movie() -> TypedDict[{"name": str, "year": int, "production": TypedDict[{"name": str, "location": str}]}]: ...

The grammar definition proposed above by Jelle already implies this, but the spec should make it clear that using nested dictionary expressions (like in the current code sample) is not allowed and should result in a type checker error.

I understand this might seem needlessly verbose, but if we allow dictionary expressions in type expressions outside of a TyepdDict type argument, it will create a cascade of exceptions and future composability issues. For example, we'd need to support raw dictionary expressions within Required, NotRequired, and ReadOnly.

Indeed using a nested inlined typed dictionary is more consistent! Changed.

(Using typing.Dict instead of TypedDict might help reducing verbosity here).

peps/pep-9995.rst

erictraut · 2024-10-29T20:45:24Z

peps/pep-9995.rst

+    def test_values() -> {"int": int, "str": str}:
+        return {"int": 42, "str": "test"}
+
+Pyright has added support in version `1.1.297`_ (using :class:`dict`), but was later


Pyright 1.1.387, which will be published in the next day, includes experimental support for the latest draft of this PEP.

Amazing, updated to reference the future release.

Some GH discussions are left opened for now, waiting for an answer.

sydney-runkle

@Viicos, awesome job here! So exciting to see this awesome contribution!!

peps/pep-0764.rst

sydney-runkle · 2024-10-31T12:50:52Z

peps/pep-0764.rst

+Although it is not possible to specify any class arguments such as ``total``,
+any :external+typing:term:`type qualifier` can be used for individual fields::
+
+    Movie = TypedDict[{'name': NotRequired[str], 'year': ReadOnly[int]}]


I wonder, is there a world in which eventually you could do something like:

Movie = Annotated[TypedDict[{'name': str, 'year': int}], Total(False)]

Or something like that for class args?

This would require:

introducing a typing.Total class/function

setting a precedent regarding the meaning of Annotated metadata, which is currently ignored by type checkers.

peps/pep-0764.rst

sydney-runkle · 2024-10-31T12:57:09Z

peps/pep-0764.rst

+While using :class:`dict` isn't ideal, we could make use of
+:class:`typing.Dict` with a single argument::
+
+    def get_movie() -> Dict[{'title': str}]: ...


Hmm, I like the strict parametrization of TypedDict better, I feel like this would set an odd precedent and confuse some users.

ZeroIntensity · 2024-10-31T16:40:30Z

Apologies for the small (now undone) edit of one of the checkmarks, I was reading the PR and pushed the wrong button 😅

Inlined typed dictionaries PEP

c364510

AA-Turner closed this Oct 24, 2024

erictraut reviewed Oct 24, 2024

View reviewed changes

JelleZijlstra reopened this Oct 24, 2024

hugovk added the new-pep A new draft PEP submitted for initial review label Oct 25, 2024

hugovk reviewed Oct 25, 2024

View reviewed changes

peps/pep-9995.rst Outdated Show resolved Hide resolved

peps/pep-9995.rst Outdated Show resolved Hide resolved

hugovk changed the title ~~Inlined typed dictionaries PEP~~ PEP 764: Inlined typed dictionaries Oct 25, 2024

Starting incorporating feedback

ac5786b

JelleZijlstra reviewed Oct 28, 2024

View reviewed changes

peps/pep-9995.rst Outdated Show resolved Hide resolved

erictraut reviewed Oct 29, 2024

View reviewed changes

Viicos added 2 commits October 29, 2024 22:09

Applied remaining feedback

c058d0e

Some GH discussions are left opened for now, waiting for an answer.

Rename PEP, fix headers

76dc942

Viicos force-pushed the inlined-typed-dictionaries branch 2 times, most recently from f87336c to 2386695 Compare October 29, 2024 21:46

Misc. fixes

2e0c379

Viicos force-pushed the inlined-typed-dictionaries branch from 2386695 to 2e0c379 Compare October 30, 2024 08:29

sydney-runkle reviewed Oct 31, 2024

View reviewed changes

Sydney's feedback

2857d69

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PEP 764: Inlined typed dictionaries #4082

PEP 764: Inlined typed dictionaries #4082

Viicos commented Oct 24, 2024 •

edited by ZeroIntensity

Loading

AA-Turner commented Oct 24, 2024

erictraut left a comment

erictraut Oct 24, 2024

Viicos Oct 29, 2024

erictraut Oct 24, 2024

JelleZijlstra Oct 28, 2024

erictraut Oct 24, 2024

tomasr8 Oct 27, 2024

JelleZijlstra Oct 27, 2024

tomasr8 Oct 28, 2024

Viicos Oct 29, 2024

erictraut Oct 24, 2024

Viicos Oct 29, 2024

erictraut commented Oct 24, 2024

JelleZijlstra commented Oct 24, 2024

Viicos commented Oct 25, 2024

hugovk left a comment

erictraut Oct 29, 2024 •

edited

Loading

Viicos Oct 29, 2024 •

edited

Loading

erictraut Oct 29, 2024

Viicos Oct 29, 2024

sydney-runkle left a comment

sydney-runkle Oct 31, 2024

Viicos Oct 31, 2024

sydney-runkle Oct 31, 2024

ZeroIntensity commented Oct 31, 2024


		The new inlined syntax can be used to resolve these problems::

		def get_movie() -> TypedDict[{'name': str, year: int, 'production': {'name': str, 'location': str}}]:

PEP 764: Inlined typed dictionaries #4082

Are you sure you want to change the base?

PEP 764: Inlined typed dictionaries #4082

Conversation

Viicos commented Oct 24, 2024 • edited by ZeroIntensity Loading

Basic requirements (all PEP Types)

Standards Track requirements

AA-Turner commented Oct 24, 2024

erictraut left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

erictraut commented Oct 24, 2024

JelleZijlstra commented Oct 24, 2024

Viicos commented Oct 25, 2024

hugovk left a comment

Choose a reason for hiding this comment

erictraut Oct 29, 2024 • edited Loading

Choose a reason for hiding this comment

Viicos Oct 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sydney-runkle left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ZeroIntensity commented Oct 31, 2024

Viicos commented Oct 24, 2024 •

edited by ZeroIntensity

Loading

erictraut Oct 29, 2024 •

edited

Loading

Viicos Oct 29, 2024 •

edited

Loading