What should be in Pyrefly V1?

[rchen152]

Starting a discussion about what should be in Pyrefly's V1 Release. Here's a list of ideas we kicked around in yesterday's eng sync:

• Reasonably complete implementation of the typing specification, as measured by how conformant Pyrefly is
• Excellent error messages
• A "gradual" mode or config options to disable certain types of type inference (e.g., empty container inference)
• Support for libraries with mypy plugins

My thoughts:

• We should aim for feature completeness but not target a particular conformance % - I worry about over-focusing on the edge cases in the conformance tests, which aren't necessarily representative of real-world code.
• We definitely need better error messages.
• Gradual (and strict) modes can be post-V1. We should instead focus on fixing issues with our type inference like Make None generalise to Optional[T] #226 and Infer type based on all usages, rather than first use #382 to make the out-of-the-box experience with Pyrefly the best it can be.
• I'm on the fence about how to prioritize plugins. We've gotten a ton of questions about them - mypy's support for a lot of dynamic patterns is clearly valuable. On the other hand, the most widely used libraries (by number of PyPI downloads per month) that I could find that have mypy plugins are numpy, pydantic, attrs, and sqlalchemy. Numpy and sqlalchemy have deprecated their plugins (1, 2), while for pydantic and attrs, the plugins appear to improve types but not be critical (3, 4).

Thoughts? Is there anything I missed that folks want to see in V1?

[yangdanny97]

the most widely used libraries (by number of PyPI downloads per month) that I could find that have mypy plugins are numpy, pydantic, attrs, and sqlalchemy.

what about Django? I feel like we got a lot of requests for Django specifically, both in person and online. #168 #211

We should aim for feature completeness but not target a particular conformance %

Agreed, tho I think we should aim for near-perfect conformance for anything related to generics/callables.

In my mind there's a list of "tricky" problems that we should solve before v1. This is similar to the list of big features we were talking about during planning. Happy to hear suggestions for anything to add/deprioritize:

• overload conformance (possibly backtracking/checkpointing the solver state)
• collect types and pin at the end instead of eagerly pinning type vars Function invocation pins type of arguments too early #105
• generic callable/alias subtyping Handle callable subtyping for generic callables #73 + generic paramspec
• handle unions in contextual typing [Bug][CLI]: dict -> TypedDict narrowing fails in Union context #164
• constrained type vars conformance (I think steven mentioned something about splitting the representation of type vars to make a type form type) Validate typevar instantiation against constraints and bounds #111 [BUG] Calling a method on a TypeVar fails #150 Constrained typevar cannot be attribute-accessed #318
• better type simplification/understand equivalence (such as tuple[()] | tuple[int, *tuple[int, ...]] should be equivalent to tuple[int, ...] #273, but also generally we don't simplify unions enough)
• some behaviors that happen in the binding phase (narrowing, control flow) may need to depend on types from the answers phase (like Never/NoReturn narrowing #361)

[rchen152]

In my mind there's a list of "tricky" problems that we should solve before v1.

Agreed, these are important to solve.

what about Django?

The situation with Django seems fairly messy. Best I can tell:

1. Django itself is intentionally untyped.
2. There's a community-maintained django-stubs project that provides stubs and a plugin targeted at mypy.
3. There's also another community-maintained fork of django-stubs called django-types that removes the plugin and makes the stubs work for type checkers other than mypy.

[yangdanny97]

Interesting...

Maybe aiming to support django-types would be good enough for v1 then?

Ideally we'd want some mechanism to prompt the user (or automatically install) the correct package, or at the very least ignore django-stubs if it's installed.

[rchen152]

Oh that's a good idea, seems very doable and a clear improvement

[martindemello]

good environment/project support (venv, uv, ide configs, PYTHONPATH) perhaps

[yangdanny97]

yes, these (+sensible defaults/search heuristics) will be very important for things to work out of the box for as many people as possible.

[rchen152]

@connernilsen has already put a lot of work into this and has been actively fixing bugs!

[lolpack]

I would argue that type checking bugs that prevent go to definition should be prioritized over bugs that yield errors, but do not impact IDE navigation

[rchen152]

Yeah, we should definitely make sure go-to-def works well. That does raise an interesting question for me - my (perhaps naive) understanding is that resolving/inferring more types = better go-to-def. I wonder how that will interact with injecting Any types in a gradual mode.

[lolpack]

There are for sure cases where losing type information does impact our ability to resolve a symbol back to a location. Or possibly ambiguous symbols that would resolve to multiple locations where accurate types would give us the right one. Let me see if I can find a concrete example.

[kinto0]

per #544 it sounds like rename is a must-have for the IDE. I think we should support everything in #344 before calling it v1

[rchen152]

Added #344 to the V1 milestone!

[lolpack]

What do we think about GitHub first development once we are V1?

[rchen152]

What would we hope to get out of GitHub-first development that we don't get out of having our milestones and issue tracker on GitHub?

[lolpack]

Here is another spicy one... type errors are off by default for VSCode unless there is a pyrefly configuration present. We could prompt an init flow via the extension, but it should not show them by default. @kinto0 @samwgoldman

[sjet47]

AFAIK Pyrefly doesn't support type comments for type hints like others do, such as Pylance. Although this isn't an issue for those using Python 3.5+, type comments are really helpful for me and possibly others who struggle with projects in older versions of Python.

Pyrefly does offer great performance improvements, which is why I switched to it. However, the lack of support for type comments is really a pain point for me, and I hope Pyrefly consider adding this feature🙏

---

Some examples, more info here

a = 1  # type: int

def foo(a, b):  # type: (int, int) -> int
    return a + b

[rchen152]

Quick update for anyone who's following along: we're working toward a beta release later this year (V1 will come after beta). Milestone here: https://github.com/facebook/pyrefly/milestone/4.

[yangdanny97]

should VSCode notebook support be a part of the beta release? might be a good project for a our new team member to work on

cc @kinto0