Some options on improving teal-types dx

[FourierTransformer]

Improving teal-type's DX

I've been thinking a little about teal-type's DX, and have some time to possibly work on it. I feel like there are a a lot of different ways to tackle this, these are just the couple I came up with. All the commands and the how could be changed, and likely other better ways are out there. I'm just putting my ideas out there. This is sort've in relation to what @mtdowling brought up in #947

Keeping a few things in mind is key:

• As low friction as possible to get type information
• Low maintenance overhead - the lower the maintenance the better!
• Compatibility with the the Lua ecosystem.

---

The LuaRocks options

1. Check teal-types when someone does a LuaRocks install

• Command: luarocks install lua-cjson
• How it could work:
  • LuaRocks would need to be made more teal-aware.
• Changes:
  • Update LuaRocks to determine if in a teal project, then get teal-type info, and add it to the tlconfig source_dir
• Maintenance:
  • Have to update LuaRocks with quite a bit of additional functionality

2. Create "shadow" teal-ified rock that also install the .d.tl files

• Command: luarocks install teal-lua-cjson
• How it could work:
  • For every type in teal-types create a new rockspec file with teal-[original rock name]
    • the rockspec version would likely need to match the version of the real library
    • Add the real library as a dependency and the .d.tl file in the install step
• Changes:
  • Need to generate a bunch of rockspec files (could be automated. I think the neovim guys do a lot of auto-gen/uploading rockspecs via GHA)
• Maintenance:
  • Have to keep all the new versions in sync. Maybe there's an easy-ish way to handle this? Maybe the teal- version always gets the latest installed version?
  • If version updates automated using GHA, likely some maintenance headaches there.

3. Create just a "type" file rock

• Command luarocks install lua-cjson-types
  Very similar to above, but doesn't worry about the dependency syncing aspect. Just installs the teal type
• Changes:
• Likely still need to generate/update rockspec files, but the syncing back to the lua version wouldn't need to happen.

---

The cyan options

4. Add a new options to cyan for installing LuaRocks

• Command: cyan add lua-cjson and/or cyan install lua-cjson
• How it could work:
  1. install lua-cjson via LuaRocks (as a lib would be nice, shell out worst case?)
  2. Check the types repo for lua-cjson types, and install it if exists (or print a warning if now)
  3. If called with add, it could update the dependencies in the local rockspec file. Otherwise just install it (similar to uv add vs uv pip install)
• Changes:
  • Cyan would need some networking capabilities
  • Get cyan to be able to call LuaRocks
  • Cyan needs to be able to interface with the teal-types repo

5. Add an option to just get the type information

• Command: cyan get-type lua-cjson
• How it could work:
  1. Check the repo for the given type, and add it to the source_dir
• Changes:
  • Cyan needs to be able to interface with the teal-types repo

---

Final Thoughts

• All the LuaRocks options (1, 2, and 3) keep the package management and the teal management separate
• LuaRocks already handles a lot of networking/os things
• There are other package managers in the Lua space (and at least one new one being developed) and allowing teal to interface with them might be nice
• Some of the options can lead to the other, and could be implemented in stages.
  • Option 3 could become Option 2
  • Option 5 could become Option 4
• My personal preference would be to see cyan improved and would start with Option 5, with the goal towards 4, but would want to hear community opinions and if @euclidianAce would be up for that level of change.

Rankings (subjective, based on my understandings)

Option	DX	Change Effort	Maint effort
1. luarocks install lua-cjson	1	Med-High?	Medium
2. luarocks install teal-lua-cjson	2	High	High
3. luarocks install lua-cjson-types	3	Med-High	Med-High
4. cyan add lua-cjson	1	Med-High	Medium-Low
5. cyan get-type lua-cjson	3	Med-High	Low

DX rankings:

1. One command to install
2. One command, but different package name
3. Two commands

[FourierTransformer]

So, one of the other things I was thinking about was where to install the .d.tl files. I feel like some folks would probably want it with alongside their rocks, and others in their projects. I know npm installs things locally "in-project" by default and could switch to globally with a "-g" flag. I suppose making it flag would be reasonable.

[catwell]

I think most of the time you'd want the types alongside your rocks if they're, but the rocks themselves might be in the project. (Personally I never use global rock trees, I have a different one per project.)

Personally I'd like option 3 best, and I would probably publish those rocks on a specific manifest. Ultimately it would be even better if this wasn't necessary at all and Teal types were added by Rocks authors - ideally with a specific mechanism in rockspecs instead of the current hack so it is simpler to choose whether to install them or not with a command-line flag.

[FourierTransformer]

Yeah, thinking about it a little more, adding a .d.tl into the source of a project is more of a workaround since they aren't installed alongside the rocks. I have been playing a little with the available compression/networking libraries and I think it would be a big add into cyan (esp from a long-term maintenance perspective).

I think more for a "first-class" experience, option 3 might actually be a better fit. Over the next few days I might try to look into some of the neovim tooling and see what they're doing.

[hishamhm]

@FourierTransformer Awesome initiative!

My 2 cents: I think I was always assuming that I would implement Option 1 eventually, but I think in terms of development effort that would be the highest effort, just because of LuaRocks's tech debt — it's always more work to add features there than it seems at first. Nowadays, I don't see myself doing it any time soon, so I'm super happy to see you stepping up on this topic!

I also like Option 4, which I think would ultimately be the simplest UX for users — making cyan act like frontend for LuaRocks (or other package manager!). I guess I've gotten used to the cargo experience. But of course, that is @euclidianAce 's call.

Option 3 sounds like the best first step indeed. It could evolve into Option 2, and it could become the "backend" for Option 4 too.

[lenscas]

If I understand it properly, 3 is basically how it works in the npm ecosystem for typescript.

[mtdowling]

We should consider versioning of types provided externally from the actual package. If the types are defined external from the source, which is the case for Option 1 and 3, then there is potential for the source Lua/C code to drift from the type definitions. This matters most in the case of major version bumps. Option 3, a types rock, solves this because you can just ship a new rock with a new version, and even maintain rocks for all versions.

IMO for Option 1 to work, it needs to essentially become Option 3. That is: if we keep a teal-types repo, ideally it can be a single repo that vends out a rock for each major version of each library it provides types for. This feels like the best of both worlds -- easy to maintain single repo for community owned Teal types, but integration with LuaRocks because we literally ship rocks for each MV of each library.

If we can get to this outcome, then all LuaRocks would need really is a way to place Teal types somewhere Cyan can find them and work with them. It also is probably better to flip the relationship around -- maybe Cyan should get LuaRocks integration and be able to find and hook up Teal types based on the rock tree than the other way around (basically option 4). This seems like a cleaner relationship since Cyan already understands tlconfig and source_dir semantics, and we don't need to teach LuaRocks about it.

[hishamhm]

if we keep a teal-types repo, ideally it can be a single repo that vends out a rock for each major version of each library it provides types for. This feels like the best of both worlds -- easy to maintain single repo for community owned Teal types, but integration with LuaRocks because we literally ship rocks for each MV of each library.

And @FourierTransformer is making this happen as we speak! teal-language/teal-types#73