Skip to content

Commit 02a81cc

Browse files
committed
add README
1 parent fca0604 commit 02a81cc

1 file changed

Lines changed: 287 additions & 0 deletions

File tree

README.md

Lines changed: 287 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,287 @@
1+
# asyncigo: asyncio-style event loops for Go
2+
3+
[![tag](https://img.shields.io/github/tag/ecryth/asyncigo.svg)](https://github.com/ecryth/asyncigo/releases)
4+
[![Go Version](https://img.shields.io/badge/Go-%3E%3D%201.22-%23007d9c)](https://go.dev/)
5+
[![Go Reference](https://pkg.go.dev/badge/github.com/ecryth/asyncigo.svg)](https://pkg.go.dev/github.com/ecryth/asyncigo)
6+
[![Build Status](https://github.com/ecryth/asyncigo/actions/workflows/test.yml/badge.svg)](https://github.com/ecryth/asyncigo/actions)
7+
[![GitHub License](https://img.shields.io/github/license/ecryth/asyncigo)](./LICENSE.txt)
8+
9+
asyncigo is a proof of concept framework for doing event loop-based asynchronous I/O in Go, modelled after Python's `asyncio`.
10+
11+
asyncigo comes with:
12+
13+
* **Tasks** that suspend and awake coroutines in response to I/O events
14+
* **Futures** that tasks can use to wait for asynchronous results
15+
* **Await** which lets you write asynchronous code that looks synchronous
16+
* **TCP client sockets** for reading and writing data asynchronously over the internet
17+
* **Asynchronous iterators** for ergonomically iterating over I/O streams
18+
* **Iterator utilities** for mapping, filtering, chaining and otherwise manipulating functional iterators
19+
20+
## What?
21+
22+
asyncigo allows for large-scale[^1], single-threaded[^2] asynchronous I/O[^3] using asyncio-style, event loop-based coroutines, tasks and futures.
23+
The cooperative nature of the concurrency model makes your code easier to reason about and reduces the need for synchronisation primitives, while still being capable of managing many thousands of I/O-bound tasks simultaneously.
24+
25+
[^1]: Not actually tested.
26+
[^2]: Technically, each coroutine is run in its own goroutine which may be scheduled on any thread, but no two goroutines belonging to the same event loop will ever run at the same time, resulting in effectively single-threaded behaviour.
27+
[^3]: The only actual asynchronous I/O currently supported is network sockets, and only on platforms that support `epoll` (i.e. Linux).
28+
29+
## How?
30+
31+
asyncigo is based on the [iterator functions](https://go.dev/wiki/RangefuncExperiment) that were added experimentally in Go 1.22.
32+
In particular it ~~ab~~uses the fact that `iter.Pull` works by context switching between the iterator function and the calling function on each call to the `yield` and `next` functions, which can be used to emulate Python's `yield` and `send` with the help of some extra bookkeeping to track the results of the yielded futures.
33+
34+
## Why?
35+
36+
I thought it would be funny.
37+
Give event loop fanatics an inch etc.
38+
(It's me, hi, I'm the event loop fanatic.)
39+
40+
## OK, but seriously, why?
41+
42+
A single-threaded event loop model with cooperative concurrency has a number of advantages over traditional multithreading for I/O-bound tasks:
43+
44+
1. Less risk of race conditions.
45+
2. Less need for manual synchronisation.
46+
3. It's easier to reason about the code, as you know state can't change between await points.
47+
4. Coroutines can have a lower footprint than threads.
48+
49+
In the context of Go, however, goroutines (being user-space green threads) already provide some of the same benefits as coroutines compared to OS threads, including faster context switching and a lower memory footprint.
50+
Still, Go's preemptive model can be hard to work with if you're not careful, and if you're not generally bound by the CPU, it can be nice to have the guarantees that cooperative concurrency brings.
51+
52+
## Should I use it?
53+
54+
Probably not!
55+
Go prides itself on having a single concurrency model built right into the language.
56+
A library like this undermines that by introducing a parallel model, which risks fragmenting the language.
57+
That said, there are clear reasons why one might prefer the event loop model, so use your own judgement.
58+
59+
Besides, this library is just a proof of concept, is very bare-bones, has not been tested in practice, and is currently only really useful on Linux.
60+
I take no responsibility for any bricked computers, burnt-down houses, escaped pets, loss of sense of self, natural disasters or general feelings of discomfort that follow from the use of asyncigo.
61+
62+
If you do want to use this as a base for your own library, though, be my guest.
63+
You could also simply use it as inspiration for designing more ergonomic goroutine-based frameworks.
64+
65+
## How do I use it?
66+
67+
Did you even read the previous section?
68+
But OK, fine.
69+
70+
1. Fetch the module:
71+
```
72+
go get github.com/ecryth/asyncigo@latest
73+
```
74+
2. Make sure that you're compiling with the `rangefunc` experiment enabled:
75+
```
76+
GOEXPERIMENT=rangefunc go build
77+
```
78+
You may also need to enable the `goexperiment.rangefunc` build tag for your IDE to resolve the `iter` import correctly.
79+
80+
### Tasks
81+
82+
You can create and await tasks which will be run in parallel:
83+
84+
```go
85+
asyncigo.NewEventLoop().Run(context.Background(), func(ctx context.Context), error {
86+
task := asyncigo.SpawnTask(ctx, func (ctx context.Context) (int, error) {
87+
for i := range 3 {
88+
fmt.Printf("in subtask: %d\n", i)
89+
Sleep(ctx, time.Second)
90+
}
91+
return 42, nil
92+
})
93+
94+
for j := range 3 {
95+
fmt.Printf("in main task: %d\n", j)
96+
Sleep(ctx, time.Second)
97+
}
98+
99+
task.Await(ctx)
100+
})
101+
// Output:
102+
// in main task: 0
103+
// in subtask: 0
104+
// in main task: 1
105+
// in subtask: 1
106+
// in main task: 2
107+
// in subtask: 2
108+
// task result: 42
109+
```
110+
111+
You can wait for multiple tasks at the same time:
112+
113+
```go
114+
asyncigo.NewEventLoop().Run(context.Background(), func(ctx context.Context) error {
115+
fut1 := asyncigo.NewFuture[string]()
116+
117+
task1 := asyncigo.SpawnTask(ctx, func(ctx context.Context) (int, error) {
118+
asyncigo.Sleep(ctx, time.Second)
119+
fut1.SetResult("test", nil)
120+
return 20, nil
121+
})
122+
123+
task2 := asyncigo.SpawnTask(ctx, func(ctx context.Context) (float64, error) {
124+
asyncigo.Sleep(ctx, time.Second)
125+
return 25.5, errors.New("oops")
126+
})
127+
128+
var result1 string
129+
var result2 int
130+
var result3 float64
131+
err := asyncigo.Wait(
132+
ctx,
133+
asyncigo.WaitAll,
134+
fut1.WriteResultTo(&result1),
135+
task1.WriteResultTo(&result2),
136+
task2.WriteResultTo(&result3),
137+
)
138+
139+
fmt.Println("results:", result1, result2, result3)
140+
fmt.Println("error:", err)
141+
return nil
142+
})
143+
// Output:
144+
// results: test 20 25.5
145+
// error: oops
146+
```
147+
148+
### Asynchronous iterators
149+
150+
asyncigo supports asynchronous iterator functions that let you wait for asynchronous I/O events while also progressively yielding results, similar to `async` generators in Python.
151+
These can then be easily ranged over.
152+
153+
For instance, you can read chunks of data from a socket, process each chunk, and then yield the result:
154+
155+
```go
156+
if err := asyncigo.NewEventLoop().Run(context.Background(), func(ctx context.Context) error {
157+
it := asyncigo.AsyncIter(func(yield func(int) error) error {
158+
stream, _ := asyncigo.RunningLoop(ctx).Dial(ctx, "tcp", "localhost:6172")
159+
160+
for {
161+
line, err := stream.ReadLine(ctx)
162+
if errors.Is(err, io.EOF) {
163+
return nil
164+
} else if err != nil {
165+
return err
166+
}
167+
168+
unicode := []rune(strings.TrimSpace(string(line)))
169+
_ = yield(len(unicode))
170+
}
171+
})
172+
173+
for lineLength, err := range it {
174+
if err != nil {
175+
return err
176+
}
177+
178+
fmt.Println(lineLength)
179+
}
180+
return nil
181+
}); err != nil {
182+
panic(err)
183+
}
184+
// Output:
185+
// 6
186+
// 12
187+
// 18
188+
// 30
189+
```
190+
191+
Using `UntilErr`, we can iterate over asynchronous iterators a bit more ergonomically, and also combine them with other utilities that work with function iterators like `Map`, `Filter` or `Chain`:
192+
193+
```go
194+
if err := asyncigo.NewEventLoop().Run(context.Background(), func(ctx context.Context) error {
195+
loop := asyncigo.RunningLoop(ctx)
196+
197+
var err error
198+
for line := range asyncigo.Chain(
199+
loop.DialLines(ctx, "tcp", "localhost:6172").UntilErr(&err),
200+
loop.DialLines(ctx, "tcp", "localhost:6173").UntilErr(&err),
201+
) {
202+
fmt.Printf("got line: %s", line)
203+
}
204+
205+
return err
206+
}); err != nil {
207+
panic(err)
208+
}
209+
// Output:
210+
// got line: Lorem ipsum dolor sit amet.
211+
// got line: Donec non velit consequat.
212+
// got line: Donec interdum in nulla ac scelerisque.
213+
// got line: Duis commodo, neque ac luctus eleifend.
214+
// got line: Fusce lacinia id quam ac porttitor.
215+
// got line: 生麦生米生卵
216+
// got line: すもももももももものうち
217+
// got line: 東京特許許可局長今日急遽休暇許可却下
218+
// got line: 斜め77度の並びで泣く泣く嘶くナナハン7台難なく並べて長眺め
219+
```
220+
221+
### Task cancellation
222+
223+
The cancellation semantics are not yet finalised, particularly regarding to what extent a task should have the opportunity to recover or clean up following cancellation.
224+
At the moment, the coroutine itself will continue running after the task has been cancelled, but its context will be cancelled, and any further calls to `Await` will immediately return `context.Canceled`:
225+
226+
```go
227+
_ = asyncigo.NewEventLoop().Run(context.Background(), func(ctx context.Context) error {
228+
futs := make([]asyncigo.Future[int], 10)
229+
task := asyncigo.SpawnTask(ctx, func(ctx context.Context) (int, error) {
230+
for i := range futs {
231+
result, err := futs[i].Await(ctx)
232+
fmt.Printf("%d: (%v, %v)\n", i, result, err)
233+
}
234+
return 0, nil
235+
})
236+
237+
loop := asyncigo.RunningLoop(ctx)
238+
for i := range futs {
239+
if i == 5 {
240+
task.Cancel(nil)
241+
}
242+
243+
_ = loop.Yield(ctx, nil)
244+
futs[i].SetResult(i, nil)
245+
}
246+
247+
result, err := task.Await(ctx)
248+
fmt.Printf("task result: (%v, %v)", result, err)
249+
return nil
250+
})
251+
// Output:
252+
// 0: (0, <nil>)
253+
// 1: (1, <nil>)
254+
// 2: (2, <nil>)
255+
// 3: (3, <nil>)
256+
// 4: (4, <nil>)
257+
// 5: (0, context canceled)
258+
// 6: (0, context canceled)
259+
// 7: (0, context canceled)
260+
// 8: (0, context canceled)
261+
// 9: (0, context canceled)
262+
// task result: (0, context canceled)
263+
```
264+
265+
For now, it is thus the responsibility of the task itself to exit early on cancellation, but any further asynchronous operations will be ignored.
266+
267+
Additionally, cancelling a task will also cancel any future or task it's currently awaiting.
268+
If you want to prevent an awaited future from being cancelled, use `Shield`:
269+
270+
```go
271+
fut.Shield().Await(ctx)
272+
```
273+
274+
### Polyglottal functions
275+
276+
Go has no language-level distinction between coroutines and normal functions, making it possible to write functions that work both synchronously and asynchronously.
277+
For example, you could write a polyglottal sleep function as so:
278+
279+
```go
280+
func SleepPolyglot(ctx context.Context, duration time.Duration) error {
281+
if _, ok := asyncigo.RunningLoopMaybe(ctx); ok {
282+
return asyncigo.Sleep(ctx, duration)
283+
}
284+
time.Sleep(duration)
285+
return nil
286+
}
287+
```

0 commit comments

Comments
 (0)