-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Description
We want to have a single messaging service where all the components exchange work to be done and results as required. As an example, for the communication between the crawler and the cruncher:
- crawler fetches flights from a remote source (netcoupe, ...) and stores them in the db
- the cruncher has a 'picker' container that fetches from the db the entries requiring (re)processing, and pushes them to the messaging service. this container should have a single replica
- the cruncher has a 'cruncher' container that picks an entry from the messaging service, processes it, and stores the result in the db (thermal data, flights stats, etc). this container can have multiple replicas
Need to choose a valid msg service, redis pub/sub or such.
Metadata
Metadata
Assignees
Labels
No labels