Add another download queue #111
Description
The ckanpackager had a slow and fast queue which had jobs added based on number of records (can't remember the exact threshold but something like <10,000 records goes on the fast queue, everything else is slow).
It would be good to add something like this/add a locking mechanism to the code so that we can run multiple workers (this would be my preference tbh, it's easier to scale the number of workers up and have them all handle the same kind of jobs than create some two-tiered worker system).
The "locking" (this could probably take different forms and doesn't have to be locks) would presumably just need to be around specific parts of the request, perhaps the query hash? I don't know if RQ has any of that kind of functionality built in, if not we'll have to think about how to do it.
This needs a bit of thought/investigation and then an implementation after.