arana crawls GitHub webpages for delicious data
Why not...?!
A spider.
A list of the tasks that need to be performed next. A "task" is a url and the timestamp the task was added to the work queue.
[
'https://github.com/dwyl 1438948333290',
'https://github.com/orgs/dwyl/people 1438948467205',
'https://github.com/dwyl/summer-2015 1438948491989'
]We are storing the Work Queue as a simple list because we don't need anything fancy!
Question: should we add all links on a page to the work queue emediatedly or