Base Module¶
tinyCrawl.base.base module¶
Created on 2021/1/7 13:36
@author: Irvinfaith
@email: Irvinfaith@hotmail.com
-
class
tinyCrawl.base.base.
BaseCrawl
(iter_url, iter_num_range, thread_num)¶ 基类:
object
-
crawl
(*args, **kwargs)¶ Crawling task function, need to be implemented.
-
main
()¶ Main function to excute crawl and sink function.
-
run
(obj)¶ Adding a crawling function into the class
- 参数
obj – An crawling function object
-
sink
()¶ Output the crawling result, need to be implemented.
-
tinyCrawl.base.base_container module¶
Created on 2021/1/10 14:44
@author: Irvinfaith
@email: Irvinfaith@hotmail.com
-
class
tinyCrawl.base.base_container.
Container
(*args, **kwargs)¶ 基类:
list
-
classmethod
clear
()¶ Clear the data stored in the container.
-
container
= {}¶
-
classmethod
-
class
tinyCrawl.base.base_container.
RowContainer
(*args, **kwargs)¶