Input Streams are classes that generate units-of-work that your mappers will work on.
Creating them is as simple as creating a class that has a process method and a job_type argument:
After you create your input stream, just add it to a config.py file:
Then pass the file path as an argument to r3-app like this:
For more information check the documentation online.
Setting mappers to run your map/reduce tasks is an integral part of r³ and is as simple as creating a class that inherits from Mapper:
Running the mappers is pretty simple as well. Say you want to run four different mappers:
For more information check the documentation online.
Reducers are the classes that get the mapped units-of-work generated by your mappers and process them into a single coherent result.
Creating them is as simple as creating a class that has a reduce method and a job_type argument:
After you create your input stream, just add it to a config.py file:
Then pass the file path as an argument to r3-app like this:
For more information check the documentation online.
Please be advised that the link below may not work if your input stream requires additional arguments in it's URL.