[FutureLearn] Managing Big Data with R and Hadoopadmin
Understand how to use R and Hadoop to manage big data
This course will give you access to a virtual environment with installations of Hadoop, R and Rstudio to get hands-on experience with big data management. Several unique examples from statistical learning and related R code for map-reduce operations will be available for testing and learning.
Those with basic knowledge in statistical learning and R will better understand the methods behind and how to run them in parallel using map-reduce functions and Hadoop data storage. At the end of the course you will get access to RHadoop on a supercomputer at University of Ljubljana.
What topics will you cover?
- Welcome to BIG DATA
- Working with Hadoop
- First steps in R and RHadoop
- Statistical learning with RHadoop: clustering
- Statistical learning with RHadoop: regression and classification
What will you achieve?
By the end of the course, you’ll be able to…
- Explore basic functionality of Apache Hadoop and of RHadoop
- Experiment how to achieve performance of modern supercomputing
- Experiment regression, clustering and classification with RHadoop
- Investigate basic functionality of Bash terminal window
- Knowledge about statistical learning to instances of data provided by edcators
- How to do big data management with RHadoop on real supercomputer provided by Universiy of Ljubljana
Who is the course for?
This course is for people interested in data science, computational statistics and machine learning and have basic experiences with them. It will be also useful for advanced undergraduate students and first year PhD students in data analysis, statistics or bioinformatics, who wish to understand how to manage big data with Hadoop using R programming language.
We expect that the learners will also have basic experiences with linux, bash and R and are capable to download and run virtual machine.
What software or tools do you need?
All software needed to actively participate the course is provided within the virtual machine that the followers are supposed to download and run on the local machine. No extra software is needed.
You will need a modest local machine with 15GB free disk space and 2GB free RAM. You can get access to big data RStudio on a real HPC cluster after completing two weeks of exercises.