Project 2 Learnings and Improvements
What I did in the project and findings:
1) As part of the project, first and foremost I read the problem statement to better understand what needs to be done as part of the project.
2) Secondly, we assessed the datasets to see which dataset would we be interested in using and decided on using the financial dataset due to our common interest of analyzing a financial dataset.
3) Then, we created an action plan on how many end points and modification we should make, what data should we use to perform the Exploratory data analysis and who should do what.
4) As part of the project, I took the responsibility and performed the actions as mentioned below:
-To create, setup and make appropriate setting changes to setup the repository which involved setting up the repository itself, setting up github pages, connecting the repo to R studio.
-Then, I created the rmd file using the mentioned settings like github_document output type
-Created the render function which allowed us to output the .Rmd file as README.md
-Worked on creating a blueprint of how and what the functions will look like and what all end points will be used in addition to what modifications can be made
-Worked on exploratory data analysis and created graphs for scatterplot, created numerical summaries, new variable creation
-Worked on writing the narrative for part of the project
The learnings from the project were plenty. First and foremost, the learning that i found very important was how to manage and work on a project with a team/group, this allowed me to better understand how to work in a team, how to distribute work, and the benefits of working as a team. It really adds value when people with different opinions come together to solve a problems. This project very much allowed me to experience that and showed how 2 people with different strengths can come together to deliver results. From a technical standpoint, there was a lot I learned as well. Firstly, learned how to work with an API, this was something that that I had never done before and the project allowed me to work on it. Secondly, learned how to make a big program with a lot of moving parts. In this case, we made 5 end points with multiple modifications which seemed complex at first, but once things started coming together it was a wow moment for me. Third, working on different types of exploratory data analysis showed me how to actually derive insights and useful information from a dataset which has thousands and thousands of datapoints. It was interesting to see that stocks that we took as part of our analysis, technology stocks were correlated with each other and moved in somewhat tandem in terms of direction. Another interesting find was when we saw the gainers and loosers, this showed us how extremely can the value of stocks move in a single day.
Challenges and approach for next time
The challenges that we faced during the project were interesting to face and overcome, some of them being:
1) The first challenge that we faced was in terms of what dataset to use. We had intially decided to use the food data set but we had difficulty parsing some of the end points and which is why we pivoted to the financial dataset in addition to our interests.
2) The second challenge was to figure out which ends points and modifications to use so as to provide the user with useful insights and data from the API.
3) The third challenge was where we had to figure out how to map the tickers with the stock names, we were puzzled how to go about the fact that user can likely not know the exact name of the ticker and would probably use only part of the name.
4) Another challenge that we faced was while working with the scatterplot, it took us a while to come up with an idea on what to plot for this type of graph.
5) We faced challenges in creating the render function as well, this was so because we had not done this before but we were able to get a solution for this using videos and google.
6) We faced challenge in coverting the UNIX time format to use readable time format, we were not able to fix that in time and are use had we invested more time into this we would have been able to resolve this.
Approaching a similar project in the future, first of all I would understand the dataset better as that would allow me to create a better action plan as to what all end points, modifications and EDA could be potentially performed. This would also allow me to be creative with how can we manupilate the data and have better plots and data presented to the user. Additionally, I would spend more time on creating better plots and webpage in general, to make it look more presentable and good looking.
Finally, please find below the project website and please provide the appropriate feedback and comments:
https://namangoel05.github.io/ST558_Project2_2D/
The link to the repo is as follows:
https://github.com/namangoel05/ST558_Project2_2D
Sincerely,
Naman Goel