- Progress Rundown
- Posts
- 📊 Data
Big Data:
The cat-and-mouse game that's been going rogue.
Why, though?
You see, on one hand, there are people who want to protect their data and privacy, but on the other, we have those that want to gauge out statistics for their business decisions.🥸.
It's an endless cycle.
What is it, fr though?
Big data refers to large sets of information that are difficult to process using traditional methods.
(The human population is over 8 billion now, cmon💀.)
In the past, we used to collect data in a manageable way with spreadsheets, but now there's so much of it that we need new methods.
Scientifically:
A compilation of structured, semi-structured, and unstructured data collected by organizations that can be mined in machine learning projects, predictive analytics, and other applications.
Here's a list of jobs that show over a 35% growth rate for the next 10 years:
Data Scientist - Extract insights and knowledge from data using statistical and machine learning techniques.
Data Analyst - Analyze data to identify trends, patterns, and insights to inform business decisions.
Data Engineer - Build free-flowing data pipelines by combining a variety of big data technologies that enable real-time analytics.
Data Mining Expert - Apply techniques to identify patterns and relationships within large datasets.
Machine Learning Engineer - Develop and deploy algorithms to enable machines to learn from data and improve their performance.
Database Administrator - Manage and optimize the performance, security, and availability of databases.
Business Intelligence Analyst - Analyze data and use visualizations to help stakeholders understand and make decisions based on business performance.
Web Analytics Expert - Collect, analyze, and report on data about user interactions with web-based applications and websites.
Data Architect - Visualize and design an organization’s data management framework.
Breaking Into Data:
One of the many first steps is to study computer science, mathematics, statistics, business analytics, or information technology at an undergraduate level.
There definitely are a few universities that offer data science, but since it's relatively new, lots of them don't teach courses important for students to get jobs.
Later on, you can either:
Get your master's
Network to get a full-time job
Gain certifications and build your own project portfolio
Freelance
Start a new life
Or even work on your own startup
Good luck with the funding! (No sarcasm intended)
It is important to note here that everyone has their OWN PACE. 🙌
Hard skills that give you a competitive edge:
1) Apache Spark
2) AWS (Amazon Web Services)
3) Microsoft Azure
➡️ To conclude, this industry isn't reserved only for the technology sector; data people work in healthcare, media, banking, and even agriculture.
This Week's Podcast Episode:
ML Engineer who sold his startup for $60 Million:
David Ma, a quant (i.e.: rocket scientist of Wall Street) at Two Sigma, quit his job to co-found Dynasty in 2016, which provides AI solutions for real estate businesses.
Their aim was to securitize residential assets and give people with small capital some buying power.
Securitization is the process in which certain types of assets are pooled so that they can be repackaged into interest-bearing securities.
It didn't work out as there was a BIGGER problem: a lot of participants had trouble managing their assets. Maintaining real estate properties requires effort, after all.
And half of the team quit 🪦.
Did David shut down the startup?
No, they shifted their product and developed Lisa, an AI for leasing and renting.
David doesn't recommend solving a lot of problems with machine learning tools, though. For example, a question like " What's the rent?" looks very subtle, but an AI personal assistant won't answer this accurately since rates vary from site to site.
Misconception about ML Engineering:
People focus on building models and not enough on generating data and designing a business process for the algorithms.
Advice for Similar Startups:
1) It's better to build out your system before identifying spots that need ML.
There's always a chance your solution might not be that useful for the market.
2) Increase the accuracy of your model only after you've handled your false positives and negatives, which are essentially assumptions that are proven wrong.
3) Don't focus too much on academic literature (existing knowledge of problems from the behavioral science field) as it's not reproducible.
Skills that they look for in a good engineer:
1) willing to dig into details
2) understands business rather than a lot of plain ML
3) good at software engineering
**Credits: Joma Tech
RESOURCES:
The required programming languages and where to learn them:
1) Python
2) Java
3) R
4) SQL
5) Scala
Websites:
Data science bootcamps:
The most popular, all-in-one site for beginners:
Gamified programming:
Find datasets for projects:
Free courses:
Fundamentals of Data Analytics by IBM:
Introduction to Data Science by Harvard University:
Recommended Books:
1) Practical SQL: A Beginner's Guide to Storytelling with Data by Anthony Debarros:
2) Practical Statistics for Data Scientists by Peter and Andrew Bruce:
3) Becoming a Data Head by Alex J.Gutman and Jordan Goldmeier:
Social Media Platforms:
Youtube:
Instagram:
Phew.
We hope this was pretty straightforward to understand and your mind didn't crack.
'Cause ours certainly did.
Stay tuned for next week!
(Spoiler alert: Finance)
Signing out,
The Rundown Team.
Reply