Skip to main content

AWESOME DATA SCIENCE

An open source Data Science repository to learn and apply towards solving real world problems.

This is a shortcut path to start studying Data Science. Just follow the steps to answer the questions, "What is Data Science and what should I study to learn Data Science?"

What is Data Science?

Data Science is one of the hottest topics on the Computer and Internet farmland nowadays. People have gathered data from applications and systems until today and now is the time to analyze them. The next steps are producing suggestions from the data and creating predictions about the future. Here you can find the biggest question for Data Science and hundreds of answers from experts.

LinkPreview
What is Data Science @ O'reillyData scientists combine entrepreneurship with patience, the willingness to build data products incrementally, the ability to explore, and the ability to iterate over a solution. They are inherently interdisciplinary. They can tackle all aspects of a problem, from initial data collection and data conditioning to drawing conclusions. They can think outside the box to come up with new ways to view the problem, or to work with very broadly defined problems: “here’s a lot of data, what can you make from it?”
What is Data Science @ QuoraData Science is a combination of a number of aspects of Data such as Technology, Algorithm development, and data interference to study the data, analyse it, and find innovative solutions to difficult problems. Basically Data Science is all about Analysing data and driving for business growth by finding creative ways.
The sexiest job of 21st centuryData scientists today are akin to Wall Street “quants” of the 1980s and 1990s. In those days people with backgrounds in physics and math streamed to investment banks and hedge funds, where they could devise entirely new algorithms and data strategies. Then a variety of universities developed master’s programs in financial engineering, which churned out a second generation of talent that was more accessible to mainstream firms. The pattern was repeated later in the 1990s with search engineers, whose rarefied skills soon came to be taught in computer science programs.
WikipediaData science is an inter-disciplinary field that uses scientific methods, processes, algorithms and systems to extract knowledge and insights from many structural and unstructured data. Data science is related to data mining, machine learning and big data.
How to Become a Data ScientistData scientists are big data wranglers, gathering and analyzing large sets of structured and unstructured data. A data scientist’s role combines computer science, statistics, and mathematics. They analyze, process, and model data then interpret the results to create actionable plans for companies and other organizations.
a very short history of #datascienceThe story of how data scientists became sexy is mostly the story of the coupling of the mature discipline of statistics with a very young one--computer science. The term “Data Science” has emerged only recently to specifically designate a new profession that is expected to make sense of the vast stores of big data. But making sense of data has a long history and has been discussed by scientists, statisticians, librarians, computer scientists and others for years. The following timeline traces the evolution of the term “Data Science” and its use, attempts to define it, and related terms.

Learn Data Science

Our favorite programming language is Python nowadays for #DataScience. Python's - Pandas library has full functionalities for collecting and analyzing data. We use Anaconda to play with data and to create applications.

Algorithms

top

These are some Machine Learning and Data Mining algorithms and models help you to understand your data and derive meaning from it.

Supervised Learning

  • Regression
  • Linear Regression
  • Ordinary Least Squares
  • Logistic Regression
  • Stepwise Regression
  • Multivariate Adaptive Regression Splines
  • Locally Estimated Scatterplot Smoothing
  • Classification
    • k-nearest neighbor
    • Support Vector Machines
    • Decision Trees
    • ID3 algorithm
    • C4.5 algorithm
  • Ensemble Learning
  • Boosting
  • Bagging
  • Random Forest
  • AdaBoost

Unsupervised Learning

  • Clustering
    • Hierchical clustering
    • k-means
    • Fuzzy clustering
    • Mixture models
  • Dimension Reduction
    • Principal Component Analysis (PCA)
    • t-SNE
  • Neural Networks
  • Self-organizing map
  • Adaptive resonance theory
  • Hidden Markov Models (HMM)

Semi-Supervised Learning

  • S3VM
  • Clustering
  • Generative models
  • Low-density separation
  • Laplacian regularization
  • Heuristic approaches

Reinforcement Learning

  • Q Learning
  • SARSA (State-Action-Reward-State-Action) algorithm
  • Temporal difference learning

Data Mining Algorithms

  • C4.5
  • k-Means
  • SVM
  • Apriori
  • EM
  • PageRank
  • AdaBoost
  • kNN
  • Naive Bayes
  • CART

Deep Learning architectures

  • Multilayer Perceptron
  • Convolutional Neural Network (CNN)
  • Recurrent Neural Network (RNN)
  • Boltzmann Machines
  • Autoencoder
  • Generative Adversarial Network (GAN)
  • Self-Organized Maps

COLLEGES

top

Intensive Programs

MOOC's

top

Tutorials

top

Free Courses

Toolboxes - Environment

top

LinkDescription
The Data Science Lifecycle ProcessThe Data Science Lifecycle Process is a process for taking data science teams from Idea to Value repeatedly and sustainably. The process is documented in this repo
Data Science Lifecycle Template RepoTemplate repository for data science lifecycle project
RexMexA general purpose recommender metrics library for fair evaluation.
ChemicalXA PyTorch based deep learning library for drug pair scoring.
PyTorch Geometric TemporalRepresentation learning on dynamic graphs.
Little Ball of FurA graph sampling library for NetworkX with a Scikit-Learn like API.
Karate ClubAn unsupervised machine learning extension library for NetworkX with a Scikit-Learn like API.
ML WorkspaceAll-in-one web-based IDE for machine learning and data science. The workspace is deployed as a Docker container and is preloaded with a variety of popular data science libraries (e.g., Tensorflow, PyTorch) and dev tools (e.g., Jupyter, VS Code)
Neptune.aiCommunity-friendly platform supporting data scientists in creating and sharing machine learning models. Neptune facilitates teamwork, infrastructure management, models comparison and reproducibility.
steppyLightweight, Python library for fast and reproducible machine learning experimentation. Introduces very simple interface that enables clean machine learning pipeline design.
steppy-toolkitCurated collection of the neural networks, transformers and models that make your machine learning work faster and more effective.
Datalab from Googleeasily explore, visualize, analyze, and transform data using familiar languages, such as Python and SQL, interactively.
Hortonworks Sandboxis a personal, portable Hadoop environment that comes with a dozen interactive Hadoop tutorials.
Ris a free software environment for statistical computing and graphics.
RStudioIDE – powerful user interface for R. It’s free and open source, works on Windows, Mac, and Linux.
Python - Pandas - AnacondaCompletely free enterprise-ready Python distribution for large-scale data processing, predictive analytics, and scientific computing
Pandas GUIPandas GUI
Scikit-LearnMachine Learning in Python
NumPyNumPy is fundamental for scientific computing with Python. It supports large, multi-dimensional arrays and matrices and includes an assortment of high-level mathematical functions to operate on these arrays.
VaexVaex is a Python library that allows you to visualize large datasets and calculate statistics at high speeds.
SciPySciPy works with NumPy arrays and provides efficient routines for numerical integration and optimization.
Data Science ToolboxCoursera Course
Data Science ToolboxBlog
Wolfram Data Science PlatformTake numerical, textual, image, GIS or other data and give it the Wolfram treatment, carrying out a full spectrum of data science analysis and visualization and automatically generating rich interactive reports—all powered by the revolutionary knowledge-based Wolfram Language.
DatadogSolutions, code, and devops for high-scale data science.
VarianceBuild powerful data visualizations for the web without writing JavaScript
Kite Development KitThe Kite Software Development Kit (Apache License, Version 2.0) , or Kite for short, is a set of libraries, tools, examples, and documentation focused on making it easier to build systems on top of the Hadoop ecosystem.
Domino Data LabsRun, scale, share, and deploy your models — without any infrastructure or setup.
Apache FlinkA platform for efficient, distributed, general-purpose data processing.
Apache HamaApache Hama is an Apache Top-Level open source project, allowing you to do advanced analytics beyond MapReduce.
WekaWeka is a collection of machine learning algorithms for data mining tasks.
OctaveGNU Octave is a high-level interpreted language, primarily intended for numerical computations.(Free Matlab)
Apache SparkLightning-fast cluster computing
Hydrosphere Mista service for exposing Apache Spark analytics jobs and machine learning models as realtime, batch or reactive web services.
Data MechanicsA data science and engineering platform making Apache Spark more developer-friendly and cost-effective.
CaffeDeep Learning Framework
TorchA SCIENTIFIC COMPUTING FRAMEWORK FOR LUAJIT
Nervana's python based Deep Learning Framework.
SkaleHigh performance distributed data processing in NodeJS
AerosolveA machine learning package built for humans.
Intel frameworkIntel® Deep Learning Framework
DatawrapperAn open source data visualization platform helping everyone to create simple, correct and embeddable charts. Also at github.com
Tensor FlowTensorFlow is an Open Source Software Library for Machine Intelligence
Natural Language ToolkitAn introductory yet powerful toolkit for natural language processing and classification
nlp-toolkit for node.js.
Juliahigh-level, high-performance dynamic programming language for technical computing
IJuliaa Julia-language backend combined with the Jupyter interactive environment
Apache ZeppelinWeb-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more
FeaturetoolsAn open source framework for automated feature engineering written in python
OptimusCleansing, pre-processing, feature engineering, exploratory data analysis and easy ML with PySpark backend.
AlbumentationsА fast and framework agnostic image augmentation library that implements a diverse set of augmentation techniques. Supports classification, segmentation, detection out of the box. Was used to win a number of Deep Learning competitions at Kaggle, Topcoder and those that were a part of the CVPR workshops.
DVCAn open-source data science version control system. It helps track, organize and make data science projects reproducible. In its very basic scenario it helps version control and share large data and model files.
Lambdois a workflow engine which significantly simplifies data analysis by combining in one analysis pipeline (i) feature engineering and machine learning (ii) model training and prediction (iii) table population and column evaluation.
FeastA feature store for the management, discovery, and access of machine learning features. Feast provides a consistent view of feature data for both model training and model serving.
PolyaxonA platform for reproducible and scalable machine learning and deep learning.
LightTagText Annotation Tool for teams
UBIAIEasy-to-use text annotation tool for teams with most comprehensive auto-annotation features. Supports NER, relations and document classification as well as OCR annotation for invoice labeling
TrainsAuto-Magical Experiment Manager, Version Control & DevOps for AI
HopsworksOpen-source data-intensive machine learning platform with a feature store. Ingest and manage features for both online (MySQL Cluster) and offline (Apache Hive) access, train and serve models at scale.
MindsDBMindsDB is an Explainable AutoML framework for developers. With MindsDB you can build, train and use state of the art ML models in as simple as one line of code.
LightwoodA Pytorch based framework that breaks down machine learning problems into smaller blocks that can be glued together seamlessly with an objective to build predictive models with one line of code.
AWS Data WranglerAn open-source Python package that extends the power of Pandas library to AWS connecting DataFrames and AWS data related services (Amazon Redshift, AWS Glue, Amazon Athena, Amazon EMR, etc).
Amazon RekognitionAWS Rekognition is a service that lets developers working with Amazon Web Services add image analysis to their applications. Catalog assets, automate workflows, and extract meaning from your media and applications.
Amazon TextractAutomatically extract printed text, handwriting, and data from any document.
Amazon Lookout for VisionSpot product defects using computer vision to automate quality inspection.Identify missing product components, vehicle and structure damage, and irregularities for comprehensive quality control.
Amazon CodeGuruAutomate code reviews and optimize application performance with ML-powered recommendations.
CMLAn open source toolkit for using continuous integration in data science projects. Automatically train and test models in production-like environments with GitHub Actions & GitLab CI, and autogenerate visual reports on pull/merge requests.
DaskAn open source Python library to painlessly transition your analytics code to distributed computing systems (Big Data)
StatsmodelsA Python-based inferential statistics, hypothesis testing and regression framework
GensimAn open-source library for topic modeling of natural language text
spaCyA performant natural language processing toolkit
Grid StudioGrid studio is a web-based spreadsheet application with full integration of the Python programming language.
Python Data Science HandbookPython Data Science Handbook: full text in Jupyter Notebooks
ShapleyA data-driven framework to quantify the value of classifiers in a machine learning ensemble.
DAGsHubA platform built on open source tools for data, model and pipeline management.
DeepnoteA new kind of data science notebook. Jupyter-compatible, with real-time collaboration and running in the cloud.
ValohaiAn MLOps platform that handles machine orchestration, automatic reproducibility and deployment.
PyMC3A Python Library for Probabalistic Programming (Bayesian Inference and Machine Learning)
PyStanPython interface to Stan (Bayesian inference and modeling)
hmmlearnUnsupervised learning and inference of Hidden Markov Models
Chaos GeniusML powered analytics engine for outlier/anomaly detection and root cause analysis
NimbleboxA full-stack MLOps platform designed to help data scientists and machine learning practitioners around the world discover, create, and launch multi-cloud apps from their web browser.
TowheeA Python library that helps you encode your unstructured data into embeddings.

Machine Learning in General Purpose

Deep Learning

pytorch

tensorflow

keras

Visualization Tools - Environments

Journals, Publications and Magazines

top

Presentations

top

Podcasts

top

Books

top

Socialize

top

Bloggers

top

Facebook Accounts

top

Twitter Accounts

top

TwitterDescription
Big Data CombineRapid-fire, live tryouts for data scientists seeking to monetize their models as trading strategies
Big Data ManiaData Viz Wiz , Data Journalist , Growth Hacker , Author of Data Science for Dummies (2015)
Big Data ScienceBig Data, Data Science, Predictive Modeling, Business Analytics, Hadoop, Decision and Operations Research.
Charlie GreenbackerDirector of Data Science at @ExploreAltamira
Chris SaidData scientist at Twitter
Clare CorthellDev, Design, Data Science @mattermark #hackerei
DADI Charles-Abner#datascientist @Ekimetrics. , #machinelearning #dataviz #DynamicCharts #Hadoop #R #Python #NLP #Bitcoin #dataenthousiast
Data Science CentralData Science Central is the industry's single resource for Big Data practitioners.
Data Science LondonData Science. Big Data. Data Hacks. Data Junkies. Data Startups. Open Data
Data Science ReneeDocumenting my path from SQL Data Analyst pursuing an Engineering Master's Degree to Data Scientist
Data Science ReportMission is to help guide & advance careers in Data Science & Analytics
Data Science TipsTips and Tricks for Data Scientists around the world! #datascience #bigdata
Data VizzardDataViz, Security, Military
DataScienceX
deeplearning4j
DJ PatilWhite House Data Chief, VP @ RelateIQ.
Domino Data Lab
Drew ConwayData nerd, hacker, student of conflict.
Emilio Ferrara#Networks, #MachineLearning and #DataScience. I work on #Social Media. Postdoc at @IndianaUniv
Erin BartoloRunning with #BigData--enjoying a love/hate relationship with its hype. @iSchoolSU #DataScience Program Mgr.
Greg RedaWorking @ GrubHub about data and pandas
Gregory PiatetskyKDnuggets President, Analytics/Big Data/Data Mining/Data Science expert, KDD & SIGKDD co-founder, was Chief Scientist at 2 startups, part-time philosopher.
Hadley WickhamChief Scientist at RStudio, and an Adjunct Professor of Statistics at the University of Auckland, Stanford University, and Rice University.
Hakan KardasData Scientist
Hilary MasonData Scientist in Residence at @accel.
Jeff HammerbacherReTweeting about data science
John Myles WhiteScientist at Facebook and Julia developer. Author of Machine Learning for Hackers and Bandit Algorithms for Website Optimization. Tweets reflect my views only.
Juan Miguel LavistaPrincipal Data Scientist @ Microsoft Data Science Team
Julia EvansHacker - Pandas - Data Analyze
Kenneth CukierThe Economist's Data Editor and co-author of Big Data (http://www.big-data-book.com/).
Kevin DavenportOrganizer of https://www.meetup.com/San-Diego-Data-Science-R-Users-Group/
Kevin MarkhamData science instructor, and founder of Data School
Kim ReesInteractive data visualization and tools. Data flaneur.
Kirk BorneDataScientist, PhD Astrophysicist, Top #BigData Influencer.
Linda RegberData story teller, visualizations.
Luis ReiPhD Student. Programming, Mobile, Web. Artificial Intelligence, Intelligent Robotics Machine Learning, Data Mining, Natural Language Processing, Data Science.
Mark StevensonData Analytics Recruitment Specialist at Salt (@SaltJobs) Analytics - Insight - Big Data - Datascience
Matt HarrisonOpinions of full-stack Python guy, author, instructor, currently playing Data Scientist. Occasional fathering, husbanding, organic gardening.
Matthew RussellMining the Social Web.
Mert NuhoğluData Scientist at BizQualify, Developer
Monica RogatiData @ Jawbone. Turned data into stories & products at LinkedIn. Text mining, applied machine learning, recommender systems. Ex-gamer, ex-machine coder; namer.
Noah IliinskyVisualization & interaction designer. Practical cyclist. Author of vis books: https://www.oreilly.com/pub/au/4419
Paul MillerCloud Computing/ Big Data/ Open Data Analyst & Consultant. Writer, Speaker & Moderator. Gigaom Research Analyst.
Peter SkomorochCreating intelligent systems to automate tasks & improve decisions. Entrepreneur, ex Principal Data Scientist @LinkedIn. Machine Learning, ProductRei, Networks
Prash ChanSolution Architect @ IBM, Master Data Management, Data Quality & Data Governance Blogger. Data Science, Hadoop, Big Data & Cloud.
Quora Data ScienceQuora's data science topic
R-BloggersTweet blog posts from the R blogosphere, data science conferences and (!) open jobs for data scientists.
Rand Hindi
Randy OlsonComputer scientist researching artificial intelligence. Data tinkerer. Community leader for @DataIsBeautiful. #OpenScience advocate.
Recep ErolData Science geek @ UALR
Ryan OrbanData scientist, genetic origamist, hardware aficionado
Sean J. TaylorSocial Scientist. Hacker. Facebook Data Science Team. Keywords: Experiments, Causal Inference, Statistics, Machine Learning, Economics.
Silvia K. Spiva#DataScience at Cisco
Harsh B. GuptaData Scientist at BBVA Compass
Spencer NelsonData nerd
Talha OzEnjoys ABM, SNA, DM, ML, NLP, HI, Python, Java. Top percentile kaggler/data scientist
Tasos SkarlatidisComplex Event Processing, Big Data, Artificial Intelligence and Machine Learning. Passionate about programming and open-source.
Terry TimkoInfoGov; Bigdata; Data as a Service; Data Science; Open, Social & Business Data Convergence
Tony BaerIT analyst with Ovum covering Big Data & data management with some systems engineering thrown in.
Tony OjedaData Scientist , Author , Entrepreneur. Co-founder @DataCommunityDC. Founder @DistrictDataLab. #DataScience #BigData #DataDC
Vamshi AmbatiData Science @ PayPal. #NLP, #machinelearning; PhD, Carnegie Mellon alumni (Blog: https://allthingsds.wordpress.com )
Wes McKinneyPandas (Python Data Analysis library).
WileyEdSenior Manager - @Seagate Big Data Analytics @McKinsey Alum #BigData + #Analytics Evangelist #Hadoop, #Cloud, #Digital, & #R Enthusiast
WNYC Data News TeamThe data news crew at @WNYC. Practicing data-driven journalism, making it visual and showing our work.
Alexey GrigorevData science author

Newsletters

top

Youtube Videos & Channels

top

Telegram Channels

top

  • Open Data Science – First Telegram Data Science channel. Covering all technical and popular staff about anything related to Data Science: AI, Big Data, Machine Learning, Statistics, general Math and the applications of former.
  • Loss function porn — Beautiful posts on DS/ML theme with video or graphic vizualization.
  • Machinelearning – Daily ML news.

Slack Communities

top

Github Groups

Competitions

Some data mining competition platforms

Fun

Infographic

PreviewDescription
Key differences of a data scientist vs. data engineer
A visual guide to Becoming a Data Scientist in 8 Steps by DataCamp (img)
Mindmap on required skills (img)
Swami Chandrasekaran made a Curriculum via Metro map.
by @kzawadz via twitter
By Data Science Central
Data Science Wars: R vs Python
How to select statistical or machine learning techniques
Choosing the Right Estimator
The Data Science Industry: Who Does What
Data Science Venn Euler Diagram
Different Data Science Skills and Roles from this article by Springboard
A simple and friendly way of teaching your non-data scientist/non-statistician colleagues how to avoid mistakes with data. From Geckoboard's Data Literacy Lessons.

Data Sets

Comics

Awesome Data Science

Hobby

Other Lists

Contribute to this list: https://github.com/academic/awesome-datascience