CSCI 443: Introduction to Data Science and Numpy, Slides of Computer Science

Download CSCI 443: Introduction to Data Science and Numpy and more Slides Computer Science in PDF only on Docsity!

CSCI 443: LECTURE 1

INTRODUCTION, AND

NUMPY

Professor David Harrison

TODAY: INTRODUCTION

• Who am I?
• Syllabus
• What is data science?
• What is data engineering?
• What does this course cover?
• What tools will we use?
• Some review of practical statistics
• Chapter 1 Practical Statistics
  • Up to page 18 in chapter 1. 2024 PRESENTATION TITLE 2

SYLLABUS

2024 CSCI 443 4

WHAT IS DATA SCIENCE?

2024 CSCI 443 5

Data science encompasses the entire lifecycle of data

processing and analysis, including data collection,

cleaning, exploration, modeling, and interpretation. Its

focus is on extracting insights and knowledge from data

and involves developing methods of recording, storing,

and analyzing data to effectively extract useful

information.

WHAT IS DATA SCIENCE?

2024 CSCI 443 7

AI

Machine Learning

Deep Learning

Data Science

Big Data

Data Analytics, Visualization

This

class

focuses

on data

science.

DATA SCIENTISTS VS DATA ENGINEERS

• Typically 2-5 data engineers to each data scientist.
• Data scientist is often both internally and externally facing.
• Data scientist interfaces with key stakeholders to
  • Define a hypothesis, problem, question, …
  • Design metrics
  • Design the experiments to answer the question.
  • Work with data engineers to understand, clean, and analyze data.
• Data engineers build it:
  • Data collection
  • Data Warehousing / Data Lakes
  • Cloud computing
  • Data pipelines
  • Dashboards and automated reporting
  • Data governance and security. 2024 CSCI 443 8

DATA SCIENTISTS VS DATA ANALYSTS

• Data scientists are both engineers and statisticians. - work on more complex and abstract tasks, such

as developing new analytics methods, predictive

models, and machine learning algorithms.

• Often involved in research and development.
• Data analysts are skilled in data manipulation and visualization.
• Often use processes put in place by a data

scientist.

• Often use tools implemented by data engineers.
• Often support executives and sales
• May be externally facing.
• Often not well versed in engineering 2024 CSCI 443 10

TOOLS

20XX CSCI 443 11 .

GITHUB 20XX CSCI 443^13 Example files I create during class will be put on github. The project is at https://github.com/dosirrah/CSCI443_25S _AdvancedDataScience You will need to create a Github account independent of your olemiss accounts. GitHub is free for our purposes. I highly recommend committing any code you create to GitHub.

NO GITLAB…

20XX CSCI 443 14 Last year I used the department Gitlab. This semester I will only use github.

DATABRICKS

20XX CSCI 443^16 Community edition is free. Offers a single instance with limited cap abilities, but should be adequate for teaching.

DATABRICKS

20XX CSCI 443^17 Community edition is free. You do not need an AWS or Azure account. You do not need to sign up for the 14 - day trial.

DATABRICKS

20XX CSCI 443^19 https://community.cloud.databricks.com Once logged in, you should see options to start a notebook and to imp ort data. Ignore “Upgrade now”

WHY DATABRICKS?

20XX CSCI 443^20 Databricks provides cluster management and a notebook (akin to Jupyter) interface to Apache Spark. Spark unifies:

• Batch processing
• Real-time data processing
• Stream analytics (trending, dashboards, etc.)
• Machine learning
• Interactive SQL
• Successor and extension to what was traditionally done with Hadoop or other map- reduce systems.