NETS 2120: Scalable and Cloud Computing (Fall 2025)
What is the "cloud"? How do we build software systems and components that scale to
millions of users and petabytes of data, and are "always available"?
In the modern Internet, virtually all large Web services run atop multiple geographically
distributed data centers: Google, Yahoo, Facebook, iTunes, Amazon, eBay, Bing, etc.
Services must scale across thousands of machines, tolerate faults, and support thousands
of concurrent requests. Increasingly, the major providers (including Amazon, Google,
Microsoft, HP, and IBM) are looking at "hosting" third-party applications in their
data centers - forming so-called "cloud computing" services. A significant number
of these services also process "streaming" data: geocoding information from cell
phones, tweets, streaming video, etc.
This course, aimed at a sophomore with exposure to basic programming within the context
of a single machine, focuses on the issues and programming models related to such cloud
and distributed data processing technologies: data partitioning, storage schemes, stream
processing, and "mostly shared-nothing" parallel algorithms.
NETS2120 is
a core requirement for the Data
Science Minor. It also counts as a project elective for
CSCI and
ASCS, and
as an AI Project Elective for the AI BSE.
Instructor
Andreas Haeberlen
Office hours: TBA (Levine 560)
Teaching assistants
Format
The format will be two 1.5-hour lectures per week, plus assigned readings. There will be regular
homework assignments, two midterms, and a term project. We will use an online forum for course-related discussions.
Time and location
Tuesdays/Thursdays 10:15-11:45am (AGH 203)
Prerequisites
CIS 1200, Programming Languages and Techniques
CIS 1600, Mathematical Foundations of Computer Science
Textbooks
Spark: The Definitive Guide, by Bill Chambers and Matei Zaharia (O'Reilly)
ISBN 9781491912218; read online for free, or buy for approx. $54.
Additional materials will be provided as handouts or in the form of light technical papers.
Grading
Homework 30%, Term project 30%, Exams 35%, Participation/quizzes 5%
Policies
You can find a list of key course policies here.
Assignments
Homework assignments will be available for download; solutions should be submitted via GradeScope.
Tentative schedule
| Date | Topic | Details | Reading | Remarks |
|---|
| Aug 26 |
Introduction |
Course introduction
"What is the Cloud, and why is it interesting?"
Data-centric computing
Course goals
Logistics
Policies
Overview of topics
|
|
|
| Aug 28 |
The Cloud |
What is the Cloud?
Cloud hardware
Problems with classical scaling
Utility computing
Kinds of clouds
Virtualization
Cloud challenges
|
Armbrust: A view of cloud computing |
HW0 released |
| Sep 2 |
Concurrency |
Scalability and parallelization; Amdahl's law
Synchronization/concurrency/consistency
Mutual exclusing and locking
"NUMA, shared-nothing"
"Frontend/backend, sharding"
|
Vogels: Eventually consistent |
|
| Sep 4 | No class (Andreas out of town) |
| Sep 9 |
The Internet |
The Internet; packet switching
Path properties; TCP
HW1 overview
|
MDN: A re-introduction to JavaScript |
HW0 due; HW1 released |
| Sep 9 | Last day to add |
| Sep 11 |
Faults and Failures |
Fault models
Examples of non-crash faults
Replication; durability and availability
Primary-backup replication
Quorum replication
Network partitions; CAP theorem
|
Tseitlin: The antifragile organization |
|
| Sep 16 |
Cloud basics |
History of cloud computing
Interacting with the cloud
EC2 basics
EBS basics
Overview of some other AWS services
|
"Cloud computing features, issues, and challenges: a big picture" |
HW1MS1 due |
| Sep 18 |
Cloud storage |
Key-value stores
KVS and concurrency
KVS and the Cloud
Case study: S3
Case study: DynamoDB
|
Cooper et al.: PNUTS to Sherpa - Lessons from Yahoo!'s Cloud Database |
|
| Sep 23 |
Spark |
Introduction to scalable analytics
MapReduce
The Streams API
Apache Spark
Lambdas and serialization
|
"Spark textbook, Chapter 2 and 3" |
HW1MS2 due |
| Sep 25 |
Programming in Spark |
Spark jobs
Working with files
Spark transformations
Spark actions
The Structured API
Distributed shared variables
|
"Spark textbook, Chapters 4-8" |
HW2 released |
| Sep 30 | First midterm exam |
| Oct 2 |
Understanding Spark |
Origins of Spark
The HDFS file system
Using HDFS
Apache Livy
|
Zaharia et al.: Cluster Computing with Working Sets |
|
| Oct 6 | Last day to drop |
| Oct 7 |
Graph algorithms |
Distributed graph algorithms
Distributed graphs
Graph algorithms in Spark
Single-source shortest path
K-Means clustering
Naive Bayes learning
|
"Lin & Dyer, Chapter 5" |
HW2MS1 due |
| Oct 9-12 | Fall Break |
| Oct 14 |
Random-walk algorithms |
Random-surfer model
Naive PageRank
Full PageRank
Adsorption / label propagation
|
Baluja et al.: Video Suggestion and Discovery for YouTube |
HW2MS2 due; HW3 released |
| Oct 16 |
Iterative processing |
Iterative processing
Bulk synchronous parallelism
Pregel and graph processing
Overview of deep neural nets
MXnet
|
Malewicz et al.: 'Pregel - A System for Large-Scale Graph Processing' |
|
| Oct 21 |
Web programming |
Web overview
HTML and CSS
Client/server model
The Domain Name System
|
"Cloudflare: HTTP/3: The past, the present, and the future" |
|
| Oct 23 |
Web programming (continued) |
HTTP and HTTPS
HTTP/2 and HTTP/3
Server design
|
|
HW3 due |
| Oct 24 | Last day to designate course as pass/fail |
| Oct 28 |
Node.js |
Motivation: CGI and servlets
Node.js; basic operation
Hello world with Node
Accessing data
Cookies and sessions
|
"Node at LinkedIn: the pursuit of thinner, lighter, faster" |
HW4 and project handout released |
| Oct 30 |
Dynamic content |
Project overview
Project advice
The Document Object Model
XMLHttpRequest
|
React: Facebook's Functional Turn on Writing JavaScript |
|
| Nov 3 | Last day to withdraw |
| Nov 4 |
AJAX |
AJAX overview
AJAX with jQuery
socket.io and async
Working with APIs
|
|
HW4MS1 due; team formation deadline; project begins |
| Nov 6 |
Web services and XML |
Web services
Data interchange; challenges
Data formats
XML
Working with XML
DTDs
XML Schema
XML DOM
|
|
First project check-in |
| Nov 11 |
Security |
Cryptography; RSA
Digital signatures
Attacks and Defenses (Part 1)
Attacks and Defenses (Part 2)
|
Current OWASP Top 10 |
HW4MS2 due |
| Nov 13 |
Databases |
Motivations for databases and data management
"Relational model, data streams"
SQL basics; declarative approach; query optimization
Transactions; ACID
|
F1: A Distributed SQL Database That Scales |
HW4MS3 due; second project check-in |
| Nov 18 |
Peer-to-peer |
Decentralization
Partly centralized systems; BitTorrent
Unstructured overlays; epidemic protocols
Structured overlays; consistent hashing; KBR
Case study: Pastry
Security challenges
|
Rodrigues and Druschel: P2P systems |
Third project check-in |
| Nov 20 |
Case study: Bitcoin |
Distributed ledgers
Bitcoin and Proof-of-Work
Bitcoin Script
Challenges in Bitcoin
|
Nakamoto: Bitcoin |
|
| Nov 25 |
Case study: Facebook |
Facebook's TAO
Scalability in TAO
Fault handing in TAO
Facebook's Haystack
Haystack design
|
Bronson et al.: TAO: Facebook's Distributed Data Store for the Social Graph |
Fourth project check-in |
| Nov 27 | No class (Thanksgiving) |
| Dec 2 |
Special topics |
Differential privacy
Federated analytics
|
Differential Privacy: The Pursuit of Protections by Default |
|
| Dec 4 | Second midterm exam |
| Dec 11-18 | Project demos; written reports due |
|
