FS21
Challenge Task Winner Distributed Systems & Blockchain FS 2021
The winning group is: Andy Pfister, Leandro Ceriani, Michael Brändli
Andy, Leandro, and Michael implemented MegaMeme, which is a voting meme platform with Svelte and Ruby on Rails. They used 3 locations to run their platform redundantly, used docker swarm, and for a unified filesystem GlusterFS. For the database they used CockroachDB. (Announcement)
The price is sponsored by Axelra, a tech venture builder that accelerates digital business models.
- Lecture dates: 22.02.2021 - 04.06.2021
- Q&A week: 09.08.2021 - 13.08.2021
- Exam sessions: 16.08.2021 - 17.09.2021
- Deadline Challenge Task: 17.05.2021, 23:59 (CET)
Distributed Systems & Blockchain (DS1)
| Nr | Date Lecture | Date Exercise | Topics | Slides | Milestones |
|---|---|---|---|---|---|
| 01 | 23.02.2021 | - | Admin, Introduction | - | Group forming, idea for services |
| 02 | 02.03.2021 | 02/03.03.2021 | Protocols, Basics | - | Group forming, idea for services |
| 03 | 09.03.2021 | 09/10.03.2021 | Containers and VMs | - | Information flow, setup dev environment |
| 04 | 16.03.2021 | 16/17.03.2021 | Distributed Applications: load balancing | - | Load balancer |
| 05 | 23.03.2021 | 23/24.03.2021 | Authorization (basic, digest, session-based, JWT, OAuth) | - | JWT Authorization |
| 06 | 30.03.2021 | 30/31.03.2021 | Application Protocols and RPC | - | Storage backend |
| 07 | - | - | Eeaster Vacations | - | |
| 08 | 13.04.2021 | 13/14.04.2021 | Distributed Version Control : git | - | |
| 09 | 20.04.2021 | 20/21.04.2021 | Deployment I, Blockchain, Bitcoin I | - | |
| 10 | 27.04.2021 | 27/28.04.2021 | Deployment II, Blockchain, Bitcoin II | - | Frontend |
| 11 | 04.05.2021 | 04/05.05.2021 | Blockchain, Bitcoin, Ethereum III | - | Connecting everything |
| 12 | 11.05.2021 | 11/12.05.2021 | Real-world examples | - | |
| 13 | 18.05.2021 | 18/19.05.2021 | Challenge Task Presentations I | - | |
| 14 | 25.05.2021 | 25/26.05.2021 | Challenge Task Presentations II | - | |
| 15 | 01.06.2021 | - | Challenge Task Award Winner Announcement, Q&A | - | |
| 16 | 10.08.2021 | - | Beratungswoche |
Online lectures
Due to COVID-19, the DS1 lecture will be held online. I'm preparing online videos as a replacement for the lectures on Tuesdays 13:10 - 14:50. The length of the videos will be ~ 80 - 100 minutes (2 x 45min. lectures) divided into segments. For the exercises, I will be available online on Tuesday from 15:10 - 16:50 and Wednesday 10:10 - 16:50 on MS Teams. Team code: ngh0p3h. Please contact me if you have any questions regarding the challenge task or also just to say Hi :) During exercises, you can work on your challenge task.
The online lectures and PDFs can be accessed from outside (also on Youtube), the excercise material only with VPN.
Lecture 1
The admin part gives an overview over this lecture and presents the challenge task for this course (Slides: DS1-FS21-01-Admin.pdf). The second video is the segment "distributed systems in the news", where I present recent articles and blog posts related to distributed systems and blockchains. This video presents a rant involving ECC by Linus Torvalds, a scaling advice, and about the technical complexity in blockchain projects (Slides: DS1-FS21-01-News.pdf). The third video starts with the distributed systems introduction (Slides: DS1-FS21-01-Intro.pdf). [116min]
Lecture 2
The admin part is about the challenge task and the groups (Slides: DS1-FS21-02-Admin.pdf). The second video is the segment "distributed systems in the news", where I talk about Kubernetes complexity and that SSD will eventually fail (Slides: DS1-FS21-02-News.pdf). The third video starts with 8 fallacies (on slide 21) of distributed systems and the fourth video presents protocols (Slides: DS1-FS21-01-Intro_v2.pdf, DS1-FS21-02-Proto_v2.pdf). [117min]
Lecture 3
The admin part is about the challenge task and the groups (Slides: DS1-FS21-03-Admin.pdf). The second video is the segment "distributed systems in the news", with an article about bitsquatting and a reminder that hardware in outer space needs to be built much more robust. The third article is about the GPU shortage and potential solutions. The fourth article is a reminder that speed matters and the last article is a nice example of a HTTP server in golang (Slides: DS1-FS21-03-News.pdf).The third video introduces Containers and VMs (Slides: DS1-FS21-03-Virtualization.pdf). [99min]
Lecture 4
The first video is the segment "distributed systems in the news", with an articles about cryptominers, EIP-1559, expensive cloud services, and the fire at SBG2 (Slides: DS1-FS21-04-News.pdf). The second video introduces Load Balancing (Slides: DS1-FS21-04-Loadbalancing.pdf). [96min]
Lecture 5
The first video is the segment "distributed systems in the news", with an a discussion about complexity and some performance number of Stack Exchange. The next article is about migrating a load balancer at Slack, following the $60 million auction of an digital image with an NFT, with some interesting insights (Slides: DS1-FS21-05-News.pdf). The second video introduces Authentication (Slides: DS1-FS21-05-Auth.pdf). [87min]
Lecture 6
The admin part shows the current planning and gives an update on round-robin load-balancing with Traefik and Nginx. (Slides: DS1-FS21-06-Admin.pdf). The second video is the segment "distributed systems in the news" with more NFTs and DeFi news and articles (Slides: DS1-FS21-06-News.pdf). The third video presents application protocols and RPC mechanisms. (Slides: DS1-FS21-06-App-proto.pdf). [118min]
Lecture 7
Easter Vacations...
Lecture 8
The admin part shows the current planning, gives an update on the challenge task hand-in, and provide more details on HTTP keep-alive, health checks, and distroless docker images. (Slides: DS1-FS21-08-Admin.pdf). The second video is the segment "distributed systems in the news" with Monero mining with Github Actions, Docker without Docker, and why hardware can fail (Slides: DS1-FS21-08-News.pdf). The third video presents distributed version control with git. (Slides: DS1-FS21-08-Git.pdf). [100min]
Lecture 9
The first video is the segment "distributed systems in the news" with Firefox and Quick, CORS, and lots of blockchain news. (Slides: DS1-FS21-09-News.pdf). The second video is the first part of the deployment lecture (Slides: DS1-FS21-09-Deployment.pdf). The third video starts with Bitcoins and Blockchains. (Slides: DS1-FS21-09-Bitcoin-Blockchain.pdf). [121min]
Lecture 10
The first video is the segment "distributed systems in the news" with a tiny docker container, leaked credentials and Monero mining, and Thodex, a Turkish crypto exange. (Slides: DS1-FS21-10-News.pdf). The second video is the second part of the deployment lecture (Slides: DS1-FS21-10-Deployment2.pdf). The third video continues with Bitcoins and Blockchains. (Slides: DS1-FS21-10-Bitcoin-Blockchain2.pdf). [74min]
Lecture 11
The first video is the segment "distributed systems in the news" with the Chia blockchain, more NFTs, and a library to build your own DNS server. (Slides: DS1-FS21-11-News.pdf). The second video is the third part of the blockchain lecture with Ethereum and Solidity. This video is the last lecture with content relevant for the exam, the lectures 12 and 15 will be shorter and lectures 13 and 14 will be the challenge task presentations. (Slides: DS1-FS21-11-Bitcoin-Blockchain3.pdf). The third video is the admin part. (Slides: DS1-FS21-11-Admin.pdf, DS1_Exam_2020-FS_v3.pdf). [147min]
Lecture 12
The first video is the segment "distributed systems in the news" with the Chia blockchain, some critical thoughts about the crypto world, and the Pinecone overlay network for P2P communication. (Slides: DS1-FS21-12-News.pdf). The second video is the interview with Jani Helin, Technical Director at Moflixgroup, where Jani talkes about the architeckure of the Swype platform. The third video shows swype in action. The fourth video is the interview with Guilherme Sperb Machado, who is the founder of AxLabs. The fifth video is the interview with the Axelra team.[85min]
Lecture 13
The first video is the segment "distributed systems in the news" with Elon Musk and cryptocurrencies. (Slides: DS1-FS21-13-News.pdf). [XYmin]
Lecture 14
The first video is the segment "distributed systems in the news" with more tweets from Elon Musk, more cryptocurrencies, and a technical article from Vitalik Buterin on blockchain scalability. (Slides: DS1-FS21-14-News.pdf). [XYmin]
Lecture 15
The first video is the segment "distributed systems in the news" with the Chia cryptocurrency and the released specification of QUIC. (Slides: DS1-FS21-15-News.pdf). [XYmin]
Lecture 16
Challenge Task FS 2021
This semester's challenge task (CT) is the design and implementation of a simple distributed system, with the following components:
- Simple Frontend (e.g., HTML, Vue, React, Svelte)
- Loadbalancer(e.g., traefik, nginx, HAproxy, Caddy)
- Two instances of a service (your choice)
- One storage backend (Postgres, simple KV memory storage)
- JWT Authentication
VirtualBox Image
You can deploy on any machine you want (AWS, Azure, or your own machine). If you don't want to deploy on your maschine, you can use the following VirtualBox image (139MB). The alpine image is a VirtualBox image, that can be imported. Install VirtualBox and click on "File -> Import Appliance" to import the ova image. Once the image is started, you can login with root/ost12345 or ssh into the image with ssh root@localhost -p 2222. Since docker and docker-compose is already installed, you can try the command docker run hello-world. You should see a "Hello from Docker!".
To use password less login, use ssh-copy-id -p 2222 root@localhost, which copies your public key to your VM and on the next login, you don't need to provide a password.
Requirements
All requirements below must be met in order to pass the challenge task.
- Load balancing
- Scalable service
- JWT authentication
- Dockerized
- Use latest stable releases of chosen libraries and frameworks
- Storage does not need to be scalable (you can build it scalable if you want)
- The solution may use existing libraries and code, but those must open software software
Deliverables: 17.05.2021, 23:59 (CET) - well documented infrastructure, presentation (slides) of the application, also showing the architecture and design decisions via email to thomas.ost-at-bocek.ch (the OST mailserver cannot handle attachements with source code). The code and configuration should be easy to read and/or well documented, the presentation (slides or text) should show the architecture, components, and design decisions. During the weeks 18.05./25.05., you will present and demo your solutions via MS Teams.
You are allowed to use any language, framework, and platforms. However, the supervisors are familiar with those: Java, Golang, JavaScript, Linux.
Groups
There will be groups of 2 - 3 for the challenge task. During the challenge task, the group shall meet every week during exercise hours to work on the task and discuss the next steps. The groups shall utilize their homework times to work on the CT, besides the exercise time slots assigned on Tuesday/Wednesday. You do not have be physically present at the exercises. The groups shall determine and set-up an internal project plan and shall distribute the workload so that each group member gets a fair load of work. Results are only accessible via VPN.