FS20
DS1/VSS Challenge Task FS 2020 Winner
The winning group is: Dominic Klinger, Lukas Leuenberger, and Jan Ruch.
They gathered weather data from multiple locations and they used many components: ActiveMQ, Eureka discovery services, Kibana, Docker compose, running on self-hosted gitlab
On the 04.08.2020, 08:10 - 09:50, we will have the "Beratungswoche" online via Microsoft Teams. The access code for this session is: wdfade1. In order to enter the code, click join or create a team below your teams list and look for the Join a team with a code card.
- Lecture dates 17.02.2020 - 29.05.2020
- Exam sessions 10.08.2020 - 11.09.2020
- Deadline Challenge Task 12.05.2020 23:59
Verteilte SW-Systeme (VSS)
| Nr | Date Lecture | Date Exercise | Topics | Slides | Milestones |
|---|---|---|---|---|---|
| 01 | 18.02.2020 | - | Admin, Introduction | VSS-FS20-01-Admin_v1.pdf, VSS-FS20-01-Introduction_v2.pdf, alpine-fs20_v3.ova (VirtualBox Image - root/hsr12345 (47MB)) | Group forming idea for services |
| 02 | 25.02.2020 | 25/26.02.2020 | Protocols, Basics, Scalability | VSS-FS20-02-Proto_v4.pdf | Setup dev environment |
| 03 | 03.03.2020 | 03/04.03.2020 | Application Protocols | VSS-FS20-03-App-proto_v3.pdf | Information flow |
| 04 | 10.03.2020 | 10/11.03.2020 | Distributed Applications: load balancing (slide 1-19) | VSS-FS20-04-Dist-app_v3.pdf | Load balancer |
| - | (lecture and exercise canceled) | - | |||
| 05 | 24.03.2020 | 24/25.03.2020 | Distributed Applications: CORS (slide 20), gRPC (slide 18-19), git (slide 22-26) | (online lectures: videos are below) VSS-FS20-05-News.pdf, VSS-FS20-04-Dist-app_v3.pdf | |
| 06 | 31.03.2020 | 31/01.04.2020 | Authorization (basic auth, digest auth, session-based, JWT), message queues | (online lectures: videos are below) VSS-FS20-06-News.pdf, VSS-FS20-06-Auth.pdf, VSS-FS20-06-Queues.pdf | Storage backend |
| 07 | 07.04.2020 | 07/08.03.2020 | Containers and VMs | (online lectures: videos are below) VSS-FS20-07-News.pdf, VSS-FS20-07-Docker.pdf | |
| 08 | - | 15.04.2020 | - | ||
| 09 | 21.04.2020 | - | Blockchain, Bitcoin, Ethereum I | (online lectures: videos are below) VSS-FS20-09-News.pdf, VSS-FS20-09-Bitcoin.pdf | |
| 10 | 28.04.2020 | 28/29.04.2020 | Blockchain, Bitcoin, Ethereum II | (online lectures: videos are below) VSS-FS20-10-News.pdf, VSS-FS20-10-Bitcoin2.pdf | Frontend |
| 11 | 05.05.2020 | 05/06.05.2020 | Docker, Docker Compose, Connecting everything | (online lectures: videos are below), VSS-FS20-11-News.pdf, VSS-FS20-11-Docker-Deployment.pdf | Connecting everything |
| 12 | 12.05.2020 | 12/13.05.2020 | Real-world examples, P2P OTC trading with sibex.io and COVID-19 traffic light system with livealytics.com, promissory note with FQX fqx.ch with Daniel Killenberger - daniel-at-fqx.ch | (online lectures: videos are below, no more exam relevant material!), VSS-FS20-12-News.pdf, VSS-FS20-12-Real-HTLC.pdf, VSS-FS20-12-Real-Counter.pdf | |
| 13 | 19.05.2020 | - | Challenge Task Presentations | (online lectures, no more exam relevant material!) VSS-FS20-13-News.pdf, VSS-FS20-13-CT.pdf Please send me any questions you have (if any) until 25.5.2020 via email | |
| 14 | 26.05.2020 | - | Challenge Task Award Winner Announcement, Q&A | (online lectures, no more exam relevant material!) VSS-FS20-14-CT.pdf | |
| 15 | 04.08.2020 | - | Beratungswoche |
Online lectures
Due to COVID-19, the VSS lecture will be held online. I'm preparing online videos as a replacement for the lectures on Tuesdays 08:10 - 09:50. The length of the videos will be ~80 - 90 minutes (2 x 45min. lectures) divided into segments. For the exercises, I will be available online on Tuesday from 10:00 - 17:00 and Wednesday 10:00 - 17:00 on https://whereby.com/tombocek. Please contact me if you have any questions regarding the challenge task or also just to say Hi :)
Disclaimer: the material below is only available within HSR and publishing outside of HSR is not allowed. Make sure that you connect via VPN, as access outside HSR is restricted.
Challenge Task FS2020 hand in
Due to COVID-19, your challenge task code, configuration, and documentation needs to be sent via email to thomas.bocek-at-hsr.ch. The code and configuration should be easy to read and/or well documented, the presentation (slides or text) should show the architecture, components, and design decisions. The hand in starts now (21.4.2020) and the deadline is 12.05.2020 23:59. In case of questions with respect to the challenge task, I will contact you directly.
Lecture 5
In the News discusses the crypto currency markets, as well as some positive news from South Korea and India with respect to crypto currency regulations. Also MakerDAO showed weaknesses during the crypto currency stress test. Slides: VSS-FS20-05-News.pdf
Since this topic was not completed in lecture 4, the NGINX, Traefik, HAProxy segment show how to setup these 3 load balancers. Slides: 12-17 from VSS-FS20-04-Dist-app_v3.pdf.
Some students asked if they can load balance the storage as well. This segment shows how to load balance gRPC traffic for a KV-storage backend. Slides: 18-19 from VSS-FS20-04-Dist-app_v3.pdf.
When developing a distributed system, you may run into security policy issues (as some students did). This segment shows how to deal with CORS (Cross-Origin Resource Sharing). Slides: 20 from VSS-FS20-04-Dist-app_v3.pdf.
Git is a distributed version-control system, which is used by many challenge task groups. This segment shows basic commands for git, as not all students are familiar with git. Slides: 22 - 26 from VSS-FS20-04-Dist-app_v3.pdf.
Lecture 6
In the News discusses the Discrod incident as well as the announcement of the Ethereum 2.0 roadmap. Slides: VSS-FS20-06-News.pdf
Authentication dives into the various authentication topics from basic auth, to session-based authentication to JWT. Examples for both, JWT and session-based authentication are presented. Finally, an short overview over OAuth is given. Slides: VSS-FS20-06-Auth.pdf
Since some people were asking, here is my setup for recording these lectures.
In this segment, message queues are discussed and how they could be applied in your challenge task. As an example, RabbitMQ with a simple producer and consumer is shown. Slides: VSS-FS20-06-Queues.pdf
Lecture 7
In the News discusses the .crypto domains and a brief reminder about concurrency vs parallelism. Slides: VSS-FS20-07-News.pdf
Some students are using docker containers for their challenge task, so in this segment, I'm presenting docker containers and how they could be used in your challenge task. A very convenient way to start your application with all your components is to provide a docker-compose configuration file, which starts all the necessary containers. Slides: VSS-FS20-07-Docker.pdf
Lecture 9
In the News discusses the provider "war" between UPC and Init7. Slides: VSS-FS20-09-News.pdf
This segment provides a high level overview over Bitcoin. Slides: VSS-FS20-09-Bitcoin.pdf
Lecture 10
In the News presents a Netflix article about TLS1.3 and a Cloudflare article about HTTP/2 vs. HTTP/3. Slides: VSS-FS20-10-News.pdf
This segment closes the overview over Bitcoin and provides a high level overview over Ethereum. Slides: VSS-FS20-10-Bitcoin2.pdf
Lecture 11
In the News presents a comparison of TCP+TLS with QUIC and briefly mentions git branching conventions and clouds. Slides: VSS-FS20-11-News.pdf
This segment combines all parts together and shows how a potential solution to the challenge task could look like. The solution uses docker and docker-compose. Slides: VSS-FS20-11-Docker-Deployment.pdf
Lecture 12
In the News talks about erasure pipeline of Twitter as an example of complexity, and briefly discusses the Bitcoin halving. Slides: VSS-FS20-12-News.pdf
P2P OTC trading with sibex.io. This segment shows what cross-chain atomic swaps are and briefly shows the product that integrated these swaps. Slides: VSS-FS20-12-Real-HTLC.pdf
COVID-19 traffic light system with livealytics.com. This segment shows WebSockets and briefly shows the product that used those for a COVID-19 counter. Slides: VSS-FS20-12-Real-Counter.pdf
Daniel Killenberger (daniel-at-fqx.ch) is showing how distributed systems are used at FQX. He talks about promissory notes, Kubernetes, deployments, and garden.io.
Lecture 13
In the News talks about the unattributable "db8151dd" data breach (which is now attributed: Covve), about the cost of TCP and the UPbit hack. Slides: VSS-FS20-13-News.pdf
This segment is the first part of the challenge task hand-in by the students, which shows many different technical solutions. While the architecture is similar accross the solutions, the libraries and frameworks differ greatly. I'm excited to look into the rest of the hand-ins. Slides: VSS-FS20-13-CT.pdf
This segment is the second part of the challenge task hand-in by the students, which shows many different technical solutions. The solutions with a docker-compose.yml are very convenient to setup and typically runs with a couple of minutes. Slides: VSS-FS20-13-CT.pdf
This segment is the third part of the challenge task hand-in by the students. Slides: VSS-FS20-13-CT.pdf
This segment is the fourth part of the challenge task hand-in by the students. Slides: VSS-FS20-13-CT.pdf
Lecture 14
The challenge task winners are: Dominic Klinger, Lukas Leuenberger & Jan Ruch. Congratulations! Slides: VSS-FS20-14-CT.pdf
Challenge Task FS 2020
This semester's challenge task (CT) is the design and implementation of a simple distributed system, with the following components:
- Simple Frontend (e.g., HTML, Vue, React)
- Loadbalancer(e.g., traefik)
- Two instances of a service (your choice) (e.g., REST / HTTP / Protobuf)
- e.g., coin tracker, survey / form, monitor application health, ...
- One KV storage backend (e.g., Protobuf)
- Authentication (e.g., basic auth)
VirtualBox Image
The alpine image is a VirtualBox image, that can be imported. Install VirtualBox and click on "File -> Import Appliance" to import the ova image. Once the image is started, you can login with root/hsr12345 or ssh into the image with ssh root@localhost -p 2222.
If you see Received disconnect from 127.0.0.1 port 2222:2: Too many authentication failures Disconnected from 127.0.0.1 port 2222, you probably
have to many ssh keys in .ssh. Either login with ssh -o PubkeyAuthentication=no -p 2222 root@127.0.0.1 or set MaxAuthTries 20 (something higher than 6) in /etc/ssh/sshd_config in the alpine image.
To use password less login, use ssh-copy-id -p 2222 root@127.0.0.1, which copies your public key to your VM and on the next login, you don't need to provide a password.
Requirements
All requirements below must be met in order to pass the challenge task.
- Use at least one text protocol, at least one binary protocol
- Load balancing
- Scalable service with authentication
- Storage does not need to be scalable (you can build it scalable if you want)
- Self-hosted preferred
- The solution may use existing libraries and code, but those must be allowed to be published under an open software license
Deliverables: 12.05.2020 - well documented code and infrastructure, presentation (slides) and demo of the application, also showing the architecture and design decisions. You are allowed to use any language, framework, and platforms. However, the supervisors are familiar with those: Java, Golang, JavaScript, Linux.
Groups
There will be groups of 3 for the challenge task. During the challenge task, the group shall meet every week during exercise hours to work on the task and discuss the next steps. The groups shall utilize their homework times to work on the CT, besides the exercise time slots assigned on Tuesday/Wednesday. You do not have be physically present at the exercises, the supervisors will be at HSR during exercise hours. The groups shall determine and set-up an internal project plan and shall distribute the workload so that each group member gets a fair load of work.
| Nr | Names | Status |
|---|---|---|
| 01 | Raphael J., Gino C., Timothée M. | Email ✓ |
| 02 | Dario C., Marco C., Martin I. | Log ✓ |
| 03 | Pascal S., Pascal S.,Florian F. | Pricechecker ✓ |
| 04 | Abdullah A., Ricky R., Ursin Z. | Shopping list ✓ |
| 05 | Dominic K., Lukas L.r, Jan R. | Weather Data ✓ |
| 06 | Jonas H., Christoph S., Rafael F. | BikePlanner ✓ |
| 07 | Armend L., Christian B., Marco E. | Deceater ✓ |
| 08 | Yves B., David L., Eliane S. | |
| 09 | Marco G., Oliver G., Carlo K. | ClickJack ✓ |
| 10 | Yael S., Nadine S., Saskia S. | Basar ✓ |
| 11 | Laurin Z., Lukas S., Simeon H. | iGETiT ✓ |
| 12 | Felix K., Marc S., Patrick K. | HappyFace ✓ |
| 13 | Fabian T., Yannick V., Robin E. | URL shortener 2 ✓ |
| 14 | Bojan D., Fabian T. | Expense Tracker ✓ |
| 15 | Mike S., Sascha G., Luca T. | Shopping list 2 ✓ |
| 16 | Kevin M., Danusan P., Aynkaran S. | Grade ✓ |
| 17 | Leonard O., Daniel E., Loris K. | EventHub ✓ |
| 18 | Benjamin P., Yanick R. | Chat ✓ |
| 19 | Benny V., Moritz S., David K. | Jass ✓ |
| 20 | Anina B., Pascal S. | Cookbook ✓ (not all req.) |
| 21 | Fabian G., Fabio C., Philippe W. | URL shortener ✓ |
| 22 | Aaron S., Nebai Kidane T. | Todo ✓ |
| 23 | Lukas B., Fabian Z., Pascal P. | ROM ✓ |