Operating Systems Design and Implementation

CPSC 436A

In this course, the students will obtain a thorough understanding of the challenges and issues related to the design and implementation of modern multicore operating systems. The students will apply and extend their knowledge in systems, software engineering, project management, and team work.

The course covers the design and implementation of various operating systems concepts such as memory management, scheduling, inter-process communication, inter-core synchronization, protection, device drivers, file systems, and networking. Moreover, the course pays particular attention to the design of system software architectures that differ from the traditional monolithic arrangements of Unix/Linux and Windows.

During the course, the students will work together in small groups to build a fairly complete operating system.

Credit: This course is based on 263-​3800-00L @ ETH Zurich taught by Prof. Timothy Roscoe. Additional and huge thanks to Reto Achermann who made delivering this course at UBC possible.

This is a demanding course, read the prerequisites below.

Goals

  • Teach general operating systems principles, using a real research operating system to illustrate them and by reading the research papers which propose some of the ideas that the particular OS builds on.
  • Give a broader perspective on operating systems which do not look like Linux, Unix, or Windows.
  • Provide exposure to the practical experience of working on OS code on real “metal”, including debugging, hardware access, etc. This kind of experience is hard to gain merely from reading books or papers.
  • Introduce a sense of the complexity of a real OS, rather than simplified teaching OSes often used in more basic courses.

Prerequisites

This course is about the design and implementation of operating systems, and thus requires strong background in computer systems, software engineering, and programming.

Students should have taken an undergraduate systems and software engineering course course. Ideally this means the following:

  • CPSC 313 (or equivalent) with 85% or above.
  • CPSC 310 (or equivalent) with 85% or above.
  • CPSC 317 (or equivalent) with 85% or above.

(See the course catalogue for hard requirements)

Additional requirements that are helpful for successfully completing the course.

  • Proficiency in C programming is assumed
  • Knowledge of computer architecture: caches, MMU, interrupts, devices
  • Not afraid to read assembly code
  • Navigate technical documentation and manuals
  • Familiarity with OS concepts: concurrency, asynchrony, process management, memory management, virtual memory, networking, filesystem
  • Linux development tools: shell, editors, make, gcc, gdb, …
  • General interest in system software and low-level hacking

Schedule

The lectures and labs are designed around the class project. They are designed to help the students to succeed in completing each milestone and progress through the construction of their operating system.

Classes

Monday (1pm-4pm ICICS/CS 005) – Tuesday (9am-11am MCLD 2002) – Thursday (9am–11am MCLD 2002)

Optional (office hours): Wednesday (2pm-3pm ICICS/CS 304) – Friday (2pm-3pm ICICS/CS 304)

This may seem like a lot of hours, but unlike a lot of courses, this course focus on hand-on experience. You will spend a significant amount of those hours working as a team or discussing with the instruction team. The activities during the term are divided between the following categories:

Lecture: During a lecture slot, we will cover the necessary background information, present an overview of the up-coming milestone, and provide hints and tips for the next project milestone. The students are expected to have at least looked at the next milestone. The students are encouraged to ask questions regarding the project during lectures.

Presence is strongly encouraged.

Lab: This is a office-hours like lab session. Course staff will be present to answer project related questions. During this slot, the students are expected to meet with their team and work on the project. Some groups will be asked to present the progress they have made on the current milestone.

Presence is encouraged. Presence is mandatory if your group is presenting. Failure to show up without giving notice will result in a zero in the associated assignment.

Tutorial: This is similar to the Lab sessions; but we’ll have a short tutorial with some additional information that may be useful for the project. The remainder of the session will be available to work on the project and ask questions.

Presence is encouraged.

Milestones demonstrations: During this slot the students will present and explain their milestone solutions to the course staff.

Presence is mandatory. Failure to show up without giving notice will result in a zero in the associated assignment.

Office Hours: During this slot you can ask questions about the course and/or your project.

Presence is optional.

Quizzes: Short knowledge test organized in the CBTF rooms.

Presence is mandatory. Failure to show up without giving notice will result in a zero in the associated assignment.

Week Start End Monday Tuesday Thursday
1 4-sep 8-sep Lecture: Introduction
2 11-sep 15-sep Lab: environment setup Tutorial: Barrelfish and Tools Lecture: Capabilities
3 18-sep 22-sep M0 - Presentation
Drop deadline
M0 Code deadline
Tutorial: Capability Operations
Group Formation Deadline
Lecture: Memory Management
4 25-sep 29-sep Lab
Quizz
Tutorial: Debugging and Autograder Lecture: Virtual Memory/Paging I
5 2-oct 6-oct Q&A Lecture: Virtual Memory/Paging II
6 9-oct 13-oct Q&A M1 - Presentation
M1 Code deadline
7 16-oct 20-oct M2 - Presentation
M2 Code deadline
Tutorial Managing foreign
VSpaces and CSpaces
Lecture: Process
8 23-oct 27-oct Lab
Quizz
Q&A Lecture: LRPC, IPC, Upcalls
9 30-oct 3-nov M3 - Presentation
M3 Code deadline
Tutorial: LMP Q&A
10 6-nov 10-nov M4 - Presentation
M4 Code deadline
Q&A Lecture: Booting Cores
11 13-nov 17-nov Lecture: User-level message passing
Mid-term report deadline
12 20-nov 24-nov Lab
Quizz
Q&A Q&A
13 27-nov 1-dec M5 - Presentation
M5 Code deadline
Q&A Lecture: Individual Milestones
14 4-dec 8-dec M6 - Presentation
M6 Code deadline
Q&A Lecture: Research Projects (TBC)
15 11-dec 15-dec Final Report Submission
16 18-dec 22-dec Final Code Submission

Instructors

Teaching Assistants

Avatar

Ilias Karimalis

Teaching Assistant

Avatar

Xuechun "Joyce" Cao

Teaching Assistant

Grading

Individual Grades (35%)

15% Quizzes. The grade for the three quizzes.

5% Milestone 0 Demonstration. These are the points awarded during the individual milestone presentation. The students are expected to demonstrate the required milestone functionality, and explain their design and implementation.

5% Milestone 7 Integration Test. These are the results of the integration tests run on the final submission, focused on the individual milestone.

10% Milestone 7 Report This includes the quality and completeness of the final report for the final individual report.

Group Grades (65%)

25% Milestone Demonstration. These are the points awarded during the group milestone presentations. The students are expected to demonstrate the required milestone functionality, and explain their design and implementation.

5% Intermediate Group Report. This includes the quality and completeness of the intermediate group report.

10% Final Group Report. This includes the quality and completeness of the final group report.

20% Integration Test. These are the results of the integration tests run on the final submission.

5% Class Presentation. How well the group did during their presentation(s).

Further Information

Extra Challenges. Milestones come with extra challenges that award the student extra points and additionally provides the basis for a good reference letter. The completion of the extra challenges is not required for obtaining a maximum grade. To receive the extra points, the extra challenge must be presented before the normal deadline. No points will be awarded for completed extra challenges on late submissions, or if the base or target tasks have not been met.

All milestones must be completed.

This means presenting the stated base functionality for each of the milestones. This is extremely important as the milestones are cumulative. The base functionality are the minimum required to implement subsequent milestones.

Late submissions. Generally, submissions must be completed by the stated deadline. Late submissions will be graded at 75% (one week delay) and 50% (two weeks delay) of the base points respectively. To help the students priorize the taskes, the course materials indicate the base functionality for each milestone. Due to the schedule and to ensure enough time for grading, there is no late hand-in for milestone M0 and the final submission.

Joker Cards. Life happens, and sometimes it’s good to get a little extra time.

  • Each group will receive two joker cards that extend the deadline of group milestones – no question asked. The instructor will set the date for the postponed demonstration. There are no jokercards for the individual milestones at the beginning or at the end of the course.
  • The final hand-in deadline is a hard deadline and cannot be extended

Joker cards must be redeemed the day before the deadline as posted on the schedule.

Why is there so many deadlines?

The goal is to cut the overall project into digestible chunks. The course staff can rapidly identify struggling students and provide adequate support. Furthermore, each individual submission is relatively low stake and you can easily course correct when performing bellow your expectations. While this course is challenging and have a significant workload, you are extremely unlikely to fail if you put in the effort.

Use of third-party libraries, code and tools

This is not an algorithms course, or a coding interview. There won’t be any points awarded for correctly implementing a self-balancing tree, or a solution based on dynamic programming. The students are allowed to make reasonable use of third-party libraries.

The use of third-party code need to be approved.

The use of third-party code (e.g., libraries, data structures, …) must be approved by the instructor and properly cited in the report. Standard plagiarism rules apply.

You may use third-party code in the project, or discuss with other students in the class. However, you must attribute and cite any third-party code that you use in your project. Moreover, you must implement a non-trivial fraction of the code on your own (or as a team). Using code from other students (teams) is not accepted. In any case, all members of the groups must be able to explain the design and implementation.

Code sharing.

The students agree not to publish their code on the Internet, or distribute it in other ways.

Generative AI. The course aims to keep up with a changing landscape and generative AI (e.g., ChatGPT or copilot) is likely to become one of the many tools available to developers. As such the use of such tools is not prohibited. However, you should be able to understand all code you submit and be able to explain it. If you use generative AI during your project, you should explain clearly how you used it (e.g., prompt and any other setup) as well as the strategy you adopted to ensure the correctness of the generated code.

Please, be aware that such tool are unlikely to lead to good results.

I would be happy to be proved wrong, especially if you report details how you got it to work well. However, please do not expect to use those tools at the last minute and to get your deliverable miraculously done. Getting such tools to work in the context of this project is likely to require an equal amount of effort as doing the actual work would have taken.

How to do well

Here we summarize some advice that may be useful to successfully complete the project.

Lectures

To get most out of the lectures, it is highly recommended reading the next chapter in the book before the class. This enables an engaged discussion during class and to help you ask relevant questions.

The lectures are also your chance to ask questions about the next milestone early on in the week. So it’s good to think about the upcoming tasks before the lecture.

Project

Think before you code. Think about the functionality that need to be implemented. Write down your architecture and keep design notes (this is helpful for your presentations and reports). This is especially important, especially when you work as a team.

Read ahead. Each milestone builds upon the functionality of the previous one. It is a good idea to read a bit ahead to get an overview of the entire project.

Read the milestone descriptions carefully.

Are there any hints and explanations? (your answer should be: yes, there is)

What is the interface to be implemented?

What are the required functionalities?

You are significantly more likely to succeed if you read the instructions, follow the suggested task order, and identify the hints you are given.

Write tests. This is one way to make sure your code works as intended. This is especially important, as later milestones build upon earlier ones. Also, those tests are a fantastic way to demonstrate your milestones.

Git. Try to use version control wisely. Use meaningful and descriptive commit messages, and clean patches to help your teammates understand what you have done. Be strategic in how you use branches, we have seen students constructing diverging branches that soon became impossible to merge. A significant portion of the difficulty inherent to this course is about working effectively with others.

Milestone Presentations

Each milestone will be demonstrated to one of the course staff. This will take about 15mins. The students will need to show that they have implemented all required functionality, and be prepared to answer question about their implementation. Any member of a group should be able to explain what is going on.

Reports

Write the report as you go, or at least keep notes. You will need to submit 1-2 pages of notes on their design decisions, implementation details, performance evaluation, etc. with every group milestones.

Presentation

Make it entertaining and interesting, do a demo. What has worked well, what have you learned. What turned out to be a bad decision? The class should be a supportive environment. You are going through the same (difficult?!) experience and it is a good opportunity to share! This class is not a zero-sum game, grades are unscaled, if everyone learns and finishes the class with an A+, it will be a resounding success.

Reading Material

Source code and other materials (the course book, manuals, specifications, etc. ) will be distributed through a git repository. The students will use the provided git repository to submit their code and reports.

What do they say about the course?

Feedback from 2022 UBC students.

It’s a self learning style course. You learn a ton by jumping into the deep end. The beginning of the course is extremely painful but gets better as time goes on.

This is one of the most interesting courses that I have taken. It has a very unique approach that allows you to think through new solutions. I am very happy that this course was added and it’s a lot better than the old OS Course.

The project, its complexities, and challenges were a highlight. While many students struggled, and many of us complained, I think we’ve grown so much as engineers and computer scientists.

I really liked the approach of this course since it is very different from what we have experienced in other classes at UBC, it teaches us what it takes to do real design and implementation of a fairly large system without just being told what to do at every stage. As a 4th year course, it also required us to use our knowledge from other courses which I found to be a great strength, especially when many other CS courses seem very disjoint.

I did have completed 310 and 313 but I still feel that I am not well–prepared for this course. To be honest, the workload for this course is much larger than other courses I have taken.

Best CS course I’ve taken! With the vast majority of my courses, I come out feeling unsatisfied, like the assignments are trivial toy problems, that I didn’t accomplish or learn much, or that I could’ve read a textbook and picked it up on my own in a few weeks. But with 436A I feel like I learned a ridiculous amount in just a couple of months and it made me feel like doing a degree in CS was worth it, and that this was an experience I wouldn’t have gotten by messing around with computers in my free time. Overall, I was really happy with the class :)

It’s too much work, it’s about 3 course load worth of work. I spend at most 4 hours finishing assignments from CPSC 314 and CPSC 322 every other week. For this course, I’m spending 30+ hours per week (all tracked via wakatime extension).

This is definitely the most difficult course I’ve taken, the project is very time consuming and the concepts are difficult. The course is centered around a project which forces you to experiment and focus on design rather than just following a recipe. As a result, you are rewarded with a much deeper understanding of OS concepts that were introduced in CPSC 313.

The course was paced quite aggressively. For most milestones I found that it took me about half the allotted time for the content to simply sink in and to figure out what exactly I was supposed to do. By that point, there was very little time left so my code was often messier than I would have liked.

Weekly feedback and answered questions well without giving away answers so that we could work through the problems ourselves. Probed us with questions to guide us in the right direction. Also provided good lectures.

At Monad I got to learn and work with Rust. I built their RPC server, which CPSC 436A was really handy for ;-)

Past Grade Distribution