Next:
Welcome
Osdev Notes
Ivan G.
Dean T.
cover art by Koobin Bingku
Table Of Contents
Welcome
Structure Of The Book
Topics covered
Assumed Knowledge
About The Authors
Our Projects
Build Process
Freestanding Environment
Cross Compilation
Differences Between GCC and Clang
Setting up a build environment
Getting a Cross Compiler
Building C Source Files
Building C++ Source Files
Linking Object Files Together
Building with Makefiles
Quick Addendum: Easily Generating a Bootable Iso
Testing with An Emulator
Building and Using Debugging Symbols
Multiboot 2
Limine Protocol
ELFs Ahead, Beware!
Locating The Symbol Table
Boot Protocols
What about the earlier versions?
Why A Bootloader At All?
Multiboot 2
Creating a Boot Shim
Creating a Multiboot 2 Header
Limine Protocol
Fancy Features
Creating a Limine Header
Finding Bootloader Tags
Multiboot 2
Limine
Makefiles
GNUMakefile vs Makefile
Simple Makefile Example
Built In Variables
Complex Makefile Example (with recursion!)
Think Bigger!
Linker Scripts
Anatomy of a Script
LMA (Load Memory Address) vs VMA (Virtual Memory Address)
Adding Symbols
Program Headers
Example
Sections
The ‘.’ Operator
Incoming vs Outgoing Sections
Common Options
Complete Example
Generating A Bootable Iso
Xorriso
Grub (Multiboot 2)
Grub.cfg
Limine (Limine Protocol, Multiboot 2)
Limine.conf
Kernel Build Process & Booting
Architecture And Drivers
Address Spaces
Higher and Lower Halves
The GDT
How The CPU Executes Code
Interrupts
Drivers
Hello World
Printing to Serial
Initialization
Sending a string
Printing Digits
Troubleshooting
Higher Half Kernel
A Very Large Address
Loading a Higher Half Kernel
The Global Descriptor Table
Overview
GDT Changes in Long Mode
Terminology
Segmentation
Segment Registers
Segmentation and Paging
Segment Descriptors
Using the GDT
Interrupt Handling on x86_64
The Interrupt Flag and Cli/Sti
Non-Maskable Interrupts
Setting Up For Handling Interrupts
Interrupt Descriptors
Loading an IDT
Interrupt Handler Stub
An Example Stub
Sending EOI
Interrupt Dispatch
Reserved Vectors
Troubleshooting
Remapping The PICs
Halt not Halting
ACPI Tables
RSDP and RSDT/XSDT
RSDP
RSDT Data structure and fields
Some useful infos
APIC
What is APIC
Types of APIC
Local APIC
Disabling The PIC8259
Discovering the Local APIC
Enabling the Local APIC, and The Spurious Vector
Reading APIC Id and Version
Local Vector Table
X2 APIC
Handling Interrupts
Sending An Inter-Processor Interrupt
I/O APIC
Configure the I/O APIC
Getting the I/O APIC address
I/O APIC Registers
Reading data from I/O APIC
Interrupt source overrides
IO Redirection Table (IOREDTBL)
Timer
Types and Characteristics
Calibrating Timers
Programmable Interval Timer (PIT)
Theory Of Operation
Example
High Precision Event Timer (HPET)
Discovery
Theory Of Operation
Comparators
Example
Local APIC Timer
Example
Timestamp Counter (TSC)
Useful Abstractions
Adding keyboard support
Keyboard Overview
Handling The Keyboard Interrupt
IRQ and IOAPIC
Driver Information
Sending Commands To The Keyboard
Identifying The Scancode Set
About Scancodes
Handling Keyboard Interrupts
Keyboard Driver Implementation
High Level Overview
Storing A History Of Pressed Keys
Handling Multi-Byte Scancodes
Handling Modifier keys
Translation
Architecture
Video Output and Framebuffer
Requesting a Framebuffer Mode (Using Grub)
Accessing the Framebuffer
Framebuffer Type
Plotting A Pixel
Drawing An Image
Drawing Fonts
Embedding a PSF File In The Kernel
Parsing the PSF
PSF v1 Structure
PSF v2 Structure
Glyph
Video Output
Memory Management
A Word of Wisdom
PMM - Physical Memory Manager
Paging
VMM - Virtual Memory Manager
Heap Allocator
An Example Workflow
Physical Memory Manager
The Bitmap
Returning An Address
Freeing A Page
Paging
What is Paging?
Page
Page Directories and Tables
Virtual (or Logical) Address
Paging in Long Mode
Page Directories and Table Structure
Loading the root table and enable paging
PML4 & PDPR & PD
Page Table
Page Table/Directory Entry Fields
Address translation
Address Translation Using 2MB Pages
Address translation Using 4KB Pages
Page Fault
Accessing Page Tables and Physical Memory
Recursive Paging
Direct Map
Troubleshooting
Virtual Memory Manager
An Overview
Virtual Memory
Concepts
How Many VMMs Is Enough?
Managing An Address Space
Allocating Objects
The Extra Argument
Freeing Objects
Workflow
Example 1: Allocating A Temporary Buffer
Example 2: Accessing MMIO
Next Steps
Final Notes
Heap Allocation
Introduction
To Avoid Confusion
A Quick Recap: Allocating Memory
Allocating and Freeing
Overview
Part 1: Allocating Memory
Part 2: Adding free()
Part 3: Actually Adding Free()
Part 4: Re-Using Freed Memory
Part 5: Coalescing (Merging)
Part 6: Splitting
Part 7: Heap Initialization
Part 8: Heap Expansion
Memory Management
Scheduling And Tasks
The Scheduler
What Is It?
Thread Selection
Overview
Prerequisites and Initialization
Triggering the Scheduler
Checking For Pre-Emption
Process Selection
Saving and Restoring Context
The States Of A Process
The Idle Process
Wrapping Up
Processes And Threads
Definitions and Terminology
Processes
Identifying A Process
Creating A New Process
Virtual Memory Allocator
Resources
Priorities
From Processes To Threads
Changes Required
Exiting A Thread
Thread Sleep
Advanced Designs
Critical Sections and Locks
Introduction
The Problem
Implementing A Lock
First Implementation
Atomic Operations
Side Effects
Locks and Interrupts
Next Steps
Scheduling
All About Userspace
Some Terminology
A Change in Perspective
Switching Modes
Getting to User Mode
What to Push Onto The Stack
Extra Considerations
Actually Getting to User Mode
Getting Back to Supervisor Mode
Handling Interrupts
The Why
The How
Loading a TSS
Putting It All Together
The TSS and SMP
Software Interrupts
System Calls
The System Call ABI
Using Interrupts
Using Software Interrupts
A Quick Example
Using Dedicated Instructions
Compatibility Issues
Using Syscall & Sysret
Handler Function
Example System Call ABI
Register Interface
Example In Practice
Summary and Next Steps
Resource Management
Resource Abstractation
Per Process Resource Table
Resource Lifecycle
Generic API
Data Copying
The Userspace
Inter-Process Communication
Shared Memory vs Message Passing
Single-Copy vs Double-Copy
IPC via Shared Memory
Overall Design
IPC Manager
Creating Shared Memory
Accessing Shared Memory
Potential Issues
Cleaning Up
Interesting Applications
Access Protection
IPC via Message Passing
How It Works
Initial Setup
Removing An Endpoint
Sending A Message
Multiple Messages
Receiving
Additional Notes
Lock Free Designs
Inter-Process Communication
Virtual File System
The VFS and File Systems
A Quick Recap
The Virtual File System
How The VFS Works
The VFS in Detail
Mounting a File System
Mounting and umounting
Accessing A File
What About Directories?
Conclusions And Suggestions
The Tar File System
Introduction
Implementation
The Header
Searching For A File
Reading From A File
Closing A File
And Now from A VFS Point Of View
Example loading a ramfs using multiboot2
Where To Go From Here
Virtual File System
Executable Linker Format
ELF Overview
Layout Of An ELF
Section Headers
Program Headers
Loading Theory
Loading A Program Header
Program Header Flags
Loading and Running an ELF
Steps Required
Verifying an ELF file
Example: Creating a simple program for our kernel
Caveats
Loading ELFs
Going Beyond
Introduction
Command Line Interface
Prerequisites
Implementing a CLI
Graphical User Interface
Implementing A GUI
Private Framebuffers
In Conclusion…
Libc (A Standard Library)
Porting A Libc
Hosted Cross Compiler
Networking
Prerequisites
Few final words
Going Beyond
Troubleshooting
Unexpected UD/Undefined Opcode exception in x86_64
Why does this happen?
The easy solution
The hard solution
Tips and Tricks
Don’t Forget About Unions
Bitfields? More like minefields.
The solution?
C Language useful information
Pointer arithmetic
Inline assembly
Dealing With Literals and Bitwise Operation
C +(+) assembly together - Calling Conventions
How to actually use this?
Use of
volatile
The Why
The How
A Final Note
Some information about NASM
Macros
Declaring Variables
Calling C from Nasm
About Sizes
If Statement
Switch Statement
Loop
Data Structures
Cross Platform Building
Why Use A Cross Compiler?
Binutils and Compilers
Prerequisites
Binutils
GCC
Clang and LLVM
Emulator (QEMU)
GDB
Debugging
GDB
Remote Debugging
Useful Commands
Navigation
Print and Examine Memory
How Did I Get Here?
Breakpoints
Variables
TUI - Text User Interface
Virtual Box
Useful Commands
Debugging a Virtual Machine
Qemu
Qemu Interrupt Log
Qemu Monitor
Debugcon
Memory Protection
WP bit
SMAP and SMEP
Page Heap
Useful Resources
Build Process
Architecture
Video Output
Memory Management
Scheduling
Userspace
IPC
Virtual File System
Loading Elfs
C Language Infos
Nasm
Debugging
Communities
Books and Manuals
Acknowledgments
Updates to the Printed Edition
First edition
Revision 0
Revision 1
Revision 2
Revision 3
Revision 4
Revision 5
Revision 6
Appendices