CR3
?x86_64
version 0.4.0 or later. Update dependencies in our Cargo.toml
: [dependencies] x86_64 = "0.4.0" # or later
4 KiB
, we get access to the virtual address 4 KiB
, and not to the physical address where the table of the 4th level pages is stored. If we want to access the physical address of 4 KiB
, then we need to use a virtual address that is translated into it.28 KiB
region, because it will rest on the already occupied page at 1004 KiB
. Therefore, we will have to look further until we find a suitable large fragment, for example, with 1008 KiB
. The same problem of fragmentation arises as in segmented memory.1008 KiB
. Now we can no longer use any frame with a physical address between 1000 KiB
and 2008 KiB
, because it cannot be identically displayed.32 KiB
into the physical frame at 32 KiB
, thereby identically displaying the level 1 table itself. The figure shows this with a horizontal arrow.24 KiB
. This created a temporary mapping of the virtual page at 0 KiB
address 0 KiB
with the physical frame of the level 2 page table, indicated by the dotted arrow. Now the kernel can access the level 2 table by writing to a page that starts at 0 KiB
.511
in the table of level 4, which is compared with the physical frame 4 KiB
, which is in the table itself.Virtual address for | Address structure ( octal ) |
---|---|
Page | 0o_SSSSSS_AAA_BBB_CCC_DDD_EEEE |
Entry in table level 1 | 0o_SSSSSS_RRR_AAA_BBB_CCC_DDDD |
Entry in table level 2 | 0o_SSSSSS_RRR_RRR_AAA_BBB_CCCC |
Entry in table level 3 | 0o_SSSSSS_RRR_RRR_RRR_AAA_BBBB |
Entry in table level 4 | 0o_SSSSSS_RRR_RRR_RRR_RRR_AAAA |
ААА
is a level 4 index, ВВВ
is level 3, ССС
is level 2, and DDD
is level 1 index for the displayed frame, EEEE
is its offset. RRR
is the index of the recursive entry. The index (three digits) is converted to an offset (four digits) by multiplying by 8 (the size of the page table entry). With this offset, the resulting address directly points to the corresponding page table entry.SSSS
are the bits of the sign bit extension, that is, they are all copies of bit 47. This is a special requirement for valid addresses in the x86_64 architecture, which we discussed in the previous article .0xffff_ffff_ffff_f000
address 0xffff_ffff_ffff_f000
. If we convert this address to octal and compare it with the table above, we see that it corresponds exactly to the structure of the entry of table level 4 with RRR
= 0o777
, AAAA
= 0
and the bits of the character expansion 1
:structure: 0o_SSSSSS_RRR_RRR_RRR_RRR_AAAA Address: 0o_177777_777_777_777_777_0000
// in src/lib.rs pub mod memory;
// in src/memory.rs use x86_64::PhysAddr; use x86_64::structures::paging::PageTable; /// Returns the physical address for the given virtual address, or `None` if the /// virtual address is not mapped. pub fn translate_addr(addr: usize) -> Option<PhysAddr> { // introduce variables for the recursive index and the sign extension bits // TODO: Don't hardcode these values let r = 0o777; // recursive index let sign = 0o177777 << 48; // sign extension // retrieve the page table indices of the address that we want to translate let l4_idx = (addr >> 39) & 0o777; // level 4 index let l3_idx = (addr >> 30) & 0o777; // level 3 index let l2_idx = (addr >> 21) & 0o777; // level 2 index let l1_idx = (addr >> 12) & 0o777; // level 1 index let page_offset = addr & 0o7777; // calculate the table addresses let level_4_table_addr = sign | (r << 39) | (r << 30) | (r << 21) | (r << 12); let level_3_table_addr = sign | (r << 39) | (r << 30) | (r << 21) | (l4_idx << 12); let level_2_table_addr = sign | (r << 39) | (r << 30) | (l4_idx << 21) | (l3_idx << 12); let level_1_table_addr = sign | (r << 39) | (l4_idx << 30) | (l3_idx << 21) | (l2_idx << 12); // check that level 4 entry is mapped let level_4_table = unsafe { &*(level_4_table_addr as *const PageTable) }; if level_4_table[l4_idx].addr().is_null() { return None; } // check that level 3 entry is mapped let level_3_table = unsafe { &*(level_3_table_addr as *const PageTable) }; if level_3_table[l3_idx].addr().is_null() { return None; } // check that level 2 entry is mapped let level_2_table = unsafe { &*(level_2_table_addr as *const PageTable) }; if level_2_table[l2_idx].addr().is_null() { return None; } // check that level 1 entry is mapped and retrieve physical address from it let level_1_table = unsafe { &*(level_1_table_addr as *const PageTable) }; let phys_addr = level_1_table[l1_idx].addr(); if phys_addr.is_null() { return None; } Some(phys_addr + page_offset) }
0o777
) and the sign extension bits (each is 1). Then we calculate the indexes of the page tables and the offset through bitwise operations, as indicated in the illustration:PageTable
links PageTable
. These are unsafe operations, since the compiler cannot know that these addresses are valid.addr
not associated with any physical memory. So we return None
. Otherwise, we know that a level 3 table exists. Then we repeat the procedure as in the previous level._start
function: // in src/main.rs #[cfg(not(test))] #[no_mangle] pub extern "C" fn _start() -> ! { […] // initialize GDT, IDT, PICS use blog_os::memory::translate_addr; // the identity-mapped vga buffer page println!("0xb8000 -> {:?}", translate_addr(0xb8000)); // some code page println!("0x20010a -> {:?}", translate_addr(0x20010a)); // some stack page println!("0x57ac001ffe48 -> {:?}", translate_addr(0x57ac001ffe48)); println!("It did not crash!"); blog_os::hlt_loop(); }
Тип RecursivePageTable
RecursivePageTable
type, which implements safe abstractions for various page table operations. With this type, you can much more concisely implement the translate_addr
function: // in src/memory.rs use x86_64::structures::paging::{Mapper, Page, PageTable, RecursivePageTable}; use x86_64::{VirtAddr, PhysAddr}; /// Creates a RecursivePageTable instance from the level 4 address. /// /// This function is unsafe because it can break memory safety if an invalid /// address is passed. pub unsafe fn init(level_4_table_addr: usize) -> RecursivePageTable<'static> { let level_4_table_ptr = level_4_table_addr as *mut PageTable; let level_4_table = &mut *level_4_table_ptr; RecursivePageTable::new(level_4_table).unwrap() } /// Returns the physical address for the given virtual address, or `None` if /// the virtual address is not mapped. pub fn translate_addr(addr: u64, recursive_page_table: &RecursivePageTable) -> Option<PhysAddr> { let addr = VirtAddr::new(addr); let page: Page = Page::containing_address(addr); // perform the translation let frame = recursive_page_table.translate_page(page); frame.map(|frame| frame.start_address() + u64::from(addr.page_offset())) }
RecursivePageTable
type completely encapsulates an insecure crawl of page tables, so the unsafe
code is no longer needed in the translate_addr
function. The init
function remains insecure due to the need to ensure the correctness of the passed level_4_table_addr
._start
function should be updated for the new function signature as follows: // in src/main.rs #[cfg(not(test))] #[no_mangle] pub extern "C" fn _start() -> ! { […] // initialize GDT, IDT, PICS use blog_os::memory::{self, translate_addr}; const LEVEL_4_TABLE_ADDR: usize = 0o_177777_777_777_777_777_0000; let recursive_page_table = unsafe { memory::init(LEVEL_4_TABLE_ADDR) }; // the identity-mapped vga buffer page println!("0xb8000 -> {:?}", translate_addr(0xb8000, &recursive_page_table)); // some code page println!("0x20010a -> {:?}", translate_addr(0x20010a, &recursive_page_table)); // some stack page println!("0x57ac001ffe48 -> {:?}", translate_addr(0x57ac001ffe48, &recursive_page_table)); println!("It did not crash!"); blog_os::hlt_loop(); }
LEVEL_4_TABLE_ADDR
to translate_addr
and accessing page tables via unsafe raw pointers, we pass references to the RecursivePageTable
type. Thus, we now have a safe abstraction and clear semantics of ownership. This ensures that we cannot accidentally change the page table in shared access, because to change it we need to take possession of the RecursivePageTable
exclusively.memory::init
is an insecure function: a block is required to call it unsafe
, because the caller must ensure that certain requirements are met. In our case, the requirement is that the transmitted address is exactly matched to the physical frame of the table of the level 4 pages.unsafe
contains the whole body of the unsafe function so that all types of operations are performed without creating additional blocks unsafe
. Therefore, we do not need an insecure block for dereferencing level_4_table_ptr
: pub unsafe fn init(level_4_table_addr: usize) -> RecursivePageTable<'static> { let level_4_table_ptr = level_4_table_addr as *mut PageTable; let level_4_table = &mut *level_4_table_ptr; // <- this operation is unsafe RecursivePageTable::new(level_4_table).unwrap() }
RecursivePageTable::new
we cannot say whether it is safe or not. It is so easy to accidentally skip some unsafe code. // in src/memory.rs pub unsafe fn init(level_4_table_addr: usize) -> RecursivePageTable<'static> { /// Rust currently treats the whole body of unsafe functions as an unsafe /// block, which makes it difficult to see which operations are unsafe. To /// limit the scope of unsafe we use a safe inner function. fn init_inner(level_4_table_addr: usize) -> RecursivePageTable<'static> { let level_4_table_ptr = level_4_table_addr as *mut PageTable; let level_4_table = unsafe { &mut *level_4_table_ptr }; RecursivePageTable::new(level_4_table).unwrap() } init_inner(level_4_table_addr) }
unsafe
again required for dereference level_4_table_ptr
, and we immediately see that these are the only unsafe operations. Currently, Rust has an RFC open for modifying this unfortunate feature of unsafe functions.0x1000
. As the desired frame, use the 0xb8000
VGA text buffer frame. So easy to check how our address translation works.create_maping
in the module memory
: // in src/memory.rs use x86_64::structures::paging::{FrameAllocator, PhysFrame, Size4KiB}; pub fn create_example_mapping( recursive_page_table: &mut RecursivePageTable, frame_allocator: &mut impl FrameAllocator<Size4KiB>, ) { use x86_64::structures::paging::PageTableFlags as Flags; let page: Page = Page::containing_address(VirtAddr::new(0x1000)); let frame = PhysFrame::containing_address(PhysAddr::new(0xb8000)); let flags = Flags::PRESENT | Flags::WRITABLE; let map_to_result = unsafe { recursive_page_table.map_to(page, frame, flags, frame_allocator) }; map_to_result.expect("map_to failed").flush(); }
RecursivePageTable
(it will change it) and FrameAllocator
, which is explained below. Then it applies the function map_to
in Tray Mapper
to map the page to the address 0x1000
with the physical frame to the address 0xb8000
. The function is insecure, as it is possible to violate the memory security with invalid arguments.page
and frame
, the function map_to
takes two more arguments. The third argument is a set of flags for the page table. We set the flag PRESENT
required for all valid entries, and the flag WRITABLE
for recording.FrameAllocator
. This argument is needed by the method.map_to
, because creating new page tables may require unused frames. The implementation requires the argument trait Size4KiB
, as types Page
and PhysFrame
are universal for the trait PageSize
, working with standard 4 KiB pages and with enormous pages 2 MiB / 1 GiB.map_to
may fail, therefore returns Result
. Since this is just an example of code that should not be reliable, we simply use it expect
with a panic when an error occurs. If successful, the function returns a type MapperFlush
that provides an easy way to clear the newly matched page from the associative translation buffer (TLB) method flush
. LikeResult
, the type uses the attribute #[must_use]
and issues a warning if we accidentally forget to use it.0x1000
does not require new page tables, it FrameAllocator
can always return None
. To test the function, create the following EmptyFrameAllocator
: // in src/memory.rs /// A FrameAllocator that always returns `None`. pub struct EmptyFrameAllocator; impl FrameAllocator<Size4KiB> for EmptyFrameAllocator { fn allocate_frame(&mut self) -> Option<PhysFrame> { None } }
allocate_frame
is not a member of trait FrameAllocator
' appears , you need to upgrade x86_64
to version 0.4.0.) // in src/main.rs #[cfg(not(test))] #[no_mangle] pub extern "C" fn _start() -> ! { […] // initialize GDT, IDT, PICS use blog_os::memory::{create_example_mapping, EmptyFrameAllocator}; const LEVEL_4_TABLE_ADDR: usize = 0o_177777_777_777_777_777_0000; let mut recursive_page_table = unsafe { memory::init(LEVEL_4_TABLE_ADDR) }; create_example_mapping(&mut recursive_page_table, &mut EmptyFrameAllocator); unsafe { (0x1900 as *mut u64).write_volatile(0xf021f077f065f04e)}; println!("It did not crash!"); blog_os::hlt_loop(); }
0x1000
, calling the function create_example_mapping
with a variable reference to the instance RecursivePageTable
. This translates the page 0x1000
into a VGA text buffer, so we will see some result on the screen.0xf021f077f065f04e
, which corresponds to the string “New!” On a white background. Just do not need to write this value immediately to the top of the page 0x1000
, because the top line will move next from the screen println
, and we will write it at the offset 0x900
, which is approximately in the middle of the screen. As we know from the VGA Text Mode article , writing to the VGA buffer should be volatile, so we use the method write_volatile
.0x1000
. If we try to translate a page for which such a table does not yet exist, the function map_to
will return an error, because it will try to allocate frames from to create new page tables EmptyFrameAllocator
. We will see this if we try to translate the page 0xdeadbeaf000
instead 0x1000
: // in src/memory.rs pub fn create_example_mapping(…) { […] let page: Page = Page::containing_address(VirtAddr::new(0xdeadbeaf000)); […] } // in src/main.rs #[no_mangle] pub extern "C" fn _start() -> ! { […] unsafe { (0xdeadbeaf900 as *mut u64).write_volatile(0xf021f077f065f04e)}; […] }
panicked at 'map_to failed: FrameAllocationFailed', .../result.rs:999zh
FrameAllocator
. But how do you know which frames are free and how much physical memory is available?_start
gives a link to the information structure of the boot. Add this argument to our function: // in src/main.rs use bootloader::bootinfo::BootInfo; #[cfg(not(test))] #[no_mangle] pub extern "C" fn _start(boot_info: &'static BootInfo) -> ! { // new argument […] }
BootInfo
still being finalized, so do not be surprised at the failures when upgrading to future versions of the bootloader that are not compatible with semver . He currently has three fields p4_table_addr
, memory_map
and package
:p4_table_addr
contains the recursive virtual address of the level 4 page table. Due to this, it is not necessary to prescribe the address strictly 0o_177777_777_777_777_777_0000
.memory_map
is of the most interest because it contains a list of all memory areas and their type (unused, reserved or others).package
is the current function for associating additional data with the loader. The implementation is not completed, so we can ignore it for now.memory_map
to create the correct one FrameAllocator
, we want to ensure the correct type of argument boot_info
.entry_point
_start
is called externally, the function’s signature is not verified. This means that arbitrary arguments will not lead to compilation errors, but may cause failure or undefined behavior at run time.bootloader
uses the entry_point
type-tested macro to define the Rust function as an entry point . Let's rewrite our function under this macro: // in src/main.rs use bootloader::{bootinfo::BootInfo, entry_point}; entry_point!(kernel_main); #[cfg(not(test))] fn kernel_main(boot_info: &'static BootInfo) -> ! { […] // initialize GDT, IDT, PICS let mut recursive_page_table = unsafe { memory::init(boot_info.p4_table_addr as usize) }; […] // create and test example mapping println!("It did not crash!"); blog_os::hlt_loop(); }
extern "C"
or no_mangle
, since the macro sets a real low-level entry point _start
. The function has kernel_main
now become a completely normal Rust function, so we can choose an arbitrary name for it. It is important that it is already typed, so a compilation error will occur if you change the signature of the function, for example, by adding an argument or changing its type.memory::init
non- hardcoded address as well boot_info.p4_table_addr
. Thus, the code will work even if the future version of the loader chooses another level 4 page table entry for the recursive display. // in src/memory.rs pub struct BootInfoFrameAllocator<I> where I: Iterator<Item = PhysFrame> { frames: I, } impl<I> FrameAllocator<Size4KiB> for BootInfoFrameAllocator<I> where I: Iterator<Item = PhysFrame> { fn allocate_frame(&mut self) -> Option<PhysFrame> { self.frames.next() } }
frames
initialized with an arbitrary frame iterator . This allows you to simply delegate calls alloc
to the Iterator :: next method .BootInfoFrameAllocator
occurs in a new function init_frame_allocator
: // in src/memory.rs use bootloader::bootinfo::{MemoryMap, MemoryRegionType}; /// Create a FrameAllocator from the passed memory map pub fn init_frame_allocator( memory_map: &'static MemoryMap, ) -> BootInfoFrameAllocator<impl Iterator<Item = PhysFrame>> { // get usable regions from memory map let regions = memory_map .iter() .filter(|r| r.region_type == MemoryRegionType::Usable); // map each region to its address range let addr_ranges = regions.map(|r| r.range.start_addr()..r.range.end_addr()); // transform to an iterator of frame start addresses let frame_addresses = addr_ranges.flat_map(|r| r.into_iter().step_by(4096)); // create `PhysFrame` types from the start addresses let frames = frame_addresses.map(|addr| { PhysFrame::containing_address(PhysAddr::new(addr)) }); BootInfoFrameAllocator { frames } }
iter
для преобразования карты распределения памяти в итератор MemoryRegion
. Затем используем метод filter
, чтобы пропустить зарезервированные или недоступные регионы. Загрузчик обновляет карту распределения памяти для всех созданных им сопоставлений, поэтому фреймы, используемые нашим ядром (код, данные или стек) или для хранения загрузочной информации, уже помечены как InUse
или аналогично. Таким образом, мы можем быть уверены, что используемые фреймы не используются где-то ещё.map
и синтаксис range Rust для преобразования нашего итератора областей памяти в итератор диапазонов адресов.into_iter
, and then select each 4096th address with step_by
. Since the page size is 4096 bytes (4 KiB), we get the start address of each frame. The bootloader page aligns all used memory areas, so we do not need an alignment or rounding code. Replacing map
on flat_map
, we get Iterator<Item = u64>
instead Iterator<Item = Iterator<Item = u64>>
.PhysFrame
in order to construct the required one Iterator<Item = PhysFrame>
. Then use this iterator to create and return a new one BootInfoFrameAllocator
.kernel_main
to pass an instance BootInfoFrameAllocator
instead of EmptyFrameAllocator
: // in src/main.rs #[cfg(not(test))] fn kernel_main(boot_info: &'static BootInfo) -> ! { […] // initialize GDT, IDT, PICS use x86_64::structures::paging::{PageTable, RecursivePageTable}; let mut recursive_page_table = unsafe { memory::init(boot_info.p4_table_addr as usize) }; // new let mut frame_allocator = memory::init_frame_allocator(&boot_info.memory_map); blog_os::memory::create_mapping(&mut recursive_page_table, &mut frame_allocator); unsafe { (0xdeadbeaf900 as *mut u64).write_volatile(0xf021f077f065f04e)}; println!("It did not crash!"); blog_os::hlt_loop(); }
map_to
creates the missing page tables as follows:frame_allocator
.create_maping
is just an example, we can now create new mappings for arbitrary pages. This is very useful when allocating memory and implementing multithreading in future articles.Source: https://habr.com/ru/post/439066/