"borrow checker invariants" section of the "leveraging the type system" chapter

src/SUMMARY.md

@@ -437,6 +437,11 @@
     - [Semantic Confusion](idiomatic/leveraging-the-type-system/newtype-pattern/semantic-confusion.md)
     - [Parse, Don't Validate](idiomatic/leveraging-the-type-system/newtype-pattern/parse-don-t-validate.md)
     - [Is It Encapsulated?](idiomatic/leveraging-the-type-system/newtype-pattern/is-it-encapsulated.md)
+  - [Borrow checking invariants](idiomatic/leveraging-the-type-system/borrow-checker-invariants.md)
+    - [Generalizing "Ownership"](idiomatic/leveraging-the-type-system/borrow-checker-invariants/generalizing-ownership.md)
+    - [Single-use values](idiomatic/leveraging-the-type-system/borrow-checker-invariants/single-use-values.md)
+    - [Aliasing XOR Mutability](idiomatic/leveraging-the-type-system/borrow-checker-invariants/aliasing-xor-mutability.md)
+    - [Lifetime Relationships & External Resources](idiomatic/leveraging-the-type-system/borrow-checker-invariants/lifetime-connections.md)
 
 ---
 

src/idiomatic/leveraging-the-type-system/borrow-checker-invariants.md

@@ -0,0 +1,59 @@
+---
+minutes: 0
+---
+
+# Using the Borrow checker to enforce Invariants
+
+The logic of the borrow checker, while tied to "memory ownership", can be abstracted away from this central use case to model other problems and prevent API misuse.
+
+```rust,editable
+fn main() {
+// Doors can be open or closed, and you need the right key to lock or unlock one.
+// Modelled with Shared Key and Owned Door. Nothing to do with "memory safety"!
+pub struct DoorKey { pub key_shape: u32 }
+pub struct LockedDoor { lock_shape: u32 }
+pub struct OpenDoor { lock_shape: u32 }
+
+fn open_door(key: &DoorKey, door: LockedDoor) -> Result<OpenDoor, LockedDoor> {
+    if door.lock_shape == key.key_shape {
+        Ok(OpenDoor{lock_shape: door.lock_shape})
+    } else {
+        Err(door)
+    }
+}
+
+fn close_door(key: &DoorKey, door: OpenDoor) -> Result<LockedDoor, OpenDoor> {
+    if door.lock_shape == key.key_shape {
+        Ok(LockedDoor{lock_shape: door.lock_shape})
+    } else {
+        Err(door)
+    }
+}
+
+let key = DoorKey{ key_shape: 7 };
+let closed_door = LockedDoor{ lock_shape: 7};
+let opened_door = open_door(&key, closed_door);
+if let Ok(opened_door) = opened_door {
+    println!("Opened the door with key shape '{}'", key.key_shape);
+} else {
+    eprintln!("Door wasn't opened! Your key only opens locks with shape '{}'", key.key_shape);
+}
+}
+```
+
+<details>
+
+<!-- TODO: link to typestate when that gets merged. -->
+- The borrow checker has been used to prevent use-after-free and multiple mutable references up until this point, and we've used types to shape and restrict use of APIs already using the "typestate" pattern.
+
+- This example uses the ownership & borrowing rules to model the locking and unlocking of a door. We can try to open a door with a key, but if it's the wrong key the door is still closed (here represented as an error) and the key persists regardless.
+
+- The rules of the borrow checker exist to prevent developers from accessing, changing, and holding onto data in memory in unpredictable ways without being so restrictive to the point where it prevents _writing software_. The underlying logical system does not "know" what memory is. All it does is enforce a specific set of rules of how different operations affect what possible later operations are.
+
+- Those rules can apply to many other cases, so we can piggy-back onto the rules of the borrow checker to design APIs to be harder or impossible to misuse. Even when there's little or no actual "memory safety" concerns in the problem domain.
+
+- This section will walk through some of those different domains.
+
+- Interior, private mutability or reference counting in data types may let an API designer shift the meaning of ownership to a different (but analogous) set of semantics as they need to, even to the point where some API designers have managed to model self-referential types this way.
+
+</details>

src/idiomatic/leveraging-the-type-system/borrow-checker-invariants/aliasing-xor-mutability.md

@@ -0,0 +1,55 @@
+---
+minutes: 0
+---
+
+# Mutually Exclusive References, or "Aliasing XOR Mutability"
+
+We can use the mutual exclusion of `&T` and `&mut T` references for a single value to model some constraints.
+
+```rust,editable
+pub struct Transaction(/* some kind of interior state */);
+pub struct QueryResult(String);
+
+pub struct DatabaseConnection {
+    transaction: Transaction,
+    query_results: Vec<QueryResult>,
+}
+
+impl DatabaseConnection {
+    pub fn new() -> Self { Self { transaction: Transaction(/* again, pretend there's some interior state */), query_results: vec![] } }
+    pub fn get_transaction(&mut self) -> &mut Transaction { &mut self.transaction }
+    pub fn results(&self) -> &[QueryResult] { &self.query_results }
+    pub fn commit(&mut self) { println!("Transaction committed!") }
+}
+
+pub fn do_something_with_transaction(transaction: &mut Transaction) {}
+
+fn main() {
+    let mut db = DatabaseConnection::new();
+    let mut transaction = db.get_transaction();
+    do_something_with_transaction(transaction);
+    let assumed_the_transactions_happened_immediately = db.results(); // ❌🔨
+    do_something_with_transaction(transaction);
+    // Works, as the lifetime of "transaction" as a reference ended above.
+    let assumed_the_transactions_happened_immediately_again = db.results();
+    db.commit();
+}
+```
+
+<details>
+
+- Aliasing XOR Mutability is a shorthand for "we can have multiple immutable references, a single mutable reference, but not both."
+
+- This example shows how we can use the mutual exclusion of these kinds of references when it comes to prevent a user from reading query results while using the transaction API, something that might happen if the user is working under the false assumption that the queries being written to the transaction happen "immediately" rather than being queued up and performed together.
+
+- As laid out in [generalizing ownership](generalizing-ownership.md) we can look at the ways Mutable References and Shareable References interact to see if they fit with the invariants we want to uphold for an API.
+
+- By having the query results not public and placed behind a getter function, we can enforce the invariant "users of this API are not looking at the query results at the same time as they are writing to a transaction."
+
+- The example API can still be circumvented, how so?
+<details>
+    - The user could access the transaction solely through `db.get_transaction()`, leaving the lifetime too temporary to prevent access to `db.results()`.
+    - How could we avoid this by working in other concepts from "Leveraging the Type System"?
+</details>
+
+</details>

src/idiomatic/leveraging-the-type-system/borrow-checker-invariants/generalizing-ownership.md

@@ -0,0 +1,52 @@
+---
+minutes: 0
+---
+
+# Generalizing Ownership
+
+The logic of the borrow checker, while modelled off "memory ownership", can be abstracted away from that use case to model other problems where we want to prevent API misuse.
+
+```rust,editable
+// An internal data type to have something to hold onto.
+pub struct Internal;
+// The "outer" data.
+pub struct Data(Internal);
+
+fn shared_use(value: &Data) -> &Internal {
+    &value.0
+}
+
+fn exclusive_use(value: &mut Data) -> &mut Internal {
+    &mut value.0
+}
+
+fn deny_future_use(value: Data) {}
+
+let mut value = Data(Internal);
+let deny_mut = shared_use(&value); 
+let try_to_deny_immutable = exclusive_use(&mut value); // ❌🔨
+let more_mut_denial = &deny_mut;
+deny_future_use(value);
+let even_more_mut_denial = shared_use(&value); // ❌🔨
+```
+
+<details> 
+
+- This example re-frames the borrow checker rules away from references and towards semantic meaning in non-memory-safety settings. Nothing is being mutated, nothing is being sent across threads. 
+
+- To use the borrow checker as a problem solving tool, we will need to "forget" that the original purpose of it is to prevent mutable aliasing in the context of concurrency & dangling pointers, instead imagining and working within situations where the rules are the same but the meaning is slightly different.
+
+- In rust's borrow checker we have access to three different ways of "taking" a value:
+
+<!-- TODO: actually link to the RAII section when it has been merged. -->
+- Owned value. Very permissive case of what you can do with it, but demands that nothing else is using it in any context and "drops" the value when scope ends (unless that scope returns this value) (see: RAII.)
+
+- Mutable Reference. While holding onto a mutable reference we can still "dispatch" to methods and functions that take an immutable, shared reference of the value but only as long as we're not aliasing immutable, shared references to related data "after" that dispatch.
+
+- Shared Reference. Allows aliasing but prevents mutable access while any of these exist. We can't "dispatch" to methods and functions that take mutable reference when all we have is a shared reference.
+
+- Important to remember that every `&T` and `&mut T` has an _implicit lifetime._ We get to avoid annotating a lot of lifetimes because the rust compiler can infer the majority of them. See: [Lifetime Elision](../../../lifetimes/lifetime-elision.md).
+
+- Potentially relevant: show how we can replace a lot of the `&` and `&mut` here with `&'a` and `&'a mut`.
+
+</details>

src/idiomatic/leveraging-the-type-system/borrow-checker-invariants/lifetime-connections.md

@@ -0,0 +1,59 @@
+---
+minutes: 0
+---
+
+# Lifetime "Connections" & External Resources
+
+Using `PhantomData` in conjunction with lifetimes lets us say "this value may own its data, but it can only live as long as the value that generated it" in rust's type system.
+
+```rust,editable
+fn main() {
+use std::marker::PhantomData;
+pub struct Tag;
+pub struct ErasedData<'a>{data: String, _phantom: PhantomData<&'a ()>}
+impl <'a> ErasedData<'a>  {
+    pub fn get(&self) -> &str {
+        &self.data
+    }
+}
+pub struct TaggedData<T>{data: String, _phantom: PhantomData<T>}
+impl <T> TaggedData<T> {
+    pub fn new(data: String) -> Self {Self {data, _phantom: PhantomData} }
+    pub fn consume(self) {}
+    pub fn get_erased(&self) -> ErasedData<'_> {
+        // has an owned String, but _phantom holds onto the lifetime of the TaggedData
+        // that created it.
+        ErasedData { data: self.data.clone(), _phantom: PhantomData }
+    }
+}
+
+let tagged_data: TaggedData<Tag> = TaggedData::new("Real Data".to_owned());
+// Get the erased-but-still-linked data.
+let erased_owned_and_linked = tagged_data.get_erased();
+tagged_data.consume();
+// The data is owned by `erased_owned_and_linked` but still connected to `tagged_data`.
+println!("{}", erased_owned_and_linked.get()); // ❌🔨
+}
+```
+
+<details>
+
+- PhantomData lets developers "tag" types with type and lifetime parameters that are not "really" present in the struct or enum.
+
+- PhantomData can be used with the Typestate pattern to have data with the same structure i.e. `TaggedData<Start>` can have methods or trait implementations that `TaggedData<End>` doesn't.
+
+- But it can also be used to encode a connection between the lifetime of one value and another, while both values still maintain separate owned data within them.
+
+- This is really useful for modelling a bunch of relationships between data, where we want to establish that while a type has owned values within it is still connected to another piece of data and can only live as long as it.
+
+- Consider a case where you want to return owned data from a method, but you don't want that data to live longer than the value that created it.
+
+- [`BorrowedFd`](https://rust-lang.github.io/rfcs/3128-io-safety.html#ownedfd-and-borrowedfdfd) uses these captured lifetimes to enforce the invariant that "if this file descriptor exists, the OS file descriptor is still open" because a `BorrowedFd`'s lifetime parameter demands that there exists another value in your program that has the same lifetime as it, and this has been encoded by the API designer to mean _that value is what keeps the access to the file open_. Its counterpart `OwnedFd` is instead a file descriptor that closes that file on drop.
+
+- Lifetimes need to come from somewhere! We can't build functions of the form `fn lifetime_shenanigans<'a>(owned: OwnedData) -> &'b Data` (without tying `'b` to `'a` in some way). Lifetime elision hides where a lot of lifetimes come from, but that doesn't mean the explicitly named lifetimes "come from nowhere."
+
+- This way of encoding information in types is _exceptionally powerful_ when combined with unsafe, as the ways one can manipulate lifetimes becomes almost arbitrary. This is also dangerous, but when combined with tools like external, mechanically-verified proofs _we can safely encode cyclic/self-referential types while encoding lifetime & safety expectations in the relevant data types._
+
+- The [GhostCell (2021)](https://plv.mpi-sws.org/rustbelt/ghostcell/) paper and its [relevant implementation](https://gitlab.mpi-sws.org/FP/ghostcell) show this kind of work off. While the borrow checker is restrictive, there are still ways to use escape hatches and then _show that the ways you used those escape hatches are consistent and safe._
+
+</details>

src/idiomatic/leveraging-the-type-system/borrow-checker-invariants/single-use-values.md

@@ -0,0 +1,42 @@
+---
+minutes: 0
+---
+
+# Single-use values
+
+In some circumstances we want values that can be used _exactly once_. One critical example of this is in cryptography: "Nonces."
+
+```rust,editable
+fn main() {
+pub struct Key;
+
+// Pretend this is a cryptographically unique, use-once number.
+pub struct Nonce(u32);
+
+let nonce = Nonce(1337);
+let data_1: [u8; 4] = [1, 2, 3, 4];
+let data_2: [u8; 4] = [4, 3, 2, 1];
+let key = Key;
+
+// The key and data can be re-used, copied, etc. but the nonce cannot.
+fn encrypt(nonce: Nonce, key: &Key, data: &[u8]) {}
+
+encrypt(nonce, &key, &data_1);
+encrypt(nonce, &key, &data_2); // 🛠️❌
+}
+```
+<details>
+
+- Owned "consumption" lets us model single-use values.
+
+- Not implementing clone/copy here and making the interior type opaque (as per the newtype pattern) is _intentional_, as it prevents multiple uses of the same, API-controlled value.
+
+- I.e. A Nonce is a additional piece of random, unique data during an encryption process that helps prevent "replay attacks".
+
+- In practice people have ended up re-using nonces in circumstances where security is important, making it possible for private key information to be derived by attackers.
+
+- By tying nonce creation and consumption up in rust's ownership model, and by not implementing clone/copy on sensitive single-use data, we can prevent this kind of dangerous misuse.
+
+- Cryptography Nuance: There is still the case where a nonce may be used twice if it's created through purely a pseudo-random process with no additional metadata, and that circumstance can't be avoided through this particular method. This kind of API prevents one kind of misuse, but not all kinds.
+
+</details>