Update from obsidian - thinkpad
Affected files: .obsidian/plugins/obsidian-omnivore/data.json Read Later/2024-02-18 - Git Tips 1- Oldies but Goodies.md Read Later/2024-02-18 - Git Tips 2- New Stuff in Git.md Read Later/2024-02-18 - Git Tips 3- Really Large Repositories.md Read Later/2024-03-26 - How Do Open Source Software Lifecycles Work-.md Read Later/2024-04-18 - Unix sockets, the basics in Rust - Emmanuel Bosquet.md Read Later/2024-05-10 - Bullet Journal in 5 Minutes a Day (for busy people).md
This commit is contained in:
parent
a6c324506a
commit
993f1f4795
7 changed files with 1166 additions and 11 deletions
|
|
@ -0,0 +1,239 @@
|
|||
---
|
||||
id: 3f72e613-eb8e-446b-93b5-fdb275a781c0
|
||||
title: |
|
||||
Unix sockets, the basics in Rust - Emmanuel Bosquet
|
||||
status: ARCHIVED
|
||||
tags:
|
||||
- read-later
|
||||
date_added: 2024-04-18 20:15:21
|
||||
url_omnivore: |
|
||||
https://omnivore.app/me/unix-sockets-the-basics-in-rust-emmanuel-bosquet-18ef3b52a37
|
||||
url_original: |
|
||||
https://emmanuelbosquet.com/2022/whatsaunixsocket/
|
||||
---
|
||||
|
||||
# Unix sockets, the basics in Rust - Emmanuel Bosquet
|
||||
|
||||
## Notes
|
||||
|
||||
This will be useful someday...
|
||||
when I want to try to build my own TCP server or something like that
|
||||
## Original
|
||||
|
||||
## Contents
|
||||
|
||||
* [What is a unix socket?](#what-is-a-unix-socket)
|
||||
* [Create a socket, server side](#create-a-socket-server-side)
|
||||
* [Waiting for connections, server side](#waiting-for-connections-server-side)
|
||||
* [Connecting to the socket, client side](#connecting-to-the-socket-client-side)
|
||||
* [Writing on the socket, client side](#writing-on-the-socket-client-side)
|
||||
* [Reading from the socket, server side](#reading-from-the-socket-server-side)
|
||||
* [Launch the whole thing!](#launch-the-whole-thing)
|
||||
* [Respond to a message, server side](#respond-to-a-message-server-side)
|
||||
* [Listen to responses, client side](#listen-to-responses-client-side)
|
||||
* [Launch the whole thing, again!](#launch-the-whole-thing-again)
|
||||
* [Browse the code](#browse-the-code)
|
||||
|
||||
I found myself wondering about unix sockets while working on [Sōzu](https://github.com/sozu-proxy/sozu), a reverse proxy written in Rust. A bunch of Sōzu issues led me to[dig into Sōzu channels](https://github.com/Keksoj/stream%5Fstuff%5Fon%5Fa%5Fsozu%5Fchannel), which themselves make use of[Metal I/O ’s implementation of unix sockets](https://tokio-rs.github.io/mio/doc/mio/net/struct.UnixListener.html).
|
||||
|
||||
Here are the questions, summed up:
|
||||
|
||||
* what are unix sockets?
|
||||
* how can we create them in Rust?
|
||||
* how do we use them to stream data?
|
||||
|
||||
So here we go.
|
||||
|
||||
It is _not_ a web socket like `127.0.0.1:8080`.
|
||||
|
||||
You may have heard that in unix,[everything is a file](https://www.youtube.com/watch?v=dDwXnB6XeiA). Unix sockets seem to be a good example of this principle. They are empty files of sorts, only there to be written to, and read from.
|
||||
|
||||
Sockets are a core feature of unix. In fact, if you type
|
||||
|
||||
```ebnf
|
||||
man unix
|
||||
|
||||
```
|
||||
|
||||
in your terminal, you should land on an ancient man page:
|
||||
|
||||
| 1 2 3 4 5 | UNIX(7) Linux Programmer's Manual UNIX(7) NAME unix - sockets for local interprocess communication |
|
||||
| --------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
|
||||
that explains how sockets are declared in C in the kernel, how they are created with the `AF_UNIX` system call, and many more thing that go far beyond my limited understanding.
|
||||
|
||||
Creating a socket is not as easy as creating just any file, using, say, `touch`. They are tools available in the command line, but most of the time, sockets are created and used by processes, not by users. Looking up how to create one will land you on a tutorial in C, or in python. So let’s see how to do it in Rust.
|
||||
|
||||
The Rust standard library has a [std::os::unix module](https://doc.rust-lang.org/std/os/unix/index.html)to interact with unix processes, unix files, and so on. Within it, we want to look at the `net` module, named that way because unix sockets are used to do networking between processes.
|
||||
|
||||
The `std::os::unix::net` module contains, among other things:
|
||||
|
||||
* [UnixListener](https://doc.rust-lang.org/std/os/unix/net/struct.UnixListener.html)
|
||||
* [UnixStream](https://doc.rust-lang.org/std/os/unix/net/struct.UnixStream.html)
|
||||
|
||||
Both those entities are unsafe wrappers of the `libc` library to perform the very same unix system calls you would write in C. They both wrap a unix file descriptor, but they are distinct in order to separate higher-level concerns.
|
||||
|
||||
* `UnixListener` is used to create sockets, (`libc::bind()` and `libc::listen()`)
|
||||
* `UnixStream` is there to connect to a socket (`libc::connect()`), to read from it and write on it.
|
||||
|
||||
Let’s use those.[Install Rust and Cargo](https://www.rust-lang.org/tools/install),[Learn the basics of Rust](https://doc.rust-lang.org/book/), and then do:
|
||||
|
||||
```haxe
|
||||
cargo new unix_sockets
|
||||
|
||||
```
|
||||
|
||||
Add this to `Cargo.toml` (makes error propagation easier):
|
||||
|
||||
| 1 2 | \# Cargo.toml anyhow \= "^1.0.42" |
|
||||
| --- | --------------------------------- |
|
||||
|
||||
In the `src` directory, create a `bin` directory, in which you will create a `server.rs` file.
|
||||
|
||||
| 1 2 3 4 5 6 7 8 9 10 11 12 13 | // src/bin/server.rs use std::os::unix::net::{UnixListener, UnixStream}; use anyhow::Context; fn main() -> anyhow::Result<()> { let socket\_path = "mysocket"; let unix\_listener = UnixListener::bind(socket\_path).context("Could not create the unix socket")?; Ok(()) } |
|
||||
| ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
|
||||
Then do
|
||||
|
||||
```routeros
|
||||
cargo run --bin server
|
||||
|
||||
```
|
||||
|
||||
Which should run smoothly, and then do `ls -l` in your directory, you should have a line like this:
|
||||
|
||||
```routeros
|
||||
srwxr-xr-x 1 emmanuel users 0 Jan 7 13:08 mysocket
|
||||
|
||||
```
|
||||
|
||||
The `s` stands for _socket_. Congratulations!
|
||||
|
||||
Do one more `cargo run --bin server` and you have a neat, self-explanatory OS error:
|
||||
|
||||
| 1 2 3 4 | Error: Could not create the unix socket Caused by: Address already in use (os error 98) |
|
||||
| ------- | ------------------------------------------------------------------------------------------- |
|
||||
|
||||
I guess we’ll have to destroy it and recreate it each time.
|
||||
|
||||
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | // src/bin/server.rs use std::os::unix::net::{UnixListener, UnixStream}; use anyhow::Context; fn main() -> anyhow::Result<()> { let socket\_path = "mysocket"; // copy-paste this and don't think about it anymore // it will be hidden from there on if std::fs::metadata(socket\_path).is\_ok() { println!("A socket is already present. Deleting..."); std::fs::remove\_file(socket\_path).with\_context(\|| { format!("could not delete previous socket at {:?}", socket\_path) })?; } let unix\_listener = UnixListener::bind(socket\_path).context("Could not create the unix socket")?; Ok(()) } |
|
||||
| ---------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
|
||||
The `UnixListener` struct has an `accept()` method that waits for other processes to connect to the socket. Once a connections comes, `accept()` returns a tuple containing a `UnixStream` and a `SocketAddr`.
|
||||
|
||||
As mentioned above, `UnixStream` implements `Read` and `Write`. We will handle this stream to:
|
||||
|
||||
* read what another process will send through the socket
|
||||
* write responses on the socket
|
||||
|
||||
Add the loop and the `handle_stream` function to the server code:
|
||||
|
||||
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | // src/bin/server.rs use std::os::unix::net::{UnixListener, UnixStream}; fn main() -> anyhow::Result<()> { let socket\_path = "mysocket"; let unix\_listener = UnixListener::bind(socket\_path).context("Could not create the unix socket")?; // put the server logic in a loop to accept several connections loop { let (mut unix\_stream, socket\_address) = unix\_listener .accept() .context("Failed at accepting a connection on the unix listener")?; handle\_stream(unix\_stream)?; } Ok(()) } fn handle\_stream(mut stream: UnixStream) -> anyhow::Result<()> { // to be filled Ok(()) } |
|
||||
| ------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
|
||||
Remove the existing socket and run the code:
|
||||
|
||||
```routeros
|
||||
cargo run --bin server
|
||||
|
||||
```
|
||||
|
||||
it should hang. Perfect! The server is waiting for connections!
|
||||
|
||||
The client process wants to connect to an existing socket, read and write from it.
|
||||
|
||||
Next to `server.rs`, create the `client.rs` file. The client will merely consist of a `UnixStream`:
|
||||
|
||||
| 1 2 3 4 5 6 7 8 9 10 11 12 | // src/bin/client.rs use std::os::unix::net::{UnixListener, UnixStream}; use anyhow::Context; fn main() -> anyhow::Result<()> { let socket\_path = "mysocket"; let mut unix\_stream = UnixStream::connect(socket\_path).context("Could not create stream")?; Ok(()) |
|
||||
| ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
|
||||
We need to import the `Read` and `Write` traits.
|
||||
|
||||
| 1 2 | // src/bin/client.rs use std::io::{Read, Write}; |
|
||||
| --- | ------------------------------------------------ |
|
||||
|
||||
And now we can write onto the stream. Below the `unix_stream` declaration, add the write logic:
|
||||
|
||||
| 1 2 3 | unix\_stream .write(b"Hello?") // we write bytes, &\[u8\] .context("Failed at writing onto the unix stream")?; |
|
||||
| ----- | ---------------------------------------------------------------------------------------------------------------------------- |
|
||||
|
||||
Be sure to import `Read` and `Write` in `server.rs`:
|
||||
|
||||
| 1 2 | // src/bin/server.rs use std::io::{Read, Write}; |
|
||||
| --- | ------------------------------------------------ |
|
||||
|
||||
Now let’s fill the `handle_stream` function with ordinary read logic:
|
||||
|
||||
| 1 2 3 4 5 6 7 8 9 10 | // src/bin/server.rs fn handle\_stream(mut unix\_stream: UnixStream) -> anyhow::Result<()> { let mut message = String::new(); unix\_stream .read\_to\_string(&mut message) .context("Failed at reading the unix stream")?; println!("{}", message); Ok(()) } |
|
||||
| ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
|
||||
Make sure you have the server running in a terminal:
|
||||
|
||||
```routeros
|
||||
cargo run --bin server
|
||||
|
||||
```
|
||||
|
||||
And in a separate terminal, run the client:
|
||||
|
||||
```routeros
|
||||
cargo run --bin client
|
||||
|
||||
```
|
||||
|
||||
If all is well, the hello message should display on the server side.
|
||||
|
||||
Let’s answer something every time the server receives anything.
|
||||
|
||||
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | // src/bin/server.rs fn handle\_stream(mut unix\_stream: UnixStream) -> anyhow::Result<()> { let mut message = String::new(); unix\_stream .read\_to\_string(&mut message) .context("Failed at reading the unix stream")?; println!("We received this message: {}\\nReplying...", message); unix\_stream .write(b"I hear you!") .context("Failed at writing onto the unix stream")?; Ok(()) } |
|
||||
| ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
|
||||
Introducing the same reading logic we used on the server **will not work**. Why? After writing on a stream, we need to shut down the writing, if we want to read from it.
|
||||
|
||||
Let’s segregate the write and read logic into distinct functions. Oh, and we pass mutable references (`&mut`) of the unix stream to the function, because… Rust. Don’t worry about it.
|
||||
|
||||
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | // src/bin/client.rs use std::io::{Read, Write}; use std::os::unix::net::{UnixListener, UnixStream}; use anyhow::Context; fn main() -> anyhow::Result<()> { let socket\_path = "mysocket"; let mut unix\_stream = UnixStream::connect(socket\_path).context("Could not create stream")?; write\_request\_and\_shutdown(&mut unix\_stream)?; read\_from\_stream(&mut unix\_stream)?; Ok(()) } |
|
||||
| ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
|
||||
The `shutdown()` method takes a `Shutdown` enum we would otherwise use on TCP streams. Write below the main function:
|
||||
|
||||
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 | fn write\_request\_and\_shutdown(unix\_stream: &mut UnixStream) -> anyhow::Result<()> { unix\_stream .write(b"Hello?") .context("Failed at writing onto the unix stream")?; println!("We sent a request"); println!("Shutting down writing on the stream, waiting for response..."); unix\_stream .shutdown(std::net::Shutdown::Write) .context("Could not shutdown writing on the stream")?; Ok(()) } |
|
||||
| ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
|
||||
The stream is now clean to be read from.
|
||||
|
||||
| 1 2 3 4 5 6 7 8 9 | fn read\_from\_stream(unix\_stream: &mut UnixStream) -> anyhow::Result<()> { let mut response = String::new(); unix\_stream .read\_to\_string(&mut response) .context("Failed at reading the unix stream")?; println!("We received this response: {}", response); Ok(()) } |
|
||||
| ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
|
||||
Have the server running in a terminal:
|
||||
|
||||
```routeros
|
||||
cargo run --bin server
|
||||
|
||||
```
|
||||
|
||||
And in a separate terminal, run the client:
|
||||
|
||||
```routeros
|
||||
cargo run --bin client
|
||||
|
||||
```
|
||||
|
||||
If all is well,
|
||||
|
||||
* the hello message should display on the server side
|
||||
* the “I hear you” response should display on the client side
|
||||
|
||||
You can run the client as many times as you want, since the server runs in a loop.
|
||||
|
||||
This tutorial comes with a [github repository](https://github.com/Keksoj/unix%5Fsockets%5Fbasics)that contains the above code.
|
||||
|
||||
Feel free to write an issue for any comment, criticism, or complaint you may have. Fork and do pull requests as you please.
|
||||
|
||||
This blog post is a sum-up of what I learned trying to understand unix sockets while working on Sōzu. A more elaborate version of the code is available[in this other repo](https://github.com/Keksoj/unix%5Fsocket%5Fbased%5Fserver%5Fclient), with additional features:
|
||||
|
||||
* a `UnixListener`\-wrapping library with a glorious `SocketBuilder` helper (permissions! blocking/nonblocking!)
|
||||
* a `Message` module with serializable `Request` and `Response` structs. The Response has a status that is either `Ok`, `Error` or `Processing`
|
||||
* a client loop that continues reading the stream as long as responses come with a `Processing` status, to stops only at `Ok` or `Error`
|
||||
|
||||
All this happened thanks to my employer, [Clever Cloud](https://clever-cloud.com/), who allows me to learn my job in the best possible conditions. Much gratitude.
|
||||
Loading…
Add table
Add a link
Reference in a new issue