Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

First draft of the 'Safety Pamphlet' #212

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

adfernandes
Copy link

No description provided.

Copy link
Contributor

@vapdrs vapdrs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think is a good document, with an appropriate length. To make it much longer would risk MEGO (my eyes glaze over) for a reader hoping to learn to what this subcommittee is for.

What you may be missing is a conclusion, another opportunity to mention Rust again after the case study. Essentially summarizing that "safety-critical-rust" is specific for the intended application, that Rust code can be Safe or unsafe depending on language keywords and scope, and that Rust code can be safe or unsafe based on its quality.


## Unsafe Code

For example, despite the fear-inducing label and _unlike_ other languages such as C or C++, `unsafe` Rust [_cannot_ cause undefined behavior](https://doc.rust-lang.org/std/keyword.unsafe.html#unsafe-abilities). In fact, to a first-order approximation, the the unsafe superset of safe Rust has only three main extra abilities:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The linked document claims that

Safe Rust can’t cause Undefined Behavior

not unsafe

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vapdrs 🤦🏽 Thank you!

So I was looking at this wondering "What the heck were you thinking?" when I typed that out, while a friend made quips about me hallucinating... 😆

But that made me remember that what I meant to say was something along the lines of "if soundness is externally guaranteed in the unsafe block, then the whole thing is sound". Unsafe only means that soundness is guaranteed by the human, not the compiler.

Let me figure out some decent wording and I'll fix. Appreciated!

Copy link
Contributor

@iglesias iglesias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice text!

I think the first section addresses the fit into the bigger picture in safety-critical Rust (at the least, in explaining why Rust is appealing in safety-critical).
If wanted to emphasize or improve in that fit, as I think vapdrs already suggested, I think a (short) conclusion after the case study mentioning Rust and recalling its explicit unsafe, and maybe also mentioning its memory-safety, could be suitable.

Thanks for sharing.

@PLeVasseur
Copy link
Collaborator

@adfernandes -- I've asked @pelico to read through here and see if he thinks he should incorporate pieces of the LLD document he started here or keep it separate. Just wanted to give you a heads-up.

@PLeVasseur PLeVasseur added the coding guidelines Related to work in the Coding Guidelines Subcommittee label Mar 4, 2025
adfernandes and others added 3 commits March 4, 2025 19:26
…oding-safety-critical-code.md

Co-authored-by: Fernando José | フェルナンド ホセ <[email protected]>
…oding-safety-critical-code.md

Co-authored-by: Fernando José | フェルナンド ホセ <[email protected]>
Copy link
Collaborator

@PLeVasseur PLeVasseur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for contributing this write-up @adfernandes! Could you take a look at my comments?


## Unsafe Code

For example, despite the fear-inducing label and _unlike_ other languages such as C or C++, `unsafe` Rust [_cannot_ cause undefined behavior](https://doc.rust-lang.org/std/keyword.unsafe.html#unsafe-abilities). In fact, to a first-order approximation, the unsafe superset of safe Rust has only three main extra abilities:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fairly sure unsafe can cause UB. Part of the usefulness of miri.

* the ability to mutate global variables, and
* the ability to access fields of `union` types.

Note that none of these are generally considered "unsafe" in other languages, even if they are considered "unwise" or indicative of poor programming practices, and may be unavoidably necessary such as accessing low-level hardware registers in embedded programming.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see where you're coming from here. Indeed safety-critical software is often going to be doing low-level things. Perhaps something explicit either here or in an earlier section clarifying that's the case for safety-critical.


Similarly, the term "safety-critical" has no universally agreed-upon definition. As a trivial example, consider a simple reminder alarm application. Criticality depends on the context in which the application is used. If used to remind someone to water the garden, a software failure may mean wilted plants. If used to remind a critically ill patient to take lifesaving medication, a software failure may imply a trip to the hospital or worse.

Even moving to a more formal language, a bank's notion of "safety-critical" may be focused on financial loss, while an airline might be more concerned with loss of life, or a medical company would need to ensure that drug delivery occurs _exactly_ as specified. There is no "one-size-fits-all" notion of safety-critical software practices!
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great context.

- ISO 26262 — This standard is specifically for functional safety in road vehicles. It addresses the safety of electrical and electronic systems within automobiles.
- ISO / SAE 21434 — A cybersecurity standard jointly developed by [ISO](https://en.wikipedia.org/wiki/ISO) and [SAE](https://en.wikipedia.org/wiki/SAE_International) working groups that proposes cybersecurity measures for the development lifecycle of road vehicles.
- AUTOSAR (AUTomotive Open System ARchitecture) — an industry development partnership that develops and establishes an open and standardized software architecture for automotive electronic control units.
- MISRA (Motor Industry Software Reliability Association) — an organization that produces guidelines for the software developed for electronic components used in the automotive industry.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this automotive only? Could have sworn I saw it discussed in other contexts too.

- ISO 14971 — Application of risk management to medical devices.
- FDA [510(k)](https://www.fda.gov/medical-devices/overview-device-regulation/regulatory-controls) — What is needed to support a submission for approval.

These standards often require rigorous development processes, documentation, testing, and risk assessment. Functional safety standards often utilize risk assessment and safety integrity levels (SILs, or ASILs for Automotive SILs) to determine the rigor of the development process that must be followed. Coding standards like those published by the MISRA organization are frequently used in conjunction with these safety standards to improve code quality and reliability. Thus "writing the software" is usually only a very small part of what "safety-critical" software programming practices need to address in order to meet a particular field's and a particular use-case's standard of care.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great writeup, nice nuance as to where software sits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
coding guidelines Related to work in the Coding Guidelines Subcommittee
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants