Skip to content

Long term plan to unify public zephyr APIs using ONE prefix #64627

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nashif opened this issue Oct 31, 2023 · 40 comments · May be fixed by #89765
Open

Long term plan to unify public zephyr APIs using ONE prefix #64627

nashif opened this issue Oct 31, 2023 · 40 comments · May be fixed by #89765
Labels
Architecture Review Discussion in the Architecture WG required RFC Request For Comments: want input from the community

Comments

@nashif
Copy link
Member

nashif commented Oct 31, 2023

Introduction

Our APIs are all over the place with various prefixes and legacy conventions that do not have any consistency or real meaning. We do mix internal API prefixes with public APIs in many places and some have decvided to use z_ as the prefix for public APIs in some places, adding more fuel to the fire.

Proposal is to pick some prefix and enforce it for all new public C identifiers, over time, (scale of years) migrate all APIs to this prefix.

Ideally we want to release LTS4 with a completely overhauled and consistent API using one prefix for all APIs.

Problem description

see

Proposed change

Come up with a prefix and enforce it for all new APIs and APIs that need to be changed because of conflicts and misuse of private prefixes. Possible prefixes:

  • zphr_
  • zph_

Alternatives

None

@nashif nashif added the RFC Request For Comments: want input from the community label Oct 31, 2023
@carlescufi
Copy link
Member

For completeness, adding some to the list:

  • zphr_
  • zph_
  • zp_
  • zr_

@carlescufi
Copy link
Member

From the TSC face to face meeting:

  • zy_ is proposed
  • zephyr_ is proposed

@nashif
Copy link
Member Author

nashif commented Apr 21, 2024

another option:

  • zrt_

@nashif
Copy link
Member Author

nashif commented Apr 21, 2024

Note the emphasis on long-term in the proposal. Goal is to agree on the prefix and start migrating to it slowly, starting with new APIs being introduced in the tree, some APIs with wrong and misleading prefixes, i.e. https://github.com/zephyrproject-rtos/zephyr/blob/main/include/zephyr/sys/fdtable.h which is a public API using z_ and so on.
Goal is to have most APIs migrated for LTS4.

@RobinKastberg
Copy link
Contributor

Can we also take a look at the use of __ prefixed keywords that are reserved for the implementation. E.g. __packed and __ramfunc straight up collides with IAR keywords. They are easy enough to rename, but they are used in many places, but to what? And how much breakage in modules could be expected?

@butok
Copy link
Collaborator

butok commented Mar 18, 2025

Proposing.

  • Short option:
    z_

  • Long option:
    zep_

@carlescufi
Copy link
Member

carlescufi commented Mar 18, 2025

Architecture WG:

  • @jhedberg talks about the need of having a prefix hierarchy for subsystems, aside from a global prefix. E.g. zy_util_count_bits()
  • @nashif highlights the necessity for a global prefix in order to be able to resolve the conflicts that arise (i.e. count_bits), once we have chosen the prefix we will be able to make progress
  • @carlescufi is worried about partial use of the new prefix because it may lead to some parts transitioning to it, some others may never transition to it
  • @aescolar agres with @nashif
  • @carlescufi states that using z_ for private functions was a mistake, and we should not exclude the option of using it and choosing another private prefix should be on the table
  • A discussion on the options for the private prefix ensues, with _ being put on the table again, as well as _z_
  • There is a wide discussion about the benefits of using an underscore due to the clash with C libraries

Prefix options

  • zphr_
  • zph_
  • zp_
  • zr_
  • z4r_
  • zy_
  • zephyr_
  • z_
  • zep_

@decsny
Copy link
Member

decsny commented Mar 18, 2025

another idea : keep z_ for private and use Z_ for public

@Thalley
Copy link
Collaborator

Thalley commented Mar 18, 2025

Also worth to keep in mind that C reserves (some) identifiers starting with underscore: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf

Image

@nashif
Copy link
Member Author

nashif commented Mar 18, 2025

Also worth to keep in mind that C reserves (some) identifiers starting with underscore: open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf

this is how threadx and others are using something like tx or txe, so, although ThreadX functions start with an underscore, they do not follow the exact pattern restricted by MISRA Rule 21.2. _tx_thread_create() does not start with _ followed by an uppercase letter or contain double underscores (__), which avoids violating the strictest interpretations of the rule.

in any case, grepping the tree for z_ yields more than 5k ocurrances, this is the heavy lifti that needs to be done and that might be very disruptive. It is doable, but we will not be able to use z_ for public any time soon. Also, if we are going to use underscore (_) anyways, why not drop the z completely, i.e. now we have z_tls_data_size, i think _tls_data_size works and is way nicer than _z_tls_data_size.

from a practical point of view and to be able to move in parallel on both fronts, private and public we should try and pick something other than z_ for public and start using it, work on cleaning up naming of private namespace and remove z_ and use a leading underscore.

@aescolar
Copy link
Member

Also, if we are going to use underscore (_) anyways,

Are we going to use _ for private? :-p

why not drop the z completely, i.e. now we have z_tls_data_size, i think _tls_data_size works

Just adding 1 _ before functions is very likely to hit names used by others, including the C library. So I would very strongly advice against that.

from a practical point of view and to be able to move in parallel on both fronts, private and public we should try and pick something other than z_ for public and start using it

I agree. There is way too many internal things using z_

@butok
Copy link
Collaborator

butok commented Mar 19, 2025

and pick something other than z_ for public and start using it

The second most popular option during the Arch. meeting vote is zep_

@pdgendt
Copy link
Collaborator

pdgendt commented Mar 19, 2025

The second most popular option during the Arch. meeting vote is zep_

I voted, zp_, but would also be in favor of zep_.

@nashif
Copy link
Member Author

nashif commented Mar 19, 2025

why not drop the z completely, i.e. now we have z_tls_data_size, i think _tls_data_size works

Just adding 1 _ before functions is very likely to hit names used by others, including the C library. So I would very strongly advice against that.

how? This is going to be internal to Zephyr, and whoever else is using it will probably have it internal to them, if we start thinking this way then we will never be able to integrate with anyone.

Btw, and for fun, someone (supporting zephyr) already uses z_ and _z_

https://github.com/eclipse-zenoh/zenoh-pico/blob/main/include/zenoh-pico/net/subscribe.h

@aescolar
Copy link
Member

aescolar commented Mar 19, 2025

how? This is going to be internal to Zephyr, and whoever else is using it will probably have it internal to them

We have a single namespace while linking. There is no such a thing as "internal" to Zephyr if it is linkable from other Zephyr areas.
A function like count_bits is very likely to hit somebodies else's, but _count_bits is also, because adding just a _ as prefix is way too typical. We need something much less likely to collide. The longer the better, a 3 letter combination like zep_ would be quite good.

@nashif
Copy link
Member Author

nashif commented Mar 19, 2025

We have a single address space while linking. There is no such a thing as "internal" to Zephyr if it is linkable from other Zephyr areas.
A function like count_bits is very likely to hit somebodies else's, but _count_bits is also, because adding just a _ as prefix is way too typical. We need something much less likely to collide. The longer the better, a 3 letter combination like zep_ would be quite good.

ok, having a distinct prefix for private/internal stuff that starts with _ might actually help with deviation from MISRA, so we could do this:

option 1:

  • leave internal/private namespace as is
  • use a new prefix for new public interfaces (zep_) and migrate existing code to this prefix over time.

pros:

  • we can start immeidiatly,

cons:

  • We are left with two prefixes that might look like being public (z_ and zep_)

option 2:

  • use _ for internal/private, ie.. use _z_. This wil make it clear this API is internal/ there will not be confusion about this z_ being public, when used side by side with what we have below
  • use a new prefix for new public interfaces (zep_ or zeph_ or zephyr_ or ...) and migrate existing code to this prefix over time.

pros:

  • less chance of mistakes and confusion between public and private
  • we can start using this model immediatly

cons:

  • requires mass rename of internal/private identifiers

option 3:

  • use _ for internal/private, ie.. use _z_. This wil make it clear this API is internal/ there will not be confusion about this z_ being public operating side by side with what we have below
  • use a z_ prefix for new public interfaces (z_) and migrate existing code to this prefix over time.

cons:

pros:
cant think of any

@tbursztyka
Copy link
Collaborator

zep_ does not directly relate to zephyr, phonetically speaking, where the 'p' has no meaning without the 'h'.

zeph_ or zphr_, in that regard, are much better. (4 letters vs 3 is imho not an issue at all)

I would go option 2, if mass-renaming can be obtained easily via coccinelle for instance. Or else, we are going to live with 2 systems for a long long time.

@jfischer-no
Copy link
Collaborator

I am still confused as to what is going on here.
Where did the idea or requirement come from that the internal and user APIs should be distinguished by prefix?
Why is this better than putting user headers in specific folders like uapi and not prefixing internal API at all?
What else can we do besides mass renaming to really challenge users?

@butok
Copy link
Collaborator

butok commented Mar 19, 2025

(4 letters vs 3 is imho not an issue at all)

IMHO: 3 letters is more practical. Just try pronounce 4 letters zphr ;)

@butok
Copy link
Collaborator

butok commented Mar 19, 2025

I am still confused as to what is going on here. Where did the idea or requirement come from that the internal and user APIs should be distinguished by prefix? Why is this better than putting user headers in specific folders like uapi and not prefixing internal API at all? What else can we do besides mass renaming to really challenge users?

Having a common prefix is a good practice:

  • act as a namespace mechanism to prevent or minimize name collisions.
  • indicate which module or subsystem a function/variable belongs to.
  • provide immediate context about what you're working with.
  • contribute to a uniform coding style.

@nashif
Copy link
Member Author

nashif commented Mar 19, 2025

I am still confused as to what is going on here.

look at the introduction in this issue and linked issues.

@henrikbrixandersen
Copy link
Member

zep_ does not directly relate to zephyr, phonetically speaking, where the 'p' has no meaning without the 'h'.

I agree with this. Both phonetically and on writing, "zep" does not immediately read "zephyr-related" to me.

zeph_ or zphr_, in that regard, are much better. (4 letters vs 3 is imho not an issue at all)

I like "zphr" in writing, but as pointed out by @butok it is very difficult to pronounce when talking about code.

I would go option 2, if mass-renaming can be obtained easily via coccinelle for instance. Or else, we are going to live with 2 systems for a long long time.

I prefer option 2 as well. We choose one prefix for public APIs and use the same prepended with a single underscore for private APIs.

@decsny
Copy link
Member

decsny commented Mar 19, 2025

zep_ does not directly relate to zephyr, phonetically speaking, where the 'p' has no meaning without the 'h'.

zeph_ or zphr_, in that regard, are much better. (4 letters vs 3 is imho not an issue at all)

I would go option 2, if mass-renaming can be obtained easily via coccinelle for instance. Or else, we are going to live with 2 systems for a long long time.

I like "zphr" in writing, but as pointed out by @butok it is very difficult to pronounce when talking about code

Your mistake is thinking logic can be applied to phonetics of english words, when actually the language is all based on the spiritual vibes the combinations of the letters makes you feel. Therefore my vote on this bikeshed would be for zep_ over anything with an h. Because the abbreviations with h is making it seem like the zephyr wind is howling through around haunted mansion, while the zep is feeling like a cyberpunk airship gliding around the sky (potentially due to being the start of zeppelin, which we can rename the project to Zeppelin RTOS while we're at all this to be phonetically consistent too). This is a technical opinion based on objective facts. Source: am a monolingual English speaker.

@jfischer-no
Copy link
Collaborator

Having a common prefix is a good practice:

* act as a namespace mechanism to prevent or minimize name collisions.
* indicate which module or subsystem a function/variable belongs to.
* provide immediate context about what you're working with.
* contribute to a uniform coding style.

I did not ask for best practice advice. Which subsystem in the project does not already have its own namespace prefix? This proposal will rename all C identifiers that already have their own namespaces, e.g. to zphr_sys_le16_to_cpu, for no good reason, and will end up being a huge pain for users.

@jhedberg
Copy link
Member

C identifiers that already have their own namespaces, e.g. to zphr_sys_le16_to_cpu

Unrelated to the specific point that's being debated here, I think we should consider skipping the intermediate namespace for the existing "general purpose" namespaces like sys_* and k_* and just put those directly under the new common namespace, in order to keep the names short & simple (e.g. zep_le16_to_cpu() or zep_thread_create())

@tbursztyka
Copy link
Collaborator

Your mistake is thinking logic can be applied to phonetics of english words, when actually the language is all based on the spiritual vibes the combinations of the letters makes you feel. Therefore my vote on this bikeshed would be for zep_ over anything with an h. Because the abbreviations with h is making it seem like the zephyr wind is howling through around haunted mansion, while the zep is feeling like a cyberpunk airship gliding around the sky (potentially due to being the start of zeppelin, which we can rename the project to Zeppelin RTOS while we're at all this to be phonetically consistent too). This is a technical opinion based on objective facts. Source: am a monolingual English speaker.

:D
Not sure magic fumes would apply on naming things though, as a general rule I mean.
(just to continue on that tone, in french zep has a meaning - it's an anagram - and is not so well connoted. Should we check every language actually?)

(way) More seriously, zeph is a classic short version of zephyr. The prefix would be kind of obvious. I am surprised it did not land in the list of possibilities.

@aescolar
Copy link
Member

aescolar commented Mar 19, 2025

@jfischer-no

Which subsystem in the project does not already have its own namespace prefix?

The majority of sys_util.h is probably the worst.
The main problem I think we need to fix is that we need to stop adding new symbols with way too common names that then force users to change their current code.

This proposal will rename all C identifiers that already have their own namespaces, e.g. to zphr_sys_le16_to_cpu, for no good reason, and will end up being a huge pain for users.

I fret the idea of such a wholesale rename, and I would only consider it acceptable if we are sure we will stop this problem once and for all.
But honestly given the pain involved, I'd be happier to grandfather all the old APIs, and say that whatever we agree would apply only to new APIs, or even only for new APIs in areas which do not have any prefix.
Specially for areas we have already prefixed reasonably, I think it would be reasonable to keep using those prefixe (k_ and sys_).

@tannewt
Copy link

tannewt commented Mar 19, 2025

I like zephyr_ best. It is by far the clearest prefix and is only 6 letters. Most editors should be able to autocomplete or tab complete it anyway. It is also what you type every time you actually talk about the project.

@nashif
Copy link
Member Author

nashif commented Mar 19, 2025

This proposal will rename all C identifiers that already have their own namespaces, e.g. to zphr_sys_le16_to_cpu, for no good reason, and will end up being a huge pain for users.

This is not being exactly proposed here. The proposal is: "Proposal is to pick some prefix and enforce it for all new public C identifiers, over time, (scale of years) migrate all APIs to this prefix."

As discussed yesterday, the migration part after picking a prefix and using it for new identifiers is TBD and will be done on a case-per-case bais, there will be no mass prefixing of APIs.

This proposal will rename all C identifiers that already have their own namespaces, e.g. to zphr_sys_le16_to_cpu, for no good reason, and will end up being a huge pain for users.

I fret the idea of such a wholesale rename, and I would only consider it acceptable if we are sure we will stop this problem once and for all. But honestly given the pain involved, I'd be happier to grandfather all the old APIs, and say that whatever we agree would apply only to new APIs, or even only for new APIs in areas which do not have any prefix. Specially for areas we have already prefixed reasonably, I think it would be reasonable to keep using those prefixe (k_ and sys_).

Yes, agree. If at some point we decide we need to overhaul k_ or whatever, moving to the new prefix would be considered as an option, so the move would be part of a larger overhaul and not the only goal in the case of established subsystems with existing prefixes.

This proposal is not about mass migration and renames, it is a corrective measure to ensure we do not keep digging ourselves deeper in the mud.

I like zephyr_ best. It is by far the clearest prefix and is only 6 letters. Most editors should be able to autocomplete or tab complete it anyway. It is also what you type every time you actually talk about the project.

we all like Zephyr :-)

I am fine with that as well, and linguistically is probably the least controversial, its not like there are many things called Zephyr out there that might collide ;-)

@Thalley
Copy link
Collaborator

Thalley commented Apr 1, 2025

Just FYI, I've reopened a PR for count_bits using the zephyr_ prefix: #87943

The PR can be used to help determine the final choice, and probably shouldn't get merged until there's a conclusion here (although it can fairly easily be renamed until the next release if merged)

@aescolar
Copy link
Member

aescolar commented Apr 1, 2025

(This discussion is dying again)
Let's just try to agree ASAP, that new symbols, which are not already part of an API which already has a prefix, will be prefixed with zephyr_.

@nashif
Copy link
Member Author

nashif commented Apr 1, 2025

@carlescufi this was on the agenda last week but we did not get to it, can we please make this first item on the agenda today?

@carlescufi
Copy link
Member

Architecture WG:

  • Proposal from @aescolar to agree to dismiss option 3 and so
  • @jfischer-no asks what do we consider here to be an identifier, whether only functions, structure names and macros?
  • @henrikbrixandersen agress that we should decide on the scope (which symbols and which subsystems/areas)
  • @aescolar suggests we stick to the decision that concerns new identifiers/symbols, and we can decide about the rest as we go along
  • @bjarki-andreasen has concerns about the length of identifiers if we prefix them, should we extend the max line length with it?
  • @henrikbrixandersen wants to ensure that we use only a single prefix in the future
  • @carlescufi asks for clarification regarding the scope: agreement to apply this to all functions, types and macros in public headers
  • @jfischer-no disagrees with prefixing at all: he claims prefixes can always collide and therefore the situation will not be fixed. @aescolar and @nashif disagree, because it is relatively easy to ask others to change their code once you have cleaned up house and have a consistent approach to naming in Zephyr.

@carlescufi
Copy link
Member

carlescufi commented Apr 15, 2025

I have been thinking a bit more about this particular topic, and I have personally come to the conclusion that we should not prefix everything in Zephyr to avoid clashes with other code. Instead, I believe we should only rename those functions that are confirmed to be clashing with external implementations.
Here is why:

  1. In my humble opinion, Zephyr is not a portable library to include in your application, it is opinionated. It imposes its own build system, its own configuration system, its hardware description language and much more. Zephyr is actually the operating system (by definition, since we call ourselves an RTOS). This does not mean we do not want other libraries to be able to be integrated with Zephyr but, in my opinion, we should treat it for what it is.
  2. I think that imposing this to all public symbols would do more harm than good, at least in the mid term. There would be a truly long (maybe never ending) period where we would live with the inconsistency of having only parts of our public API prefixed.
  3. We have not actually had that many instances of name clashing (at least to my knowledge). This is not an everyday occurrence, at least not one that is reported to the upstream project. This has happened just a couple of times in the last decade (again, unless I am wrong).
  4. Since we adopted the policy of "not changing Zephyr" for macro name collisions we have not heard (at least to my knowlege) of negative impacts to users.
  5. We should definitely prefix those that can be considered very generic (count_bits()), that do not have a natural subystem or area prefix (e.g. no k_ or bt_ or i2c_) or that clash with standardized names (e.g. C standard library). In this way we can help users that have a real problem combining Zephyr with other libraries, but without renaming our whole massive API surface.

@butok
Copy link
Collaborator

butok commented Apr 15, 2025

  • ThreadX - Uses "tx_" prefix
  • embOS - Uses "OS_" prefix
  • µC/OS - Uses "OS_" prefix
  • RT-Thread - Uses "rt_" prefix
  • CMSIS-RTOS - Uses "os" prefix (like osDelay, osThreadCreate)

@carlescufi
Copy link
Member

carlescufi commented Apr 15, 2025

Architecture WG meeting:

  • Discussing the comment by @carlescufi

    • @aescolar says that we should not consider us the centre of the world. We should not have non-prefixed symbols in our headers to avoid clashes with users. Additional option would be to have both bt_ and zephyr_bt_ in parallel.
    • @nashif says that the original issue we were trying to fix was to differentiate between internal and public APIs and also to provide a path for new APIs to have a set of guidelines.
    • @nashif considers that the most urgent things to solve are both the unprefixed functions (e.g. sys/util.h) and then the private prefix
    • @nordicjm agrees with the comment in general, private functions and area-less functions are the most urgent topics to handle
    • @jfischer-no disagrees with z_ being an issue, but the functions that do not belong to an area could be prefixed with sys_
  • @nashif suggest we come up with rules to define new APIs, both for area-specific APIs and area-less ones

  • @carlescufi suggests a vote: 9/12 present (Quorum)
    "Should Zephyr adopt a universal prefix for all of its public (everything in include/) symbols?"
    @pdgendt : No
    @fabiobaltieri: No
    @kartben: Abstain
    @nashif: Yes
    @carlescufi: No
    @jhedberg: No
    @erwango: No
    @cfriedt: Abstain
    @decsny: No

The preliminary conclusions from this meeting is therefore:

  • No universal prefix for all public symbols
  • We need a prefix for the "area-less" APIs that are currently an issue because they clash. This includes (but may not be limited to) everything in include/zephyr/sys/util*.h. This has to be decided.
  • We need a decision on how to replace the current z_ prefix for private APIs that are present in include/zephyr/. This is tracked in Drop z_ prefix as identifier for internal/private functions and revert back to _  #58007

@decsny
Copy link
Member

decsny commented Apr 15, 2025

I have been thinking a bit more about this particular topic, and I have personally come to the conclusion that we should not prefix everything in Zephyr to avoid clashes with other code. Instead, I believe we should only rename those functions that are confirmed to be clashing with external implementations. Here is why:

1. In my humble opinion, Zephyr is not a portable library to include in your application, it is opinionated. It imposes its own build system, its own configuration system, its hardware description language and much more. Zephyr is actually the operating system (by definition, since we call ourselves an RTOS). This does not mean we do not want other libraries to be able to be integrated with Zephyr but, in my opinion, we should treat it for what it is.

2. I think that imposing this to all public symbols would do more harm than good, at least in the mid term. There would be a truly long (maybe never ending) period where we would live with the inconsistency of having only parts of our public API prefixed.

3. We have not actually had that many instances of name clashing (at least to my knowledge). This is not an everyday occurrence, at least not one that is reported to the upstream project. This has happened just a couple of times in the last decade (again, unless I am wrong).

4. Since we adopted the policy of "not changing Zephyr" for [macro name collisions](https://docs.zephyrproject.org/latest/contribute/coding_guidelines/index.html#rule-a-3-macro-name-collisions) we have not heard (at least to my knowlege) of negative impacts to users.

5. We should definitely prefix those that can be considered very generic (`count_bits()`), that do not have a natural subystem or area prefix (e.g. no `k_` or `bt_` or `i2c_`) or that clash with standardized names (e.g. C standard library). In this way we can help users that have a real problem combining Zephyr with other libraries, but without renaming our whole massive API surface.

It seems like the summary of the main idea of what you are proposing is to introduce a default prefix for any public APIs that do not fall into an area that already has a prefix.

@Thalley
Copy link
Collaborator

Thalley commented Apr 16, 2025

@carlescufi I guess this means that we still haven't settled on a prefix for util macros and functions like count_bits?

@carlescufi
Copy link
Member

@carlescufi I guess this means that we still haven't settled on a prefix for util macros and functions like count_bits?

Correct, see my updated comment for more details.

carlescufi added a commit to carlescufi/zephyr that referenced this issue May 10, 2025
Add a new section with Zephyr-wide naming conventions, as well as a
first rule for C code: area prefixes.

Closes zephyrproject-rtos#64627.

Signed-off-by: Carles Cufi <[email protected]>
carlescufi added a commit to carlescufi/zephyr that referenced this issue May 12, 2025
Add a new section with Zephyr-wide naming conventions, as well as a
first rule for C code: area prefixes.

Closes zephyrproject-rtos#64627.

Signed-off-by: Carles Cufi <[email protected]>
@carlescufi
Copy link
Member

carlescufi commented May 13, 2025

Architecture WG meeting:

Discussion about the definition of public and private symbols:

From bottom to top in terms of private vs public:

  • Symbols defined and used within a compilation unit (.c file) (private)
  • Symbols defined and used within a single subsystem (private)
  • Symbols defined and used as an interface between the kernel and architecture, porting, hooks, and similar low-level components (e.g. arch_, soc_, ...) (internal)
  • Symbols defined and used by any part of Zephyr (public)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Architecture Review Discussion in the Architecture WG required RFC Request For Comments: want input from the community
Projects
Status: In Progress
Status: No status
Development

Successfully merging a pull request may close this issue.