--- id: 1 state: approved created: 2018-08-20 placement: category: meta order: 10 js_scripts: - /assets/js/graphviz/viz.js - /assets/js/graphviz/lite.render.js - /assets/js/aip/aip-graphviz.js --- # AIP Purpose and Guidelines As the corpus of Google APIs has grown and the API Governance team has grown to meet the demand of supporting them, it is increasingly necessary to have a corpus of documentation for API producers, reviewers, and other interested parties to reference. The API style guide and introductory One Platform documentation are intentionally terse and high-level. The AIP collection offers a method to provide consistent documentation for API design guidance. ## What is an AIP? AIP stands for **API Improvement Proposal**, which is a design document providing high-level, concise documentation for API development. They are to serve as the source of truth for API-related documentation at Google and the means by which API teams discuss and come to consensus on API guidance. AIPs are maintained as Markdown files in the [AIP GitHub repository][]. ## Types of AIPs There are several different types of AIPs, described below. The list of AIP types may evolve over time as necessary. ### Guidance These AIPs describe guidance on API design. These are provided as instruction for API producers to help write simple, intuitive, and consistent APIs, and are used by API reviewers as a basis for review comments. ### Process These AIPs describe a process surrounding API design. They often affect the AIP process itself and are used to enhance the way in which AIPs are handled. ## Stakeholders As with any process there are many different stakeholders when it comes to reviewing and working with AIPs. Below is a summary of the escalation path starting with the API producer. ```graphviz digraph d_front_back { rankdir=BT; ranksep=0.3; node [ style="filled,solid" shape=box fontname="Roboto" ]; producer [ label="API Producer" ]; editors [ label="AIP Editors" ]; tl_infra [ label="Infrastructure TL" ]; tl_design [ label="Design TL" ]; tl [ label="TL" ]; producer -> editors; editors -> tl_infra -> tl; editors -> tl_design -> tl; } ``` As noted in the diagram above, the TL is the final decision-maker on the AIP process and the final point of escalation if necessary. ### Editors The editors are the set of people who make decisions on AIPs. The general goal is that the AIP process is collaborative and that we largely work on the basis of consensus. However, a limited number of designated approvers is necessary, and these Googlers will be approvers for each AIP in the general scope. The list of AIP editors is currently: - Angie Lin ([@alin04][]) - Jon Skeet ([@jskeet][]) - Jose Juan Zavala Iglesias ([@itsStrobe][]) - Louis Dejardin ([@loudej][]) - Noah Dietz ([@noahdietz][]) - Sam Levenick ([@slevenick][]) - Sam Woodard ([@shwoodard][]) The editors are also responsible for the administrative and editorial aspects of shepherding AIPs and managing the AIP pipeline and workflow. They approve PRs to AIPs, assign proposal numbers, manage the agenda, set AIP states, and so forth. They also ensure that AIPs are readable (proper spelling, grammar, sentence structure, markup, etc.). AIP editorship is by invitation of the current editors. ## Domain-specific AIPs Some AIPs may be specific to a particular domain (for example, only to APIs within a certain PA, or even a certain team). In this situation, the group will be given a particular block of AIPs to use in accordance with AIP-2, and the applicable AIPs will clearly indicate their scope. ## States At any given time, AIPs may exist in a variety of states as they work their way through the process. The following is a summary of each state. ### Draft The initial state for an AIP is the "Draft" state. This means that the AIP is being discussed and iterated upon, primarily by the original authors. While the editors _may_ get involved at this stage, it is not necessary. **Note:** If significant, high-level iteration is required, it is recommended to draft AIPs in a Google doc instead of a PR. AIPs that are migrated into the AIP system from Google Docs **may** skip the draft state and go directly to reviewing provided there is sufficient approval. ### Reviewing Once discussion on an AIP has generally concluded, but before it is formally accepted it moves to the "Reviewing" state. This means that the authors have reached a general consensus on the proposal and the editors are now involved. At this stage the editors may request changes or suggest alternatives to the proposal before moving forward. **Note:** As a formal matter, one AIP approver (other than the author) **must** provide formal signoff to advance an AIP to the reviewing state. Additionally, there **must not** be formal objections ("changes requested" on the GitHub PR) from other approvers. ### Approved Once an approved AIP has been agreed upon, it enters "approved" state and is considered "best current practice". **Note:** As a formal matter, two AIP approvers (other than the author) **must** provide formal signoff to advance an AIP to the approved state. Additionally, there **must not** be formal objections ("changes requested" on the GitHub PR) from other approvers. ### Withdrawn If an AIP is withdrawn by the author or champion, it enters "withdrawn" state. AIPs that are withdrawn may be taken up by another champion. ### Rejected If an AIP is rejected by the AIP editors, it enters "rejected" state. AIPs that are rejected remain, and provide documentation and reference to inform future discussions. ### Deferred If an AIP has not been acted upon for a significant period of time, the editors may mark it as "deferred". ### Replaced If an AIP has been replaced by another AIP, it enters "replaced" state. AIP editors are responsible to provide a notice explaining the replacement and rationale (the replacement AIP should also clearly explain the rationale). In general, API producers should rely primarily on AIPs in the "approved" state. ## Workflow The following workflow describes the process for proposing an AIP, and moving an AIP from proposal to implementation to final acceptance. ### Overview ```graphviz digraph d_front_back { rankdir=LR; node [ style="filled,solid" shape=box fontname="Roboto" ]; draft [ label="Draft" fillcolor="orange" ]; reviewing [ label="Reviewing" fillcolor="lightskyblue" ]; approved [ label="Approved" fillcolor="palegreen" ]; withdrawn [ label="Withdrawn" fillcolor="mistyrose" ]; rejected [ label="Rejected" fillcolor="mistyrose" ]; deferred [ label="Deferred" fillcolor="lightsteelblue" ]; replaced [ label="Replaced" fillcolor="lightsteelblue" ]; draft -> reviewing; draft -> withdrawn [ style=dashed, color=mistyrose3 ]; draft -> rejected [ style=dashed, color=mistyrose3 ]; reviewing -> approved; reviewing -> withdrawn [ style=dashed, color=mistyrose3 ]; reviewing -> rejected [ style=dashed, color=mistyrose3 ]; draft -> deferred [ style=dashed, color=lightsteelblue3 ]; reviewing -> deferred [ style=dashed, color=lightsteelblue3 ]; approved -> replaced [ style=dashed, color=lightsteelblue3 ]; reviewing -> replaced [ style=dashed, color=lightsteelblue3 ]; } ``` ### Proposing an AIP In order to propose an AIP, first [open an issue][] to circulate the fundamental idea for initial feedback. It should generally be possible to describe the idea in a couple of pages. When proposing a new AIP or changes to an existing one, it is best to reference prior art and/or example use cases that the proposal will impact, so as to ensure that the proposal is grounded in a realistic problem space. So, proposals **should** provide concrete references and/or well-defined examples. Appropriate material includes, but is not limited to, the following: - Existing external RFCs or standards - A corpus of APIs that have aligned on a similar pattern e.g. `Search` methods - A concrete use case that has yet to be solved that exists or could exist in one or more APIs e.g. adding an AIP-202 Format for AIP-143 Unicode CLDR region codes Once ready, create a PR with a new file in the AIP directory using a file titled `aip/new.md`. Ensure that the PR is editable by maintainers. In most circumstances, the editors will assign the proposal an AIP number and submit the PR with the AIP in the "Reviewing" state. The editors may reject an AIP outright if they have an obvious reason to do so (e.g. the proposal was already discussed and rejected in another AIP or is fundamentally unsound), in which case the PR is not merged. ### Discussing an AIP Once the PR is merged, the AIP author is responsible for championing the AIP on a follow-up approval pull request. This means that the author is responsible for pushing towards consensus around the proposal. This may involve a discussion at the regularly scheduled meetings for the API Governance team. The AIP author may modify the AIP over the course of discussion by submitting follow-up commits to the PR. ### Accepting an AIP The editors will work together to ensure that qualified proposals do not linger in review. To gain final approval, an AIP **must** be approved by, at minimum, the TL with responsibility over the domain covered by the AIP (either design or infrastructure) and at least one other editor, with no editors actively requesting changes. **Note:** If an AIP editor is the primary author of an AIP, then at least two _other_ editors must approve it. Once the AIP is approved, the editors will update the state of the AIP to reflect this and submit the PR. ### Withdrawing or Rejecting an AIP The author of an AIP may decide, after further consideration, that an AIP should not advance. If so, the author may withdraw the AIP by updating the PR adding a notice of withdrawal with an explanation of the rationale. Additionally, the author may be unable to get consensus among the group and the AIP editors may elect to reject the AIP. In this situation, the AIP editors shall amend the PR adding a notice of rejection with an explanation of the rationale. In both cases, the AIP editors update the state accordingly and submit the PR. ### Replacing an AIP In rare cases, it may be necessary to replace an AIP with another one. This is not general practice: minor edits to approved AIPs are acceptable, and will be the common way to tweak guidance. However, if new guidance fundamentally alters the old guidance in some way, then the AIP editors shall create a new AIP that, once approved, will replace the old one. The old one then enters "Replaced" state, and will link to the new, current AIP. ## Changelog - **2025-01-09**: Add requirement to include references/examples in proposals. - **2024-09-04**: Updated names of current editors and remove TLs. - **2023-05-10**: Updated names of current and editors and TLs. - **2019-07-30**: Further clarified AIP quorum requirements. - **2019-05-12**: Collapsed AIP approvers and editors into a single position, relaxed approval rules from full quorum. - **2019-05-04**: Updated the AIP to refer to GitHub processes, rather than internal processes. [aip github repository]: https://github.com/googleapis/aip [open an issue]: https://github.com/googleapis/aip/issues [@alin04]:https://github.com/alin04 [@jskeet]: https://github.com/jskeet [@loudej]: https://github.com/loudej [@noahdietz]: https://github.com/noahdietz [@slevenick]: https://github.com/slevenick [@shwoodard]: https://github.com/shwoodard [@itsStrobe]: https://github.com/itsStrobe --- id: 2 state: approved created: 2018-08-23 placement: category: meta order: 20 --- # AIP Numbering The AIP system provides a mechanism to index and have a single source of truth for API Improvement Proposals, as well as iterate on them collaboratively and transparently. This document describes the AIP numbering system. ## Assigning AIP Numbers The AIP editors (see [AIP-1](./0001.md)) are responsible for assigning a number to each AIP when it is accepted as a draft for review. Importantly, _all_ AIPs have numbers, not just approved ones. The [AIP Index](/) clearly delineates which AIPs are approved and binding and which are under discussion. The editors may decide to reserve a specific block of numbers for groups of AIPs that are related in some way (for example, that are only scoped to a specific subset of APIs). Beyond this, AIP numbers are assigned arbitrarily. In general, the editors will take the next AIP number off of the stack to assign to a draft AIP, but occasionally may use a special/joke number if useful for mnemonic or other reasons. ## AIP Blocks Currently recognized blocks of AIP numbers are: ### Generally Applicable - **1-99:** Reserved for meta-AIPs (generally process-related). - **100-999:** General API design guidance ### Google Product Areas - **2700-2799:** Apps (Google Workspace) - **2500-2599:** Cloud - **3000-3099:** Actions on Google - **3200-3299:** Firebase - **4100-4199:** Auth libraries - **4200-4299:** Client libraries - **4600-4699:** Geo To request a block for a specific team that is publishing API guidance or documentation germane to that specific team, reach out to api-editors@. ## Changelog - **2019-10-03:** The 3000-3099 block was assigned to Actions on Google. - **2019-01-26:** The general API design guidance block was expanded to include 100-199. - **2018-10-24:** The 4600-4699 block was assigned to Google Geo. - **2018-10-02:** The 2500-2599 block was assigned to Google Cloud Platform. - **2018-10-02:** The 2700-2799 block was assigned to Google Workspace. --- id: 3 state: approved created: 2023-03-28 placement: category: meta order: 30 --- # AIP Versioning This AIP defines the versioning scheme of the AIPs. ## Guidance - The AIPs **must** be versioned by date, using the [ISO-8601 format of `YYYY-MM-DD`](https://www.w3.org/TR/NOTE-datetime), that corresponds to the date the version was added. - AIPs versions **must** be available as a tag on the source control system used to store the AIPs, of the format `v{version}`. Example: `v2023-03-28`. - The AIPs **must** have a new version when there is a significant change to one or more AIPs. - Each AIP **must** include a changelog section, with the date the change was made and a short description. ## Rationale Versions serve as reference points to AIPs at a specific point in time. They are crucial since guidance on an AIP can be reversed, or include significant changes such that they are no longer similar to the original design. APIs using AIPs may need to reference older AIP guidance to justify their design choice. ### Rationale for date-based versioning With date-based versioning, a client can easily find the AIP guidance at the time an API was authored. ### Why not individually versioned AIPs An alternative to a universal version is to have specific versions attached to each AIP. AIPs often cross-reference one another. If each AIP had a specific version, then cross-references would also have to specify specific versions of those referenced AIPs to provide complete guidance.--- id: 8 state: approved created: 2019-05-28 placement: category: meta order: 40 --- # AIP Style and Guidance AIP stands for **API Improvement Proposal**, which is a design document providing high-level, concise documentation for API design and development. The goal is for these documents to serve as the source of truth for API-related documentation at Google and the way API teams discuss and come to consensus on API guidance. AIPs are most useful when they are clear and concise, and cover a single topic or inquiry well. In the same way that AIPs describe consistent patterns and style for use in APIs, they also _follow_ consistent patterns and style. ## Guidance - AIPs **must** cover a single, discrete topic, and provide clear, actionable guidance. - AIPs **must not** duplicate or contradict guidance in another AIP. - AIPs **may** also cover what _not_ to do, but **should not** cover _only_ anti-patterns. - If AIP guidance is conditional (e.g. a design pattern such as Jobs), the guidance **must** clearly explain under what conditions the guidance should be followed. Guidance contained within an AIP **must** be beneficial to one or more types of clients or their authors, including but not limited to: - Asset inventories which can be used to audit and analyze resources. - Command line interfaces for exploration and simple automation. - Custom controllers (e.g. auto-scalers) which poll live state and adjust resource configuration accordingly. - [IaC][] clients for orchestration and automation of multiple resources. - Recommendation tools which provide guidance on which APIs are useful for specific use cases, and how to use them. - SDKs to interact with an API from a programming language, often used heavily for data-plane operations. - Security orchestration, automation, and remediation tools. - Simple scripts to automate or orchestrate tasks. - Test frameworks. - Tools that operate on resource data at rest. - Visual User Interfaces for visualization and one-off manual actions. - Users. Examples of improvements include: - Requiring new proto annotations that enable more descriptive interfaces on clients (e.g. requiring `singular` and `plural` on a `google.api.resource` annotation). AIP guidance **must not** be a significant detriment to a client's usability or implementation difficulty, or maintenance difficulty. Examples of detriments include: - Introduction of a non-uniform pattern in a standard method such that all clients must introduce additional code without sufficient benefit (e.g. List behaves like this *except* for resources that start with the name Foo). - Renames of well-established fields for minor improvements in readability (e.g. rename `expire_time` to `lapse_time` since `lapse` is a common term in my service). While the length of AIPs will necessarily vary based on the complexity of the question, most AIPs **should** be able to cover their content in roughly two printed pages. ### File structure AIPs **must** be written in Markdown, and **must** be named using their four-digit number (example: `0008.md`). AIPs that serve a specific scope **must** be in the subdirectory for that scope. AIPs **must** have appropriate front matter. ```yaml --- id: 8 state: reviewing created: 2019-05-28 permalink: /8 redirect_from: - /08 - /008 - /0008 --- ``` Front matter for AIPs **must** include: - The `aip` key: - `id`: Required. The ID for the given AIP, as an integer. - `state`: Required. The current state of the AIP, in all lower-case. The valid states are listed in [AIP-1][], and common states are `draft`, `reviewing`, and `approved`. - `created`: Required. The ISO-8601 date (`yyyy-mm-dd`) when the AIP was originally drafted, with no quotes. - `updated`: The ISO-8601 date (`yyyy-mm-dd`) when the AIP was last revised. - `scope`: The scope for the AIP. This **must** match the directory name for that scope. Required for AIPs with IDs >= 1000, prohibited otherwise. - The `permalink` key (required): This **must** be set to `/{aip.scope}/{aip.id}`. If there is no scope, use `/{aip.id}` instead. - The `redirect_from` key: This should include a list of any `/{aip.id}` permutations that a reader would be likely to enter, including: - `/{aip.id}` (for AIPs where the permalink includes the scope) - AIP IDs with zero-padding, for each level of zero-padding up to four digits (for example: `/08`, `/008`, `/0008`). ### Document structure AIPs **must** begin with a top-level heading with the AIP's title (`# Title`). The title **should** be a noun (not an imperative). For example, "Bad API precedents" not "Avoid breaking API precedent". AIPs **should** then begin with an introduction (with no additional heading), followed by a `## Guidance` heading. If necessary, the AIP **may** include any of the following after the guidance, in the following order: - "Further reading" is a bulleted list of links to other AIPs that are useful to fully understand the current AIP. - "Appendices" covering further explanation in the same AIP. These are relatively rare but are important in cases where an AIP requires a lot of justification for the decision. Often this is primarily an explanation of alternatives considered to help explain the guidance. - "Changelog" is a bulleted list of changes made to the AIP since the first writing. The guidance section **may** include subsections that elaborate further on details. Subsections will automatically create an entry in the table of contents, and an anchor for citations. Below is an example AIP shell that uses each major section: ```md # AIP title The introductory text explains the background and reason why the AIP exists. It lays out the basic question, but does not tell the reader what to do. ## Guidance The "guidance" section helps the reader know what to do. A common format for the guidance section is a high-level imperative, followed by an example, followed by a bulleted list explaining the example. ### Subsection Individual subsections can be cited individually, and further elaborate details. ## Rationale The "rationale" section is optional, and helps the reader understand the motivation behind specific guidance within the AIP. Deeper explanations of design justification and tradeoffs **must** be in the rationale instead of other sections, to ensure the rest of the document acts as an easily actionable reference. ## History The "history" section is optional, and documents events and context around a significant edit to an AIP. For example, explanation of rewrite would be included in this section While the changelog is a dotted list of one-line summaries of changes to an AIP, the history section should elaborate on significant events in a descriptive format. The section **must not** be used to exhaustively enumerate all changes. This is what the changelog provides. ## Further reading A bulleted list of (usually) other AIPs, in the following format: - [AIP-1](./0001.md): AIP purpose and guidelines ## Changelog A bulleted list of changes in reverse chronological order, using the following format: - **2020-02-18**: Specified ordering. - **2019-07-01**: Added a subsection clarifying XYZ. ``` AIPs **should** attempt to follow this overall format if possible, but AIPs **may** deviate from it if necessary (in particular, if the AIP would be more difficult to understand, even for a reader already accustomed to reading AIPs in the usual format). **Note:** Except for the title, AIPs **must** only use the second heading level (`##`) and above. AIPs **should** only use the second and third heading levels (`##`, `###`). ### Requirement keywords AIPs **should** use the following requirement level keywords: "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY", which are to be interpreted as described in [RFC 2119][]. When using these terms in AIPs, they **must** be lower-case and **bold**. These terms **should not** be used in other ways. If "SHOULD" or "SHOULD NOT" are used, they **must** include valid examples of where other concerns may override the guidance. **Important:** If rationale is used, it exists to provide background and a more complete understanding, but **must not** contain guidance (and RFC-2119 terms **must not** be used). ### Code examples API design examples in AIPs **should** use [protocol buffers][]. Examples **should** cover only enough syntax to explain the concept. When using RPCs in examples, a `google.api.http` annotation **should** be included. ### Referencing AIPs When AIPs reference other AIPs, the prosaic text **must** use the format `AIP-XXXX` without zero-padding (e.g., `AIP-8`, not `AIP-0008`), and **must** link to the relevant AIP. AIP links **may** point to a particular section of the AIP if appropriate. **Important:** AIP links **must** use the relative path to the file in the repository (such as `./0008.md` for core AIPs, or `../0008.md` for AIPs in a subdirectory); this ensures that the link works both on the AIP site, when viewing the Markdown file on GitHub, using the local development server, or a branch. ## Rationale ### Designing for a broad set of clients API guidance, similar to any software, is most beneficial when there is a clear purpose and target beneficiary. The beneficiaries of improved API design are users. These users interact with APIs via a variety of clients, depending on their use case as enumerated above. API guidance must in turn consider the impact broadly across these clients. [aip-1]: ./0001.md [Infrastructure as Code]: ./0009.md#iac [IaC]: ./0009.md#iac [protocol buffers]: https://developers.google.com/protocol-buffers/ [rfc 2119]: https://www.ietf.org/rfc/rfc2119.txt ## Changelog - **2023-05-20**: Increase API guidance scope to include broad set of clients. - **2023-03-30**: Removed appendix, added rationale and history to the template. - **2020-02-18**: Specified reverse chronological ordering for changelog items. - **2019-08-23**: Added guidance for internal AIP links.--- id: 9 state: approved created: 2019-08-01 placement: category: meta order: 100 --- # Glossary In the name of brevity, this AIP defines some common terminology here rather than in each AIP individually. ## Guidance The following terminology **should** be used consistently throughout AIPs. ### API Application programming interface. This can be a local interface (such as a client library) or a Network API (defined below). ### API backend A set of servers and related infrastructure that implements the business logic for an API service. An individual API backend server is often called an API server. ### API consumer The entity that consumes an API service. For Google APIs, it typically is a Google project that owns the client application or the server resource. ### API definition The definition of an API, usually defined in a Protocol Buffer service. An API definition can be implemented by any number of API services. ### API frontend A set of servers plus related infrastructure that provides common functionality across API services, such as load balancing and authentication. An individual API frontend server is often called an API proxy. **Note:** the API frontend and the API backend may run next to each other or far away from each other. In some cases, they can be compiled into a single application binary and run inside a single process. ### API interface The element of an API specification IDL that groups API methods, such as a Protocol Buffers `service` definition. It is typically mapped to a similar high level grouping mechanism in most programming languages, like a `class` or `interface`. ### API method An individual operation within an API. It is typically represented in Protocol Buffers by an `rpc` definition, and is mapped to a function in the API in most programming languages. ### API producer The entity that produces an API service. For Google APIs, it typically is a Google team responsible for the API service. ### API product An API service and its related components, such as Terms of Service, documentation, client libraries, and service support, are collectively presented to customers as a API product. For example, Google Calendar API. **Note:** people sometimes refer to an API product simply as an API. ### API service A deployed implementation of one or more APIs, exposed on one or more network addresses, such as the Cloud Pub/Sub API. ### API service definition The combination of API definitions (`.proto` files) and API service configurations (`.yaml` files) used to define an API service. The schema for Google API service definition is `google.api.Service`. ### API service endpoint Refers to a network address that an API service uses to handle incoming API requests. One API service may have multiple API service endpoints, such as `https://pubsub.googleapis.com` and `https://content-pubsub.googleapis.com`. ### API service name Refers to the logical identifier of an API service. Google APIs use RFC 1035 DNS compatible names as their API service names, such as `pubsub.googleapis.com`. ### API title Refers to the user-facing product title of an API service, such as "Cloud Pub/Sub API". ### API request A single invocation of an API method. It is often used as the unit for billing, logging, monitoring, and rate limiting. ### API version The version of an API or a group of APIs if they are defined together. An API version is often represented by a string, such as "v1", and presents in API requests and Protocol Buffers package names. ### Client Clients are programs that perform a specific tasks by calling an API or generic tools, such as CLIs, that expose the API in a user-accessible fashion or operate on resource data at rest. Examples of clients include the following: - Command line interfaces - Libraries, such as an SDK for a particular programming language - Scripts that operates on a JSON representation of a resource after reading it from an API - Tools, such as a [Declarative clients][] - Visual UIs, such as a web application ### Google API A Network API exposed by a Google service. Most of these are hosted on the `googleapis.com` domain. It does not include other types of APIs, such as client libraries and SDKs. ### Declarative Clients Declarative Clients, also known as Infrastructure as Code (IaC), describes a category of clients that consumes a markup language or code that represents resources exposed by an API, and executes the appropriate imperative actions to drive the resource to that desired state. To determine what changes to make and if a set of updates was successful a declarative client compares server side resource attributes with client defined values. The comparison feature ensures accuracy of a creation or an update but it requires services to treat the client set fields as read-only and diligently preserve those values. Examples of complexities that declarative clients abstract away include: - Determining the appropriate imperative action (create / update / delete) to achieve desired state. - Ordering of these imperative actions. [Terraform][] is an example of such a client. ### User A human being which is using an API directly, such as with cURL. This term is defined to differentiate usage in the AIPs between a human *user* and a programmatic *client*. ### Network API An API that operates across a network of computers. Network APIs communicate using network protocols including HTTP, and are frequently produced by organizations separate from those that consume them. [Declarative clients]: #declarative-clients [Terraform]: https://www.terraform.io/ ## Changelog - **2025-08-13**: Add API inteface entry - **2024-12-18**: Downcase headings and terms as per dev docs style - **2024-10-23**: Add API Title entry - **2023-07-24**: Rename IaC to Declarative Clients - **2023-04-01**: Adding definition of IaC - **2023-03-24**: Reformatting content to include anchor links. --- id: 100 state: approved created: 2018-08-27 placement: category: process order: 10 js_scripts: - /assets/js/graphviz/viz.js - /assets/js/graphviz/lite.render.js - /assets/js/aip/aip-graphviz.js --- # API Design Review FAQ API design review exists to ensure a simple, intuitive, and consistent API experience throughout our API corpus. ## Do I need API design approval? **TL;DR:** You usually need API design approval if you are launching an API that users can code against (either now or in the future) at the beta or GA quality level. API design review is fundamentally about ensuring we provide a simple and consistent experience for our users, and therefore is only expected for APIs that users code directly against. The following flowchart illustrates whether or not your API needs to go through the design review process: ```graphviz digraph { node [ style="filled,solid" shape=box fontname="Roboto" ]; graph [ splines=ortho, nodesep=0.2 ]; audience [ label="Who should code directly\nagainst this API?" shape=diamond fillcolor=bisque ]; subgraph audience_responses { rank = "same"; node [ shape=oval fillcolor=orange ]; googlers [ label="Googlers\nOnly" fillcolor=lightcoral ]; public [ label="Anyone" fillcolor=limegreen ]; partners [ label="Partners\nOnly" fillcolor=deepskyblue ]; partners -> public -> googlers [ style=invisible arrowhead=none ]; } subgraph and_ever { rank = "same"; node [ shape=diamond fillcolor=bisque ]; forever_partners [ label="Forever?" shape=diamond fillcolor=bisque ]; forever_googlers [ label="Forever?" shape=diamond fillcolor=bisque ]; } subgraph forever_responses { rank = "same"; node [ shape=oval fillcolor=orange ]; forever_partners_yes [ label="Yes" fillcolor=deepskyblue ]; forever_no [ label="No, anyone\neventually" fillcolor=limegreen ]; forever_googlers_yes [ label="Yes" fillcolor=lightcoral ]; } release_level [ label="What release\nlevel?" shape=diamond fillcolor=bisque ]; # Not required is not in the subgraph with the other outcomes because # it makes the graph much smaller if it can be higher in the image. subgraph release_levels { rank = "same"; node [ shape=oval ]; alpha [ label="Alpha" fillcolor=darkorange ]; beta [ label="Beta" fillcolor=goldenrod1 ]; ga [ label="GA" fillcolor=limegreen ]; alpha -> beta -> ga [ style=invisible arrowhead=none ]; } changes [ label="Any changes\nfrom beta?" shape=diamond fillcolor=bisque ]; subgraph changes_bool { rank = "same"; node [ shape=oval ]; changes_yes [ label="Yes" fillcolor=goldenrod1 ]; changes_no [ label="No" fillcolor=limegreen ]; } subgraph outcome { rank = "same"; node [ style="rounded,filled" ]; fyi [ label="FYI" fillcolor=lightblue ]; recommended [ label="Recommended" fillcolor=limegreen ]; required [ label="⚠ Required" fillcolor=goldenrod1 ]; not_required [ label="Not Required" fillcolor=lightgrey style="rounded,filled" ]; recommended -> required -> fyi -> not_required [style=invisible arrowhead=none]; } audience -> googlers [ arrowhead=none style=dashed color=grey ]; audience -> partners [ arrowhead=none ]; audience -> public [ arrowhead=none ]; partners -> forever_partners; googlers -> forever_googlers [ style=dashed color=grey ]; public -> release_level; forever_partners -> forever_partners_yes [ arrowhead=none ]; forever_googlers -> forever_googlers_yes [ arrowhead=none style=dashed color=grey ]; forever_partners -> forever_no [ arrowhead=none ]; forever_googlers -> forever_no [ arrowhead=none ]; forever_partners_yes -> recommended; forever_googlers_yes -> not_required [ style=dashed color=grey ]; forever_no -> release_level; release_level -> alpha [ arrowhead=none ]; release_level -> beta [ arrowhead=none ]; release_level -> ga [ arrowhead=none ]; alpha -> recommended; beta -> required; ga -> changes; changes -> changes_yes [ arrowhead=none ]; changes -> changes_no [ arrowhead=none ]; changes_yes -> required; changes_no -> fyi; } ``` ### Who should code directly against it? One of the more complex questions is, "Who should code directly against this API?" API design review is primarily concerned about the API's audience. This means we care about who is permitted to write their own HTTP/gRPC calls against the service, and who is able to see the documentation. (We do _not_ care about questions such as whether the service is exposed on the public network.) Design review is expected if the general public is intended to read documentation and write code that interacts with the service. The following situations **do not** require design review: - An API which will only ever be used by Googlers, or internal tools (for example, Pantheon). - An API which will only ever be called by an executable program released by Google (even if the API could be reverse-engineered from the executable). - An API which will only ever be called by a single customer or small set of customers under contract, and which will _never_ be made more widely available. (Design review is still recommended in this case, but not required.) ### Alpha For alpha, API design review is optional but recommended. It may often make sense to endeavor to get initial feedback from customers quickly, and launching an alpha can be a way of gaining data to determine the best answer to some usability questions; therefore, bypassing review may be expedient. On the other hand, launching an alpha requires building an implementation which then takes engineering effort to update if the API design review at the beta stage raises concerns. Because API design review can precede implementation work, we recommend a design review for alpha. ## Why is design review important? **TL;DR:** Product excellence. Our design review process exists to ensure that the APIs that we present to customers are **simple**, **intuitive**, and **consistent**. Your reviewer approaches your API from the standpoint of a naïve user, thinks through the resources and actions that your API provides, and attempts to make the surface as accessible and extensible as possible. Your design reviewer is not only evaluating your API, but also checking to ensure that your API is consistent with Google's existing corpus of APIs. Many customers use multiple APIs, and therefore it is important that our conventions and naming choices line up with customer expectations. ## What should I expect? ### How long does the review process take? Reviewers make an effort to keep up with their assigned reviews and offer feedback frequently, so as not to cause unnecessary delay, but it's generally best to begin the review process early in case there are delays. The design review process varies based on the size and complexity of the underlying API surface: - Incremental changes to existing APIs generally take a few days. - Small APIs usually take around a week. - Entirely new APIs with large surfaces tend to take no less than a week. In cases with extraordinarily large surfaces (e.g., Cloud AutoML), reviews may take a month or more to go through design review. ### How do reviewers approach my API? API reviewers seek to approach your API the same way that your users will, by focusing primarily on the API surface and its user-facing documentation. In an ideal world, your API reviewer will ask the types of design questions that users will ask (and nudge the API toward raising fewer of those questions in the first place). ### What is precedent? In general, we want Google APIs to be as consistent as possible. Once customers learn their first Google API, it should be easier to learn the second (and then the third, and so on) because we are using the same patterns consistently. We refer to **precedent** to mean decisions that have already been made by previous APIs, which generally should be binding upon newer APIs in similar situations. The most common example of this is naming: we have a list of [standard fields][] that dictate how we use common terms like `name`, `create_time`, and so forth, and which also dictates that we always attach _the same_ name to the same concept. Precedent also applies to _patterns_. All APIs should implement pagination the same way. Ditto for long-running operations, import and export, and so on. Once a pattern has been established, we seek to implement that pattern the same way wherever it is germane. ## What should I do? ### ...if I have a launch on a tight deadline? The best thing that you can do is to engage design review as early as possible. Additionally, make your reviewers aware of your timeline so that they are aware, and can endeavor to provide you the best possible service. We _want_ you to make your deadline if at all possible. For time-sensitive _alpha_ launches, an API **may** launch without receiving design review approval. Such launches **must** be limited to a known set of users. In this case, the reviewers will provide notes for the API team to take under consideration for subsequent stages. **Warning:** Launching an API in alpha with an incomplete design review **does not** enshrine that API's decisions. Design review will be required to promote the API to beta, and API reviewers will block your beta launch if there are issues. For launch stages after alpha, the API design review is mandatory due to its impact on user experience across the board. Your team's inconsistencies affect more than just your team. In some cases, there is a difficult choice to be made between product excellence and either engineering effort or deadlines. These are difficult business decisions and we understand that they are sometimes necessary; however, a director or VP must make an explicit choice to put these other concerns ahead of product excellence when choosing to bypass design review or disregard reviewers' feedback. ### ...to make my review go faster? A few tips: - Begin API review as early as possible, and follow up frequently. - Run the [API linter][] beforehand. (If you are disabling the linter at any point, explain why. Reviewers often find that the linter is disabled because it did its job.) - Ensure that every message, RPC, and field is _usefully_ commented. Comments should be in valid American English and say something meaningful. - If your API reviewer asks you to explain something, add the explanation _in the proto comments_, rather than the code review conversation. This will very often save you a round trip. ### ...if one of my API reviewers is unresponsive? Reach out to the reviewer on Chat. If that fails, reach out to the other reviewer, who will coordinate accordingly. If that fails also, escalate according to [AIP-1][]. ### ...if I have a design question? The first places to look at the [API style guide][], the [AIP index][], and other public APIs within Google. Other public APIs are particularly valuable; it is common that someone has encountered a situation that is germane to your question. ### ...if I have a question not covered there? Reach out to api-design@google.com with your question. This generally works best when you are seeking guidance on a specific question related to API design, and when you clearly explain your use case and provide examples. **Note:** The membership of this list comprises almost exclusively volunteers, who spend the majority of their time doing something else. We do our best to be responsive, but please be patient with us. ### ...if a question is complex and languishing in a CL? While the code review interface is the best way to resolve questions when practical, sometimes there are issues that are sufficiently complicated that working them out in the code review tool is not feasible. In this situation, reach out to your reviewers and ask to schedule a meeting. In general, most issues can be discussed in 30 minutes. When this happens, make sure that someone documents what is discussed in the CL, so that the history is preserved. ### ...if my API needs to violate a standard? Clearly document (using an internal comment in the proto) that you are violating an API design guideline and your rationale for doing so. This comment **must** be prefixed with `aip.dev/not-precedent`. In general, your rationale for the design guideline violation **should** be in accordance with one of the enumerated reasons listed in [AIP-200][]. If it is not, work together with your API reviewer to determine the right thing to do. ### ...if a reviewer is bringing up a previously-settled issue? If you have a different reviewer from your API's previous stages, this might happen. In general, the best approach is simply to reference the code review where the issue was decided. Reviewers want to avoid causing you churn, and therefore usually give deference to previous reviews. This is usually sufficient to resolve the question promptly. Occasionally, the reviewer may believe that the previous reviewer made a significant mistake, and that correcting it is important. In this case, you and your reviewer should work together to determine the best course of action. ### ...if the team and the reviewers strongly disagree? Escalate according to [AIP-1][]. ## Does my PA or team have any particular guidelines? The Cloud PA has specific guidelines to ensure additional uniformity across Cloud, and Cloud APIs have their own reviewer pool. Other teams may adopt similar (but not necessarily identical) rules and systems. Some teams that produce multiple APIs (for example, machine learning) may also have guidelines that apply to that group of APIs. In all cases, we endeavor to make these guidelines available as AIPs; the higher AIP numbers are reserved for specific PA and team use (see [AIP-2][]), and these AIPs are listed in the [AIP index][]. [aip-1]: ./0001.md [aip-2]: ./0002.md [aip-200]: ./0200.md [aip index]: / [api linter]: https://github.com/googleapis/api-linter [api style guide]: https://cloud.google.com/apis/design/ [standard fields]: https://cloud.google.com/apis/design/standard_fields --- id: 111 state: approved created: 2023-06-17 placement: category: api-concepts order: 50 --- # Planes Resources and methods on an API can be divided into the *plane* that they reside or perform operations upon. For the context of APIs, the following planes are defined: - Management plane: a uniform, resource-oriented API that primarily configures and allows retrieval of resources. - Data plane: a heterogenous API (ideally resource-oriented) that reads and write user data. Often connects to entities provisioned by the management plane, such as virtual machines. The term "plane" was originally used in networking architecture. Although system and network architecture often defines additional planes (e.g. control plane or power planes), as the AIPs are focused on the interface, they are not defined in this AIP. ## Guidance ### Management Plane Management resources and methods exist primarily to provision, configure, and audit the resources that the data plane interfaces with. As an example, the following are considered management resources for a cloud provider: - virtual machines - virtual private networks - virtual disks - a blob store instance - a project or account ### Data Plane Methods on the data plane operate on user data in a variety of data formats, and generally interface with a resource provisioned via a management plane API. Examples of data plane methods include: - writing and reading rows in a table - pushing to or pulling from a message queue - uploading blobs to or downloading blobs from a blob store instance Data plane APIs **may** be heterogenous across a larger API surface, due to requirements including high throughput, low latency, or the need to adhere to an existing interface specification (e.g. ANSI SQL). - For convenience, resources and methods that operate on the data plane **may** expose themselves via resource-oriented management APIs. If so, those resources and methods **must** adhere to the requirements of the management plane as specified in the other AIPs ([AIP-131][] through [AIP-135][]). ### Major distinctions between management and data plane - [Declarative clients][] operate on the management plane exclusively. - Data planes are often on the critical path of user-facing functionality, and therefore: - Have higher availability requirements than management planes. - Are more performance-sensitive than management planes. - Require higher-throughput than management planes. [AIP-131]: ./0131.md [AIP-135]: ./0135.md [Declarative clients]: ./0009.md#declarative-clients ## Changelog - **2023-06-10**: Introduction of this AIP. --- id: 121 state: approved created: 2019-01-26 placement: category: resource-design order: 10 --- # Resource-oriented design Resource-oriented design is a pattern for specifying [RPC][] APIs, based on several high-level design principles (most of which are common to recent public HTTP APIs): - The fundamental building blocks of an API are individually-named _resources_ (nouns) and the relationships and hierarchy that exist between them. - A small number of standard _methods_ (verbs) provide the semantics for most common operations. However, custom methods are available in situations where the standard methods do not fit. - Stateless protocol: Each interaction between the client and the server is independent, and both the client and server have clear roles. Readers might notice similarities between these principles and some principles of [REST][]; resource-oriented design borrows many principles from REST, while also defining its own patterns where appropriate. ## Guidance When designing an API, consider the following (roughly in logical order): - The resources (nouns) the API will provide - The relationships and hierarchies between those resources - The schema of each resource - The methods (verbs) each resource provides, relying as much as possible on the standard verbs. ### Resources A resource-oriented API **should** generally be modeled as a resource hierarchy, where each node is either a simple resource or a collection of resources. A _collection_ contains resources of _the same type_. For example, a publisher has the collection of books that it publishes. A resource usually has fields, and resources may have any number of sub-resources (usually collections). **Note:** While there is some conceptual alignment between storage systems and APIs, a service with a resource-oriented API is not necessarily a database, and has enormous flexibility in how it interprets resources and methods. API designers **should not** expect that their API will be reflective of their database schema. In fact, having an API that is identical to the underlying database schema is actually an anti-pattern, as it tightly couples the surface to the underlying system. ### Methods Resource-oriented APIs emphasize resources (data model) over the methods performed on those resources (functionality). A typical resource-oriented API exposes a large number of resources with a small number of methods on each resource. The methods can be either the standard methods ([Get][], [List][], [Create][], [Update][], [Delete][]), or [custom methods][]. If the request to or the response from a standard method (or a custom method in the same *service*) **is** the resource or **contains** the resource, the resource schema for that resource across all methods **must** be the same. | Standard method | Request | Response | | --------------- | --------------------- | ---------------------- | | Create | Contains the resource | Is the resource | | Get | None | Is the resource | | Update | Contains the resource | Is the resource | | Delete | None | None | | List | None | Contains the resources | *The table above describes each standard method's relationship to the resource, where "None" indicates that the resource neither **is** nor **is contained** in the request or the response* A resource **must** support at minimum [Get][]: clients must be able to validate the state of resources after performing a mutation such as [Create][], [Update][], or [Delete][]. A resource **must** also support [List][], except for [singleton resources][] where more than one resource is not possible. **Note:** A custom method in resource-oriented design does _not_ entail defining a new or custom HTTP verb. Custom methods use traditional HTTP verbs (usually `POST`) and define the custom verb in the URI. APIs **should** prefer standard methods over custom methods; the purpose of custom methods is to define functionality that does not cleanly map to any of the standard methods. Custom methods offer the same design freedom as traditional RPC APIs, which can be used to implement common programming patterns, such as database transactions, import and export, or data analysis. ### Strong Consistency For methods that operate on the [management plane][], the completion of those operations (either successful or with an error, long-running operation, or synchronous) **must** mean that the state of the resource's existence and all user-settable values have reached a steady-state. [Output only][] values unrelated to the resource [state][] **should** also have reached a steady-state for values that are related to the resource [state][]. Examples include: - Following a successful create that is the latest mutation on a resource, a get request for a resource **must** return the resource. - Following a successful update that is the latest mutation on a resource, a get request for a resource **must** return the final values from the update request. - Following a successful delete that is the latest mutation on a resource, a get request for a resource **must** return `NOT_FOUND` (or the resource with the `DELETED` state value in the case of [soft delete][]) Clients of resource-oriented APIs often need to orchestrate multiple operations in sequence (e.g. create resource A, create resource B which depends on A), and ensuring that resources immediately reflect steady user state after an operation is complete ensures clients can rely on method completion as a signal to begin the next operation. [Output only][] fields ideally would follow the same guidelines, but as these fields can often represent a resource's live state, it's sometimes necessary for these values to change after a successful mutation operation to reflect a state change. ### Stateless protocol As with most public APIs available today, resource-oriented APIs **must** operate over a [stateless protocol][]: The fundamental behavior of any individual request is independent of other requests made by the caller. This is to say, each request happens in isolation of other requests made by that client or another, and resources exposed by an API are directly addressable without needing to apply a series of specific requests to "reach" the desired resource. In an API with a stateless protocol, the server has the responsibility for persisting data, which may be shared between multiple clients, while clients have sole responsibility and authority for maintaining the application state. ### Cyclic References The relationship between resources, such as with [resource references][], **must** be representable via a [directed acyclic graph][]. The parent-child relationship also **must** be acyclic, and as per [AIP-124][] a given resource instance will only have one canonical parent resource. A cyclic relationship between resources increases the complexity of managing resources. Consider resources A and B that refer to each other. The process to create said resources are: 1. create resource A without a reference to B. Retrieve id for resource A. 2. create resource B with a reference to A. Retrieve id for resource B. 3. update resource A with the reference to B. The delete operation may also become more complex, due to reasoning about which resource must be dereferenced first for a successful deletion. This requirement does not apply to relationships that are expressed via [output only][] fields, as they do not require the user to specify the values and in turn do not increase resource management complexity. [AIP-124]: ./0124.md [create]: ./0133.md [custom methods]: ./0136.md [delete]: ./0135.md [directed acyclic graph]: https://en.wikipedia.org/wiki/Directed_acyclic_graph [get]: ./0131.md [list]: ./0132.md [management plane]: ./0111.md#management-plane [output only]: ./0203.md#output-only [rest]: https://en.wikipedia.org/wiki/Representational_state_transfer [resource references]: ./0122.md#fields-representing-another-resource [rpc]: https://en.wikipedia.org/wiki/Remote_procedure_call [singleton resources]: ./0156.md [soft delete]: ./0164.md [state]: ./0216.md [stateless protocol]: https://en.wikipedia.org/wiki/Stateless_protocol [update]: ./0134.md ## Changelog - **2024-07-08**: Clarify acyclic nature of parent-child relationship. - **2023-08-24**: Added guidance on consistency guarantees of methods. - **2023-07-23**: Clarify stateless protocol definition. - **2023-01-21**: Explicitly require matching schema across standard methods. - **2022-12-19**: Added a section requiring Get and List. - **2022-11-02**: Added a section restricting resource references. - **2019-08-01**: Changed the examples from "shelves" to "publishers", to present a better example of resource ownership. --- id: 122 state: approved created: 2019-01-26 placement: category: resource-design order: 20 --- # Resource names Most APIs expose _resources_ (their primary nouns) which users are able to create, retrieve, and manipulate. Additionally, resources are _named_: each resource has a unique identifier that users use to reference that resource, and these names are what users should _store_ as the canonical names for the resources. ## Guidance All resource names defined by an API **must** be unique within that API. (See the section on [full resource names](#full-resource-names) below for more information on referring to resources across APIs.) Resource names are formatted according to the [URI path schema][], but without the leading slash: publishers/123/books/les-miserables users/vhugo1802 - Resource name components **should** usually alternate between collection identifiers (example: `publishers`, `books`, `users`) and resource IDs (example: `123`, `les-miserables`, `vhugo1802`). - Resource names **must** use the `/` character to separate individual segments of the resource name. - Non-terminal segments of a resource name **must not** contain a `/` character. - The terminal segment of a resource name **should not** contain a `/` character. - Resource names **should** only use characters available in DNS names, as defined by [RFC-1123](https://tools.ietf.org/html/rfc1123). - Additionally, resource IDs **should not** use upper-case letters. - If additional characters are necessary, resource names **should not** use characters that require URL-escaping, or characters outside of ASCII. - If Unicode characters can not be avoided, resource names **must** be stored in Normalization Form C (see [AIP-210][]). - Resources **must** expose a `name` field that contains its resource name. - Resources **may** provide the resource ID as a separate field (e.g. `book_id`). This field **must** apply the [`OUTPUT_ONLY`](./0203.md#output-only) field behavior classification. - Resources **may** expose a separate, system-generated unique ID field [(`uid`)](./0148.md#uid). This field **must** apply the [`OUTPUT_ONLY`](./0203.md#output-only) field behavior classification. - Resources **must not** expose tuples, self-links, or other forms of resource identification. - All ID fields **should** be strings. **Note:** Resource names as described here are used within the scope of a single API (or else in situations where the owning API is clear from the context), and are only required to be unique within that scope. For this reason, they are sometimes called _relative resource names_ to distinguish them from _full resource names_ (discussed below). [aip-210]: ./0210.md [uri path schema]: https://datatracker.ietf.org/doc/html/rfc3986#appendix-A ### Collection identifiers The collection identifier segments in a resource name **must** be the plural form of the noun used for the resource. (For example, a collection of `Publisher` resources is called `publishers` in the resource name.) - Collection identifiers **must** be concise American English terms. - Collection identifiers **must** be in `camelCase`. - Collection identifiers **must** begin with a lower-cased letter and contain only ASCII letters and numbers (`/[a-z][a-zA-Z0-9]*/`). - Collection identifiers **must** be plural. - In situations where there is no plural word ("info"), or where the singular and plural terms are the same ("moose"), the non-pluralized (singular) form is correct. Collection segments **must not** "coin" words by adding "s" in such cases (e.g, avoid "infos"). - Within any given single resource name, collection identifiers **must** be unique. (e.g. `people/xyz/people/abc` is invalid) #### Nested collections If a resource name contains multiple levels of a hierarchy, and a parent collection's name is used as a prefix for the child resource's name, the child collection's name **may** omit the prefix. For example, given a collection of `UserEvent` resources that would normally be nested underneath `users`: ``` users/vhugo1802/userEvents/birthday-dinner-226 ``` An API **should** use the less-redundant form: ``` users/vhugo1802/events/birthday-dinner-226 ``` In this situation, the _message_ and _resource type_ are still called `UserEvent`; only the collection and resource identifiers in the pattern(s) are shortened. Since the _resource type_ is not shortened, the `singular` and `plural` are similarly _not shortened_. ``` message UserEvent { option (google.api.resource) = { type: "example.googleapis.com/UserEvent" // Only the collection & resource identfiers in the `pattern` are shortened. pattern: "projects/{project}/users/{user}/events/{event}" singular: "userEvent" plural: "userEvents" }; string name = 1; } ``` **Note:** APIs wishing to do this **must** follow this format consistently throughout all of its `pattern` entries defined and anywhere else the resource is referenced in the API, or else not at all. ### Resource ID segments A resource ID segment identifies the resource within its parent collection. In the resource name `publishers/123/books/les-miserables`, `123` is the resource ID for the publisher, and `les-miserables` is the resource ID for the book. - If resource IDs are user-specified, the API **must** document allowed formats. User-specified resource IDs **should** conform to [RFC-1034][]; which restricts to letters, numbers, and hyphen, with the first character a letter, the last a letter or a number, and a 63 character maximum. - Additionally, user-specified resource IDs **should** restrict letters to lower-case (`^[a-z]([a-z0-9-]{0,61}[a-z0-9])?$`). - Characters outside of ASCII **should not** be permitted; however, if Unicode characters are necessary, APIs **must** follow guidance in [AIP-210][]. - If resource IDs are not user-settable, the API **should** document the basic format, and any upper boundaries (for example, "at most 63 characters"). - For more information, see the [create][] standard method. [create]: ./0133.md#user-specified-ids [rfc-1034]: https://tools.ietf.org/html/rfc1034 ### Resource ID aliases It is sometimes valuable to provide an alias for common lookup patterns for resource IDs. For example, an API with `users` at the top of its resource hierarchy may wish to provide `users/me` as a shortcut for retrieving information for the authenticated user. APIs **may** provide programmatic aliases for common lookup patterns. However, all data returned from the API **must** use the canonical resource name. ### Full resource names In most cases, resource names are used within a single API only, or else they are used in contexts where the owning API is clear (for example, `string pubsub_topic`). However, sometimes it is necessary for services to refer to resources in an arbitrary API. In this situation, the service **should** use the _full resource name_, a schemeless URI with the owning [API's service name](./0009.md#api-service-name), followed by the relative resource name: ``` //library.googleapis.com/publishers/123/books/les-miserables //calendar.googleapis.com/users/vhugo1802 ``` **Note:** The full resource name **should not** be used for cross-API references where the owning API is clear; it is only used if a field refers to resources in multiple APIs where ambiguity is possible. ### Resource URIs The full resource name is a schemeless URI, but slightly distinct from the full URIs we use to access a resource. The latter includes the protocol (HTTPS), the API version, and the specific [service endpoint](./0009.md#api-service-endpoint) to target: ``` https://library.googleapis.com/v1/publishers/123/books/les-miserables https://calendar.googleapis.com/v3/users/vhugo1802 ``` The version is not included in the full resource name because the full resource name is expected to persist from version to version. Even though the API surface may change between major versions, multiple major versions of the same API are expected to use the same underlying data. **Note:** The correlation between the full resource name and the service's endpoint is by convention. In particular, one service is able to have multiple endpoints (example use cases include regionalization, MTLS, and private access), and the full resource name does not change between these. ### Fields representing resource names When defining a resource, the first field **should** be the resource name, which **must** be of type `string` and **must** be called `name` for the resource name. The message **should** include a `google.api.resource` annotation declaring the type (see [AIP-123][] for more on this). ```proto // A representation of a book in the library. message Book { option (google.api.resource) = { type: "library.googleapis.com/Book" pattern: "publishers/{publisher}/books/{book}" }; // The resource name of the book. // Format: publishers/{publisher}/books/{book} string name = 1 [(google.api.field_behavior) = IDENTIFIER]; // Other fields... } ``` When defining a method that retrieves or acts on an already-existing resource (such as `GetBook` or `ArchiveBook`), the first field of the request message **should** be the resource name, which **must** be of type `string` and **must** be called `name` for the resource name. The field **should** also be annotated with the `google.api.resource_reference` annotation, referencing the resource type ([AIP-123][]). ```proto // Request message for ArchiveBook message ArchiveBookRequest { // The book to archive. // Format: publishers/{publisher}/books/{book} string name = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { type: "library.googleapis.com/Book" }]; // Other fields... } ``` **Note:** Fields **must not** be called `name` except for this purpose. For other use cases, either use a different term or prepend an adjective (for example: `display_name`). [aip-123]: ./0123.md ### Fields representing a resource's parent When defining a method that retrieves resources from a collection or adds a new resource to a collection (such as `ListBooks` or `CreateBook`), the first field of the request message **should** be of type `string` and **should** be called `parent` for the resource name of the collection. The `parent` field **should** also be annotated with the `google.api.resource_reference` annotation, referencing the parent's resource type ([AIP-123][]). ```proto // Request message for ListBooks. message ListBooksRequest { // The publisher to list books from. // Format: publishers/{publisher_id} string parent = 1 [(google.api.resource_reference) = { type: "library.googleapis.com/Publisher" }]; // Other fields (e.g. page_size, page_token, filter, etc.)... } ``` If there is more than one possible parent type, the `parent` field **should** be annotated with the `child_type` key on `google.api.resource_reference` instead: ```proto // Request message for ListBooks. message ListBooksRequest { // The parent to list books from. // Format: // - publishers/{publisher_id} // - authors/{author_id} string parent = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { child_type: "library.googleapis.com/Book" }]; // Other fields (e.g. page_size, page_token, filter, etc.)... } ``` **Note:** Fields **should not** be called `parent` except for this purpose. For other use cases, use a synonymous term if possible. ### Fields representing another resource When a field represents another resource, the field **should** be of type `string` and accept the resource name of the other resource. The field name **should** be equivalent to the corresponding message's name in snake case. - Field names **may** include a leading adjective if appropriate (such as `string dusty_book`). - Field names **should not** use the `_name` suffix unless the field would be ambiguous without it (e.g., `crypto_key_name`) - Fields representing another resource **should** provide the `google.api.resource_reference` annotation with the resource type being referenced. - If using the resource name is not possible and using the ID component alone is strictly necessary, the field **should** use an `_id` suffix (e.g. `shelf_id`). The field **should not** be of type `message` using the `message` that implements the resource, __except__ for one of following conditions: - The API is internal-only, has tight lifecycle relationships, and has a permission model that enables inherited access to embedded resources. - The embedding of the resource is done as part of the AIP-162 revisions pattern. Example of a resource reference: ```proto // A representation of a book in a library. message Book { option (google.api.resource) = { type: "library.googleapis.com/Book" pattern: "publishers/{publisher}/books/{book}" }; // Name of the book. // Format is `publishers/{publisher}/books/{book}` string name = 1 [(google.api.field_behavior) = IDENTIFIER]; // The shelf where the book currently sits. // Format is `shelves/{shelf}`. string shelf = 2 [(google.api.resource_reference) = { type: "library.googleapis.com/Shelf" }]; // Other fields... } ``` ## Further reading - For evolving resource names over time, see [AIP-180](./0180.md#changing-resource-names). - For resource types, see [AIP-123][]. ## Rationale ### Using names instead of IDs For any large system, there are many kinds of resources. To use simple resource IDs to identify a resource, we'd actually need use a resource-specific tuple to reliably identify it, such as `(bucket, object)` or `(user, album, photo)`. This creates several issues: * Developers have to understand and remember such anonymous tuples. * Passing tuples is generally harder than passing strings. * Centralized infrastructures, such as logging and access control systems, don't understand specialized tuples. * Specialized tuples limit API design flexibility, such as providing reusable API interfaces. For example, [Long Running Operations][aip-151] can work with many other API interfaces because they use flexible resource names. [aip-151]: ./0151.md ### Standardizing on `name` The concept of resource names is not a new one, and is formalized in Uniform Resource Names (URN) in conjunction with Uniform Resource Identifiers (URI) and Uniform Resource Locators (URL). Considering that the term "name" is so heavily overloaded in general, usage outside of a very well-defined meaning would be confusing for developers. So, the field name `name` is reserved in the context of AIP-compliant APIs so as to eliminate any confusion with resource names, and force other would be "name" fields to use a more specific field name. ### Disallow embedding of resources Using a resource message directly as the type of a field within another resource is problematic for a number of reasons, which are as follows: * Complicates the resource lifecycle: If the dependency resource is deleted, what happens to the embedded reference in the dependent resource? Data retention and clean up operations will be significantly complicated. * Bypasses permissions: If every resource has its own set of permissions, a user with read permission on the dependent resource that doesn't have the same permission on the dependency resource suddenly cannot see the full resource. * Tightly couples resources in all aspects: Changing the requirements in the schema, permissions, or otherwise for either resource impacts the other, significantly increasing complexity of roll outs. Referencing by name, as is recommended, eliminates all of this complexity by preventing resource data duplication, and forcing the owning service to be involved in the resolution of the reference (via Standard Methods), guaranteeing isolation of logical concerns per-resource. ## History ### Disallowing UUIDs in user-specified IDs As part of an effort to make APIs more declarative-friendly, APIs were required to allow user-specified resource ID on creation. As part of this, a supporting field `string uid` containing a UUID was also encouraged. Guidance was then added to discourage the _user-specified resource ID_ from being a UUID or even resembling one. Ostensibly, the fact that `uid` contained a service-generated UUID motivated this restriction on the user-specified resource ID, forcing the user-specified resource ID to be a more "meaningful" value. However, we no longer saw value in this requirement, hence its removal. ## Changelog - **2025-03-10**: Drop guidance disallowing UUID in user-specified ID. - **2024-10-15**: Add some rationale we found for use of `name` as a field and instead of IDs as an identifier. - **2024-06-14**: Clarify resource annotation shortening rules for nested collections. - **2023-09-19**: Prohibit duplicate collection identifiers. - **2023-09-01**: Add a clause that allows embedding for revision resource messages. - **2023-08-10**: Explicitly disallow embedding resource messages in a resource. - **2023-03-24**: Correction: full resource name contains the service name rather than the service endpoint - **2023-03-17**: Add `OUTPUT_ONLY` guidance for resource ID fields. - **2020-10-06**: Added declarative-friendly guidance, and tightened character set restrictions. - **2020-10-05**: Clarified when full resource names are used. - **2020-05-19**: Clarified that resource _IDs_ avoid capital characters, not the entire resource _name_. - **2020-04-27**: Tighten the restriction on valid characters. - **2019-12-05**: Added guidance for resource annotations. - **2019-08-01**: Changed the examples from "shelves" to "publishers", to present a better example of resource ownership. Also changed the final example from a Pub/Sub example to the usual Book example. - **2019-07-30**: Changed the nested collection brevity suggestion from "may" to "should" --- id: 123 state: approved created: 2019-05-12 placement: category: resource-design order: 30 --- # Resource types Most APIs expose _resources_ (their primary nouns) which users are able to create, retrieve, and manipulate. APIs are allowed to name their resource types reasonably freely (within the requirements of this AIP), and are only required to ensure uniqueness within that API. This means that it is possible (and often desirable) for different APIs to use the same type name. For example, a Memcache and Redis API would both want to use `Instance` as a type name. When mapping the relationships between APIs and their resources, however, it becomes important to have a single, globally-unique type name. Additionally, tools such as Kubernetes or GraphQL interact with APIs from multiple providers. ## Terminology In the guidance below, we use the following terms: - **Service Name:** This is the name defined in the [service configuration][]. This usually (but not necessarily) matches the hostname that users use to call the service. Example: `pubsub.googleapis.com`. This is equivalent to an [API Group][] in Kubernetes. - **Type:** This is the name used for the type within the API e.g. the name of the Protobuf `message`. This is equivalent to an [Object][] in Kubernetes. ## Guidance APIs **must** define a resource type for each resource in the API, according to the following pattern: `{Service Name}/{Type}`. The type name **must**: - Match the containing API type's name. - Start with an uppercase letter. - Only contain alphanumeric characters. - Be of the singular form of the noun. - Use PascalCase (UpperCamelCase). ### Examples Examples of resource types include: - `pubsub.googleapis.com/Topic` - `pubsub.googleapis.com/Subscription` - `spanner.googleapis.com/Database` - `spanner.googleapis.com/Instance` - `networking.istio.io/Instance` ### Annotating resource types APIs **should** annotate the resource types for each resource in the API using the [`google.api.resource`][resource] annotation: ```proto // A representation of a Pub/Sub topic. message Topic { option (google.api.resource) = { type: "pubsub.googleapis.com/Topic" pattern: "projects/{project}/topics/{topic}" singular: "topic" plural: "topics" }; // Name and other fields... } ``` - Patterns **must** correspond to the [resource name][aip-122]. - Pattern variables (the segments within braces) **must** use `snake_case`, and **must not** use an `_id` suffix. - Pattern variables **must** conform to the format `[a-z][_a-z0-9]*[a-z0-9]`. - Pattern variables **must** be unique within any given pattern. (e.g. `projects/{abc}/topics/{abc}` is invalid; this is usually a natural corollary of collection identifiers being unique within a pattern.) - Resources with [multiple patterns][multi-pattern-resources] **must** preserve ordering: new patterns **must** be added at the end of the list, and existing patterns **must not** be removed or re-ordered, as this breaks client library backward compatibility. - Singular **must** be the lower camel case of the type. - Pattern variables **must** be the singular form of the resource type e.g. a pattern variable representing a `Topic` resource ID is named `{topic}`. - Plural **must** be the lower camel case plural of the singular. - Pattern collection identifier segments **must** match the plural of the resources, except in the case of [nested collections][]. #### Pattern uniqueness When multiple patterns are defined within a resource, these patterns **must** be mutually unique, where uniqueness is defined as being by-character identical once all resource ID path segments have been removed, leaving all `/` separators. Therefore the following two patterns **must not** be defined within the same resource: - `user/{user}` - `user/{user_part_1}~{user_part_2}` ## Rationale ### Type and message name alignment In addition to simple schema-resource coherence and alignment, a number of consumers benefit from the `{Type}` and `message` names matching. Consumers have simpler lookups, client libraries get the same in addition to aligned user experience where resource-oriented code has naming aligned with the generated `message` code, generated reference documentation aligns resources with `message` docs, etc. ### Singular and Plural Well-defined singular and plurals of a resource enable clients to determine the proper name to use in code and documentation. lowerCamelCase can be translated into other common forms of a resource name such as UpperCamelCase and snake_case. [aip-122]: ./0122.md [multi-pattern-resources]: https://google.aip.dev/client-libraries/4231#multi-pattern-resources [API Group]: https://kubernetes.io/docs/concepts/overview/kubernetes-api/#api-groups [nested collections]: ./0122.md#collection-identifiers [Object]: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#types-kinds [resource]: https://github.com/googleapis/googleapis/blob/master/google/api/resource.proto [service configuration]: https://github.com/googleapis/googleapis/blob/master/google/api/service.proto ## Changelog - **2025-01-09**: Strongly align resource type and message naming. - **2024-08-07**: Added multi-pattern ordering compatibility requirements. - **2023-09-19**: Prohibited duplicate pattern variables. - **2023-05-06**: Added requirement of singular and plural. - **2023-01-28**: Clarified guidance for the resource type name. - **2022-10-28**: Added pattern variable format guidance. - **2020-05-14**: Added pattern uniqueness. - **2019-12-05**: Added guidance on patterns. - **2019-07-17**: Fleshed out the annotation example somewhat. --- id: 124 state: approved created: 2020-03-20 placement: category: resource-design order: 40 --- # Resource association APIs sometimes have resource hierarchies that can not be cleanly expressed in the usual tree structure. For example, a resource may have a many-to-one relationship with two other resource types instead of just one. Alternatively, a resource may have a many-to-many relationship with another resource type. ## Guidance A resource **must** have at most one canonical parent, and `List` requests **must not** require two distinct "parents" to work. ### Multiple many-to-one associations If a resource has a many-to-one relationship with multiple resource types, it **must** choose at most one of them to be the canonical parent. The resource **may** be associated with other resources through other fields on the resource. ```proto message Book { // The resource name pattern for Book indicates that Publisher is the // canonical parent. option (google.api.resource) = { type: "library.googleapis.com/Book" pattern: "publishers/{publisher}/books/{book}" }; // The resource name for the book. string name = 1 [(google.api.field_behavior) = IDENTIFIER]; // The resource name for the book's author. string author = 2 [(google.api.resource_reference) = { type: "library.googleapis.com/Author" }]; } ``` When listing resources with multiple associations in this way, the RPC **must** treat the `string parent` field as required as discussed in [AIP-132][], and **must not** add additional required arguments. The RPC **should** include a `string filter` field that allows users to filter by other resource associations as discussed in [AIP-160][]. **Note:** Resource reference fields **must** accept the same resource name format that is used in the `name` field of the referenced resource. ### Many-to-many associations Many-to-many associations are less common in APIs than they are in relational databases, in part because they are more difficult to model and present over network interfaces. An API **may** contain many-to-many relationships, and **should** use a repeated field containing a list of resource names, following the principles described for repeated fields in [AIP-144][]. ```proto message Book { option (google.api.resource) = { type: "library.googleapis.com/Book" pattern: "publishers/{publisher}/books/{book}" }; string name = 1 [(google.api.field_behavior) = IDENTIFIER]; // The resource names for the book's authors. repeated string authors = 2 [(google.api.resource_reference) = { type: "library.googleapis.com/Author" }]; } ``` **Note:** See [AIP-144][] for more information on repeated fields, including how to handle common issues such as atomic changes. If the use of a repeated field is too restrictive, or if more metadata is required along with the association, an API **may** model a many-to-many relationship using a sub-resource with two one-to-many associations. ```proto message BookAuthor { // The resource pattern for BookAuthor indicates that Book is the // canonical parent. option (google.api.resource) = { type: "library.googleapis.com/BookAuthor" pattern: "publishers/{publisher}/books/{book}/authors/{book_author}" }; // The resource name for the book-author association. string name = 1 [(google.api.field_behavior) = IDENTIFIER]; // The resource name for the author. string author = 2 [(google.api.resource_reference) = { type: "library.googleapis.com/Author" }]; // Other fields... } ``` **Note:** Using subresources to model an association between resources is only recommended if additional metadata is required in the relationship, or if the restrictions around the use of a repeated field preclude the use of that approach. [aip-132]: ./0132.md [aip-144]: ./0144.md [aip-160]: ./0160.md ## Changelog - **2021-04-07**: Clarified that resource reference fields accept resource names with the same format as the `name` field of the resource. --- id: 126 state: approved created: 2019-07-24 placement: category: resource-design order: 60 --- # Enumerations It is common for a field to only accept or provide a discrete and limited set of values. In these cases, it can be useful to use enumerations (generally abbreviated "enums") in order to clearly communicate what the set of allowed values are. ## Guidance APIs **may** expose enum objects for sets of values that are expected to change infrequently: ```proto // A representation of a book. message Book { // Other fields... // Possible formats in which the book may be published. enum Format { // Default value. This value is unused. FORMAT_UNSPECIFIED = 0; // The printed format, in hardback. HARDBACK = 1; // The printed format, in paperback. PAPERBACK = 2; // An electronic book format. EBOOK = 3; // An audio recording. AUDIOBOOK = 4; } // The format of the book. Format format = 99; // Other fields... } ``` - All enum values **must** use `UPPER_SNAKE_CASE`. - The first value of the enum **should** be the name of the enum itself followed by the suffix `_UNSPECIFIED`. - An exception to this rule is if there is a clearly useful zero value. In particular, if an enum needs to present an `UNKNOWN`, it is usually clearer and more useful for it to be a zero value rather than having both. The `UNKNOWN` value **may** be prefixed by the enum name as is typical for avoiding enum value name collisions. - Enums which will only be used in a single message **should** be nested within that message. In this case, the enum **should** be declared immediately before it is used. - The non-zero values of such a nested enum definition **should not** be prefixed by the name of the enum itself. This generally requires users to write `MyState.MYSTATE_ACTIVE` in their code, which is unnecessarily verbose. - Enums which will be used by multiple messages **should** be defined at the package level and **should** be defined at the bottom of the proto file (see [AIP-191][]). - Some languages (including C++) hoist enum values into the parent namespace, which can result in conflicts for enums with the same values in the same proto package. To avoid sharing values, APIs **should** prefix package-level enum values with the name of the enum. - Enums **should** document whether the enum is frozen or they expect to add values in the future. ### When to use enums Enums can be more accessible and readable than strings or booleans in many cases, but they do add overhead when they change. Therefore, enums **should** receive new values infrequently. While the definition of "infrequently" may change based on individual use cases, a good rule of thumb is no more than once a year. For enums that change frequently, the API **should** use a string and document the format. Additionally, enums **should not** be used when there is a competing, widely-adopted standard representation (such as with [language codes][bcp-47] or [media types][]). **Note:** If an enumerated value needs to be shared across APIs, an enum **may** be used, but the assignment between enum values and their corresponding integers **must** match. ### Alternatives For enumerated values where the set of allowed values changes frequently, APIs **should** use a `string` field instead, and **must** document the allowed values. String fields with enumerated values **should** use `kebab-case` for their values. For enumerated values where there is a competing, widely-adopted standard representation (generally, but not necessarily, a string), that standard representation **should** be used. This is true even if only a small subset of values are permitted, because using enums in this situation often leads to frustrating lookup tables when trying to use multiple APIs together. Boolean fields **may** be used in situations where it is clear that no further flexibility will be needed. The default value **must** be `false`. **Note:** When using protocol buffers, it is impossible to distinguish between `false` and unset. If this is a requirement, an enum **may** be a better design choice (although `google.protobuf.BoolValue` is also available). ## Further reading - For states, a special type of enum, see [AIP-216][]. [aip-191]: ./0191.md [aip-216]: ./0216.md [bcp-47]: https://en.wikipedia.org/wiki/IETF_language_tag [media types]: https://en.wikipedia.org/wiki/Media_type ## Changelog - **2025-10-03**: Added prefixing guidance for `UNKNOWN` value exception. --- id: 127 state: approved created: 2019-08-22 placement: category: protobuf order: 10 --- # HTTP and gRPC Transcoding APIs that follow [resource-oriented design][aip-121] are defined using [RPCs][rpc], but the resource-oriented design framework allows them to also be presented as APIs that largely follow REST/JSON conventions. This is important in order to help developers use their existing knowledge: over 80% of the public APIs available follow most REST conventions, and developers are accustomed to that pattern. ## Guidance APIs **must** provide HTTP definitions for each RPC that they define, except for bi-directional streaming RPCs, which can not be natively supported using HTTP/1.1. When providing a bi-directional streaming method, an API **should** also offer an alternative method that does not rely on bi-directional streaming. ### HTTP method and path When using protocol buffers, each RPC **must** define the HTTP method and path using the `google.api.http` annotation: ```proto rpc CreateBook(CreateBookRequest) returns (Book) { option (google.api.http) = { post: "/v1/{parent=publishers/*}/books" body: "book" }; } message CreateBookRequest { // The publisher who will publish this book. // When using HTTP/JSON, this field is automatically populated based // on the URI, because of the `{parent=publishers/*}` syntax. string parent = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { child_type: "library.googleapis.com/Book" }]; // The book to create. // When using HTTP/JSON, this field is populated based on the HTTP body, // because of the `body: "book"` syntax. Book book = 2 [(google.api.field_behavior) = REQUIRED]; // The user-specified ID for the book. // When using HTTP/JSON, this field is populated based on a query string // argument, such as `?bookId=foo`. This is the fallback for fields that // are not included in either the URI or the body. // Note that clients use camelCase format to communicate the field names // to the service. string book_id = 3; } ``` - The first key (`post` in this example) corresponds to the HTTP method. RPCs **may** use `get`, `post`, `patch`, or `delete`. - RPCs **must** use the prescribed HTTP verb for each standard method, as discussed in [AIP-131][], [AIP-132][], [AIP-133][], [AIP-134][], and [AIP-135][] - RPCs **should** use the prescribed HTTP verb for custom methods, as discussed in [AIP-136][]. - RPCs **should not** use `put` or `custom`. - The corresponding value represents the URI. - URIs **must** use the `{foo=bar/*}` syntax to represent a variable that should be populated in the request proto. When extracting a [resource name][aip-122], the variable **must** include the entire resource name, not just the ID component. - URIs **may** use nested fields for their variable names. (Additionally, AIP-134 mandates this for `Update` requests.) - URIs **must** use the `*` character to represent ID components, which matches all URI-safe characters except for `/`. URIs **may** use `**` as the final segment of a URI if matching `/` is required. - The `body` key defines which single top-level field in the request will be sent as the HTTP body. If the body is `*`, then this indicates that the request object itself is the HTTP body. The request body is encoded as JSON as defined by protocol buffers' canonical [JSON encoding][]. - RPCs **must not** define a `body` at all for RPCs that use the `GET` or `DELETE` HTTP verbs. - RPCs **must** use the prescribed `body` for Create ([AIP-133][]) and Update ([AIP-134][]) requests. - RPCs **should** use the prescribed `body` for custom methods ([AIP-136][]). - The `body` **must not** contain a nested field (or use the `.` character), - The `body` **must not** be the same as a URI parameter. - The `body` **must not** be a `repeated` field. - Fields **should not** use the `json_name` annotation to alter the field name in JSON, unless doing so for backwards-compatibility reasons. **Note:** Bi-directional streaming RPCs should not include a `google.api.http` annotation at all. If feasible, the service **should** provide non-streaming equivalent RPCs. ### Multiple URI bindings Occasionally, an RPC needs to correspond to more than one URI: ```proto rpc CreateBook(CreateBookRequest) returns (Book) { option (google.api.http) = { post: "/v1/{parent=publishers/*}/books" body: "book" additional_bindings: { post: "/v1/{parent=authors/*}/books" body: "book" } additional_bindings: { post: "/v1/books" body: "book" } }; } ``` - RPCs **may** define any number of additional bindings. The structure is identical to the `google.api.http` annotation (in fact, it is a recursive reference). - RPCs **must not** define an additional binding within an additional binding. - The `body` clause **must** be identical in the top-level annotation and each additional binding. ## Changelog - **2022-08-18**: Added the comment that query string parameter names are in camelCase. - **2021-01-06**: Added clarification around `body` and nested fields. - **2019-09-23**: Added a statement about request body encoding, and guidance discouraging `json_name`. [aip-121]: ./0121.md [aip-122]: ./0122.md [aip-131]: ./0131.md [aip-132]: ./0132.md [aip-133]: ./0133.md [aip-134]: ./0134.md [aip-135]: ./0135.md [aip-136]: ./0136.md [json encoding]: https://developers.google.com/protocol-buffers/docs/proto3#json [rpc]: https://en.wikipedia.org/wiki/Remote_procedure_call --- id: 128 state: approved created: 2020-10-06 placement: category: resource-design order: 65 --- # Declarative-friendly interfaces Many services need to interact with common DevOps tools, particularly those that create and manage network-addressable resources (such as virtual machines, load balancers, database instances, and so on). These tools revolve around the principle of "configuration as code": the user specifies the complete intended landscape, and tooling is responsible for making whatever changes are necessary to achieve the user's specification. These tools are **declarative**: rather than specifying specific _actions_ to take, they specify the desired _outcome_, with the actions being derived based on the differences between the current landscape and the intended one. Furthermore, there are numerous popular DevOps tools, with more being introduced each year. Integrating hundreds of resource types with multiple tools requires uniformity, so that integration can be automated. ## Guidance ### Resources Resources that are declarative-friendly **must** use only strongly-consistent standard methods for managing resource lifecycle, which allows tools to support these resources generically, as well as conforming to other declarative-friendly guidance (see [further reading](#further-reading)). Declarative-friendly resources **should** designate that they follow the declarative-friendly style: ```proto message Book { option (google.api.resource) = { type: "library.googleapis.com/Book" pattern: "publishers/{publisher}/books/{book}" style: DECLARATIVE_FRIENDLY }; // Name and other fields... } ``` ### Reconciliation If a resource takes time (more than a few seconds) for updates to be realized, the resource **should** include a `bool reconciling` field to disclose that changes are in flight. This field **must** be output only. A resource **must** set the `reconciling` field to `true` if the current state of the resource does not match the user's intended state, and the system is working to reconcile them. This is regardless of whether the root cause of going into reconciliation was user or system action. **Note:** Services responding to a `GET` request **must** return the resource's current state (not the intended state). ## Further reading A significant amount of guidance is more strict for declarative-friendly interfaces, due to the focus on automation on top of these resources. This list is a comprehensive reference to declarative-friendly guidance in other AIPs: - Resources **should not** employ custom methods: see AIP-136. - Resources **must** use the `Update` method for repeated fields: see AIP-144. - Resources **must** include certain standard fields: see AIP-148. - Resources **must** have an `etag` field: see AIP-154. - Resources **should** provide change validation: see AIP-163. - Resources **should not** implement soft-delete. If the id cannot be re-used, the resource **must** implement soft-delete and the undelete RPC: see AIP-164 ### Annotations See [AIP-148 annotations][]. ## Changelog - **2023-07-13**: Move `annotations` from AIP-148. - **2023-06-17**: Definition of plane was removed and incorporated into AIP-111. - **2023-05-11**: Removed must on resource_id, which was upstreamed to a general must. [AIP-148 annotations]: ./0148.md#annotations --- id: 129 state: approved created: 2023-09-20 updated: 2023-10-31 placement: category: resource-design order: 65 --- # Server-Modified Values and Defaults Services often provide default values for resource fields, and occasionally normalize the user input before returning it in the response. The guidance herein describes how services document such behavior for the benefit of consumers. ## Guidance ### Single Owner Fields Fields **must** have a single owner, whether that is the client or the server. Server owned fields **must** be indicated with the `OUTPUT_ONLY` field_behavior. All other types of fields **must** be considered to be owned by the client. The server **must** respect the value (or lack thereof) for all client owned fields and not modify them. ### Effective Values There are instances where a service will allocate, generate, or calculate a value if the client chooses not to specify one. For example: a client creates a virtual machine without specifying a static IP address for the virtual machine to be available on. Such a scenario is opting into dynamic IP address allocation. Some examples of these types of fields are ones that are: * generated (UUID) * allocated (dynamic IP address) * assigned (most recent software package version) An attribute with an effective value ***must*** be expressed as two fields in the API: * a mutable field that can be optionally set by the user and **must not** be modified by the service * an `OUTPUT_ONLY` field that records the effective value decided on by the service Example: ```proto message VirtualMachine { … string ip_address = 4; string effective_ip_address = 5 [ (google.api.field_behavior) = OUTPUT_ONLY ]; } ``` #### Naming Effective values **must** be named by prefixing `effective_` to the mutable field's name. ### User-Specified Fields For user-specified fields, the value in response from the service **must** be the same as provided by the create or update request. For string fields this means returning the value unchanged, with one exception: * When a string field has a data type annotation, a normalized string that represents the given value **may** be returned. ### Normalizations A field that is normalized by the service **must** be annotated with the `google.api.field_info` extension. See ([AIP-202][]) for guidance on using this extension The allowed set of normalizations includes the following formats: * uuid * ipv4 * ipv6 * email Normalizations on fields **must** be described using the `google.api.field_info` annotation. ## Rationale Server-modified and default values often make it harder to implement [declarative clients][]. These clients are often unable to tell when their desired state matches the current state for these fields, as the rules by which a server may modify and return values are complex, not public, and not repeatable. ### Rationale for Single Owner Fields When fields do not have a single owner they can cause issues for [declarative clients][]. These clients may attempt to set values for fields that are overwritten by server set values, leading to the client entering an infinite loop to correct the change. ### Rationale for Naming Consistent naming is important for identifying standard behavior across APIs and fields. Programmatic association between user-specified and effective values depends on consistent naming. ### Rationale for Normalizations Normalizations are important to allow services to store and return values in a standard way while communicating to clients what changes are semantically identical. Normalizing a value on the service side allows the service to accept a wider range of semantically identical inputs without needing to maintain every value as a raw string. Surfacing the normalization that is being applied to clients allows for client side comparison of sent and retrieved values to check for differences. For example, in a resource that accepts an email address on a particular field a client may specify a given email address in a variety of ways. For the email `ada@example.com` a client may choose to specify `ADA@example.com`, `aDa@example.com`, or `AdA@example.com`. These are semantically identical and *should* all be accepted by the service. The service then may choose to normalize the email address for storage and retrieval through downcasing or canonicalization. Importantly, the information surfaced to clients on the normalization of a field will not describe the normalization algorithm itself, but instead the comparison method used to accurately compute if two values should be considered equal. ### Rationale for Field Value Handling For fields not using an allowed normalization, [Declarative clients][] will not be able to identify which changes are semantically meaningful. When a [Declarative client][Declarative clients] sends a particular value it will ensure that the value is being returned by the service to validate it was set correctly. ## Changelog - **2023-10-31:** Update to approved. [Declarative clients]: ./0009.md#declarative-clients [aip-202]: ./0202.md [aip-180]: ./0180.md --- id: 130 state: approved created: 2023-03-13 updated: 2023-09-05 placement: category: operations order: 9 --- # Methods An API is composed of one or more methods, which represent a specific operation that a service can perform on behalf of the consumer. ## Guidance ### Categories of Methods The following enumerates multiple categories of methods that exist, often grouped up under some object (e.g. collection or resource) that the method operates upon. | Category Name | Related AIPs | [Declarative client][] integration | CLI / UI integration | SDK integration | | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------- | ---------------------------------- | -------------------- | --------------- | | *Standard Methods* | | | | | | **Standard collection methods**: operate on a collection of resources (List or Create). | [AIP-121][], [AIP-132][], [AIP-133][] | automatable | automatable | automatable | | **Standard resource methods**: fetch or mutate a single resource (Get, Update, Delete). | [AIP-121][], [AIP-131][], [AIP-134][], [AIP-135][] | automatable | automatable | automatable | | **Batch resource methods**: fetch or mutate multiple resources in a collection by name. | [AIP-231][], [AIP-233][], [AIP-234][], [AIP-235][] | may be used to optimize queries | automatable | automatable | | **Aggregated list methods**: fetch or mutate multiple resources of the same type across multiple collections. | [AIP-159][] | not useful nor automatable | automatable | automatable | | *Custom Fetch Methods* | | | | | | **Custom collection fetch methods**: fetch information across a collection that cannot be expressed via a standard method. | [AIP-136][] | handwritten | automatable | automatable | | **Custom resource fetch methods**: fetch information for a single resource that cannot be expressed via a standard method. | [AIP-136][] | handwritten | automatable | automatable | | *Custom Mutation Methods* | | | | | | **Backing up a resource**: storing a copy of a resource at a particular point in time. | [AIP-162][] | unused or handwritten | automatable | automatable | | **Restoring a resource**: setting a resource to a version from a particular point in time. | [AIP-162][] | unused or handwritten | automatable | automatable | | **Renaming a resource**: modify the resource's name or id while preserving configuration and data. | [AIP-136][] | unused or handwritten | automatable | automatable | | **Custom collection mutation methods**: perform an imperative operation referencing a collection that may mutate one or more resources within that collection in fashion that cannot be easily achieved by standard methods (e.g. state transitions). | [AIP-136][] | unused or handwritten | automatable | automatable | | **Custom resource mutation methods**: perform an imperative operation on a resource that may mutate it in a way a standard method cannot (e.g. state transitions). | [AIP-136][] | unused or handwritten | automatable | automatable | | *Misc Custom Methods* | | | | | **Stateless Methods**: a method that has no permanent effect on any data within the API (e.g. translating text) | [AIP-136][] | unused or handwritten | automatable | automatable | | *None of the above* | | | | | | **Streaming methods**: methods that communicate via client, server, or bi-directional streams. | | handwritten | handwritten | automatable | ### Choosing a method category While designing a method, API authors should choose from the defined categories in the following order: 1. Standard methods (on collections and resources) 1. Standard batch or aggregate methods 1. Custom methods (on collections, resources, or stateless) 1. Streaming methods ## Rationale Resource-oriented standard and custom methods are recommended first, as they can be expressed in the widest variety of clients (Declarative clients, CLIs, UIs, and so on), and offer the most uniform experience that allows users to apply their knowledge of one API to another. If a standard method is unsuitable, then custom methods (that are mounted to a resource or collection) offer a lesser, but still valuable level of consistency, helping the user reason about the scope of the action and the object whose configuration is read to inform that action. Although mutative custom methods are not uniform enough to have a automated integration with exclusively resource-oriented clients such as [Declarative client][]s, they are still a pattern that can easily recognized by CLIs, UIs, and SDKs. If one cannot express their APIs in a resource-oriented fashion at all, then the operation falls in a category where the lack of uniformity makes it difficult for any client aside from SDKs to model the operation. This category is preferred last due to the fact that a user cannot rely on their knowledge of similar APIs, as well as the issue that integration with many clients will likely have to be hand-written. [AIP-121]: ./0121.md [AIP-131]: ./0131.md [AIP-132]: ./0132.md [AIP-133]: ./0133.md [AIP-134]: ./0134.md [AIP-135]: ./0135.md [AIP-136]: ./0136.md [AIP-159]: ./0159.md [AIP-162]: ./0162.md [AIP-231]: ./0231.md [AIP-233]: ./0233.md [AIP-234]: ./0234.md [AIP-235]: ./0235.md [Declarative client]: ./0009.md#declarative-clients ## Changelog - **2023-09-05**: Update to approved.--- id: 131 state: approved created: 2019-01-22 updated: 2022-06-02 placement: category: operations order: 10 --- # Standard methods: Get In REST APIs, it is customary to make a `GET` request to a resource's URI (for example, `/v1/publishers/{publisher}/books/{book}`) in order to retrieve that resource. Resource-oriented design ([AIP-121][]) honors this pattern through the `Get` method. These RPCs accept the URI representing that resource and return the resource. ## Guidance APIs **must** provide a get method for resources. The purpose of the get method is to return data from a single resource. Get methods are specified using the following pattern: ```proto rpc GetBook(GetBookRequest) returns (Book) { option (google.api.http) = { get: "/v1/{name=publishers/*/books/*}" }; option (google.api.method_signature) = "name"; } ``` - The RPC's name **must** begin with the word `Get`. The remainder of the RPC name **should** be the singular form of the resource's message name. - The request message **must** match the RPC name, with a `Request` suffix. - The response message **must** be the resource itself. (There is no `GetBookResponse`.) - The response **should** usually include the fully-populated resource unless there is a reason to return a partial response (see [AIP-157][]). - The HTTP verb **must** be `GET`. - The URI **should** contain a single variable field corresponding to the resource name. - This field **should** be called `name`. - The URI **should** have a variable corresponding to this field. - The `name` field **should** be the only variable in the URI path. All remaining parameters **should** map to URI query parameters. - There **must not** be a `body` key in the `google.api.http` annotation. - There **should** be exactly one `google.api.method_signature` annotation, with a value of `"name"`. ### Request message Get methods implement a common request message pattern: ```proto message GetBookRequest { // The name of the book to retrieve. // Format: publishers/{publisher}/books/{book} string name = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { type: "library.googleapis.com/Book" }]; } ``` - A resource name field **must** be included. It **should** be called `name`. - The field **should** be [annotated as required][aip-203]. - The field **must** identify the [resource type][aip-123] that it references. - The comment for the `name` field **should** document the resource pattern. - The request message **must not** contain any other required fields, and **should not** contain other optional fields except those described in another AIP. **Note:** The `name` field in the request object corresponds to the `name` variable in the `google.api.http` annotation on the RPC. This causes the `name` field in the request to be populated based on the value in the URL when the REST/JSON interface is used. ### Errors See [errors][], in particular [when to use PERMISSION_DENIED and NOT_FOUND errors][permission-denied]. [aip-121]: ./0121.md [aip-123]: ./0123.md [aip-157]: ./0157.md [aip-203]: ./0203.md [errors]: ./0193.md [permission-denied]: ./0193.md#permission-denied ## Changelog - **2023-03-17**: Align with AIP-122 and make Get a must. - **2022-11-04**: Aggregated error guidance to AIP-193. - **2022-06-02**: Changed suffix descriptions to eliminate superfluous "-". - **2020-06-08**: Added guidance on returning the full resource. - **2019-10-18**: Added guidance on annotations. - **2019-08-12**: Added guidance for error cases. - **2019-08-01**: Changed the examples from "shelves" to "publishers", to present a better example of resource ownership. - **2019-05-29**: Added an explicit prohibition on arbitrary fields in standard methods. --- id: 132 state: approved created: 2019-01-21 updated: 2022-06-02 placement: category: operations order: 20 --- # Standard methods: List In many APIs, it is customary to make a `GET` request to a collection's URI (for example, `/v1/publishers/1/books`) in order to retrieve a list of resources, each of which lives within that collection. Resource-oriented design ([AIP-121][]) honors this pattern through the `List` method. These RPCs accept the parent collection (and potentially some other parameters), and return a list of responses matching that input. ## Guidance APIs **must** provide a `List` method for resources unless the resource is a [singleton][]. The purpose of the `List` method is to return data from a finite collection (generally singular unless the operation supports [reading across collections][]). List methods are specified using the following pattern: ```proto rpc ListBooks(ListBooksRequest) returns (ListBooksResponse) { option (google.api.http) = { get: "/v1/{parent=publishers/*}/books" }; option (google.api.method_signature) = "parent"; } ``` - The RPC's name **must** begin with the word `List`. The remainder of the RPC name **should** be the plural form of the resource being listed. - The request and response messages **must** match the RPC name, with `Request` and `Response` suffixes. - The HTTP verb **must** be `GET`. - The collection whose resources are being listed **should** map to the URI path. - The collection's parent resource **should** be called `parent`, and **should** be the only variable in the URI path. All remaining parameters **should** map to URI query parameters. - The collection identifier (`books` in the above example) **must** be a literal string. - The `body` key in the `google.api.http` annotation **must** be omitted. - If the resource being listed is not a top-level resource, there **should** be exactly one `google.api.method_signature` annotation, with a value of `"parent"`. If the resource being listed is a top-level resource, there **should** be either no `google.api.method_signature` annotation, or exactly one `google.api.method_signature` annotation, with a value of `""`. ### Request message List methods implement a common request message pattern: ```proto message ListBooksRequest { // The parent, which owns this collection of books. // Format: publishers/{publisher} string parent = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { child_type: "library.googleapis.com/Book" }]; // The maximum number of books to return. The service may return fewer than // this value. // If unspecified, at most 50 books will be returned. // The maximum value is 1000; values above 1000 will be coerced to 1000. int32 page_size = 2; // A page token, received from a previous `ListBooks` call. // Provide this to retrieve the subsequent page. // // When paginating, all other parameters provided to `ListBooks` must match // the call that provided the page token. string page_token = 3; } ``` - A `parent` field **must** be included unless the resource being listed is a top-level resource. It **should** be called `parent`. - The field **should** be [annotated as required][aip-203]. - The field **must** identify the [resource type][aip-123] of the resource being listed. - The `page_size` and `page_token` fields, which support pagination, **must** be specified on all list request messages. For more information, see [AIP-158][]. - The comment above the `page_size` field **should** document the maximum allowed value, as well as the default value if the field is omitted (or set to `0`). If preferred, the API **may** state that the server will use a sensible default. This default **may** change over time. - If a user provides a value greater than the maximum allowed value, the API **should** coerce the value to the maximum allowed. - If a user provides a negative or other invalid value, the API **must** send an `INVALID_ARGUMENT` error. - The `page_token` field **must** be included on all list request messages. - The request message **may** include fields for common design patterns relevant to list methods, such as `string filter` and `string order_by`. - The request message **must not** contain any other required fields, and **should not** contain other optional fields except those described in this or another AIP. **Note:** List methods **should** return the same results for any user that has permission to make a successful List request on the collection. Search methods are more relaxed on this. ### Response message List methods implement a common response message pattern: ```proto message ListBooksResponse { // The books from the specified publisher. repeated Book books = 1; // A token, which can be sent as `page_token` to retrieve the next page. // If this field is omitted, there are no subsequent pages. string next_page_token = 2; } ``` - The response message **must** include one repeated field corresponding to the resources being returned, and **should not** include any other repeated fields unless described in another AIP (for example, AIP-217). - The response **should** usually include fully-populated resources unless there is a reason to return a partial response (see AIP-157). - The `next_page_token` field, which supports pagination, **must** be included on all list response messages. It **must** be set if there are subsequent pages, and **must not** be set if the response represents the final page. For more information, see AIP-158. - The message **may** include a `int32 total_size` (or `int64 total_size`) field with the number of items in the collection. - The value **may** be an estimate (the field **should** clearly document this if so). - If filtering is used, the `total_size` field **should** reflect the size of the collection _after_ the filter is applied. ### Ordering `List` methods **may** allow clients to specify sorting order; if they do, the request message **should** contain a `string order_by` field. - Values **should** be a comma separated list of fields. For example: `"foo,bar"`. - The default sorting order is ascending. To specify descending order for a field, users append a `" desc"` suffix; for example: `"foo desc, bar"`. - Redundant space characters in the syntax are insignificant. `"foo, bar desc"`, `" foo , bar desc "`, and `"foo,bar desc"` are all equivalent. - Subfields are specified with a `.` character, such as `foo.bar` or `address.street`. - The resulting list order **should** be based on the field type's natural comparator e.g. numerics ordered numerically, strings ordered lexicographically, etc. However, APIs **may** choose to use a different ordering; if so, it **must** be documented in the `order_by` definition. - Furthermore, [well-known][] types, like `Timestamp` and `Duration` are compared as their representative type; `Timestamp` is compared as time e.g. before or after, `Duration` is compared as a quantity e.g. more or less. **Note:** Only include ordering if there is an established need to do so. It is always possible to add ordering later, but removing it is a breaking change. ### Filtering List methods **may** allow clients to specify filters; if they do, the request message **should** contain a `string filter` field. Filtering is described in more detail in AIP-160. **Note:** Only include filtering if there is an established need to do so. It is always possible to add filtering later, but removing it is a breaking change. ### Soft-deleted resources Some APIs need to "[soft delete][]" resources, marking them as deleted or pending deletion (and optionally purging them later). APIs that do this **should not** include deleted resources by default in list requests. APIs with soft deletion of a resource **should** include a `bool show_deleted` field in the list request that, if set, will cause soft-deleted resources to be included. ### Errors See [errors][], in particular [when to use PERMISSION_DENIED and NOT_FOUND errors][permission-denied]. ## Further reading - For details on pagination, see [AIP-158](./0158.md). - For listing across multiple parent collections, see [AIP-159](./0159.md). [aip-121]: ./0121.md [aip-123]: ./0123.md [aip-157]: ./0157.md [aip-158]: ./0158.md [aip-203]: ./0203.md [errors]: ./0193.md [permission-denied]: ./0193.md#permission-denied [reading across collections]: ./0159.md [singleton]: ./0156.md [soft delete]: ./0135.md#soft-delete [well-known]: https://protobuf.dev/reference/protobuf/google.protobuf ## Changelog - **2025-02-25**: Require documentation for ordering not matching field type with clarification on ordering of well-known types. - **2023-03-22**: Fix guidance wording to mention AIP-159. - **2023-03-17**: Align with AIP-122 and make Get a must. - **2022-11-04**: Aggregated error guidance to AIP-193. - **2022-06-02**: Changed suffix descriptions to eliminate superfluous "-". - **2020-09-02**: Add link to the filtering AIP. - **2020-08-14**: Added error guidance for permission denied cases. - **2020-06-08**: Added guidance on returning the full resource. - **2020-05-19**: Removed requirement to document ordering behavior. - **2020-04-15**: Added guidance on List permissions. - **2019-10-18**: Added guidance on annotations. - **2019-08-01**: Changed the examples from "shelves" to "publishers", to present a better example of resource ownership. - **2019-07-30**: Added guidance about documenting the ordering behavior. - **2019-05-29**: Added an explicit prohibition on arbitrary fields in standard methods. --- id: 133 state: approved created: 2019-01-23 updated: 2022-06-02 placement: category: operations order: 30 --- # Standard methods: Create In REST APIs, it is customary to make a `POST` request to a collection's URI (for example, `/v1/publishers/{publisher}/books`) in order to create a new resource within that collection. Resource-oriented design ([AIP-121][]) honors this pattern through the `Create` method. These RPCs accept the parent collection and the resource to create (and potentially some other parameters), and return the created resource. ## Guidance APIs **should** generally provide a create method for resources unless it is not valuable for users to do so. The purpose of the create method is to create a new resource in an already-existing collection. Create methods are specified using the following pattern: ```proto rpc CreateBook(CreateBookRequest) returns (Book) { option (google.api.http) = { post: "/v1/{parent=publishers/*}/books" body: "book" }; option (google.api.method_signature) = "parent,book"; } ``` - The RPC's name **must** begin with the word `Create`. The remainder of the RPC name **should** be the singular form of the resource being created. - The request message **must** match the RPC name, with a `Request` suffix. - The response message **must** be the resource itself. There is no `CreateBookResponse`. - The response **should** include the fully-populated resource, and **must** include any fields that were provided unless they are input only (see [AIP-203][]) or there is a reason to return a partial response (see [AIP-157][]). - If the create RPC is [long-running](#long-running-create), the response message **must** be a `google.longrunning.Operation` which resolves to the resource itself. - The HTTP verb **must** be `POST`. - The collection where the resource is being added **should** map to the URI path. - The collection's parent resource **should** be called `parent`, and **should** be the only variable in the URI path. - The collection identifier (`books` in the above example) **must** be a literal string. - There **must** be a `body` key in the `google.api.http` annotation, and it **must** map to the resource field in the request message. - All remaining fields **should** map to URI query parameters. - There **should** be exactly one `google.api.method_signature` annotation, with a value of `"parent,{resource},{resource}_id"`, or "`"parent,{resource}"` if the resource ID is not required. - If the API is operating on the [management plane][], the operation should have [strong consistency][]: the completion of a create operation **must** mean that all user-settable values and the existence of the resource have reached a steady-state and reading resource state returns a consistent response. ### Request message Create methods implement a common request message pattern: ```proto message CreateBookRequest { // The parent resource where this book will be created. // Format: publishers/{publisher} string parent = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { child_type: "library.googleapis.com/Book" }]; // The ID to use for the book, which will become the final component of // the book's resource name. // // This value should be 4-63 characters, and valid characters // are /[a-z][0-9]-/. string book_id = 2 [(google.api.field_behavior) = REQUIRED]; // The book to create. Book book = 3 [(google.api.field_behavior) = REQUIRED]; } ``` - A `parent` field **must** be included unless the resource being created is a top-level resource. It **should** be called `parent`. - The field **should** be [annotated as required][aip-203]. - The field **must** identify the [resource type][aip-123] of the resource being created. - A `{resource}_id` field **must** be included for management plane resources, and **should** be included for data plane resources. - The resource field **must** be included and **must** map to the POST body. - The request message **must not** contain any other required fields and **should not** contain other optional fields except those described in this or another AIP. ### Long-running create Some resources take longer to create a resource than is reasonable for a regular API request. In this situation, the API **should** use a long-running operation (AIP-151) instead: ```proto rpc CreateBook(CreateBookRequest) returns (google.longrunning.Operation) { option (google.api.http) = { post: "/v1/{parent=publishers/*}/books" body: "book" }; option (google.longrunning.operation_info) = { response_type: "Book" metadata_type: "OperationMetadata" }; } ``` - The response type **must** be set to the resource (what the return type would be if the RPC was not long-running). - Both the `response_type` and `metadata_type` fields **must** be specified. **Important:** Declarative-friendly resources (AIP-128) **should** use long-running operations. The service **may** return an LRO that is already set to done if the request is effectively immediate. ### User-specified IDs An API **must** allow a user to specify the ID component of a resource (the last segment of the resource name) on creation if the API is operating on the [management plane][]. On the [data plane][], an API **should** allow a user to specify the ID. Exceptional cases should have the following behavior: - The data plane resource allows identical records without a need to disambiguate between the two (e.g. rows in a table with no primary key). - The data plane resource will not be exposed in [Declarative clients][]. An API **may** allow the `{resource}_id` field have the [field_behavior][] `OPTIONAL`, and generate a system-generated ID if one is not specified. For example: ``` // Using user-specified IDs. publishers/lacroix/books/les-miserables // Using system-generated IDs. publishers/012345678-abcd-cdef/books/12341234-5678-abcd ``` - The `{resource}_id` field **must** exist on the request message, not the resource itself. - The field **may** be required or optional. If it is required, it **should** include the corresponding annotation. - The `name` field on the resource **must** be ignored. - There **should** be exactly one `google.api.method_signature` annotation on the RPC, with a value of `"parent,{resource},{resource}_id"` if the resource being created is not a top-level resource, or with a value of `"{resource},{resource}_id"` if the resource being created is a top-level resource. - The documentation **should** explain what the acceptable format is, and the format **should** follow the guidance for resource name formatting in [AIP-122][]. - If a user tries to create a resource with an ID that would result in a duplicate resource name, the service **must** error with `ALREADY_EXISTS`. - However, if the user making the call does not have permission to see the duplicate resource, the service **must** error with `PERMISSION_DENIED` instead. **Note:** For REST APIs, the user-specified ID field, `{resource}_id`, is provided as a query parameters on the request URI. ### Errors See [errors][], in particular [when to use PERMISSION_DENIED and NOT_FOUND errors][permission-denied]. ## Further reading - For ensuring idempotency in `Create` methods, see [AIP-155][]. - For naming resources involving Unicode, see [AIP-210][]. ## Rationale ### Requiring user-specified ids [Declarative clients][] use the resource ID as a way to identify a resource for applying updates and for conflict resolution. The lack of a user-specified ID means a client is unable to find the resource unless they store the identifier locally, and can result in re-creating the resource. This in turn has a downstream effect on all resources that reference it, forcing them to update to the ID of the newly-created resource. Having a user-specified ID also means the client can precalculate the resource name and use it in references from other resources. [aip-121]: ./0121.md [aip-122]: ./0122.md [aip-123]: ./0123.md [aip-155]: ./0155.md [aip-157]: ./0157.md [aip-203]: ./0203.md [aip-210]: ./0210.md [data plane]: ./0111.md#data-plane [management plane]: ./0111.md#management-plane [errors]: ./0193.md [field_behavior]: ./0203.md [Declarative clients]: ./0009.md#declarative-clients [permission-denied]: ./0193.md#permission-denied [strong consistency]: ./0121.md#strong-consistency ## Changelog - **2025-10-03**: Allow use of partial responses via AIP-157. - **2023-10-20**: Clarify that {resource}_id is only required for management plane resources. - **2023-08-24**: Adding consistency requirement. - **2023-05-11**: Changing guidance around resource_id to a must. - **2022-11-04**: Referencing aggregated error guidance in AIP-193, similar to other CRUDL AIPs. - **2022-06-02**: Changed suffix descriptions to eliminate superfluous "-". - **2020-10-06**: Added declarative-friendly guidance. - **2020-08-14**: Updated error guidance to use permission denied over forbidden. - **2020-06-08**: Added guidance on returning the full resource. - **2019-11-22**: Added clarification on what error to use if a duplicate name is sent. - **2019-10-18**: Added guidance on annotations. - **2019-08-01**: Changed the examples from "shelves" to "publishers", to present a better example of resource ownership. - **2019-06-10**: Added guidance for long-running create. - **2019-05-29**: Added an explicit prohibition on arbitrary fields in standard methods. --- id: 134 state: approved created: 2019-01-24 updated: 2022-06-02 placement: category: operations order: 40 --- # Standard methods: Update In REST APIs, it is customary to make a `PATCH` or `PUT` request to a resource's URI (for example, `/v1/publishers/{publisher}/books/{book}`) in order to update that resource. Resource-oriented design ([AIP-121][]) honors this pattern through the `Update` method (which mirrors the REST `PATCH` behavior). These RPCs accept the URI representing that resource and return the resource. ## Guidance APIs **should** generally provide an update method for resources unless it is not valuable for users to do so. The purpose of the update method is to make changes to the resources without causing side effects. Update methods are specified using the following pattern: ```proto rpc UpdateBook(UpdateBookRequest) returns (Book) { option (google.api.http) = { patch: "/v1/{book.name=publishers/*/books/*}" body: "book" }; option (google.api.method_signature) = "book,update_mask"; } ``` - The RPC's name **must** begin with the word `Update`. The remainder of the RPC name **should** be the singular form of the resource's message name. - The request message **must** match the RPC name, with a `Request` suffix. - The response message **must** be the resource itself. (There is no `UpdateBookResponse`.) - The response **should** include the fully-populated resource, and **must** include any fields that were sent and included in the update mask unless they are input only (see AIP-203) or there is a reason to return a partial response (see [AIP-157][]). - If the update RPC is [long-running](#long-running-update), the response message **must** be a `google.longrunning.Operation` which resolves to the resource itself. - The method **should** support partial resource update, and the HTTP verb **should** be `PATCH`. - If the method will only ever support full resource replacement, then the HTTP verb **may** be `PUT`. However, this is strongly discouraged because it becomes a backwards-incompatible change to add fields to the resource. - The resource's `name` field **should** map to the URI path. - The `{resource}.name` field **should** be the only variable in the URI path. - There **must** be a `body` key in the `google.api.http` annotation, and it **must** map to the resource field in the request message. - All remaining fields **should** map to URI query parameters. - There **should** be exactly one `google.api.method_signature` annotation, with a value of `"{resource},update_mask"`. - If the API is operating on the [management plane][], the operation should have [strong consistency][]: the completion of an update operation **must** mean that all user-settable values and the existence of the resource have reached a steady-state and reading resource state returns a consistent response. **Note:** Unlike the other four standard methods, the URI path here references a nested field (`book.name`) in the example. If the resource field has a word separator, `snake_case` is used. ### Request message Update methods implement a common request message pattern: ```proto message UpdateBookRequest { // The book to update. // // The book's `name` field is used to identify the book to update. // Format: publishers/{publisher}/books/{book} Book book = 1 [(google.api.field_behavior) = REQUIRED]; // The list of fields to update. google.protobuf.FieldMask update_mask = 2; } ``` - The request message **must** contain a field for the resource. - The field **must** map to the `PATCH` body. - The field **should** be [annotated as required][aip-203]. - A `name` field **must** be included in the resource message. It **should** be called `name`. - The field **must** identify the [resource type][aip-123] of the resource being updated. - If partial resource update is supported, a field mask **must** be included. It **must** be of type `google.protobuf.FieldMask`, and it **must** be called `update_mask`. - The fields used in the field mask correspond to the resource being updated (not the request message). - The field **must** be optional, and the service **must** treat an omitted field mask as an implied field mask equivalent to all fields that are populated (have a non-empty value). - Update masks **must** support a special value `*`, meaning full replacement (the equivalent of `PUT`). - API producers need to be conscious of how adding new, mutable fields to a resource will be handled when consumers use `*` without knowledge of said new, mutable fields. Likewise consumers need to use `*` only when the risks of doing so are acceptable. In general, it is safest to explicitly specify the fields to update rather than use `*`. - The request message **must not** contain any other required fields, and **should not** contain other optional fields except those described in this or another AIP. ### Side effects In general, update methods are intended to update the data within the resource. Update methods **should not** trigger other side effects. Instead, side effects **should** be triggered by custom methods. In particular, this entails that [state fields][] **must not** be directly writable in update methods. ### PATCH and PUT **TL;DR:** Google APIs generally use the `PATCH` HTTP verb only, and do not support `PUT` requests. We standardize on `PATCH` because Google updates stable APIs in place with backwards-compatible improvements. It is often necessary to add a new field to an existing resource, but this becomes a breaking change when using `PUT`. To illustrate this, consider a `PUT` request to a `Book` resource: PUT /v1/publishers/123/books/456 {"title": "Mary Poppins", "author": "P.L. Travers"} Next consider that the resource is later augmented with a new field (here we add `rating`): ```proto message Book { string title = 1; string author = 2; // Subsequently added to v1 in place... int32 rating = 3; } ``` If a rating is set on a book and the existing `PUT` request was executed, it would wipe out the book's rating. In essence, a `PUT` request unintentionally wiped out data because the previous version did not know about it. ### Long-running update Some resources take longer to update a resource than is reasonable for a regular API request. In this situation, the API **should** use a long-running operation ([AIP-151][]) instead: ```proto rpc UpdateBook(UpdateBookRequest) returns (google.longrunning.Operation) { option (google.api.http) = { patch: "/v1/{book.name=publishers/*/books/*}" body: "book" }; option (google.longrunning.operation_info) = { response_type: "Book" metadata_type: "OperationMetadata" }; } ``` - The response type **must** be set to the resource (what the return type would be if the RPC was not long-running). - Both the `response_type` and `metadata_type` fields **must** be specified. **Note:** Declarative-friendly resources ([AIP-128][]) **should** use long-running update. ### Create or update If the service uses client-assigned resource names, `Update` methods **may** expose a `bool allow_missing` field, which will cause the method to succeed in the event that the user attempts to update a resource that is not present (and will create the resource in the process): ```proto message UpdateBookRequest { // The book to update. // // The book's `name` field is used to identify the book to be updated. // Format: publishers/{publisher}/books/{book} Book book = 1 [(google.api.field_behavior) = REQUIRED]; // The list of fields to be updated. google.protobuf.FieldMask update_mask = 2; // If set to true, and the book is not found, a new book will be created. // In this situation, `update_mask` is ignored. bool allow_missing = 3; } ``` More specifically, the `allow_missing` flag triggers the following behavior: - If the method call is on a resource that does not exist, the resource is created. All fields are applied regardless of any provided field mask. - However, if any required fields are missing or fields have invalid values, an `INVALID_ARGUMENT` error is returned. - If the method call is on a resource that already exists, and all fields match, the existing resource is returned unchanged. - If the method call is on a resource that already exists, only fields declared in the field mask are updated. The user **must** have the update permissions to call `Update` even with `allow_missing` set to `true`. For customers that want to prevent users from creating resources using the update method, IAM conditions **should** be used. ### Etags An API may sometimes need to allow users to send update requests which are guaranteed to be made against the most current data (a common use case for this is to detect and avoid race conditions). Resources which need to enable this do so by including a `string etag` field, which contains an opaque, server-computed value representing the content of the resource. In this situation, the resource **should** contain a `string etag` field: ```proto message Book { option (google.api.resource) = { type: "library.googleapis.com/Book" pattern: "publishers/{publisher}/books/{book}" }; // The resource name of the book. // Format: publishers/{publisher}/books/{book} string name = 1 [(google.api.field_behavior) = IDENTIFIER]; // The title of the book. // Example: "Mary Poppins" string title = 2; // The author of the book. // Example: "P.L. Travers" string author = 3; // The etag for this book. // If this is provided on update, it must match the server's etag. string etag = 4; } ``` The `etag` field **may** be either [required][] or [optional][]. If it is set, then the request **must** succeed if and only if the provided etag matches the server-computed value, and **must** fail with an `ABORTED` error otherwise. The `update_mask` field in the request does not affect the behavior of the `etag` field, as it is not a field _being_ updated. ### Expensive fields APIs sometimes encounter situations where some fields on a resource are expensive or impossible to reliably return. This can happen in a few situations: - A resource may have some fields that are very expensive to compute, and that are generally not useful to the customer on update requests. - A single resource sometimes represents an amalgamation of data from multiple underlying (and eventually consistent) data sources. In these situations, it is impossible to return authoritative information on the fields that were not changed. In this situation, an API **may** return back only the fields that were updated, and omit the rest, and **should** document this behavior if they do so. ### Errors See [errors][], in particular [when to use PERMISSION_DENIED and NOT_FOUND errors][permission-denied]. In addition, if the user does have proper permission, but the requested resource does not exist, the service **must** error with `NOT_FOUND` (HTTP 404) unless `allow_missing` is set to `true`. [aip-121]: ./0121.md [aip-128]: ./0128.md [aip-151]: ./0151.md [aip-157]: ./0157.md [aip-203]: ./0203.md [errors]: ./0193.md [management plane]: ./0111.md#management-plane [permission-denied]: ./0193.md#permission-denied [state fields]: ./0216.md [strong consistency]: ./0121.md#strong-consistency [required]: ./0203.md#required [optional]: ./0203.md#optional ## Changelog - **2025-10-03**: Allow use of partial responses via AIP-157. - **2024-12-03**: Add caveats to usage of full replacement `update_mask`. - **2024-03-14**: Make `update_mask` optional field_behaviour guidance a **must**. - **2023-08-26**: Adding consistency requirement. - **2023-07-17**: Make `update_mask` name guidance a **must**. - **2022-11-04**: Aggregated error guidance to AIP-193. - **2022-06-02**: Changed suffix descriptions to eliminate superfluous "-". - **2021-11-04**: Changed the permission check if `allow_missing` is set. - **2021-07-08**: Added error guidance for resource not found case. - **2021-03-05**: Changed the etag error from `FAILED_PRECONDITION` (which becomes HTTP 400) to `ABORTED` (409). - **2020-10-06**: Added guidance for declarative-friendly resources. - **2020-10-06**: Added guidance for `allow_missing`. - **2020-08-14**: Added error guidance for permission denied cases. - **2020-06-08**: Added guidance on returning the full resource. - **2019-10-18**: Added guidance on annotations. - **2019-09-10**: Added a link to the long-running operations AIP ([AIP-151][]). - **2019-08-01**: Changed the examples from "shelves" to "publishers", to present a better example of resource ownership. - **2019-06-10**: Added guidance for long-running update. - **2019-05-29**: Added an explicit prohibition on arbitrary fields in standard methods. --- id: 135 state: approved created: 2019-01-24 updated: 2022-06-02 placement: category: operations order: 50 --- # Standard methods: Delete In REST APIs, it is customary to make a `DELETE` request to a resource's URI (for example, `/v1/publishers/{publisher}/books/{book}`) in order to delete that resource. Resource-oriented design (AIP-121) honors this pattern through the `Delete` method. These RPCs accept the URI representing that resource and usually return an empty response. ## Guidance APIs **should** generally provide a delete method for resources unless it is not valuable for users to do so. Delete methods are specified using the following pattern: ```proto rpc DeleteBook(DeleteBookRequest) returns (google.protobuf.Empty) { option (google.api.http) = { delete: "/v1/{name=publishers/*/books/*}" }; option (google.api.method_signature) = "name"; } ``` - The RPC's name **must** begin with the word `Delete`. The remainder of the RPC name **should** be the singular form of the resource's message name. - The request message **must** match the RPC name, with a `Request` suffix. - The response message **should** be `google.protobuf.Empty`. - If the resource is [soft deleted](#soft-delete), the response message **should** be the resource itself. - If the delete RPC is [long-running](#long-running-delete), the response message **must** be a `google.longrunning.Operation` which resolves to the correct response. - The HTTP verb **must** be `DELETE`. - The request message field receiving the resource name **should** map to the URI path. - This field **should** be called `name`. - The `name` field **should** be the only variable in the URI path. All remaining parameters **should** map to URI query parameters. - There **must not** be a `body` key in the `google.api.http` annotation. - There **should** be exactly one `google.api.method_signature` annotation, with a value of `"name"`. If an etag or force field are used, they **may** be included in the signature. - If the API is operating on the [Management Plane][], the operation should have [strong consistency][]: the completion of a delete operation **must** mean that the existence of the resource has reached a steady-state and reading resource state returns a consistent response. - The API **must** fail with a `FAILED_PRECONDITION` error if child resources are present. See guidance on [Cascading Delete](#cascading-delete) if forcing deletion of parent and child resources is necessary. - If the only child resource type is a [Singleton][aip-156], deletion **must** be allowed, because the lifecycle of a Singleton is tied to that of its parent resource. This applies even if there are multiple different Singleton resource types for the same parent resource. The Delete method **should** succeed if and only if a resource was present and was successfully deleted. If the resource did not exist, the method **should** send a `NOT_FOUND` error. ### Request message Delete methods implement a common request message pattern: ```proto message DeleteBookRequest { // The name of the book to delete. // Format: publishers/{publisher}/books/{book} string name = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { type: "library.googleapis.com/Book" }]; } ``` - A `name` field **must** be included. It **should** be called `name`. - The field **should** be [annotated as required][aip-203]. - The field **must** identify the [resource type][aip-123] that it references. - The comment for the field **should** document the resource pattern. - The request message **must not** contain any other required fields, and **should not** contain other optional fields except those described in this or another AIP. ### Soft delete **Note:** This material was moved into its own document to provide a more comprehensive treatment: AIP-164. ### Long-running delete Some resources take longer to delete a resource than is reasonable for a regular API request. In this situation, the API **should** use a long-running operation instead: ```proto rpc DeleteBook(DeleteBookRequest) returns (google.longrunning.Operation) { option (google.api.http) = { delete: "/v1/{name=publishers/*/books/*}" }; option (google.longrunning.operation_info) = { response_type: "google.protobuf.Empty" metadata_type: "OperationMetadata" }; } ``` - The response type **must** be set to the appropriate return type if the RPC was not long-running: `google.protobuf.Empty` for most Delete RPCs, or the resource itself for soft delete (AIP-164). - Both the `response_type` and `metadata_type` fields **must** be specified (even if they are `google.protobuf.Empty`). ### Cascading delete Sometimes, it may be necessary for users to be able to delete a resource as well as all applicable child resources. However, since deletion is usually permanent, it is also important that users not do so accidentally, as reconstructing wiped-out child resources may be quite difficult. If an API allows deletion of a resource that may have child resources, the API **should** provide a `bool force` field on the request, which the user sets to explicitly opt in to a cascading delete. ```proto message DeletePublisherRequest { // The name of the publisher to delete. // Format: publishers/{publisher} string name = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { type: "library.googleapis.com/Publisher" }]; // If set to true, any books from this publisher will also be deleted. // (Otherwise, the request will only work if the publisher has no books.) bool force = 2; } ``` The API **must** fail with a `FAILED_PRECONDITION` error if the `force` field is `false` (or unset) and child resources are present. ### Protected delete Sometimes, it may be necessary for users to ensure that no changes have been made to a resource that is being deleted. If a resource provides an [etag][], the delete request **may** accept the etag (as either required or optional): ```proto message DeleteBookRequest { // The name of the book to delete. // Format: publishers/{publisher}/books/{book} string name = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { type: "library.googleapis.com/Book" }]; // Optional. The etag of the book. // If this is provided, it must match the server's etag. string etag = 2; } ``` If the etag is provided and does not match the server-computed etag, the request **must** fail with a `ABORTED` error code. **Note:** Declarative-friendly resources (AIP-128) **must** provide the `etag` field for Delete requests. ### Delete if existing If the service uses client-assigned resource names, `Delete` methods **may** expose a `bool allow_missing` field, which will cause the method to succeed in the event that the user attempts to delete a resource that is not present (in which case the request is a no-op): ```proto message DeleteBookRequest { // The book to delete. // Format: publishers/{publisher}/books/{book} string name = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference).type = "library.googleapis.com/Book" ]; // If set to true, and the book is not found, the request will succeed // but no action will be taken on the server bool allow_missing = 2; } ``` More specifically, the `allow_missing` flag triggers the following behavior: - If the method call is on a resource that does not exist, the request is a no-op. - The `etag` field is ignored. - If the method call is on a resource that already exists, the resource is deleted (subject to other checks). **Note:** Declarative-friendly resources (AIP-128) **should** expose the `bool allow_missing` field. ### Errors If the user does not have permission to access the resource, regardless of whether or not it exists, the service **must** error with `PERMISSION_DENIED` (HTTP 403). Permission **must** be checked prior to checking if the resource exists. If the user does have proper permission, but the requested resource does not exist, the service **must** error with `NOT_FOUND` (HTTP 404) unless `allow_missing` is set to `true`. [aip-123]: ./0123.md [aip-131]: ./0131.md [aip-132]: ./0132.md [aip-136]: ./0136.md [aip-154]: ./0154.md [aip-156]: ./0156.md [aip-203]: ./0203.md [aip-214]: ./0214.md [aip-216]: ./0216.md [management plane]: ./0111.md#management-plane [strong consistency]: ./0121.md#strong-consistency [etag]: ./0134.md#etags ## Further reading - For soft delete and undelete, see AIP-164. - For bulk deleting large numbers of resources based on a filter, see AIP-165. ## Changelog - **2024-06-11**: Add deletion behavior for parent resource deletion requests without a `force` field. - **2023-08-24**: Adding consistency requirement. - **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-". - **2022-02-02**: Changed eTag error from `FAILED_PRECONDITION` to `ABORTED` making it consistent with change to [AIP-154][] & [AIP-134][etag] on 2021-03-05. - **2020-10-06**: Added guidance for declarative-friendly resources. - **2020-10-06**: Added guidance for allowing no-op delete for missing resources. - **2020-10-06**: Moved soft delete and undelete guidance into a new AIP-164. - **2020-06-08**: Added guidance for `Get` of soft-deleted resources. - **2020-02-03**: Added guidance for error cases. - **2019-10-18**: Added guidance on annotations. - **2019-08-01**: Changed the examples from "shelves" to "publishers", to present a better example of resource ownership. - **2019-06-10**: Added guidance for long-running delete. - **2019-05-29**: Added an explicit prohibition on arbitrary fields in standard methods. --- id: 136 state: approved created: 2019-01-25 updated: 2023-03-02 placement: category: operations order: 100 --- # Custom methods Resource-oriented design (AIP-121) uses custom methods to provide a means to express arbitrary actions that are difficult to model using only the standard methods. Custom methods are important because they provide a means for an API's vocabulary to adhere to user intent. ## Guidance Custom methods **should** only be used for functionality that can not be easily expressed via standard methods; prefer standard methods if possible, due to their consistent semantics. (Of course, this only applies if the functionality in question actually conforms to the normal semantics; it is _not_ a good idea to contort things to endeavor to make the standard methods "sort of work".) While custom methods vary widely in how they are designed, many principles apply consistently: ```proto // Archives the given book. rpc ArchiveBook(ArchiveBookRequest) returns (ArchiveBookResponse) { option (google.api.http) = { post: "/v1/{name=publishers/*/books/*}:archive" body: "*" }; } ``` **Note:** The pattern above shows a custom method that operates on a specific resource. Custom methods can be associated with resources, collections, or services. The bullets below apply in all three cases. - The name of the method **should** be a verb followed by a noun. - The name **must not** contain prepositions ("for", "with", etc.). - The verb in the name **should not** contain any of the standard method verbs ([Get][], [List][], [Create][], [Update][], [Delete][]). - The name **must not** include the term `Async`. Instead, if the intention is to differentiate between immediate and long-running RPCs, the suffix `LongRunning` **may** be used for this purpose. For example, to create a long-running book creation RPC (if the standard `CreateBook` method was designed before long-running aspects were considered), a custom `CreateBookLongRunning` method could be introduced. - The HTTP method **must** be `GET` or `POST`: - `GET` **must** be used for methods retrieving data or resource state. - `POST` **must** be used if the method has side effects or mutates resources or data. - The HTTP URI **must** use a `:` character followed by the custom verb (`:archive` in the above example), and the verb in the URI **must** match the verb in the name of the RPC. - If word separation is required, `camelCase` **must** be used. - The `body` clause in the `google.api.http` annotation **should** be `"*"`. - See [HTTP and gRPC Transcoding][transcoding] for more information. - Custom methods **should** take a request message matching the RPC name, with a `Request` suffix. - Custom methods **should** return a response message matching the RPC name, with a `Response` suffix. - When operating on a specific resource, a custom method **may** return the resource itself. ### Resource-based custom methods Custom methods **must** operate on a resource if the API can be modeled as such: ```proto // Archives the given book. rpc ArchiveBook(ArchiveBookRequest) returns (ArchiveBookResponse) { option (google.api.http) = { post: "/v1/{name=publishers/*/books/*}:archive" body: "*" }; } ``` - The parameter for the resource's name **must** be called `name`, and be the only variable in the URI path. ### Collection-based custom methods While most custom methods operate on a single resource, some custom methods **may** operate on a collection instead: ```proto // Sorts the books from this publisher. rpc SortBooks(SortBooksRequest) returns (SortBooksResponse) { option (google.api.http) = { post: "/v1/{parent=publishers/*}/books:sort" body: "*" }; } ``` - If the collection's resource has a parent, that resource **must** be called `parent` and be the only variable in the URI path. - The collection key (`books` in the above example) **must** be literal. ### Stateless methods Some custom methods are not attached to resources at all. These methods are generally _stateless_: they accept a request and return a response, and have no permanent effect on data within the API. ```proto // Translates the provided text from one language to another. rpc TranslateText(TranslateTextRequest) returns (TranslateTextResponse) { option (google.api.http) = { post: "/v1/{project=projects/*}:translateText" body: "*" }; } ``` - If the method runs in a particular scope (such as a project, as in the above example), the field name in the request message **should** be the name of the scope resource. If word separators are necessary, `snake_case` **must** be used. - The URI **should** place both the verb and noun after the `:` separator (avoid a "faux collection key" in the URI in this case, as there is no collection). For example, `:translateText` is preferable to `text:translate`. - Stateless methods **must** use `POST` if they involve billing. ### Declarative-friendly resources Declarative-friendly resources usually **should not** employ custom methods (except specific declarative-friendly custom methods discussed in other AIPs), because declarative-friendly tools are unable to automatically determine what to do with them. An exception to this is for rarely-used, fundamentally imperative operations, such as a `Move` or `Rename` operation, for which there would not be an expectation of declarative support. ## Rationale ### HTTP path Similar to standard methods, a custom method that operates on a resource or collection needs a `name` or `parent` parameter to indicate the resource that it operates on. This convention allows clients to map custom methods to the appropriate resource. ### HTTP methods Allowing both `GET` and `POST` HTTP verbs allows a clear distinction for which methods do not mutate data, and which ones do. Methods that only read data have first-class concepts in some clients (DataSources in Terraform) and clearly indicate to a user which methods can be called without risk of runtime impact. ### Disallowing prepositions Generally, method names with prepositions indicate that a new method is being used where a field should instead be added to an existing method, or the method should use a distinct verb. For example, if a `CreateBook` message already exists and you are considering adding `CreateBookFromDictation`, consider a `TranscribeBook` method instead. Similarly, if there is desire for a property-specific look-up method, instead of `GetBookByAuthor` consider a `SearchBooks` with an `author` field as a search dimension. This helps prevent an explosion of hyper-focused methods that bloat API and client surfaces, and add complexity to both managing and consuming the API. [get]: ./0131.md [list]: ./0132.md [create]: ./0133.md [update]: ./0134.md [delete]: ./0135.md [transcoding]: ./0127.md ### RPC name The term "async" is commonly used in programming languages to indicate whether a specific method call is synchronous or asynchronous, including for making RPCs. That sync/async aspect is at a different abstraction level to whether the RPC itself is intended to start a long-running operation. Using "async" within the RPC name itself causes confusion, and can even cause issues for client libraries which generate both synchronous and asynchronous methods to call the RPC in some languages. ## Changelog - **2025-05-12:** Extend disallowing prepositions rationale. - **2025-01-09:** Add original rationale for disallowing prepositions in names. - **2023-11-16:** Included link to AIP-127 "HTTP and gRPC Transcoding" for guidance on body definition. - **2023-05-16:** Added prohibition of the term "async" within RPC names. - **2023-05-09:** Adding guidance for POST and GET, require parent instead of the resource singular. - **2023-03-02:** Explicitly discourage use of standard method verbs. - **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-". - **2020-10-06:** Added declarative-friendly guidance. - **2019-08-01:** Changed the examples from "shelves" to "publishers", to present a better example of resource ownership. --- id: 140 state: approved created: 2019-07-22 placement: category: fields order: 0 --- # Field names Naming fields in a way that is intuitive to users can often be one of the most challenging aspects of designing an API. This is true for many reasons; often a field name that seems entirely intuitive to the author can baffle a reader. Additionally, users rarely use only one API; they use many APIs together. As a result, a single company using the same name to mean different things (or different names to mean the same thing) can often cause unnecessary confusion, because users can no longer take what they've already learned from one API and apply that to another. In short, APIs are easiest to understand when field names are simple, intuitive, and consistent with one another. ## Guidance Field names **should** be in correct American English. Field names **should** clearly and precisely communicate the concept being presented and avoid overly general names that are ambiguous. That said, field names **should** avoid including unnecessary words. In particular, avoid including adjectives that always apply and add little cognitive value. For example, a `proxy_settings` field might be as helpful as `shared_proxy_settings` if there is no unshared variant. **Important:** Field names often appear in generated client surfaces. Ensure they are appropriately descriptive and of suitable length. ### Case Field definitions in protobuf files **must** use `lower_snake_case` names. These names are mapped to an appropriate naming convention in JSON and in generated code. Additionally, each word in the field **must not** begin with a number, because it creates ambiguity when converting between snake case and camel case. Similarly, fields **must not** contain leading, trailing, or adjacent underscores. ### Uniformity APIs **should** endeavor to use the same name for the same concept and different names for different concepts wherever possible. This includes names across multiple APIs, in particular if those APIs are likely to be used together. ### Repeated fields Repeated fields **must** use the proper plural form, such as `books` or `authors`. On the other hand, non-repeated fields **should** use the singular form such as `book` or `author`. This implies that resource names **should** use the singular form as well, since the field name should follow the resource name (e.g., use `repeated Book books`, **not** `Books books = 1`). ### Prepositions Field names **should not** include prepositions (such as "with", "for", "at", "by", etc). For example: - `error_reason` (**not** `reason_for_error`) - `author` (**not** `written_by`) It is easier for field names to match more often when following this convention. Additionally, prepositions in field names may also indicate a design concern, such as an overly-restrictive field or a sub-optimal data type. This is particularly true regarding "with": a field named `book_with_publisher` likely indicates that the book resource may be improperly structured and worth redesigning. **Note:** The word "per" is an exception to this rule. See [AIP-141][] for guidance in the case where "per" is used as part of a unit (e.g. "miles per hour"). ### Abbreviations For well known name abbreviations among software developers, such as "config" and "spec", the abbreviations **should** be used in API definitions instead of the full spelling. This will make the source code easy to read and write. Examples: - `config` (**not** `configuration`) - `id` (**not** `identifier`) - `info` (**not** `information`) - `spec` (**not** `specification`) - `stats` (**not** `statistics`) Furthermore, well known abbreviations for units **should** be used in field names. See [AIP-141][] for more guidance on fields that represent quantities. Examples: - `distance_km` (**not** `distance_kilometers`) - `width_px` (**not** `width_pixels`) ### Adjectives For uniformity, field names that contain both a noun and an adjective **should** place the adjective _before_ the noun. For example: - `collected_items` (**not** `items_collected`) - `imported_objects` (**not** `objects_imported`) ### Verbs Field names **must not** be named to reflect an intent or action. They **must not** be verbs. Rather, because the field defines the *desired value* for mutations, e.g. Create and Update, and the *current value* for reads, e.g. Get and List, the name **must** be a noun. It defines what is so, not what to do. - `collected_items` (**not** `collect_items`) - `disabled` (**not** `disable`) In contrast, method names, whether standard or custom, change facets of resources and are named as verbs. ### Booleans Boolean fields **should** omit the prefix "is". For example: - `disabled` (**not** `is_disabled`) - `required` (**not** `is_required`) **Note:** Field names that would otherwise be [reserved words](#reserved-words) are an exception to this rule. For example, `is_new` (**not** `new`). ### String vs. bytes When using `bytes`, the contents of the field are base64-encoded when using JSON on the wire. Services **should** use `bytes` when there is a need to send binary contents over the wire, and **should not** ask the user to manually base64-encode a field into a `string` field. The one exception is when the API is designed to handle data that is meant to be base64-encoded at rest and the double base64-encoding as a result of using `bytes` is undesirable - in this case, services **may** use a `string`. ### URIs Field names representing arbitrary URIs **should** use `uri`. In particular, note that URLs are URIs but not all URIs are URLs. Field names **may** use a prefix in front of `uri` as appropriate. Field names that can only represent a URL **should** use `url`. ```proto message Book { string name = 1 [(google.api.field_behavior) = IDENTIFIER]; // A URL pointing to an image of the book. string image_url = 2; // A URI identifying the book. // This could be an ISBN or a URL. string uri = 3; } ``` **Note:** APIs that have previously used `uri` for URL fields may continue to do so to avoid unnecessary API changes and to preserve local consistency. ### Reserved words Field names **should** avoid using names that are likely to conflict with keywords in common programming languages, such as `new`, `class`, `function`, `import`, etc. Reserved keywords can cause hardship for developers using the API in that language. ### Conflicts Messages **should not** include a field with the same name as the enclosing message (ignoring case transformations). This causes conflicts when generating code in some languages. ### Display names Many resources have a human-readable name, often used for display in UI. This field **should** be called `display_name`, and **should not** have a uniqueness requirement. If an entity has an official, formal name (such as a company name or the title of a book), an API **may** use `title` as the field name instead. The `title` field **should not** have a uniqueness requirement. ## Further reading - For naming resource fields, see [AIP-122][]. - For naming fields representing quantities, see [AIP-141][]. - For naming fields representing time, see [AIP-142][]. ## Rationale ### URI vs URL The guidance itself mentions that all URLs are URIs, but not all URIs are URLs. The goal of aligning on URI is to enable a more generalizable field, that can handle a variety of use cases, as well as drive standardization across APIs, creating a more coherent surface. At the same time, the requirement being a **should** allows for more specific terms when it truly merits it. ## Changelog - **2025-10-23**: Call out AIP-141 unit abbreviations as preferred. - **2025-03-10**: Add rationale for URI guidance. - **2024-12-20**: Copy over abbreviations guidance from old design site. - **2024-08-26**: Codify exception to string and base64 guidance - **2024-05-18**: Documented the effect of field names on client surfaces. - **2023-04-25**: Field names **must not** be expressed as verbs. - **2021-07-12**: Normalized display name guidance to "should". - **2021-04-07**: Added base64 and bytes guidance. - **2021-03-05**: Added prohibition on leading, trailing, or adjacent underscores. - **2020-06-10**: Added prohibition on starting any word with a number. - **2020-05-29**: Added guidance around URIs. - **2020-03-24**: Added guidance around conflicting field and message names. - **2020-01-30**: Added guidance around `display_name` and `title`. [aip-122]: ./0122.md [aip-141]: ./0141.md [aip-142]: ./0142.md --- id: 141 state: approved created: 2019-07-18 placement: category: fields order: 20 --- # Quantities Many services need to represent a discrete quantity of items (number of bytes, number of miles, number of nodes, etc.). ## Guidance Quantities with a clear unit of measurement (such as bytes, miles, and so on) **must** include the unit of measurement as the suffix. When appropriate, units **should** use generally accepted abbreviations, and abbreviations **should not** be pluralized (for example, `distance_km` rather than `distance_kilometers`). ```proto // A representation of a non-stop air route. message Route { // The airport where the route begins. string origin = 1; // The destination airport. string destination = 2; // The distance between the origin and destination airports. // This value is also used to determine the credited frequent flyer miles. int32 distance_miles = 3; } ``` If the quantity is a number of items (for example, the number of nodes in a cluster), then the field **should** use the suffix `_count` (**not** the prefix `num_`): ```proto // A cluster of individual nodes. message Cluster { // The number of nodes in the cluster. int32 node_count = 1; } ``` **Note:** Fields **must not** use unsigned integer types, because many programming languages and systems do not support them well. ### Compound units Quantities with compound units of measurement **may** use separating underscores between units as needed for clarity. Unabbreviated units **must** be separated. Abbreviated units **should not** be separated unless otherwise ambiguous. Compound units **should** be in plural form, with all component units in singular form except for the final component unit, which should be in plural form unless abbreviated. - `energy_kwh` (**not** `energy_kw_h`) - `energy_kw_fortnights` (**not** `energy_kwfortnight` or `energy_kw_fortnight`) **Note:** Metric prefixes **must not** be separated from their base unit. ### Inverse units Quantities with units of measurement that are or include inverse units **should** indicate all inverse units as a compound unit after a compound of any non-inverse units, separated by the word "per". The inverse compound unit **should** be in singular form. - `speed_miles_per_hour` (**not** `speed_mph`) - `speed_meters_per_second` (**not** `speed_meters_per_seconds` or `speed_meter_per_second`) - `event_count_per_hour` (**not** `events_per_hour`, `event_counts_per_hour`, or `hourly_events`) - `price_per_kwh` (using [`google.type.Money`][money]) **Note:** This guidance does not apply in cases where generally accepted derived units with special names and symbols exist for inverse quantities. For example, the derived unit 'hertz' **should** be used when appropriate for reciprocal time. ### Specialized messages It is sometimes useful to create a message that represents a particular quantity. This is particularly valuable in two situations: - Grouping two or more individual quantities together. Example: [`google.protobuf.Duration`][duration] - Representing a common concept where the unit of measurement may itself vary. Example: [`google.type.Money`][money] APIs **may** create messages to represent quantities when appropriate. When using these messages as fields, APIs **should** use the name of the message as the suffix for the field name if it makes intuitive sense to do so. ## Changelog - **2025-07-09**: Added guidance for compound units. - **2025-07-09**: Clarified guidance to not pluralize abbreviated units. - **2025-07-09**: Clarified guidance to use per_ prefix to represent inverse units. - **2019-09-13**: Added the prohibition on uint and fixed types. [duration]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/duration.proto [money]: https://github.com/googleapis/googleapis/blob/master/google/type/money.proto --- id: 142 state: approved created: 2019-07-16 placement: category: fields order: 30 --- # Time and duration Many services need to represent the concepts surrounding time. Representing time can be challenging due to the intricacies of calendars and time zones, as well as the fact that common exchange formats (such as JSON) lack a native concept of time. ## Guidance Fields representing time **should** use the common, generally used components (such as [`google.protobuf.Timestamp`][timestamp] or [`google.type.Date`][date]) for representing time or duration types. These types are [common components][aip-213], and using them consistently allows infrastructure and tooling to provide a better experience when interacting with time values. ### Timestamps Fields that represent an absolute point in time (independent of any time zone or calendar) **should** use the [`google.protobuf.Timestamp`][timestamp] type, (which uses UNIX timestamps under the hood and hold nanosecond precision). These fields **should** have names ending in `_time`, such as `create_time` or `update_time`. For repeated fields, the names **should** end in `_times` instead. Many timestamp fields refer to an activity (for example, `create_time` refers to when the applicable resource was created). For these, the field **should** be named with the `{imperative}_time` form. For example, if a book is being published, the field storing the time when this happens would use the imperative form of the verb "to publish" ("publish") resulting in a field called `publish_time`. Fields **should not** be named using the past tense (such as `published_time`, `created_time` or `last_updated_time`). ### Durations Fields that represent a span between two points in time (independent of any time zone or calendar) **should** use the [`google.protobuf.Duration`][duration] type. To illustrate the distinction between timestamps and durations, consider a flight record: ```proto // A representation of a (very incomplete) flight log. message FlightRecord { // The absolute point in time when the plane took off. google.protobuf.Timestamp takeoff_time = 1; // The length (duration) of the flight, from takeoff to landing. google.protobuf.Duration flight_duration = 2; } ``` **Note:** Observant readers may notice that the [timestamp][] and [duration][] messages have the same structure (`int64 seconds` and `int32 nanos`). However, the distinction between these is important, because they have different semantic meaning. Additionally, tooling is able to base behavior off of which message is used. For example, a Python-based tool could convert timestamps to [datetime][py_datetime] objects and durations to [timedelta][py_timedelta] objects. ### Relative time segments In some cases, it may be necessary to represent a time segment inside a stream. In these cases, the [`google.protobuf.Duration`][duration] type **should** be used, and the field name **should** end with `_offset`. To ensure that the meaning is clear, the field **must** have a comment noting the point that the offset is relative to. To illustrate this, consider a resource representing a segment of an audio stream: ```proto message AudioSegment { // The duration relative to the start of the stream representing the // beginning of the segment. google.protobuf.Duration start_offset = 1; // The total length of the segment. google.protobuf.Duration segment_duration = 2; } ``` ### Civil dates and times Fields that represent a calendar date or wall-clock time **should** use the appropriate common components: - Civil date: [`google.type.Date`][date] - Wall-clock time: [`google.type.TimeOfDay`][time_of_day] Fields representing civil dates **should** have names ending in `_date`, while fields representing civil times or datetimes **should** have names ending in `_time`. **Note:** Both the `Date` and `TimeOfDay` components are timezone-naïve. Fields that require timezone-awareness **should** use `DateTime` (see below). ### Civil timestamps Fields that represent a civil timestamp (date and time, optionally with a time zone) **should** use the [`google.type.DateTime`][datetime] component, and the field name **should** end in `_time`. ### Compatibility Occasionally, APIs are unable to use the common structures for legacy or compatibility reasons. For example, an API may conform to a separate specification that mandates that timestamps be integers or [ISO-8601][] strings. In these situations, fields **may** use other types. If possible, the following naming conventions apply: - For integers, include the meaning (examples: `time`, `duration`, `delay`, `latency`) **and** the unit of measurement (valid values: `seconds`, `millis`, `micros`, `nanos`) as a final suffix. For example, `send_time_millis`. - For strings, include the meaning (examples: `time`, `duration`, `delay`, `latency`) but no unit suffix. In all cases, clearly document the expected format, and the rationale for its use. [aip-213]: ./0213.md [date]: https://github.com/googleapis/googleapis/blob/master/google/type/date.proto [datetime]: https://github.com/googleapis/googleapis/blob/master/google/type/datetime.proto [duration]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/duration.proto [iso-8601]: https://www.iso.org/iso-8601-date-and-time-format.html [py_datetime]: https://docs.python.org/3/library/datetime.html#datetime.datetime [py_timedelta]: https://docs.python.org/3/library/datetime.html#datetime.timedelta [time_of_day]: https://github.com/googleapis/googleapis/blob/master/google/type/timeofday.proto [timestamp]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/timestamp.proto --- id: 143 state: approved created: 2019-07-24 placement: category: fields order: 40 --- # Standardized codes Many common concepts, such as spoken languages, countries, currency, and so on, have common codes (usually formalized by the [International Organization for Standardization][iso]) that are used in data communication and processing. These codes address the issue that there are often different ways to express the same concept in written language (for example, "United States" and "USA", or "Español" and "Spanish"). ## Guidance For concepts where a standardized code exists and is in common use, fields representing these concepts **should** use the standardized code for both input and output. ```proto // A message representing a book. message Book { // Other fields... // The IETF BCP-47 language code representing the language in which // the book was originally written. // https://en.wikipedia.org/wiki/IETF_language_tag string language_code = 99; } ``` - Fields representing standardized concepts **must** use the appropriate data type for the standard code (usually `string`). - Fields representing standardized concepts **should not** use enums, even if they only allow a small subset of possible values. Using enums in this situation often leads to frustrating lookup tables when using multiple APIs together. - Fields representing standardized concepts **must** indicate which standard they follow, preferably with a link (either to the standard itself, the Wikipedia description, or something similar). - The field name **should** end in `_code` or `_type` unless the concept has an obviously clearer suffix. - When accepting values provided by users, validation **should** be case-insensitive unless this would introduce ambiguity (for example, accept both `en-gb` and `en-GB`). When providing values to users, APIs **should** use the canonical case (in the example above, `en-GB`). ### Content types Fields representing a content or media type **must** use [IANA media types][]. For legacy reasons, the field **should** be called `mime_type`. ### Countries and regions Fields representing individual countries or nations **must** use the [Unicode CLDR region codes][cldr] ([list][]), such as `US` or `CH`, and the field **must** be called `region_code`. **Important:** Please read the [rationale](#countryregion-field-naming) for this requirement. ### Currency Fields representing currency **must** use [ISO-4217 currency codes][iso-4217], such as `USD` or `CHF`, and the field **must** be called `currency_code`. **Note:** For representing an amount of money in a particular currency, rather than the currency code itself, use [`google.type.Money`][money]. ### Language Fields representing spoken languages **must** use [IETF BCP-47 language codes][bcp-47] ([list][]), such as `en-US` or `de-CH`, and the field **must** be called `language_code`. ### Time zones Fields representing a time zone **should** use the [IANA TZ][] codes, and the field **must** be called `time_zone`. Fields also **may** represent a UTC offset rather than a time zone (note that these are subtly different). In this case, the field **must** use the [ISO-8601 format][] to represent this, and the field **must** be named `utc_offset`. ## Rationale ### Country/region field naming The use of `region_code` instead of `country_code` is critical to being able to convey regions that are distinct from any country and to avoid any political disputes associated with said region regarding their sovereignty or affiliation. Google and many other companies are supporters of Unicode CLDR and standardize their product internationalization efforts on Unicode CLDR, and APIs are no different here. Furthermore, many of the values supported by Unicode CLDR are not countries on their own, so using a more generic name is actually more compatible with the specification. ## Changelog - **2024-12-03**: Strengthen rationale of country/region field naming - **2024-11-12**: Change country/region code list to CLDR list from IANA list - **2020-05-12**: Replaced `country_code` guidance with `region_code`, correcting an original error. [bcp-47]: https://en.wikipedia.org/wiki/IETF_language_tag [cldr]: http://cldr.unicode.org/ [iana media types]: https://www.iana.org/assignments/media-types/media-types.xhtml [iana tz]: http://www.iana.org/time-zones [iso]: https://www.iso.org/ [iso-4217]: https://en.wikipedia.org/wiki/ISO_4217 [iso-8601 format]: https://en.wikipedia.org/wiki/ISO_8601#Time_offsets_from_UTC [list]: https://www.unicode.org/cldr/charts/46/supplemental/territory_information.html [money]: https://github.com/googleapis/googleapis/blob/master/google/type/money.proto --- id: 144 state: approved created: 2020-03-19 updated: 2022-06-02 placement: category: fields order: 50 --- # Repeated fields Representing lists of data in an API is trickier than it often appears. Users often need to modify lists in place, and longer data series within a single resource pose a challenge for pagination. ## Guidance Resources **may** use repeated fields where appropriate. ```proto message Book { option (google.api.resource) = { type: "library.googleapis.com/Book" pattern: "publishers/{publisher}/books/{book}" }; string name = 1 [(google.api.field_behavior) = IDENTIFIER]; repeated string authors = 2; } ``` - Repeated fields **must** use a plural field name. - If the English singular and plural words are identical ("moose", "info"), the dictionary word **must** be used rather than attempting to coin a new plural form. - Repeated fields **should** have an enforced upper bound that will not cause a single resource payload to become too large. A good rule of thumb is 100 elements. - If repeated data has the chance of being too large, the API **should** use a sub-resource instead. - Repeated fields **must not** represent the body of another resource inline. Instead, the message **should** provide the resource names of the associated resources. ### Scalars and messages Repeated fields **should** use a scalar type (such as `string`) if they are certain that additional data will not be needed in the future, as using a message type adds significant cognitive overhead and leads to more complicated code. However, if additional data is likely to be needed in the future, repeated fields **should** use a message instead of a scalar proactively, to avoid parallel repeated fields. ### Update strategies A resource **may** use two strategies to enable updating a repeated field: direct update using the [standard `Update`][aip-134] method, or custom `Add` and `Remove` methods. A standard `Update` method has one key limitation: the user is only able to update _the entire_ list. Field masks are unable to address individual entries in a repeated field. This means that the user must read the resource, make modifications to the repeated field value as needed, and send it back. This is fine for many situations, particularly when the repeated field is expected to have a small size (fewer than 10 or so) and race conditions are not an issue, or can be guarded against with [ETags][aip-154]. **Note:** Declarative-friendly resources **must** use the standard `Update` method, and not introduce `Add` and `Remove` methods. If declarative tools need to reason about particular relationships while ignoring others, consider using a subresource instead. If atomic modifications are required, the API **should** define custom methods using the verbs `Add` and `Remove`: **Note:** If both of these strategies are too restrictive, consider using a subresource instead. ```proto rpc AddAuthor(AddAuthorRequest) returns (Book) { option (google.api.http) = { post: "/v1/{book=publishers/*/books/*}:addAuthor" body: "*" }; } rpc RemoveAuthor(RemoveAuthorRequest) returns (Book) { option (google.api.http) = { post: "/v1/{book=publishers/*/books/*}:removeAuthor" body: "*" }; } ``` - The data being added or removed **should** be a primitive (usually a `string`). - For more complex data structures with a primary key, the API **should** use a map with the `Update` method instead. - The RPC's name **must** begin with the word `Add` or `Remove`. The remainder of the RPC name **should** be the singular form of the field being added. - The request message **must** match the RPC name, with a `Request` suffix. - The response message **should** be the resource itself, unless there is useful context to provide in the response, in which case the response message must match the RPC name, with a `Response` suffix. - When the response is the resource itself, it **should** include the fully-populated resource. - The HTTP verb **must** be `POST`, as is usual for [custom methods][aip-136]. - The HTTP URI **must** end with `:add*` or `:remove*`, where `*` is the snake-case singular name of the field being added or removed. - The request message field receiving the resource name **should** map to the URI path. - The HTTP variable **should** be the name of the resource (such as `book`) rather than `name` or `parent`. - That variable **should** be the only variable in the URI path. - The body clause in the `google.api.http` annotation **should** be `"*"`. - If the data being added in an `Add` RPC is already present, the method **must** error with `ALREADY_EXISTS`. - If the data being removed in a `Remove` RPC is not present, the method **must** error with `NOT_FOUND`. #### Request Message ```proto message AddAuthorRequest { // The name of the book to add an author to. string book = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference).type = "library.googleapis.com/Book" ]; string author = 2 [(google.api.field_behavior) = REQUIRED]; } message RemoveAuthorRequest { // The name of the book to remove an author from. string book = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference).type = "library.googleapis.com/Book" ]; string author = 2 [(google.api.field_behavior) = REQUIRED]; } ``` - A resource field **must** be included. It **should** be the name of the resource (such as `book`) rather than `name` or `parent`. - The field **should** be [annotated as required][aip-203]. - The field **should** identify the [resource type][aip-123] that it references. - A field for the value being added or removed **must** be included. It **should** be the singular name of the field. - The field **should** be [annotated as required][aip-203]. - The request message **must not** contain any other required fields, and **should not** contain other optional fields except those described in this or another AIP. ## Changelog - **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-". - **2020-10-17**: Recommended returning the resource itself in Add and Remove RPCs over separate response types. - **2020-10-17**: Added guidance for Add and Remove RPCs and requests. [aip-123]: ./0123.md [aip-134]: ./0134.md [aip-136]: ./0136.md [aip-154]: ./0154.md [aip-203]: ./0203.md --- id: 145 state: approved created: 2020-05-28 placement: category: fields order: 60 --- # Ranges Services often need to represent ranges of discrete or continuous values. These have wide differences in meaning, and come in many types: integers, floats, and timestamps, just to name a few, and the expected meaning of a range can vary in subtle ways depending on the type of range being discussed. ## Guidance A resource or message representing a range **should** ordinarily use two separate fields of the same type, with prefixes `start_` and `end_`: ```proto // A representation of a chapter in a book. message Chapter { string title = 1; // The page where this chapter begins. int32 start_page = 2; // The page where the next chapter or section begins. int32 end_page = 3; } ``` ### Inclusive or exclusive ranges Fields representing ranges **should** use inclusive start values and exclusive end values (half-closed intervals) in most situations; in interval notation: `[start_xxx, end_xxx)`. Exclusive end values are preferable for the following reasons: - It conforms to user expectations, particularly for continuous values such as timestamps, and avoids the need to express imprecise "limit values" (e.g. `2012-04-20T23:59:59`). - It is consistent with most common programming languages, including C++, Java, Python, and Go. - It is easier to reason about abutting ranges: `[0, x), [x, y), [y, z)`, where values are chainable from one range to the next. ### Timestamp intervals The following section describes the use of the [google.type.Interval][interval] type, found amongst the common protos that are described in [AIP-213][]. This type represents a range between two timestamps, with an inclusive start value and exclusive end value. Ranges between two timestamps which conform to the expectations of the `Interval` message **should** use this rather than having separate start and end fields. This allows client code to be written against the `Interval` message (such as checking whether a given timestamp occurs within the interval) and reused across multiple intervals in the same API, or even across multiple APIs. APIs **may** use start and end timestamp fields instead. In particular, if a message within an API is inherently describing an interval with extra information about that interval, the additional level of nesting introduced by using the `Interval` message may be undesirable. ### Exceptions In some cases, there is significant colloquial precedent for inclusive start and end values (closed intervals), to the point that using an exclusive end value would be confusing even for people accustomed to them. For example, when discussing dates (not to be confused with timestamps), most people use inclusive end: a conference with dates "April 21-23" is expected to run for three days: April 21, April 22, and April 23. This is also true for days of the week: a business that is open "Monday through Friday" is open, not closed, on Fridays. In this situation, the prefixes `first` and `last` **should** be used instead: ```proto // A representation of a chapter in a book. message Chapter { string title = 1; // The first page of the chapter. int32 first_page = 2; // The last page of the chapter. int32 last_page = 3; } ``` Fields representing ranges with significant colloquial precedent for inclusive start and end values **should** use inclusive end values with `first_` and `last_` prefixes for those ranges only. The service **should** still use exclusive end values for other ranges where this does not apply, and **must** clearly document each range as inclusive or exclusive. [aip-213]: ./0213.md [interval]: https://github.com/googleapis/googleapis/blob/master/google/type/interval.proto --- id: 146 state: approved created: 2019-05-28 placement: category: fields order: 70 --- # Generic fields Most fields in any API, whether in a request, a resource, or a custom response, have a specific type or schema. This schema is part of the contract that developers write their code against. However, occasionally it is appropriate to have a generic or polymorphic field of some kind that can conform to multiple schemata, or even be entirely free-form. ## Guidance While generic fields are generally rare, a service **may** introduce generic field where necessary. There are several approaches to this depending on how generic the field needs to be; in general, services **should** attempt to introduce the "least generic" approach that is able to satisfy the use case. ### Oneof A `oneof` **may** be used to introduce a type union: the user or service is able to specify one of the fields inside the `oneof`. Additionally, a `oneof` **may** be used with the same type (usually strings) to represent a semantic difference between the options. Because the individual fields in the `oneof` have different keys, a developer can programmatically determine which (if any) of the fields is populated. A `oneof` preserves the largest degree of type safety and semantic meaning for each option, and services **should** generally prefer them over other generic or polymorphic options when feasible. However, the `oneof` construct is ill-suited when there is a large (or unlimited) number of potential options, or when there is a large resource structure that would require a long series of "cascading oneofs". **Note:** Adding additional possible fields to an existing `oneof` is a non-breaking change, but moving existing fields into or out of a `oneof` is breaking (it creates a backwards-incompatible change in Go protobuf stubs). ### Maps Maps **may** be used in situations where many values _of the same type_ are needed, but the keys are unknown or user-determined. Maps are usually not appropriate for generic fields because the map values all share a type, but occasionally they are useful. In particular, a map can sometimes be suited to a situation where many objects of the same type are needed, with different behavior based on the names of their keys (for example, using keys as environment names). ### Struct The [`google.protobuf.Struct`][struct] object **may** be used to represent arbitrary nested JSON. Keys can be strings, and values can be floats, strings, booleans, arrays, or additional nested structs, allowing for an arbitrarily nested structure that can be represented as JSON (and is automatically represented as JSON when using REST/JSON). A `Struct` is most useful when the service does not know the schema in advance, or when a service needs to store and retrieve arbitrary but structured user data. Using a `Struct` is convenient for users in this case because they can easily get JSON objects that can be natively manipulated in their environment of choice. If a service needs to reason about the _schema_ of a `Struct`, it **should** use [JSONSchema][] for this purpose. Because JSONSchema is itself JSON, a valid JSONSchema document can itself be stored in a `Struct`. ### Any The [`google.protobuf.Any`][any] object can be used to send an arbitrary serialized protocol buffer and a type definition. However, this introduces complexity, because an `Any` becomes useless for any task other than blind data propagation if the consumer does not have access to the proto. Additionally, even if the consumer _does_ have the proto, the consumer has to ensure the type is registered and then deserialize manually, which is an often-unfamiliar process. Because of this, `Any` **should not** be used unless other options are infeasible. [any]: https://github.com/protocolbuffers/protobuf/tree/master/src/google/protobuf/any.proto [struct]: https://github.com/protocolbuffers/protobuf/tree/master/src/google/protobuf/struct.proto [JSONSchema]: https://json-schema.org/ --- id: 147 state: approved created: 2020-07-24 placement: category: fields order: 80 --- # Sensitive fields Sometimes APIs need to collect sensitive information such as private encryption keys meant to be _stored_ by the underlying service but not intended to be _read_ after writing due to the sensitive nature of the data. For this type of data, extra consideration is required for the representation of the sensitive data in API requests and responses. ## Guidance If the sensitive information is _required_ for the resource as a whole to exist, the data **should** be accepted as an [input-only field][input-only] with no corresponding output field. Because the sensitive data must be present for the resource to exist, users of the API may assume that existence of the resource implies storage of the sensitive data. For example: ```proto message SelfManagedKeypair { string name = 1 [(google.api.field_behavior) = IDENTIFIER]; // The public key data in PEM-encoded form. bytes public_key = 2; // The private key data in PEM-encoded form. bytes private_key = 3 [ (google.api.field_behavior) = INPUT_ONLY]; } ``` If the sensitive information is _optional_ within the containing resource, an [output-only][] boolean field with a postfix of `_set` **should** be used to indicate whether or not the sensitive information is present. For example: ```proto message Integration { string name = 1 [(google.api.field_behavior) = IDENTIFIER]; string uri = 2; // A secret to be passed in the `Authorization` header of the webhook. string shared_secret = 3 [ (google.api.field_behavior) = INPUT_ONLY]; // True if a `shared_secret` has been set for this Integration. bool shared_secret_set = 4 [ (google.api.field_behavior) = OUTPUT_ONLY]; } ``` If it is important to be able to identify the sensitive information without allowing it to be read back entirely, a field of the same type with an `obfuscated_` prefix **may** be used instead of the boolean `_set` field to provide contextual information about the sensitive information. The specific nature of the obfuscation is outside the scope of this AIP. For example: ```proto message AccountRecoverySettings { // An email to use for account recovery. string email = 1 [ (google.api.field_behavior) = INPUT_ONLY]; // An obfuscated representation of the recovery email. For example, // `ada@example.com` might be represented as `a**@e*****e.com`. string obfuscated_email = 2 [ (google.api.field_behavior) = OUTPUT_ONLY]; } ``` [input-only]: ./0203.md#input-only [output-only]: ./0203.md#output-only --- id: 148 state: approved created: 2020-10-06 placement: category: fields order: 90 --- # Standard fields Certain concepts are common throughout any corpus of APIs. In these situations, it is useful to have a standard field name and behavior that is used consistently to communicate that concept. ## Guidance Standard fields **should** be used to describe their corresponding concept, and **should not** be used for any other purpose. ### Resource names and IDs #### name Every resource **must** have a `string name` field, used for the resource name (AIP-122), which **should** be the first field in the resource. **Note:** The `_name` suffix **should not** be used to describe other types of names unless otherwise covered in this AIP. #### parent The `string parent` field refers to the resource name of the parent of a collection, and **should** be used in most `List` (AIP-132) and `Create` (AIP-133) requests. ### Other names #### display_name The `string display_name` field **must** be a mutable, user-settable field where the user can provide a human-readable name to be used in user interfaces. [Declarative-friendly resources][] **should** include this field. Display names **should not** have uniqueness requirements, and **should** be limited to <= 63 characters. #### title The `string title` field **should** be the official name of an entity, such as a company's name. This is a more formal variant of `string display_name`. #### given_name The `string given_name` field **must** refer to a human or animal's given name. Resources **must not** use `first_name` for this concept, because the given name is not placed first in many cultures. #### family_name The `string family_name` field **must** refer to a human or animal's family name. Resources **must not** use `last_name` for this concept, because the family name is not placed last in many cultures. ### Timestamps #### create_time The output only `google.protobuf.Timestamp create_time` field **must** represent the timestamp when the resource was created. This **may** be either the time creation was initiated or the time it was completed. [Declarative-friendly resources][] **should** include this field. #### update_time The output only `google.protobuf.Timestamp update_time` field **must** represent the timestamp when the resource was most recently updated. Any change to the resource made by users **must** refresh this value; changes to a resource made internally by the service **may** refresh this value. [Declarative-friendly resources][] **should** include this field. #### delete_time The output only `google.protobuf.Timestamp delete_time` field **must** represent the timestamp that a resource was soft deleted. This **may** correspond to either the time when the user requested deletion, or when the service successfully soft deleted the resource. If a resource is not soft deleted, the `delete_time` field **must** be empty. Resources that support soft delete (AIP-164) **should** provide this field. #### expire_time The `google.protobuf.Timestamp expire_time` field **should** represent the time that a given resource or resource attribute is no longer useful or valid (e.g. a rotating security key). It **may** be used for similar forms of expiration as described in AIP-214. Services **may** provide an `expire_time` value that is inexact, but the resource **must not** expire before that time. #### purge_time The `google.protobuf.Timestamp purge_time` field **should** represent the time when a soft deleted resource will be purged from the system (see AIP-164). It **may** be used for similar forms of expiration as described in AIP-214. Resources that support soft delete **should** include this field. Services **may** provide a `purge_time` value that is inexact, but the resource **must not** be purged from the system before that time. ### Annotations To store small amounts of arbitrary data, a `map annotations` field **may** be added. The `annotations` field **must** use the [Kubernetes limits][] to maintain wire compatibility, and **should** require dot-namespaced annotation keys to prevent tools from trampling over one another. Examples of information that might be valuable to store in annotations include: - For CI/CD, an identifier of the pipeline run or version control identifier used to propagate. **Note:** Annotations are distinct from various forms of labels. Labels can be used by server-side policies, such as IAM conditions. Annotations exist to allow client tools to store their own state information without requiring a database. ### Well known string fields #### IP address A field that represents an IP address **must** comply with the following: * use type `string` * use the name `ip_address` or end with the suffix `_ip_address` e.g. `resolved_ip_address` * specify the IP address version format via one of the supported formats `IPV4`, `IPV6`, or if it can be either, `IPV4_OR_IPV6` (see [AIP-202][aip-202]) #### uid The output only `string uid` field refers to a system-assigned unique identifier for a resource. When provided, this field **must** be a [UUID4][] and **must** specify this format via the `UUID4` format extension (see [AIP-202][aip-202]). [Declarative-friendly resources][] **should** include this field. ## Further reading - For standardized codes, see AIP-143. - For the `etag` field, see AIP-154. - For the `request_id` field, see AIP-155. - For the `filter` field, see AIP-160. - For fields related to resource revisions, see AIP-162. - For the `validate_only` field, see AIP-163. - For fields related to soft delete and undelete, see AIP-164. ## Rationale ### Well known string fields Some fields represent very well defined concepts or artifacts that sometimes also have strict governance of their semantics. For such fields, presenting an equally standardized API surface is important. This enables development of improved API consumer tools and documentation, as well as a more unified user experience across the platform. ## History Before 2023-07, `purge_time` for soft-deleted resources was also called `expire_time`. `purge_time` was introduced to reduce user confusion. ## Changelog - **2023-10-05**: Introduce well known string fields with IP Address and `uid`. - **2023-08-14**: Introduce the term `annotations` from AIP-128. - **2023-07-13**: Introduce the term `purge_time`. - **2021-04-06**: Require output only field behavior for `uid` and `delete_time` fields. [aip-202]: ./0202.md [declarative-friendly resources]: ./0128.md#resources [kubernetes limits]: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/#syntax-and-character-set [uuid4]: https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random)--- id: 149 state: approved created: 2021-02-12 placement: category: fields order: 100 --- # Unset field values In many messages, many fields are optional: the user is not required to provide them, or for output fields, the service might not populate the field. In most cases, there is no meaningful difference between setting it to a default value (such as `0`) as opposed to not setting it at all; however, occasionally this distinction is meaningful. ## Guidance Services defined in protocol buffers **should** use the `optional` keyword for primitives if and only if it is necessary to distinguish setting the field to its default value (`0`, `false`, or empty string) from not setting it at all: ```proto // A representation of a book in a library. message Book { option (google.api.resource) = { type: "library.googleapis.com/Book" pattern: "publishers/{publisher}/books/{book}" }; // The name of the book. string name = 1 [(google.api.field_behavior) = IDENTIFIER]; // The rating for the book, from 0 to 5. // 0 is distinct from no rating. optional int32 rating = 2; } ``` **Important:** Services **should not** need to distinguish between the default value and unset most of the time; if an alternative design does not require such a distinction, it is usually preferred. In practice, this means `optional` **should** only ever be used for integers and floats. **Important:** Tracking field presence is *not* the same as documenting API field behavior as defined in [AIP-203][]. For example, a field labeled with `optional` for presence tracking **may** also be annotated as `google.api.field_behavior = REQUIRED` if the field must be set. If you only want to document the server perceived behavior of a field, read [AIP-203][]. ### Backwards compatibility It is a backwards incompatible change to add or remove the `optional` qualifier to an existing field. This is because the compiled src API is changed (in some languages). For example, in Golang, adding `optional` changes the field type of primitives to be the pointer variant of their original type, e.g. a field formerly of type `string` becomes `*string`, etc. Accordingly, this change requires that both clients and servers update their usage of the changed field in unison, which is risky and error prone. Additional information is [documented by Protobuf][proto docs]. ## Rationale ### field behavior and `optional` The field behavior annotation and `optional` label are not mutually exclusive, because they address different problems. The former, `google.api.field_behavior`, focuses on communicating the server's perception of a field within the API e.g. if it is required or not, if it is immutable, etc. The latter, proto3's `optional`, is a wire format and code generation option that is strictly for toggling field presence tracking. While it might be confusing for a field to be simultaneously annotated with `google.api.field_behavior = REQUIRED` and labeled as `optional`, they are unrelated in practice and can reasonably be used together. ## Changelog - **2024-06-05**: Add backwards compatibility considerations - **2023-06-20**: Differentiate from field behavior documentation [AIP-203]: ./0203.md [proto docs]: https://protobuf.dev/programming-guides/field_presence/#considerations-for-change-compatibility --- id: 151 state: approved created: 2019-07-25 placement: category: operations order: 110 --- # Long-running operations Occasionally, an API may need to expose a method that takes a significant amount of time to complete. In these situations, it is often a poor user experience to simply block while the task runs; rather, it is better to return some kind of promise to the user and allow the user to check back in later. The long-running operations pattern is roughly analogous to a [Python Future][], or a [Node.js Promise][]. Essentially, the user is given a token that can be used to track progress and retrieve the result. ## Guidance Individual API methods that might take a significant amount of time to complete **should** return a [`google.longrunning.Operation`][lro] object instead of the ultimate response message. ```proto // Create a book. rpc CreateBook(CreateBookRequest) returns (google.longrunning.Operation) { option (google.api.http) = { post: "/v1/{parent=publishers/*}/books" body: "book" }; option (google.longrunning.operation_info) = { response_type: "Book" metadata_type: "OperationMetadata" }; } ``` - The response type **must** be `google.longrunning.Operation`. The `Operation` proto definition **must not** be copied into individual APIs. - The response **must not** be a streaming response. - The method **must** include a `google.longrunning.operation_info` annotation, which **must** define both response and metadata types. - The response and metadata types **must** be defined in the file where the RPC appears, or a file imported by that file. - If the response and metadata types are defined in another package, the fully-qualified message name **must** be used. - The response type **should not** be `google.protobuf.Empty` (except for [`Delete`][aip-135] methods), unless it is certain that response data will _never_ be needed. If response data might be added in the future, define an empty message for the RPC response and use that. - The metadata type is used to provide information such as progress, partial failures, and similar information on each `GetOperation` call. The metadata type **should not** be `google.protobuf.Empty`, unless it is certain that metadata will _never_ be needed. If metadata might be added in the future, define an empty message for the RPC metadata and use that. - APIs with messages that return `Operation` **must** implement the [`Operations`][lro] service. Individual APIs **must not** define their own interfaces for long-running operations to avoid non-uniformity. - If an RPC supports a [validate-only mode][aip-163], the response to a validation request **must** be one of the following: - A successful response with an `Operation` which is already complete, with the `done` field set to `true`, and a valid (but potentially empty) response message in the `response` field, wrapped in a `google.protobuf.Any` message. The `name` field **may** be empty, to avoid the service having to maintain state for successful validation. - An immediate error response (typically "bad request") - An `Operation` with the `done` field set to `false`, to indicate long-running validation. In this case, the `name` field **must** be set, to allow clients to poll the long-running validation operation until it has completed. Successful validation **must** eventually be represented by an operation with `done=true` and a valid (but potentially empty) wrapped response message in the `response` field. Unsuccessful validation **must** eventually be represented by an operation with `done=true` and the error details provided in the `error` field. **Note:** User expectations can vary on what is considered "a significant amount of time" depending on what work is being done. A good rule of thumb is 10 seconds. ### Standard methods APIs **may** return an `Operation` from the [`Create`][aip-133], [`Update`][aip-134], or [`Delete`][aip-135] standard methods if appropriate. In this case, the response type in the `operation_info` annotation **must** be the standard and expected response type for that standard method. When creating or deleting a resource with a long-running operation, the resource **should** be included in [`List`][aip-132] and [`Get`][aip-131] calls; however, the resource **should** indicate that it is not usable, generally with a [state enum][aip-216]. ### Parallel operations A resource **may** accept multiple operations that will work on it in parallel, but is not obligated to do so: - Resources that accept multiple parallel operations **may** place them in a queue rather than work on the operations simultaneously. - Resources that do not permit multiple operations in parallel (denying any new operation until the one that is in progress finishes) **must** return `ABORTED` if a user attempts a parallel operation, and include an error message explaining the situation. - Resources with [declarative-friendly APIs][aip-128] **may** allow subsequent updates to preempt existing operations. In this case, the latest update begins processing and previous operations are marked as `ABORTED` with an error message explaining the situation. ### Expiration APIs **may** allow their operation resources to expire after sufficient time has elapsed after the operation completed. **Note:** A good rule of thumb for operation expiry is 30 days. ### Errors Errors that prevent a long-running operation from _starting_ **must** return an error response ([AIP-193][]), similar to any other method. Operations that fail during their execution phase **must** return an error response ([AIP-193][]), placed in the `Operation.error` [google.rpc.Status][] field. Non-terminal errors that occur over the course of an operation **may** be placed in the metadata message and the field(s) **must** be [AIP-193][] compliant [google.rpc.Status][]. ### Backwards compatibility Changing either the `response_type` or `metadata_type` of a long-running operation is a breaking change. ## Rationale ### Validate-only behavior The guidance for validate-only responses comes from a tension between clients, which benefit from "fully formed" operations that can be treated uniformly, and servers, which don't wish to maintain additional state for trivial operations. It seems counterintuitive that just validating a request should generate more state, but a full operation response that can be fetched later would either require that or "special" singleton operation IDs. The guidance provided is a compromise: by returning a "done" operation, clients can use existing logic to check that the operation has completed successfully (and therefore doesn't need to be fetched for an updated status) but server don't need to maintain any additional state. [aip-128]: ./0128.md [aip-131]: ./0131.md [aip-132]: ./0132.md [aip-133]: ./0133.md [aip-134]: ./0134.md [aip-135]: ./0135.md [aip-163]: ./0163.md [aip-193]: ./0193.md [aip-216]: ./0216.md [google.rpc.Status]: https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto [lro]: https://github.com/googleapis/googleapis/blob/master/google/longrunning/operations.proto [node.js promise]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Using_promises [python future]: https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Future ## Changelog - **2025-02-04**: Clarified error propagation behavior for failures that occur during long-running operations. - **2024-04-23**: Provided pattern for validation on RPCs returning long-running operations. - **2022-05-31**: Added compatibility section. - **2020-08-24**: Clarified that responses are not streaming responses. - **2020-06-24**: Added guidance for parallel operations. - **2020-03-20**: Clarified that both `response_type` and `metadata_type` are required. - **2019-11-22**: Added a short explanation of what `metadata_type` is for. - **2019-09-23**: Added guidance on errors. - **2019-08-23**: Added guidance about fully-qualified message names when the message name is in another package. - **2019-08-01**: Changed the examples from "shelves" to "publishers", to present a better example of resource ownership. --- id: 152 state: approved created: 2020-04-27 updated: 2022-06-02 placement: category: design-patterns order: 10 --- # Jobs Occasionally, APIs may need to expose a task that takes significant time to complete, and where a transient [long-running operation][aip-151] is not appropriate. For example, a task could need to run repeatedly, or have separate permissions for configuring the task as opposed to running it. ## Guidance An API **may** define a `Job` resource to represent a particular task with distinct setup, configuration, and execution: ```proto message WriteBookJob { option (google.api.resource) = { type: "library.googleapis.com/WriteBookJob" pattern: "publishers/{publisher}/writeBookJobs/{write_book_job}" }; // Name and other fields... } ``` - The name of the resource **must** end with the word "Job". - The prefix **should** be a valid RPC name, with a verb and a noun. - The service **should** define all five of the standard methods (AIP-131, AIP-132, AIP-133, AIP-134, AIP-135), and use them as the primary way to configure the job. ### Run method The service **should** define a `Run` custom method that executes the job immediately: ```proto rpc RunWriteBookJob(RunWriteBookJobRequest) returns (google.longrunning.Operation) { option (google.api.http) = { post: "/v1/{name=publishers/*/writeBookJobs/*}:run" body: "*" }; option (google.longrunning.operation_info) = { response_type: "RunWriteBookJobResponse" metadata_type: "RunWriteBookJobMetadata" }; } ``` - The RPC's name **must** begin with the word `Run`. The remainder of the RPC name **should** be the singular form of the job resource being run. - The request message **must** match the RPC name, with a `Request` suffix. - The method **should** return a [long-running operation][aip-151], which **must** resolve to a response message that includes the result of running the job. - The response message name must match the RPC name, with a `Response` suffix. - The method **may** use any metadata message it wishes. - The HTTP verb **must** be `POST`, as is usual for [custom methods][aip-136]. - The body clause in the `google.api.http` annotation **should** be `"*"`. - The URI path **should** contain a single `name` variable corresponding to the name of the job resource being run. - The URI path **must** end with `:run`. - Errors that prevent execution of the job from _starting_ **must** return an error response (AIP-193), similar to any other method. Errors that occur over the course of the job execution **may** be placed in the metadata message. The errors themselves **must** still be represented with a [`google.rpc.Status`][status] object. ### Run request message Run methods implement a common request message pattern: ```proto message RunWriteBookJobRequest { // The name of the job to run. string name = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { type: "library.googleapis.com/WriteBookJob" }]; } ``` - A singular `string name` field **must** be included. - The field **should** be [annotated as required][aip-203]. - The field **should** identify the [resource type][aip-123] that it references. ### Executions and results Ordinarily, the API **should** provide results to the user as the final response of the `Run` method. However, this is sometimes insufficient; for example, a job that runs on a recurring schedule in the background can not deliver results to the user in this way. The service **may** store resources representing individual executions along with their result as a sub-collection of resources under the job, which allows the user to list *past* job executions. A service that does this **should** define the `Get`, `List`, and `Delete` methods for the execution resources: ```proto message WriteBookJobExecution { option (google.api.resource) = { type: "library.googleapis.com/WriteBookJobExecution" pattern: "publishers/{publisher}/writeBookJobs/{write_book_job}/executions/{execution}" }; // Name and other information about the execution, such as metadata, the // result, error information, etc. } ``` In this case, the operation returned by job's `Run` method **should** refer to the child resource. ## Changelog - **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-". - **2020-11-02**: Expanded guidance on HTTP, field behavior, and resource reference annotations and request format. [aip-123]: ./0123.md [aip-136]: ./0136.md [aip-151]: ./0151.md [aip-203]: ./0203.md [status]: https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto --- id: 153 state: approved created: 2019-12-16 placement: category: design-patterns order: 20 --- # Import and export Many users want to be able to load data into an API, or get their existing data out of an API. This is particularly important for enterprise users, who are often concerned about vendor lock-in. ## Guidance APIs **may** support import and export operations, which **may** create multiple new resources, or they **may** populate data into a single resource. ### Multiple resources Services **may** support importing and exporting multiple resources into or out of an API, and **should** implement a common pattern to do so: ```proto rpc ImportBooks(ImportBooksRequest) returns (google.longrunning.Operation) { option (google.api.http) = { post: "/v1/{parent=publishers/*}/books:import" body: "*" }; option (google.longrunning.operation_info) = { response_type: "ImportBooksResponse" metadata_type: "ImportBooksMetadata" }; } rpc ExportBooks(ExportBooksRequest) returns (google.longrunning.Operation) { option (google.api.http) = { post: "/v1/{parent=publishers/*}/books:export" body: "*" }; option (google.longrunning.operation_info) = { response_type: "ExportBooksResponse" metadata_type: "ExportBooksMetadata" }; } ``` - The method **must** return a long-running operation (see [AIP-151][]) unless the service can guarantee that it will _never_ need more than a few seconds to complete. - The HTTP verb **must** be `POST`, and the `body` **must** be `"*"`. - A `parent` field **should** be included as part of the URI. - If importing into or exporting from multiple resources is required, the API **should** keep the `parent` field and allow the user to use the `-` character to indicate multiple parents (see [AIP-159][]). - On import, if the user provides a specific parent, the API **must** reject any imported resources that would be added to a different parent. - The URI suffix **should** be `:import` or `:export`. ### Data for a single resource Services **may** support importing and exporting data into or out of a single resource, and **should** implement a common pattern to do so: ```proto rpc ImportPages(ImportPagesRequest) returns (google.longrunning.Operation) { option (google.api.http) = { post: "/v1/{book=publishers/*/books/*}:importPages" body: "*" }; option (google.longrunning.operation_info) = { response_type: "ImportPagesResponse" metadata_type: "ImportPagesMetadata" }; } rpc ExportPages(ExportPagesRequest) returns (google.longrunning.Operation) { option (google.api.http) = { post: "/v1/{book=publishers/*/books/*}:exportPages" body: "*" }; option (google.longrunning.operation_info) = { response_type: "ExportPagesResponse" metadata_type: "ExportPagesMetadata" }; } ``` - The method **must** return a long-running operation (see [AIP-151][]) unless the service can guarantee that it will _never_ need more than a few seconds to complete. - The HTTP verb **must** be `POST`, and the `body` **must** be `"*"`. - A field representing the resource that data is being imported into **should** be included as part of the URI. The field **should** be named after the resource (and **should not** be called `name`). - The URI suffix should include both the verb and a noun for the data itself, such as `:importPages` or `:exportPages`. ### Request object Imports and exports often require two fundamentally different types of configuration: 1. Configuration specific to the source or destination. 2. Configuration regarding the imported or exported data itself. Source or destination configuration should be grouped into a single message and placed inside a oneof: ```proto message ImportBooksRequest { string parent = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { child_type: "library.googleapis.com/Book" }]; oneof source { AuthorSource author_source = 2; TranslatorSource translator_source = 3; } string isbn_prefix = 4; } message ExportBooksRequest { string parent = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { child_type: "library.googleapis.com/Book" }]; oneof destination { PrinterDestination printer_destination = 2; TranslatorDestination translator_destination = 3; } string filter = 4; } ``` - The source configuration messages **must** be placed within a `oneof source` (for import) or `oneof destination` (for export), even if there is only one. (This maintains flexibility to add more later.) - Configuration related to the data itself (and therefore common across all sources) **must** be placed at the top-level of the request message. **Note:** The configuration for import and export **may** be different from one another. (For example, it would be sensible to import from a file but export to a directory.) ### Inline sources APIs **may** also permit import and export "inline", where the contents to be imported or exported are provided in the request or response. ```proto message InlineSource { repeated Book books = 1; } ``` - The source or destination **should** be named `InlineSource` or `InlineDestination`. - The message **should** include a repeated field representing the resource. However, if the resource structure is complex, the API **may** use a separate inline representation. In this situation, the same format **must** be used for both import and export. ### Partial failures While partial failures are normally discouraged, import and export RPCs **should** include partial failure information in the metadata object. Each individual error **should** be a `google.rpc.Status` object describing the error. For more on errors, see [AIP-193][]. [aip-151]: ./0151.md [aip-159]: ./0159.md [aip-193]: ./0193.md [google bigquery]: https://cloud.google.com/bigquery [google storage]: https://cloud.google.com/storage --- id: 154 state: approved created: 2019-07-24 placement: category: design-patterns order: 30 --- # Resource freshness validation APIs often need to validate that a client and server agree on the current state of a resource before taking some kind of action on that resource. For example, two processes updating the same resource in parallel could create a race condition, where the latter process "stomps over" the effort of the former one. ETags provide a way to deal with this, by allowing the server to send a checksum based on the current content of a resource; when the client sends that checksum back, the server can ensure that the checksums match before acting on the request. ## Guidance A resource **may** include an `etag` field on any resource where it is important to ensure that the client has an up to date resource before acting on certain requests: ```proto // A representation of a book. message Book { // Other fields... // This checksum is computed by the server based on the value of other // fields, and may be sent on update and delete requests to ensure the // client has an up-to-date value before proceeding. string etag = 99; } ``` - The etag field **must** be a string, and **must** be named `etag`. - The etag field on the _resource_ **should not** be given any behavior annotations. - The etag field **must** be provided by the server on output, and values **should** conform to [RFC 7232][]. - If a user sends back an etag which matches the current etag value, the service **must** permit the request (unless there is some other reason for failure). - If a user sends back an etag which does not match the current etag value, the service **must** send an `ABORTED` error response (unless another error takes precedence, such as `PERMISSION_DENIED` if the user is not authorized). - If the user does not send an etag value at all, the service **should** permit the request. However, services with strong consistency or parallelism requirements **may** require users to send etags all the time and reject the request with an `INVALID_ARGUMENT` error in this case. **Note:** ETag values **should** include quotes as described in [RFC 7232][]. For example, a valid etag is `"foo"`, not `foo`. ### Declarative-friendly resources A resource that is declarative-friendly (AIP-128) **must** include an `etag` field. ### Etags on request methods In some situations, the etag needs to belong on a request message rather than the resource itself. For example, an `Update` standard method can "piggyback" off the `etag` field on the resource, but the `Delete` standard method can not: ```proto message DeleteBookRequest { // The name of the book. string name = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { type: "library.googleapis.com/Book" }]; // The current etag of the book. // If an etag is provided and does not match the current etag of the book, // deletion will be blocked and an ABORTED error will be returned. string etag = 2 [(google.api.field_behavior) = OPTIONAL]; } ``` On a request message, the `etag` field **should** be given a behavior annotation - either `REQUIRED` or `OPTIONAL`. See AIP-203 for more information. An `etag` field **may** also be used on custom methods, similar to the example above. ### Strong and weak etags ETags can be either "strongly validated" or "weakly validated": - A strongly validated etag means that two resources bearing the same etag are byte-for-byte identical. - A weakly validated etag means that two resources bearing the same etag are equivalent, but may differ in ways that the service does not consider to be important. Resources **may** use either strong or weak etags, as it sees fit, but **should** document the behavior. Additionally, weak etags **must** have a `W/` prefix as mandated by [RFC 7232][]. [rfc 7232]: https://tools.ietf.org/html/rfc7232#section-2.3 ## Further reading - For how to retry on errors in client libraries, see AIP-194. ## Changelog - **2021-04-01**: Updated an additional reference to `FAILED_PRECONDITION` to `ABORTED`. - **2021-03-05**: Changed the etag error from `FAILED_PRECONDITION` (which becomes HTTP 400) to `ABORTED` (409). - **2020-10-06**: Added declarative-friendly resource requirement. - **2020-09-02**: Clarified that other errors may take precedence over `FAILED_PRECONDITION` for etag mismatches. - **2020-09-02**: Add guidance for etags on request messages. - **2019-09-23**: Changed the title to "resource freshness validation". --- id: 155 state: approved created: 2019-05-06 placement: category: design-patterns order: 40 --- # Request identification It is sometimes useful for an API to have a unique, customer-provided identifier for particular requests. This can be useful for several purposes, such as de-duplicating requests from parallel processes, ensuring the safety of retries, or auditing. The most important purpose for request IDs is to provide idempotency guarantees: allowing the same request to be issued more than once without subsequent calls having any effect. In the event of a network failure, the client can retry the request, and the server can detect duplication and ensure that the request is only processed once. ## Guidance APIs **may** add a `string request_id` parameter to request messages (including those of standard methods) in order to uniquely identify particular requests. ```proto message CreateBookRequest { // The parent resource where this book will be created. // Format: publishers/{publisher} string parent = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { child_type: "library.googleapis.com/Book" }]; // The ID to use for the book, which will become the final component of // the book's resource name. // // This value should be 4-63 characters, and valid characters // are /[a-z][0-9]-/. string book_id = 2 [(google.api.field_behavior) = REQUIRED]; // The book to create. Book book = 3 [(google.api.field_behavior) = REQUIRED]; // A unique identifier for this request. Restricted to 36 ASCII characters. // A random UUID is recommended. // This request is only idempotent if a `request_id` is provided. string request_id = 4 [(google.api.field_info).format = UUID4]; } ``` - Providing a request ID **must** guarantee idempotency. - If a duplicate request is detected, the server **should** return the response for the previously successful request, because the client most likely did not receive the previous response. - APIs **may** choose any reasonable timeframe for honoring request IDs. - The `request_id` field **must** be provided on the request message to which it applies (and it **must not** be a field on resources themselves). - Request IDs **should** be optional. - Request IDs **should** be able to be UUIDs, and **may** allow UUIDs to be the only valid format. The format restrictions for request IDs **must** be documented. - Request IDs that are UUIDs **must** be annotated with the `google.api.FieldInfo.Format` value `UUID4` using the extension `(google.api.field_info).format = UUID4`. See [AIP-202](./0202.md) for more. ### Stale success responses In some unusual situations, it may not be possible to return an identical success response. For example, a duplicate request to create a resource may arrive after the resource has not only been created, but subsequently updated; because the service has no other need to retain the historical data, it is no longer feasible to return an identical success response. In this situation, the method **may** return the current state of the resource instead. In other words, it is permissible to substitute the historical success response with a similar response that reflects more current data. ## Further reading - For which codes to retry, see [AIP-194](https://aip.dev/194). - For how to retry errors in client libraries, see [AIP-4221](https://aip.dev/4221). ## Rationale ### Using UUIDs for request identification When a value is required to be unique, leaving the format open-ended can lead to API consumers incorrectly providing a duplicate identifier. As such, standardizing on a universally unique identifier drastically reduces the chance for collisions when done correctly. ## Changelog - **2024-01-08**: Add book_id to request message. - **2023-10-02**: Add UUID format extension guidance. - **2019-08-01**: Changed the examples from "shelves" to "publishers", to present a better example of resource ownership. --- id: 156 state: approved created: 2019-05-12 placement: category: resource-design order: 70 --- # Singleton resources APIs sometimes need to represent a resource where exactly one instance of the resource always exists within any given parent. A common use case for this is for a config object. ## Guidance An API **may** define _singleton resources_. A singleton resource **must** always exist by virtue of the existence of its parent, with one and exactly one per parent. For example: ```proto message Config { option (google.api.resource) = { type: "api.googleapis.com/Config" pattern: "users/{user}/config" singular: "config" plural: "configs" }; // additional fields including name } ``` The `Config` singleton would have the following RPCs: ```proto rpc GetConfig(GetConfigRequest) returns (Config) { option (google.api.http) = { get: "/v1/{name=users/*/config}" }; } rpc UpdateConfig(UpdateConfigRequest) returns (Config) { option (google.api.http) = { patch: "/v1/{config.name=users/*/config}" body: "config" }; } ``` - Singleton resources **must not** have a user-provided or system-generated ID; their [resource name][aip-122] includes the name of their parent followed by one static-segment. - Example: `users/1234/config` - Singleton resources are always singular. - Example: `users/1234/thing` - Singleton resource definitions **must** provide both the `singular` and `plural` fields (see above example). - Singleton resources **may** parent other resources. - Singleton resources **must not** define the [`Create`][aip-133] or [`Delete`][aip-135] standard methods. The singleton is implicitly created or deleted when its parent is created or deleted. - Singleton resources **should** define the [`Get`][aip-131] and [`Update`][aip-134] methods, and **may** define custom methods as appropriate. - However, singleton resources **must not** define the [`Update`][aip-134] method if all fields on the resource are [output only][aip-203]. - Singleton resources **may** define the [`List`][aip-132] method, but **must** implement it according to [AIP-159][aip-159]. See the example below. - The trailing segment in the path pattern that typically represents the collection **should** be the `plural` form of the Singleton resource e.g. `/v1/{parent=users/*}/configs`. - If a parent resource ID is provided instead of the hyphen `-` as per AIP-159, then the service **should** return a collection of one Singleton resource corresponding to the specified parent resource. ```proto rpc ListConfigs(ListConfigsRequest) returns (ListConfigsResponse) { option (google.api.http) = { get: "/v1/{parent=users/*}/configs" }; } message ListConfigsRequest { // To list all configs, use `-` as the user id. // Formats: // * `users/-` // * `users/{user}` // // Note: Specifying an actual user id will return a collection of one config. // Use GetConfig instead. string parent = 1 [ (google.api.resource_reference).child_type = "api.googleapis.com/Config"]; // other standard pagination fields... } ``` ## Rationale ### Support for Standard List While Singleton resources are not directly part of a collection themselves, they can be viewed as part of their parent's collection. The one-to-one relationship of parent-to-singleton means that for every one parent there is one singleton instance, naturally enabling some collection-based methods when combined with the pattern of [Reading Across Collections][aip-159]. The Singleton can present as a collection to the API consumer as it is indirectly one based on its parent. Furthermore, presenting the Singleton resource as a pseudo-collection in such methods enables future expansion to a real collection, should a Singleton be found lacking. ### Including `plural` definition While a Singleton is by definition singular, there are certain cases where a Singleton resource may appear in a plural form e.g., if the service supports Standard List (as defined here). As such, it is better to forward declare the plural form of the Singleton resource type than to not have it when needed. ## Changelog - **2024-04-15:** Singletons must specify `singular` and `plural` in resource. - **2023-08-10:** Add Standard `List` support. - **2023-07-26:** Clarified that read-only singletons should not have `Update`. - **2021-11-02:** Added an example message and state parent eligibility. - **2021-01-14:** Changed example from `settings` to `config` for clarity. [aip-122]: ./0122.md [aip-131]: ./0131.md [aip-132]: ./0132.md [aip-133]: ./0133.md [aip-134]: ./0134.md [aip-135]: ./0135.md [aip-159]: ./0159.md [aip-203]: ./0203.md#output-only --- id: 157 state: approved created: 2019-01-26 placement: category: design-patterns order: 50 --- # Partial responses Sometimes, a resource can be either large or expensive to compute, and the API needs to give the user control over which fields it sends back. ## Guidance APIs **may** support partial responses in one of two ways: ### Field masks parameter Field masks (`google.protobuf.FieldMask`) can be used for granting the user fine-grained control over what fields are returned. An API **should** support the mask in a side channel. For example, the parameter can be specified either using an HTTP query parameter, an HTTP header, or a [gRPC metadata entry][1]. Google Cloud APIs specify field masks as a [system parameter][0]. Field masks **should not** be specified in the [request](./0157.md#read-masks-as-a-request-field). - The value of the field mask parameter **must** be a `google.protobuf.FieldMask`. - The field mask parameter **must** be optional: - An explicit value of `"*"` **should** be supported, and **must** return all fields. - If the field mask parameter is omitted, it **must** default to `"*"`, unless otherwise documented. - An API **may** allow read masks with non-terminal repeated fields (unlike update masks), but is not obligated to do so. **Note:** Changing the default value of the field mask parameter is a [breaking change](../0180.md#semantic-changes). ### View enumeration Alternatively, an API **may** support partial responses with view enums. View enums are useful for situations where an API only wants to expose a small number of permutations to the user: ```proto enum BookView { // The default / unset value. // The API will default to the BASIC view. BOOK_VIEW_UNSPECIFIED = 0; // Include basic metadata about the book, but not the full contents. // This is the default value (for both ListBooks and GetBook). BOOK_VIEW_BASIC = 1; // Include everything. BOOK_VIEW_FULL = 2; } ``` - The enum **should** be specified as a `view` field on the request message. - The enum **should** be named something ending in `-View` - The enum **should** at minimum have values named `BASIC` and `FULL` (although it **may** have values other than these). - The `UNSPECIFIED` value **must** be valid (not an error), and the API **must** document what the unspecified value will do. - For List RPCs, the effective default value **should** be `BASIC`. - For the following RPC types, the effective default value **should** be either `BASIC` or `FULL`: - Get - Create - Update - Soft Delete - Custom Method - The enum **should** be defined at the top level of the proto file (as it is likely to be needed in multiple requests, e.g. both `Get` and `List`). See [AIP-126][] for more guidance on top-level enumerations. - APIs **may** add fields to a given view over time. APIs **must not** remove a field from a given view (this is a breaking change). **Note:** If a service requires (or might require) multiple views with overlapping but distinct values, there is a potential for a namespace conflict. In this situation, the service **should** nest the view enum within the individual resource. **Note:** Having a partial response be the default of standard methods can degrade the effectiveness of declarative clients. Providing a mechanism to request the full resource be populated in the response, like this View pattern, is preferred if partial responses are deemed necessary. ### Read masks as a request field **Warning:** Read mask as a single, explicit field on the request message is **DEPRECATED** for Google APIs. The [system parameter](#field-masks-parameter) **must** be used instead. The following guidance is for the benefit of existing legacy Google and external non-Google usage. An API **may** support read masks as a single field on the request message: `google.protobuf.FieldMask read_mask`. - The read mask **must** be a `google.protobuf.FieldMask` and **should** be named `read_mask`. - The field mask **should** be optional: - An explicit value of `"*"` **should** be supported, and **must** return all fields. - If the field mask parameter is not provided, all fields **must** be returned. - An API **may** allow read masks with non-terminal repeated fields (unlike update masks), but is not obligated to do so. ## Rationale ### Deprecating `read_mask` in request messages As mentioned, Google API infrastructure implements a service-wide response field filtering mechanism, so there is no need for individual API methods to specify a `read_mask` in their request schema. Doing so is both redundant and a potential point of conflict for the client or service. ## Changelog - **2025-10-03**: Added default view guidance for other RPC types, and declarative client warning. - **2025-06-16**: Reinstate read mask guidance as historical/external reference. - **2023-05-09**: Fix top-level enum example and link to AIP-126. - **2022-03-14:** Updated guidance on default value and how to specify a read mask. - **2021-10-06:** Updated the guidance with system parameters. - **2021-03-04:** Added guidance for conflicting view enums. [0]: https://cloud.google.com/apis/docs/system-parameters [1]: https://grpc.io/docs/what-is-grpc/core-concepts/#metadata [AIP-126]: ./0126.md--- id: 158 state: approved created: 2019-02-18 placement: category: design-patterns order: 60 --- # Pagination APIs often need to provide collections of data, most commonly in the [List][] standard method. However, collections can often be arbitrarily sized, and also often grow over time, increasing lookup time as well as the size of the responses being sent over the wire. Therefore, it is important that collections be paginated. ## Guidance RPCs returning collections of data **must** provide pagination _at the outset_, as it is a [backwards-incompatible change](#backwards-compatibility) to add pagination to an existing method. ```proto // The request structure for listing books. message ListBooksRequest { // The parent, which owns this collection of books. // Format: publishers/{publisher} string parent = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { child_type: "library.googleapis.com/Book" }]; // The maximum number of books to return. The service may return fewer than // this value. // If unspecified, at most 50 books will be returned. // The maximum value is 1000; values above 1000 will be coerced to 1000. int32 page_size = 2; // A page token, received from a previous `ListBooks` call. // Provide this to retrieve the subsequent page. // // When paginating, all other parameters provided to `ListBooks` must match // the call that provided the page token. string page_token = 3; } // The response structure from listing books. message ListBooksResponse { // The books from the specified publisher. repeated Book books = 1; // A token that can be sent as `page_token` to retrieve the next page. // If this field is omitted, there are no subsequent pages. string next_page_token = 2; } ``` - Request messages for collections **should** define an `int32 page_size` field, allowing users to specify the maximum number of results to return. - The `page_size` field **must not** be required. - If the user does not specify `page_size` (or specifies `0`), the API chooses an appropriate default, which the API **should** document. The API **must not** return an error. - If the user specifies `page_size` greater than the maximum permitted by the API, the API **should** coerce down to the maximum permitted page size. - If the user specifies a negative value for `page_size`, the API **must** send an `INVALID_ARGUMENT` error. - The API **may** return fewer results than the number requested (including zero results), even if not at the end of the collection. - Request messages for collections **should** define a `string page_token` field, allowing users to advance to the next page in the collection. - The `page_token` field **must not** be required. - If the user changes the `page_size` in a request for subsequent pages, the service **must** honor the new page size. - The user is expected to keep all other arguments to the RPC the same; if any arguments are different, the API **should** send an `INVALID_ARGUMENT` error. - The response **must not** be a streaming response. - Response messages for collections **should** define a `string next_page_token` field, providing the user with a page token that may be used to retrieve the next page. - The field containing pagination results **should** be the first field in the message and have a field number of `1`. It **should** be a repeated field containing a list of resources constituting a single page of results. - If the end of the collection has been reached, the `next_page_token` field **must** be empty. This is the _only_ way to communicate "end-of-collection" to users. - If the end of the collection has not been reached (or if the API can not determine in time), the API **must** provide a `next_page_token`. - Response messages for collections **may** provide an `int32 total_size` field, providing the user with the total number of items in the list. - This total **may** be an estimate (but the API **should** explicitly document that). ### Skipping results The request definition for a paginated operation **may** define an `int32 skip` field to allow the user to skip results. The `skip` value **must** refer to the number of individual resources to skip, not the number of pages. For example: - A request with no page token and a `skip` value of `30` returns a single page of results starting with the 31st result. - A request with a page token corresponding to the 51st result (because the first 50 results were returned on the first page) and a `skip` value of `30` returns a single page of results starting with the 81st result. If a `skip` value is provided that cannot be fulfilled e.g. due to latency of querying a massive data set, the response **must** be `200 OK` with an empty result set. If it is _known_ to put the cursor beyond the total size of the collection, the response **must not** include a `next_page_token`. ### Opacity Page tokens provided by APIs **must** be opaque (but URL-safe) strings, and **must not** be user-parseable. This is because if users are able to deconstruct these, _they will do so_. This effectively makes the implementation details of your API's pagination become part of the API surface, and it becomes impossible to update those details without breaking users. **Warning:** Base-64 encoding an otherwise-transparent page token is **not** a sufficient obfuscation mechanism. For page tokens which do not need to be stored in a database, and which do not contain sensitive data, an API **may** obfuscate the page token by defining an internal protocol buffer message with any data needed, and send the serialized proto, base-64 encoded. Page tokens **must** be limited to providing an indication of where to continue the pagination process only. They **must not** provide any form of authorization to the underlying resources, and authorization **must** be performed on the request as with any other regardless of the presence of a page token. ### Expiring page tokens Many APIs store page tokens in a database internally. In this situation, APIs **may** expire page tokens a reasonable time after they have been sent, in order not to needlessly store large amounts of data that is unlikely to be used. It is not necessary to document this behavior. **Note:** While a reasonable time may vary between APIs, a good rule of thumb is three days. ## Backwards compatibility Adding pagination to an existing RPC is a backwards-incompatible change. This may seem strange; adding fields to proto messages is generally backwards compatible. However, this change is _behaviorally_ incompatible. Consider a user whose collection has 75 resources, and who has already written and deployed code. If the API later adds pagination fields, and sets the default to 50, then that user's code breaks; it was getting all resources, and now is only getting the first 50 (and does not know to advance pagination). Even if the API set a higher default limit, such as 100, the user's collection could grow, and _then_ the code would break. Additionally, [client libraries implement automatic pagination](https://google.aip.dev/client-libraries/4233), typically representing paginated RPCs using different method signatures to unpaginated ones. This means that adding pagination to a previously-unpaginated method causes a breaking change in those libraries. For this reason, it is important to always add pagination to RPCs returning collections _up front_; they are consistently important, and they can not be added later without causing problems for existing users. **Warning:** This also entails that, in addition to presenting the pagination fields, they **must** be _actually implemented_ with a non-infinite default value. Implementing an in-memory version (which might fetch everything then paginate) is reasonable for initially-small collections. [list]: ./0132.md ## Rationale ### Degraded `skip` response Large collections, complex queries, and globally distributed data can all contribute to a paginated method being unable to quickly or confidently fulfill a given `skip` request. Backend queries can timeout, data collation can take time, and the end user experience need not suffer as a result. In such cases, the pagination interface can be leveraged to keep the client engaged by providing a `next_page_token`, while the service collects an appropriate result. When the service has definitively determined that the requested `skip` exceeds the available results, the pagination interface is again applied and `next_page_token` is omitted to signal the end of results. ## Changelog - **2025-07-08**: Clarify degraded skip response guidance - **2020-05-24**: Clarified that adding pagination breaks client libraries. - **2020-05-13**: Added guidance for skipping results. - **2020-08-24**: Clarified that responses are not streaming responses. - **2020-06-24**: Clarified that page size is always optional for users. - **2019-02-12**: Added guidance on the field being paginated over. - **2019-08-01**: Changed the examples from "shelves" to "publishers", to present a better example of resource ownership. - **2019-07-19**: Update the opacity requirement from "should" to "must". --- id: 159 state: approved created: 2019-07-26 placement: category: design-patterns order: 70 --- # Reading across collections Sometimes, it is useful for a user to be able to retrieve resources across multiple collections, or retrieve a single resource without needing to know what collection it is in. ## Guidance APIs **may** support reading resources across multiple collections by allowing users to specify a `-` (the hyphen or dash character) as a wildcard character in a standard [`List`][aip-132] method: ``` GET /v1/publishers/-/books?filter=... ``` - The URI pattern **must** still be specified with `*` and permit the collection to be specified; a URI pattern **must not** hard-code the `-` character. - The method **must** explicitly document that this behavior is supported. - The resources provided in the response **must** use the canonical name of the resource, with the actual parent collection identifiers (instead of `-`). - Services **may** support reading across collections on `List` requests regardless of whether the identifiers of the child resources are guaranteed to be unique. However, services **must not** support reading across collections on `Get` requests if the child resources might have a collision. - Cross-parent requests **should not** support `order_by`. If they do, the field **must** document that it is best effort. This is because cross-parent requests introduce ambiguity around ordering, especially if there is difficulty reaching a parent (see [AIP-217][]). **Important:** If listing across multiple collections introduces the possibility of partial failures due to unreachable parents (such as when listing across locations), the method **must** indicate this following the guidance in [AIP-217][]. ### Unique resource lookup Sometimes, a resource within a sub-collection has an identifier that is unique across parent collections. In this case, it may be useful to allow a [`Get`][aip-131] method to retrieve that resource without knowing which parent collection contains it. In such cases, APIs **may** allow users to specify the wildcard collection ID `-` (the hyphen or dash character) to represent any parent collection: ``` GET https://example.googleapis.com/v1/publishers/-/books/{book} ``` - The URI pattern **must** still be specified with `*` and permit the collection to be specified; a URI pattern **must not** hard-code the `-` character. - The method **must** explicitly document that this behavior is supported. - The resource name in the response **must** use the canonical name of the resource, with actual parent collection identifiers (instead of `-`). For example, the request above returns a resource with a name like `publishers/123/books/456`, _not_ `publishers/-/books/456`. - The resource ID **must** be unique within parent collections. ## Further reading - For partial failures due to unreachable resources, see [AIP-217][]. [aip-131]: ./0131.md [aip-132]: ./0132.md [aip-217]: ./0217.md ## Changelog - **2019-08-26**: Added a reference to guidance for unreachable resources. - **2019-08-01**: Changed the examples from "shelves" to "publishers", to present a better example of resource ownership. --- id: 160 state: approved created: 2020-02-24 placement: category: design-patterns order: 80 --- # Filtering Often, when listing resources (using a list method as defined in [AIP-132][] or something reasonably similar), it is desirable to filter over the collection and only return results that the user is interested in. It is tempting to define a structure to handle the precise filtering needs for each API. However, filtering requirements evolve frequently, and therefore it is prudent to use a string field with a structured syntax accessible to a non-technical audience. This allows updates to be able to be made transparently, without waiting for UI or client updates. **Note:** Because list filters are intended for a potentially non-technical audience, they sometimes borrow from patterns of colloquial speech rather than common patterns found in code. ## Guidance APIs **may** provide filtering to users on `List` methods (or similar methods to query a collection, such as `Search`). If they choose to do so, they **should** follow the common specification for filters discussed here. The syntax is formally defined in the [EBNF grammar][]. When employing filtering, a request message **should** have exactly one filtering field, `string filter`. Filtering of related objects is handled through [traversal](#traversal-operator) or [functions](#functions). **Note:** List Filters have fuzzy matching characteristics with support for result ranking and scoring. For developers interested in deterministic evaluation of list filters, see [CEL][]. ### Literals A bare literal value (examples: "42", "Hugo") is a value to be matched against. Literals appearing alone (with no specified field) **should** usually be matched anywhere it may appear in an object's field values. However, a service **may** choose to only consider certain fields; if so, it **must** document which fields it considers. A service **may** include new fields over time, but **should** do so judiciously and consider impact on existing users. **Note:** Literals separated by whitespace are considered to have a fuzzy variant of `AND`. Therefore, `Victor Hugo` is roughly equivalent to `Victor AND Hugo`. ### Logical Operators Filtering implementations **should** provide the binary operators: | Operator | Example | Meaning | | -------- | ------------- | -------------------------------------- | | `AND` | `a AND b` | True if `a` and `b` are true. | | `OR` | `a OR b OR c` | True if any of `a`, `b`, `c` are true. | **Note:** To match common patterns of speech, the `OR` operator has higher precedence than `AND`, unlike what is found in most programming languages. The expression `a AND b OR c` evaluates: `a AND (b OR c)`. API documentation and examples **should** encourage the use of explicit parentheses to avoid confusion, but **should not** require explicit parentheses. ### Negation Operators Filtering implementations **should** provide the unary operators `NOT` and `-`. These are used interchangeably, and a service that supports negation **must** support both formats. | Operator | Example | Meaning | | -------- | ------- | ------------------------ | | `NOT` | `NOT a` | True if `a` is not true. | | `-` | `-a` | True if `a` is not true. | ### Comparison Operators Filtering implementations **should** provide the binary comparison operators `=`, `!=`, `<`, `>`, `<=`, and `>=` for string, numeric, timestamp, and duration fields (but **should not** provide them for booleans or enums). | Operator | Example | Meaning | | -------- | ------------ | ----------------------------------------------- | | `=` | `a = true` | True if `a` is true. | | `!=` | `a != 42` | True unless `a` equals 42. | | `<` | `a < 42` | True if `a` is a numeric value below 42. | | `>` | `a > "foo"` | True if `a` is lexically ordered after "foo". | | `<=` | `a <= "foo"` | True if `a` is "foo" or lexically before it. | | `>=` | `a >= 42` | True if `a` is a numeric value of 42 or higher. | **Note:** Unlike in most programming languages, field names **must** appear on the left-hand side of a comparison operator; the right-hand side only accepts literals and logical operators. Because filters are accepted as query strings, type conversion takes place to translate the string to the appropriate strongly-typed value: - Enums expect the enum's string representation (case-sensitive). - Booleans expect `true` and `false` literal values. - Numbers expect the standard integer or float representations. For floats, exponents are supported (e.g. `2.997e9`). - [Durations][] expect a numeric representation followed by an `s` suffix (for seconds). Examples: `20s`, `1.2s`. - [Timestamps][] expect an [RFC-3339][] formatted string (e.g. `2012-04-21T11:30:00-04:00`). UTC offsets are supported. **Warning:** The identifiers `true`, `false`, and `null` only carry intrinsic meaning when used in the context of a typed field reference. Additionally, when comparing strings for equality, services **should** support wildcards using the `*` character; for example, `a = "*.foo"` is true if `a` _ends with_ ".foo". ### Traversal operator Filtering implementations **should** provide the `.` operator, which indicates traversal through a message, map, or struct. | Example | Meaning | | --------------- | ----------------------------------------------------- | | `a.b = true` | True if `a` has a boolean `b` field that is true. | | `a.b > 42` | True if `a` has a numeric `b` field that is above 42. | | `a.b.c = "foo"` | True if `a.b` has a string `c` field that is "foo". | Traversal **must** be written using the field names from the resource. If a service wishes to support "implicit fields" of some kind, they **must** do so through well-documented [functions](#functions). A service **may** specify a subset of fields that are supported for traversal. If a user attempts to traverse to a field that is not defined on the message, the service **should** return an error with `INVALID_ARGUMENT`. A service **may** permit traversal to undefined keys on maps and structs, and **should** document how it behaves in this situation. When evaluating an expression involving a traversal, if any non-primitive field in the chain is not set on the entry being evaluated, the entry **should** be skipped i.e. not match the filter expression. This applies even when the comparison is a `!=`, which would imply matching on empty values. In the examples above, if resource field `a` is not set on the resource instance, that instance is skipped as a non-match. **Important:** The `.` operator **must not** be used to traverse through a repeated field or list, except for specific use with the `:` operator. ### Has Operator Filtering implementations **must** provide the `:` operator, which means "has". It is usable with collections (repeated fields or maps) as well as messages, and behaves slightly differently in each case. Repeated fields query to see if the repeated structure contains a matching element: | Example | Meaning | | ---------- | ----------------------------------------------------------- | | `r:42` | True if `r` contains 42. | | `r.foo:42` | True if `r` contains an element `e` such that `e.foo = 42`. | **Important:** Filters can not query a _specific_ element on a repeated field for a value. For example, `e.0.foo = 42` and `e[0].foo = 42` are **not** valid filters. Maps, structs, messages can query either for the presence of a field in the map or a specific value: | Example | Meaning | | ---------- | ----------------------------------- | | `m:foo` | True if `m` contains the key "foo". | | `m.foo:*` | True if `m` contains the key "foo". | | `m.foo:42` | True if `m.foo` is 42. | There are two slight distinctions when parsing messages: - When traversing messages, a field is only considered to be present if it has a non-default value. - When traversing messages, field names are snake case, although implementations **may** choose to support automatic conversion between camel case and snake case. For all aforementioned types, simply checking for the presence of a top-level resource field is possible with the `*` value: | Example | Meaning | | ---------- | ---------------------------------------- | | `r:*` | True if `repeated` field `r` is present. | | `p:*` | True if `map` field `p` is present. | | `m:*` | True if `message` field `m` is present. | **Note:** For `map` and `repeated` fields, there is no semantic difference between an unset field and "set with empty value" - they both resolve to "not present". ### Functions The filtering language supports a function call syntax in order to support API-specific extensions. An API **may** define a function using the `call(arg...)` syntax, and **must** document any specific functions it supports. ### Limitations A service **may** specify further structure or limitations for filter queries, above what is defined here. For example, a service may support the logical operators but only permit a certain number of them (to avoid "queries of death" or other performance concerns). Further structure or limitations **must** be clearly documented, and **must not** violate requirements set forth in this document. ### Validation If a non-compliant or schematically invalid `filter` string is specified, the API **should** error with `INVALID_ARGUMENT`. Wherever validation is relaxed for `filter`, the API **must** document the difference. Schematic validation refers, but is not limited to, the following: - Fields referenced in the `filter` **must** exist on the filtered schema - Field values provided in the `filter` **must** align to the type of the field - For example, for a field `int32 age` a `filter` like `"age=hello"` is invalid - Field values for bounded data types e.g. `enum` provided in the `filter` **must** be a valid value in the set - Field values for standardized types e.g. `Timestamp` **must** conform to the documented standard (see [Comparison Operators](#comparison-operators) for a list of such types) [aip-132]: ./0132.md [cel]: https://github.com/google/cel-spec [cel-cpp]: https://github.com/google/cel-cpp [cel-go]: https://github.com/google/cel-go [durations]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/duration.proto [ebnf grammar]: /assets/misc/ebnf-filtering.txt [rfc-3339]: https://tools.ietf.org/html/rfc3339 [timestamps]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/timestamp.proto ## Changelog - **2025-10-03**: Clarify top-level field has operator behavior. - **2025-01-07**: Clarify behavior for unset field in traversal operator chain. - **2024-12-11**: Move non-compliant filter guidance to Validation section. --- id: 161 state: approved created: 2021-03-01 placement: category: design-patterns order: 85 --- # Field masks Often, when updating resources (using an update method as defined in AIP-134 or something reasonably similar), it is desirable to specify exactly which fields are being updated, so that the service can ignore the rest, even if the user sends new values. It is tempting to define a mask format to handle the precise needs for each API. However, masking requirements evolve, and therefore it is prudent to use a structured syntax. This allows updates to be able to be made transparently, without waiting for UI or client updates. ## Guidance These masks of field names are called "field masks". Fields representing a field mask **must** use the `google.protobuf.FieldMask` type. Field masks are most common on Update requests (AIP-134). Field masks **must** always be relative to the resource: **Warning:** Read masks as a single field on the request message, for example: `google.protobuf.FieldMask read_mask` are **DEPRECATED**. Instead, see AIP-157. ```proto message UpdateBookRequest { // The book to update. // // The book's `name` field is used to identify the book to update. // Format: publishers/{publisher}/books/{book} Book book = 1 [(google.api.field_behavior) = REQUIRED]; // The list of fields to update. // Fields are specified relative to the book // (e.g. `title`, `rating`; *not* `book.title` or `book.rating`). google.protobuf.FieldMask update_mask = 2; } ``` ### Read-write consistency Read and write behavior for field masks **must** be self-consistent if a mask is present: - If a user updates a resource with a given mask, and then reads the same resource with the same mask, the exact same data **must** be returned. - Exception: [Output only fields](#output-only-fields). - Similarly, reading a resource with a given mask and then updating the resource with the returned data and the same mask **must** be a no-op. **Note:** This implies that any mask that is valid for either read or write **must** be valid for both. ### Specifying specific fields Field masks **must** permit the specification of specific fields in a defined struct, using the `.` character for traversal. Because field masks are always relative to the resource, direct fields on the resource require no traversal (examples: `title`, `rating`). Traversal is used when resources contain messages (example: `author.given_name`). **Note:** A user **must** be able to specify either a field as a whole, or one of its subfields: `author` and `author.given_name` are both valid. ### Map fields Field masks **may** permit the specification of specific fields in a map, if and only if the map's keys are either strings or integers, using the `.` character for traversal. Field masks **should** support string keys that contain characters that are problematic for the field mask syntax, using the backtick character. ```proto message Book { // The name of the book. // Format: publishers/{publisher}/books/{book} string name = 1; // Reviews for the back cover. The key is the author of the review, // and the value is the text of the review. // // Valid field masks: reviews, reviews.smith, reviews.`John Smith` map reviews = 2; } ``` ### Wildcards Field masks **may** permit the use of the `*` character on a repeated field or map to indicate the specification of particular sub-fields in the collection: ```proto message Book { option (google.api.resource) = { type: "library.googleapis.com/Book" pattern: "publishers/{publisher}/books/{book}" }; // The name of the book. // Format: publishers/{publisher}/books/{book} string name = 1 [(google.api.field_behavior) = IDENTIFIER]; // The author or authors of the book. // Valid field masks: authors, authors.*.given_name, authors.*.family_name // Invalid field masks: authors.0, authors.0.given_name repeated Author authors = 2; } message Author { // The author's given name. string given_name = 1; // The author's family name. string family_name = 2; } ``` **Note:** Field masks **must not** permit accessing a particular element of a repeated field by index, and **must** return an `INVALID_ARGUMENT` error if this is attempted. ### Output only fields If a user includes an output only field in an update mask indirectly (by using a wildcard or specifying an overall message that includes an output-only subfield), the service **must** ignore any output only fields provided as input, even if they are cleared or modified. If a user directly specifies an output only field in an update mask, the service **must** ignore the output only fields provided as input, even if they are cleared or modified, to permit the same field mask to be used for input and output. ### Invalid field mask entries When reading data, field masks **may** ignore entries that point to a value that can not exist (either a field that does not exist, or a map key that the service considers invalid). When writing data, field masks **should** return an `INVALID_ARGUMENT` error if an entry points to a value that can not exist; however, the service **may** permit deletions. ## Changelog - **2023-10-18**: Update guidance for presence of output only fields in update mask. - **2023-07-17**: Move `update_mask` guidance to AIP-134. --- id: 162 state: draft created: 2019-09-17 updated: 2023-09-01 placement: category: design-patterns order: 88 --- # Resource Revisions Some APIs need to have resources with a revision history, where users can reason about the state of the resource over time. There are several reasons for this: - Users may want to be able to roll back to a previous revision, or diff against a previous revision. - An API may create data which is derived in some way from a resource at a given point in time. In these cases, it may be desirable to snapshot the resource for reference later. **Note:** We use the word _revision_ to refer to a historical reference for a particular resource, and intentionally avoid the term _version_, which refers to the version of an API as a whole. ## Guidance APIs **may** store a revision history for a resource. Examples of when it is useful include: - When it is valuable to expose older versions of a resource via an API. This can avoid the overhead of the customers having to write their own API to store and enable retrieval of revisions. - Other resources depend on different revisions of a resource. - There is a need to represent the change of a resource over time. APIs implementing resources with a revision history **should** abstract resource revisions as nested collection of the resource. Sometimes, the revisions collection can be a top level collection, exceptions include: - If resource revisions are meant to have longer lifespan than the parent resource. In other words, resource revisions exist after resource deletion. ```proto message BookRevision { // The name of the book revision. string name = 1; // The snapshot of the book Book snapshot = 2 [(google.api.field_behavior) = OUTPUT_ONLY]; // The timestamp that the revision was created. google.protobuf.Timestamp create_time = 3 [(google.api.field_behavior) = OUTPUT_ONLY]; // Other revision IDs that share the same snapshot. repeated string alternate_ids = 4 [(google.api.field_behavior) = OUTPUT_ONLY]; } ``` - The `message` **must** be annotated as a resource (AIP-123). - The `message` name **must** be named `{ResourceType}Revision`. - The resource revision **must** contain a field with a message type of the parent resource, with a field name of `snapshot`. - The value of `snapshot` **must** be the configuration of the parent at the point in time the revision was created. - The resource revision **must** contain a `create_time` field (see [AIP-142][]). - The resource revision **may** contain a repeated field `alternate_ids`, which would contain a list of resource IDs that the revision is also known by (e.g. `latest`) ### Creating Revisions Depending on the resource, different APIs may have different strategies for - Create a new revision any time that there is a change to the parent resource - Create a new revision when important system state changes - Create a new revision when specifically requested APIs **may** use any of these strategies. APIs **must** document their revision creation strategy. ### Resource names for revisions When referring to specific revision of a resource, the subcollection name **must** be named `revisions`. Resource revisions have names with the format `{resource_name}/revisions/{revision_id}`. For example: ``` publishers/123/books/les-miserables/revisions/c7cfa2a8 ``` ### Server-specified Aliases Services **may** reserve specific IDs to be [aliases][alias] (e.g. `latest`). These are read-only and managed by the service. ``` GET /v1/publishers/{publisher}/books/{book}/revisions/{revision_id} ``` - If a `latest` ID exists, it **must** represent the most recently created revision. The content of `publishers/{publisher}/books/{book}/revisions/latest` and `publishers/{publisher}/books/{book}` can differ, as the latest revision may be different from the current state of the resource. ### User-Specified Aliases APIs **may** provide a mechanism for users to assign an [alias][] ID to an existing revision with a custom method "alias": ```proto rpc AliasBookRevision(AliasBookRevisionRequest) returns (Book) { option (google.api.http) = { post: "/v1/{name=publishers/*/books/*/revisions/*}:alias" body: "*" }; } ``` ```proto message AliasBookRevisionRequest { string name = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { type: "library.googleapis.com/BookRevision" }]; // The ID of the revision to alias to, e.g. `CURRENT` or a semantic // version. string alias_id = 2 [(google.api.field_behavior) = REQUIRED]; } ``` - The request message **must** have a `name` field: - The field **must** be [annotated as required][aip-203]. - The field **must** identify the [resource type][aip-123] that it references. - The request message **must** have a `alias_id` field: - The field **must** be [annotated as required][aip-203]. - If the user calls the method with an existing `alias_id`, the request **must** succeed and the alias will be updated to refer to the provided revision. This allows users to write code against a specific alias (e.g. `published`) and the revision can change with no code change. ### Rollback A common use case for a resource with a revision history is the ability to roll back to a given revision. APIs **should** handle this with a `Rollback` custom method: ```proto rpc RollbackBook(RollbackBookRequest) returns (BookRevision) { option (google.api.http) = { post: "/v1/{name=publishers/*/books/*/revisions/*}:rollback" body: "*" }; } ``` - The method **must** use the `POST` HTTP verb. - The method **should** return a resource revision. ```proto message RollbackBookRequest { // The revision that the book should be rolled back to. string name = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { type: "library.googleapis.com/BookRevision" }]; } ``` - The request message **must** have a `name` field, referring to the resource revision whose configuration the resource should be rolled back to. - The field **must** be [annotated as required][aip-203]. - The field **must** identify the [resource type][aip-123] that it references. ### Child resources Resources with a revision history **may** have child resources. If they do, there are two potential variants: - Child resources where each child resource is a child of the parent resource as a whole. - Child resources where each child resource is a child of _a single revision of_ the parent resource. APIs **should not** include multiple levels of resources with revisions, as this quickly becomes difficult to reason about. ### Standard methods Any standard methods **must** implement the corresponding AIPs (AIP-131, AIP-132, AIP-133, AIP-134, AIP-135), with the following additional behaviors: - List methods: By default, revisions in the list response **must** be ordered in reverse chronological order. User can supply `order_by` to override the default behavior. - If the revision supports aliasing, a delete method with the resource name of the alias (e.g. `revisions/1.0.2`) **must** remove the alias instead of deleting the resource. As revisions are nested under the resource, also see [cascading delete][]. ## Rationale ### Abstract revisions as nested collection Revisions being resources under nested collection make revisions a first class citizen. - Revisions can offer standard get, list, and delete methods. - It retains the flexibility of extending new fields to revision in addition to the resource message. ### Tagging to Aliases Previously, a concept of `tag` existed. This concept was redundant with that of an [alias][], and the terms were consolidated to reduce complexity in the AIPs. ### Output only resource configuration Although it was an option to have the revision take in the resource configuration as part of the create method, doing so would have allowed users to submit resource configuration for a revision that the resource was never in. `OUTPUT_ONLY` and requiring that a created revision represents the resource at current point in time eliminates that issue. ## History ### Switching from a collection extension to a subcollection In 2023-09, revisions are abstracted as a nested resource collection. Prior to this, revisions are more like extension of an existing resource by using `@` symbol. List and delete revisions were custom methods on the resource collection. A single Get method was used to retrieve either the resource revision, or the resource. Its primary advantage was allowing a resource reference to seamlessly refer to a resource, or its revision. It also had several disadvantages: - List revisions is a custom method (:listRevisions) on the resource collection - Delete revision is a custom method on the resource collection - Not visible in API discovery doc - Resource ID cannot use `@` The guidance was modified ultimately to enable revisions to behave like a resource, which reduces the users cognitive load and allows resource-oriented clients to easily list, get, create, and update revisions. ### Using resource ID instead of tag In the previous design, revisions had a separate identifer for a revision known as a `tag`, that would live in a revision. Tags were effectively a shadow resource ID, requiring methods to create, get and filter revisions based on the value of the tag. By consolidating the concept of a tag into the revision ID, the user no longer needs to be familiar with a second set of retrieval and identifier methods. ## Changelog - **2023-09-01**: AIP was updated to be a sub-collection. - **2021-04-27**: Added guidance on returning the resource from Delete Revision. [aip-142]: ./0142.md [alias]: ./0122.md#resource-id-aliases [cascading delete]: ./0135.md#cascading-delete [UUID4]: https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random) --- id: 163 state: approved created: 2019-12-16 placement: category: design-patterns order: 90 --- # Change validation Occasionally, a user wants to validate an intended change to see what the result will be before actually making the change. For example, a request to provision new servers in a fleet will have an impact on the overall fleet size and cost, and could potentially have unexpected downstream effects. ## Guidance APIs **may** provide an option to validate, but not actually execute, a request, and provide the same response (status code, headers, and response body) that it would have provided if the request was actually executed. To provide this option, the method **should** include a `bool validate_only` field in the request message: ```proto message ReviewBookRequest { string name = 1 [(google.api.resource_reference) = { type: "library.googleapis.com/Book" }]; int32 rating = 2; string comment = 3; // If set, validate the request and preview the review, but do not actually // post it. bool validate_only = 4; } ``` The API **must** perform permission checks and any other validation that would be performed on a "live" request; a request using `validate_only` **must** fail if it determines that the actual request would fail. **Note:** It may occasionally be infeasible to provide the full output. For example, if creating a resource would create an auto-generated ID, it does not make sense to do this on validation. APIs **should** omit such fields on validation requests in this situation. ### Declarative-friendly resources A resource that is declarative-friendly (AIP-128) **must** include a `validate_only` field on methods that mutate the resource. ## Changelog - **2020-10-06:** Added declarative-friendly resource requirement. --- id: 164 state: approved created: 2020-10-06 placement: category: design-patterns order: 95 --- # Soft delete There are several reasons why a client could desire soft delete and undelete functionality, but one over-arching reason stands out: recovery from mistakes. A service that supports undelete makes it possible for users to recover resources that were deleted by accident. ## Guidance APIs **may** support the ability to "undelete", to allow for situations where users mistakenly delete resources and need the ability to recover. If a resource needs to support undelete, the `Delete` method **must** simply mark the resource as having been deleted, but not completely remove it from the system. If the method behaves this way, it **should** return the updated resource instead of `google.protobuf.Empty`. Resources that support soft delete **should** have both a `delete_time` and `purge_time` field as described in AIP-148. Additionally, resources **should** include a `DELETED` state value if the resource includes a `state` field (AIP-216). ### Undelete A resource that supports soft delete **should** provide an `Undelete` method: ```proto rpc UndeleteBook(UndeleteBookRequest) returns (Book) { option (google.api.http) = { post: "/v1/{name=publishers/*/books/*}:undelete" body: "*" }; } ``` - The HTTP verb **must** be `POST`. - The `body` clause **must** be `"*"`. - The response message **must** be the resource itself. There is no `UndeleteBookResponse`. - The response **should** include the fully-populated resource unless it is infeasible to do so. - If the undelete RPC is [long-running](#long-running-undelete), the response message **must** be a `google.longrunning.Operation` which resolves to the resource itself. ### Undelete request message Undelete methods implement a common request message pattern: ```proto message UndeleteBookRequest { // The name of the deleted book. // Format: publishers/{publisher}/books/{book} string name = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference).type = "library.googleapis.com/Book"]; } ``` - A `name` field **must** be included. It **should** be called `name`. - The field **should** be [annotated as required][aip-203]. - The field **should** identify the [resource type][aip-123] that it references. - The comment for the field **should** document the resource pattern. - The request message **must not** contain any other required fields, and **should not** contain other optional fields except those described in this or another AIP. ### Long-running undelete Some resources take longer to undelete a resource than is reasonable for a regular API request. In this situation, the API **should** use a long-running operation (AIP-151) instead: ```proto rpc UndeleteBook(UndeleteBookRequest) returns (google.longrunning.Operation) { option (google.api.http) = { post: "/v1/{name=publishers/*/books/*}:undelete" body: "*" }; option (google.longrunning.operation_info) = { response_type: "Book" metadata_type: "OperationMetadata" }; } ``` - The response type **must** be set to the resource (what the return type would be if the RPC was not long-running). - Both the `response_type` and `metadata_type` fields **must** be specified. ### List and Get Soft-deleted resources **should not** be returned in `List` (AIP-132) responses by default (unless `bool show_deleted` is true). `Get` (AIP-131) requests for soft-deleted resources **should** return the resource (rather than a `NOT_FOUND` error). APIs that soft delete resources **may** choose a reasonable strategy for purging those resources, including automatic purging after a reasonable time (such as 30 days), allowing users to set an expiry time (AIP-214), or retaining the resources indefinitely. Regardless of what strategy is selected, the API **should** document when soft deleted resources will be completely removed. ### Declarative-friendly resources Soft-deletable resources have a poorer experience than hard-deleted resources in declarative clients: since an ID on a soft-deleted resource is not re-usable unless a custom method (undelete) is called, an imperative client must be introduced or hand-written code is required to incorporate the usage of the custom method. ### Errors If the user does not have permission to access the resource, regardless of whether or not it exists, the service **must** error with `PERMISSION_DENIED` (HTTP 403). Permission **must** be checked prior to checking if the resource exists. If the user does have proper permission, but the requested resource does not exist (either it was never created or already expunged), the service **must** error with `NOT_FOUND` (HTTP 404). If the user calling a soft `Delete` has proper permission, but the requested resource is already deleted, the service **must** succeed if `allow_missing` is `true`, and **should** error with `NOT_FOUND` (HTTP 404) if `allow_missing` is `false`. If the user calling `Undelete` has proper permission, but the requested resource is not deleted, the service **must** respond with `ALREADY_EXISTS` (HTTP 409). ## Further reading - For the `Delete` standard method, see AIP-135. - For long-running operations, see AIP-151. - For resource freshness validation (`etag`), see AIP-154. - For change validation (`validate_only`), see AIP-163. ## Changelog - **2024-09-24**: Included missing requirement for `delete_time`. - **2023-07-13**: Renamed overloaded `expire_time` to `purge_time`. - **2021-07-12**: Added error behavior when soft deleting a deleted resource. - **2021-07-12**: Clarified that `ALREADY_EXISTS` errors apply to `Undelete`. - **2021-07-12**: Changed the `expire_time` field to "should" for consistency with AIP-148. - **2020-09-23**: Soft delete material in AIP-135 migrated to this AIP. [aip-123]: ./0123.md [aip-203]: ./0203.md --- id: 165 state: approved created: 2019-12-18 updated: 2022-06-02 placement: category: design-patterns order: 100 --- # Criteria-based delete Occasionally, an API may need to provide a mechanism to delete a large number of resources based on some set of filter parameters, rather than requiring the individual resource name of the resources to be deleted. This is a rare case, reserved for situations where users need to delete thousands or more resources at once, in which case the normal Batch Delete pattern ([AIP-235][]) becomes unwieldy and inconvenient. ## Guidance **Important:** Most APIs **should** use only Delete ([AIP-135][]) or Batch Delete ([AIP-235][]) for deleting resources, and **should not** implement deleting based on criteria. This is because deleting is generally irreversible and this type of operation makes it easy for a user to accidentally lose significant amounts of data. An API **may** implement a Purge method to permit deleting a large number of resources based on a filter string; however, this **should** only be done if the Batch Delete ([AIP-235][]) pattern is insufficient to accomplish the desired goal: ```proto rpc PurgeBooks(PurgeBooksRequest) returns (google.longrunning.Operation) { option (google.api.http) = { post: "/v1/{parent=publishers/*}/books:purge" body: "*" }; option (google.longrunning.operation_info) = { response_type: "PurgeBooksResponse" metadata_type: "PurgeBooksMetadata" }; } ``` - The RPC's name **must** begin with the word `Purge`. The remainder of the RPC name **should** be the plural form of the resource being purged. - The request message **must** match the RPC name, with a `Request` suffix. - The response type **must** be a `google.longrunning.Operation` (see AIP-151) that resolves to a message whose name matches the RPC name, with a `Response` suffix. - The HTTP verb **must** be `POST`, and the `body` **must** be `"*"`. - The URI path **should** represent the collection for the resource. - The `parent` field **should** be included in the URI. If the API wishes to support deletion across multiple parents, it **should** accept the `-` character consistent with [AIP-159][]. ### Request message Purge methods implement a common request message pattern: ```proto message PurgeBooksRequest { // The publisher to purge books from. // To purge books across publishers, send "publishers/-". string parent = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { child_type: "library.googleapis.com/Book" }]; // A filter matching the books to be purged. string filter = 2 [(google.api.field_behavior) = REQUIRED]; // Actually perform the purge. // If `force` is set to false, the method will return a sample of // resource names that would be deleted. bool force = 3; } ``` - A singular `string parent` field **should** be included, unless the resource is top-level. - The field **should** be [annotated as required][aip-203]. - The field **should** identify the [resource type][aip-123] that it references. - A singular `string filter` field **must** be included and **must** follow the same semantics as in List methods (AIP-160). - It **should** be [annotated as required][aip-203]. - A wildcard value of `"*"` **may** be supported for deleting everything. - A singular `bool force` field **must** be included. If it is not set, the API **must** return a count of the resources that would be deleted as well as a sample of those resources, without actually performing the deletion. ### Response message Purge methods implement a common response message pattern: ```proto message PurgeBooksResponse { // The number of books that this request deleted (or, if `force` is false, // the number of books that will be deleted). int32 purge_count = 1; // A sample of the resource names of books that will be deleted. // Only populated if `force` is set to false. repeated string purge_sample = 2 [(google.api.resource_reference) = { type: "library.googleapis.com/Book" }]; } ``` - A singular `int32 purge_count` field **should** be included, and provide the number of resources that were deleted (or would be deleted). This count **may** be an estimate similar to `total_size` in AIP-158 (but the service **should** document this if so). - A `repeated string purge_sample` field **should** be included: If `force` is `false`, it **should** provide a sample of resource names that will be deleted. If `force` is true, this field **should not** be populated. - The sample **should** be a sufficient size to catch clearly obvious mistakes: A good rule of thumb is 100. The API **should** document the size, and **should** document that it is a maximum (it is possible to send fewer). - The sample **may** be random or **may** be deterministic (such as the first matched resource names). The API **should** document which approach is used. - The field **should** identify the [resource type][aip-123] that it references. **Note:** Even if `purge_count` and `purge_sample` are not included, the `force` field **must** still be included in the request. ## Changelog - **2025-07-23:** Explicitly state filter wildcard may be supported. - **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-". - **2020-10-29**: Expanded guidance on HTTP, field behavior, and resource reference annotations. [aip-123]: ./0123.md [aip-135]: ./0135.md [aip-159]: ./0159.md [aip-203]: ./0203.md [aip-235]: ./0235.md --- id: 180 state: approved created: 2019-07-23 placement: category: compatibility order: 10 --- # Backwards compatibility APIs are fundamentally contracts with users, and users often write code against APIs that is then launched into a production service with the expectation that it continues to work (unless the API has a [stability level][aip-181] that indicates otherwise). Therefore, it is important to understand what constitutes a backwards compatible change and what constitutes a backwards incompatible change. ## Guidance Existing client code **must not** be broken by a service updating to a new minor or patch release. Old clients **must** be able to work against newer servers (with the same major version number). **Important:** It is not always clear whether a change is compatible or not. The guidance here **should** be treated as indicative, rather than as a comprehensive list of every possible change. There are three distinct types of compatibility to consider: 1. Source compatibility: Code written against a previous version **must** compile against a newer version, and successfully run with a newer version of the client library. 2. Wire compatibility: Code written against a previous version **must** be able to communicate correctly with a newer server. In other words, not only are inputs and outputs compatible, but the serialization and deserialization expectations continue to match. 3. Semantic compatibility: Code written against a previous version **must** continue to receive what most reasonable developers would expect. (This can be tricky in practice, however, and sometimes determining what users will expect can involve a judgment call.) **Note:** In general, the specific guidance here assumes use of protocol buffers and JSON as transport formats. Other transport formats may have slightly different rules. **Note:** This guidance assumes that APIs are intended to be called from a range of consumers, written in multiple languages and with no control over how and when consumers update. Any API which has a more limited scope (for example, an API which is only called by client code written by the same team as the API producer, or deployed in a way which can enforce updates) should carefully consider its own compatibility requirements. ### Adding components In general, new components (interfaces, methods, messages, fields, enums, or enum values) **may** be added to existing APIs in the same major version. However, keep the following guidelines in mind when doing this: - Code written against the previous surface (and thus is unaware of the new components) **must** continue to be treated the same way as before. - New required fields **must not** be added to existing request messages or resources. - Any field being populated by clients **must** have a default behavior matching the behavior before the field was introduced. - This can be tricky to do in some cases. For example, adding pagination after the fact where previously all items were returned (i.e. `page_size` is infinite, which is not advised). If the default for the new `page_size` field is _less_ than what was previously returned, older clients will incorrectly assume all results were returned. - Any field previously populated by the server **must** continue to be populated, even if it introduces redundancy. - For enum values specifically, be aware that it is possible that user code does not handle new values gracefully. - Enum values **may** be freely added to enums which are only used in request messages. - Enums that are used in response messages or resources and which are expected to receive new values **should** document this. Enum values still **may** be added in this situation; however, appropriate caution **should** be used. **Note:** It is possible when adding a component closely related to an existing component (for example, `string foo_value` when `string foo` already exists) to enter a situation where generated code will conflict. Service owners **should** be aware of subtleties in the tooling they or their users are likely to use (and tool authors **should** endeavor to avoid such subtleties if possible). ### Removing or renaming components Existing components (interfaces, methods, messages, fields, enums, or enum values) **must not** be removed from existing APIs in the same major version. Removing a component is a backwards incompatible change. **Important:** Renaming a component is semantically equivalent to "remove and add". In cases where these sorts of changes are desirable, a service **may** add the new component, but **must not** remove the existing one. In situations where this can allow users to specify conflicting values for the same semantic idea, the behavior **must** be clearly specified. ### Moving components between files Existing components **must not** be moved between files. Moving a component from one proto file to another within the same package is wire compatible, however, the code generated for languages like C++ or Python will result in breaking change since `import` and `#include` will no longer point to the correct code location. ### Moving into oneofs Existing fields **must not** be moved into or out of a oneof. This is a backwards-incompatible change in the Go protobuf stubs. ### Changing the type of fields Existing fields and messages **must not** have their type changed, even if the new type is wire-compatible, because type changes alter generated code in a breaking way. ### Changing string length APIs **should** avoid increasing the upper bound for the size or limit (if accepted as input) of `string` fields. APIs **should** treat expected size upper bound increases as incompatible changes (see [Changing resource names](#changing-resource-names) as an example). APIs **may** pad out values with filler characters if reserving a consistent size is necessary, but this **must** be documented if done. ### Changing resource names A resource **must not** change its [name][aip-122]. Unlike most breaking changes, this affects major versions as well: in order for a client to expect to use v2.0 access a resource that was created in v1.0 or vice versa, the same resource name **must** be used in both versions. More subtly, the set of valid resource names **should not** change either, for the following reasons: - If resource name formats become more restrictive, a request that would previously have succeeded will now fail. - If resource name formats become less restrictive than previously documented, then code making assumptions based on the previous documentation could break. Users are very likely to store resource names elsewhere, in ways that may be sensitive to the set of permitted characters and the length of the name. Alternatively, users might perform their own resource name validation to follow the documentation. - For example, Amazon gave customers [a lot of warning][ec2] and had a migration period when they started allowing longer EC2 resource IDs. ### Semantic changes Code will often depend on API behavior and semantics, _even when such behavior is not explicitly supported or documented_. Therefore, APIs **must not** change visible behavior or semantics in ways that are likely to break reasonable user code, as such changes will be seen as breaking by those users. **Note:** This does involve some level of judgment; it is not always clear whether a proposed change is likely to break users, and an expansive reading of this guidance could ostensibly prevent _any_ change (which is not the intent). #### Changing value format or construction APIs **must not** change the expected format or algorithm used to construct the value of an existing field - even if the field is `OUTPUT_ONLY` and populated by the API service - within an API version. Doing so requires a new API version. For example, changing the format of a field `ip_address` conforming to IPv4 format to instead contain IPv6 values is a breaking change. #### Default values must not change Default values are the values set by servers for resources when they are not specified by the client. This section only applies to static default values within fields on resources and does not apply to dynamic defaults such as the default IP address of a resource. Changing the default value is considered breaking and **must not** be done. The default behavior for a resource is determined by its default values, and this **must not** change across minor versions. For example: ```proto message Book { // google.api.resource and other annotations and fields // The genre of the book // If this is not set when the book is created, the field will be given a value of FICTION. enum Genre { UNSPECIFIED = 0; FICTION = 1; NONFICTION = 2; } } ``` Changing to: ```proto message Book { // google.api.resource and other annotations and fields // The genre of the book // If this is not set when the book is created, the field will be given a value of NONFICTION. enum Genre { UNSPECIFIED = 0; FICTION = 1; NONFICTION = 2; } } ``` would constitute a breaking change. #### Serializing defaults APIs **must not** change the way a field with a default value is serialized. For example if a field does not appear in the response if the value is equal to the default, the serialization **must not** change to include the field with the default. Clients may depend on the presence or absence of a field in a resource as semantically meaningful, so a change to how serialization is done for absent values **must not** occur in a minor version. Consider the following proto, where the default value of `wheels` is `2`: ```proto // A representation of an automobile message Automobile { // google.api.resource and other annotations and fields // The number of wheels on the automobile. // The default value is 2, when no value is sent by the client. int wheels = 2; } ``` First the proto serializes to JSON when the value of `wheels` is `2` as follows: ```json { "name": "my-car" } ``` Then, the API service changes the serialization to include `wheel` even if the value is equal to the default value, `2` as follows: ```json { "name": "my-car", "wheels": 2 } ``` This constitutes a change that is not backwards compatible within a major version. ## Further reading - For compatibility around field behavior, see [AIP-203][]. - For compatibility around pagination, see [AIP-158][]. - For compatibility around long-running operations, see [AIP-151][]. - For understanding stability levels and expectations, see [AIP-181][]. - For compatibility with client library resource name parsing, see [AIP-4231][] - For compatibility with client library method signatures, see [AIP-4232][] - For compatibility around field presence changes, see [AIP-149][]. - For compatibility around resource types, see [AIP-123][]. ## Rationale ### Risk of string length changes End users may store resource properties, like the `name`, in a dedicated database column with a limited length. If the service starts returning values for the `name` that are twice the originally documented/observed length, this may unexpectedly break the customer's database. Furthermore, string properties that appear in URLs (including query parameters) are especially likely to have client-side limits, making them more sensitive to length changes. ### Risk of changing value format or construction Customers often depend on the format or algorithmic construction of a field for client-side parsing, hashing, or database table construction. Changing it in an existing field could break that client-side consumption. ## Changelog - **2025-10-21**: Added guidance for string length changes, changing formats, and an example for carefully adding components. - **2024-08-07**: Added reference to resource type compatibility. - **2024-06-05**: Added reference to field presence compatibility. - **2023-07-26**: Added reference to field behavior compatibility. - **2023-07-26**: Added note on APIs which have limited clients. - **2022-08-11**: Added "Moving components between files" section. - **2022-06-01**: Added more links to other AIPs with compatibility concerns - **2019-12-16**: Clarified that moving existing fields into oneofs is breaking. [aip-122]: ./0122.md [aip-123]: ./0123.md [aip-149]: ./0149.md [aip-151]: ./0151.md [aip-158]: ./0158.md [aip-181]: ./0181.md [aip-203]: ./0203.md [aip-4231]: https://google.aip.dev/client-libraries/4231 [aip-4232]: https://google.aip.dev/client-libraries/4232 [ec2]: https://aws.amazon.com/blogs/aws/theyre-here-longer-ec2-resource-ids-now-available/ --- id: 181 state: approved created: 2019-02-18 placement: category: compatibility order: 20 --- # Stability levels While different organizations (both inside Google and outside) have different product life cycles, AIPs refer to the _stability_ of an API component using the following terms. **Note:** These stability levels roughly correspond to the product launch stages (alpha, beta, GA) in Google Cloud, but are not identical. GCP imposes its own additional expectations and commitments on top of what is outlined here. ## Alpha An _alpha_ component undergoes rapid iteration with a known set of users who **must** be tolerant of change. The number of users **should** be a curated, manageable set, such that it is feasible to communicate with all of them individually. Breaking changes **must** be both allowed and expected in alpha components, and users **must** have no expectation of stability. ## Beta A _beta_ component **must** be considered complete and ready to be declared stable, subject to public testing. Beta components **should** be exposed to an unknown and potentially large set of users. In other words, beta components **should not** be behind an allowlist; instead, they **should** be available to the public. Because users of beta components tend to have a lower tolerance of change, beta components **should** be as stable as possible; however, the beta component **must** be permitted to change over time. These changes **should** be minimal but **may** include backwards-incompatible changes to beta components. Backwards-incompatible changes **must** be made only after a reasonable deprecation period to provide users with an opportunity to migrate their code. This deprecation period **must** be defined at the time of being marked beta. Beta components **should** be time-boxed and promoted to stable if no issues are found in the specified timeframe, which **should** be specified at the time of being marked beta. A reasonable time period **may** vary, but a good rule of thumb is 90 days. ## Stable A _stable_ component **must** be fully-supported over the lifetime of the major API version. Because users expect such stability from components marked stable, there **must** be no breaking changes to these components, subject to the caveats described below. ### Major versions When breaking changes become necessary, the API producer **should** create the next major version of the API, and start a deprecation clock on the existing version. Turn-down of any version containing stable components **must** have a formal process defined at the time of being marked stable. This process **must** specify a deprecation period for users which provides them with reasonable advance warning. ### Isolated changes On very rare occasions, it could be preferable to make a small, isolated breaking change, if this will only cause inconvenience to a small subset of users. (Creating a new major version is an inconvenience to all users.) In this case, the API producer **may** deprecate the component, but **must** continue to support the component for the normal turndown period for a stable component. **Important:** Making an in-place breaking change in a stable API is considered an extreme course of action, and should be treated with equal or greater gravity as creating a new major version. For example, at Google, this requires the approval of the API Governance team. ### Emergency changes In certain exceptional cases, such as security concerns or regulatory requirements, any API component **may** be changed in a breaking manner regardless of its stability level, and a deprecation is not promised in these situations. --- id: 182 state: reviewing created: 2021-08-13 placement: category: compatibility order: 30 --- # External software dependencies Some services have a particular type of dependency on external software: they allow users to create resources that run on or expose the external software in some way. For example: - A database admin service can allow users to create databases running on a particular version of a particular database engine (for example, PostgreSQL 13.4). - A virtual machine service can allow users to create VMs running a particular operating system (for example, Ubuntu 20.04). - An application or function platform service can allow users to write code that runs against a particular version of a programming language (for example, Node.js 16.6). Services that provide external software to users in this way will eventually need to address the fact that all of these types of software have release lifecycles, and the versions they currently expose will eventually reach end-of-life. ## Guidance Services that expose external software dependencies **should** allow users to create resources using any currently-supported LTS (long-term support) version of the supported software, and **may** allow users to create resources using non-LTS versions. Services **should not** indefinitely allow users to create new resources using versions that have reached end-of-life, although they **may** have a transition period between when the software version reaches end-of-life and when support for creating new resources with that version is removed. **Note:** Restricting or removing the ability to create resources using end-of-life versions of software is **not** considered a breaking change for the service for the purpose of AIP-181, even though it actually is one. However, because the change can break existing users' workflows, services **must** notify users who are using resources approaching end-of-life. If possible, services **should** allow previously-created resources to remain, and **may** warn users of the risks associated with continuing to use end-of-life software. Services **should not** proactively remove resources using end-of-life software, or impose other restrictions on existing resources, unless critical security concerns require the service to do so. ### Continued support If supporting a version that has reached end-of-life is necessary for business reasons (usually because the end-of-life software still has significant adoption), the service **may** choose to officially support the end-of-life version, but **must** take on the responsibility of patching and maintaining the software if it does so. --- id: 185 state: approved created: 2024-10-22 placement: category: compatibility order: 40 --- # API Versioning This topic describes the versioning strategies used by Google APIs. In general, these strategies apply to all Google-managed services. ## Guidance All Google API interfaces **must** provide a _major version number_, which is encoded at the end of the protobuf package, and included as the first part of the URI path for REST APIs. In the event an API needs to make an incompatible change, consult [AIP-180][] and [AIP-181][] for necessary steps based on the stability level of the surface in question. Note: The use of the term "major version number" above is taken from [semantic versioning][]. However, unlike in traditional semantic versioning, Google APIs **must not** expose minor or patch version numbers. For example, Google APIs use `v1`, not `v1.0`, `v1.1`, or `v1.4.2`. From a user's perspective, major versions are updated in place with minor/patch equivalent changes, and users receive new functionality without migration. A new major version of an API **must not** depend on a previous major version of the same API. An API surface **must not** depend on other APIs, except for in the cases outlined in [AIP-213][] and [AIP-215][]. Different versions of the same API **must** be able to work at the same time within a single client application for a reasonable transition period. This time period allows the client to transition smoothly to the newer version. An older version **must** go through a reasonable, well-communicated deprecation period before being shut down. For releases which have [alpha or beta stability][AIP-181], APIs **must** append the stability level after the major version number in the protobuf package and URI path using one of these strategies: - Channel-based versioning (recommended) - Release-based versioning - Visibility-based versioning ### Channel-based versioning A *stability channel* is a long-lived release at a given stability level that receives in-place updates. There is no more than one channel per stability level for a major version. Under this strategy, there are up to three channels available: alpha, beta, and stable. The alpha and beta channel **must** have their stability level appended to the version, but the stable channel **must not** have the stability level appended. For example, `v1` is an acceptable version for the stable channel, but `v1beta` or `v1alpha` are not. Similarly, `v1beta` or `v1alpha` are acceptable versions for the respective beta and alpha channel, but `v1` is not acceptable for either. Each of these channels receives new features and updates "in-place". The beta channel's functionality **must** be a superset of the stable channel's functionality, and the alpha channel's functionality **must** be a superset of the beta channel's functionality. #### Deprecating API functionality API elements (fields, messages, RPCs) **may** be marked deprecated in any channel to indicate that they should no longer be used: ```proto // Represents a scroll. Books are preferred over scrolls. message Scroll { option deprecated = true; // ... } ``` Deprecated API functionality **must not** graduate from alpha to beta, nor beta to stable. In other words, functionality **must not** arrive "pre-deprecated" in any channel. The beta channel's functionality **may** be removed after it has been deprecated for a sufficient period; we recommend 180 days. For functionality that exists only in the alpha channel, deprecation is optional, and functionality **may** be removed without notice. If functionality is deprecated in an API's alpha channel before removal, the API **should** apply the same annotation, and **may** use any timeframe it wishes. ### Release-based versioning **Important:** This pattern is not commonly used for new services. There are existing services that follow it, but Channel-based Versioning is the preferred mechanism. An *individual release* is an alpha or beta release that is expected to be available for a limited time period before its functionality is incorporated into the stable channel, after which the individual release will be shut down. When using release-based versioning strategy, an API may have any number of individual releases at each stability level. Note: Both the channel-based and release-based strategies update the _stable_ version in-place. There is a single stable channel, rather than individual stable releases, even when using the release-based strategy. Alpha and beta releases **must** have their stability level appended to the version, followed by an incrementing release number. For example, `v1beta1` or `v1alpha5`. APIs **should** document the chronological order of these versions in their documentation (such as comments). Each alpha or beta release **may** be updated in place with backwards-compatible changes. For beta releases, backwards-incompatible updates **should** be made by incrementing the release number and publishing a new release with the change. For example, if the current version is `v1beta1`, then `v1beta2` is released next. Alpha and beta releases **should** be shut down after their functionality reaches the stable channel. An alpha release **may** be shut down at any time, while a beta release **should** allow users a reasonable transition period; we recommend 180 days. ### Visibility-based versioning [API visibility][] is an advanced feature provided by Google API infrastructure. It allows API producers to expose multiple external API views from one internal API surface, and each view is associated with an API _visibility label_, such as: ```proto import "google/api/visibility.proto"; message Resource { string name = 1; // Preview. Do not use this feature for production. string display_name = 2 [(google.api.field_visibility).restriction = "PREVIEW"]; } ``` A visibility label is a case-sensitive string that can be used to tag any API element. By convention, visibility labels should always use UPPER case. An implicit `PUBLIC` label is applied to all API elements unless an explicit visibility label is applied as in the example above. Each visibility label is an allow-list. API producers need to grant visibility labels to API consumers for them to use API features associated with the labels. In other words, an API visibility label is like an ACL'ed API version. Multiple visibility labels **may** be applied to an element by using a comma-separated string (e.g. `"PREVIEW,TRUSTED_TESTER"`). When multiple visibility labels are used, then the client needs only _one_ of the visibility labels (logical `OR`). By default, the visibility labels granted to the API consumer are used to verify incoming requests. However, a client can send requests with an explicit visibility label as follows: ``` GET /v1/projects/my-project/topics HTTP/1.1 Host: pubsub.googleapis.com Authorization: Bearer y29.... X-Goog-Visibilities: PREVIEW ``` A single API request can specify at most one visibility label. API producers can use API visibility for API versioning, such as `INTERNAL` and `PREVIEW`. A new API feature starts with the `INTERNAL` label, then moves to the `PREVIEW` label. When the feature is stable and becomes generally available, all API visibility labels are removed from the API definition. In general, API visibility is easier to implement than API versioning for incremental changes, but it depends on sophisticated API infrastructure support. Google Cloud APIs often use API visibility for Preview features. [AIP-180]: https://aip.dev/180 [AIP-181]: https://aip.dev/181 [AIP-213]: https://aip.dev/213 [AIP-215]: https://aip.dev/215 [api visibility]: https://github.com/googleapis/googleapis/blob/master/google/api/visibility.proto [semantic versioning]: https://semver.org/ --- id: 190 state: approved created: 2025-06-10 placement: category: polish order: 0 --- # Naming conventions This topic describes the naming conventions used in Google APIs. In general, these conventions apply to all Google-managed services. ## Guidance In order to provide consistent developer experience across many APIs and over a long period of time, all names used by an API **should** be: - straightforward - intuitive - consistent This includes names of interfaces, resources, collections, methods, and messages. Since English is a second language for many developers, one goal of these naming conventions is to make every API name understandable to the majority of developers. It does this by encouraging the use of a simple, consistent, and small vocabulary when naming methods and resources. - Names used in APIs **should** be in correct American English. For example, license (instead of licence), color (instead of colour). - Commonly accepted short forms or abbreviations of long words **may** be used for brevity. For example, API is preferred over Application Programming Interface. - Unless otherwise specified, definitions **must** use UpperCamelCase names, as defined by [Google Java Style](https://google.github.io/styleguide/javaguide.html#s5.3-camel-case). - Use intuitive, familiar terminology where possible. For example, when describing removing (and destroying) a resource, delete is preferred over erase. - Use the same name or term for the same concept, including for concepts shared across APIs. - Avoid name overloading. Use different names for different concepts. - Avoid overly general names that are ambiguous within the context of the API and the larger ecosystem of Google APIs. They can lead to misunderstanding of API concepts. Rather, choose specific names that accurately describe the API concept. This is particularly important for names that define first-order API elements, such as resources. There is no definitive list of names to avoid, as every name must be evaluated in the context of other names. Instance, info, and service are examples of names that have been problematic in the past. Names chosen should describe the API concept clearly (for example: instance of what?) and distinguish it from other relevant concepts (for example: does "alert" mean the rule, the signal, or the notification?). - Carefully consider use of names that may conflict with keywords in common programming languages. Such names **may** be used but will likely trigger additional scrutiny during API review. Use them judiciously and sparingly. ### Interface names To avoid confusion with [Service Names](./0009.md#api-service-name) such as `pubsub.googleapis.com`, the term *interface name* refers to the name used when defining a `service` in a .proto file: ```proto // Library is the interface name. service Library { rpc ListBooks(...) returns (...); rpc ... } ``` You can think of the *service name* as a reference to the actual implementation of a set of APIs, while the *interface name* refers to the abstract definition of an API. An interface name **should** use an intuitive noun such as Calendar or BlobStore. The name **should not** conflict with any well-established concepts in programming languages and their runtime libraries (for example, File). In the rare case where an *interface name* would conflict with another name within the API, a suffix (for example `Api` or `Service`) **should** be used to disambiguate. ### Method names A service **may**, in its IDL specification, define one or more API methods that correspond to methods on collections and resources. The method names **should** follow the naming convention of `VerbNoun` in UpperCamelCase, where the noun is typically the resource type. Standard methods, and their Batch variants, define their naming guidance in the following documents: Method | Standard | Batch ------ | -------- | ----- `Get` | [AIP-131][] | [AIP-231][] `List` | [AIP-132][] | N/A `Create` | [AIP-133][] | [AIP-233][] `Update` | [AIP-134][] | [AIP-234][] `Delete` | [AIP-135][] | [AIP-235][] All other methods are considered Custom Methods and adhere to AIP-136 naming guidance. ### Message names Message names **should** be short and concise. Avoid unnecessary or redundant words. Adjectives can often be omitted if there is no corresponding message without the adjective. For example, the `Shared` in `SharedProxySettings` is unnecessary if there are no _unshared_ proxy settings. Message names **should not** include prepositions (e.g. "With", "For"). Generally, message names with prepositions are better represented with optional fields on the message. #### Request and response messages For request and response message names, see AIP-136 for custom methods and the appropriate AIP for [standard methods](https://google.aip.dev/general#operations). ## Further reading - For proto and language package naming, see AIP-191. - For collection ID naming conventions, see [AIP-122](./0122.md#collection-identifiers). - For Enum names, see AIP-126. - For field names, see AIP-140. - For repeated field names, see [AIP-140](./0140#repeated-fields). - For fields representing times and durations, see AIP-142. - For fields representing dates and times of day, see [AIP-142](./0142#civil-dates-and-times). - For fields representing a quantity, see AIP-141. - For the canonical `List` method `filter` field, see [AIP-132](./0132#filtering). - For the canonical `List` response message, see [AIP-132](./0132#response-message). - For well known abbreviations, see [AIP-140](./0140#abbreviations). [AIP-131]: ./0131.md [AIP-132]: ./0132.md [AIP-133]: ./0133.md [AIP-134]: ./0134.md [AIP-135]: ./0135.md [AIP-231]: ./0231.md [AIP-233]: ./0233.md [AIP-234]: ./0234.md [AIP-235]: ./0235.md --- id: 191 state: approved created: 2019-07-25 placement: category: polish order: 10 --- # File and directory structure A consistent file and directory structure, while making minimal difference technically, makes API surface definitions easier for users and reviewers to read. ## Guidance **Note:** The following guidance applies to APIs defined in protocol buffers, such as those used throughout Google. While the spirit of this guidance applies to APIs defined using other specification languages or formats, some of the particular recommendations might be irrelevant. ### Syntax APIs defined in protocol buffers **must** use `proto3` syntax. ### Single package APIs defined in protocol buffers **must** define each individual API in a single package, which **must** end in a version component. For example: ```proto syntax = "proto3"; package google.cloud.translation.v3; ``` Google APIs **must** reside in a directory that matches the protocol buffer `package` directive. For example, the package above dictates that the directory be `google/cloud/translation/v3`. ### File names It is often useful to divide API definitions into multiple files. File names **must** use `snake_case`. APIs **should** have an obvious "entry" file, generally named after the API itself. An API with a small number of discrete services (Google Cloud Pub/Sub's `Publisher` and `Subscriber` is a good example) **may** have a separate entry file per service. APIs with only one file **should** use a filename corresponding to the name of the API. API `service` definitions and associated RPC request and response `message` definitions **should** be defined in the same file. Bear in mind that the file names often become module names in client libraries, and customers use them in `import` or `use` statements. Therefore, choosing a descriptive and language keyword-free filename does matter. For example, a file called `import.proto` may be problematic in Python. **Note:** The version **must not** be used as a filename, because this creates bizarre imports in client libraries. Filenames such as `v3.proto` or `v1beta1.proto` are prohibited. ### File layout Individual files **should** place higher level and more important definitions before lower level and less important definitions. In a proto file, components **should** be in the following order, and each of these **should** be separated by a blank line: - Copyright and license notice (if applicable). - The proto `syntax` statement. - The proto `package` statement. - Any `import` statements, in alphabetical order. - Any file-level `option` statements. - Any `service` definitions. - Methods **should** be grouped by the resource they impact, and standard methods **should** precede custom methods. - Resource `message` definitions. A parent resource **must** be defined before its child resources. - The RPC request and response `message` definitions, in the same order of the corresponding methods. Each request message **must** precede its corresponding response message (if any). - Any remaining `message` definitions. - Any top-level `enum` definitions. ### Packaging annotations Protocol buffers ships with annotations to declare the package or namespace (depending on the vocabulary of the target language) of the generated files. For example, setting `go_package` or `csharp_namespace` will override the inferred package name. When defining APIs, the following rules apply: - Java - The `java_package` annotation **must** be set. The correct value is usually the proto package with the appropriate TLD prefixed. Example: `com.google.example.v1`. - The `java_multiple_files` annotation **must** be set to `true`. - The `java_outer_classname` annotation **must** be set, and **should** be set to the name of the proto filename, in `PascalCase`, with `Proto` appended. Example: `LibraryProto`. - Other languages - Package or namespace directives for other languages **must** be set either in every file in the proto package, or none of them. If they are set, the values **must** be identical in every file. - If any part of the protobuf package is a compound name (such as accessapproval), C#, Ruby and PHP options **must** be specified in order to take account of the word breaks using PascalCase (UpperCamelCase). Example: ```proto option csharp_namespace = "Google.Cloud.AccessApproval.V1"; option php_namespace = "Google\\Cloud\\AccessApproval\\V1"; option ruby_package = "Google::Cloud::AccessApproval::V1"; ``` - The `go_package` value depends directly on how the Go code is managed i.e. if the module name is based on the VCS provider or using a remote import path, but often has a consistent structure. - The module **may** differ based on product area e.g. `google.cloud.accessapproval.v1` would be in module `cloud.google.com/go/accessapproval`. - The package import path **should** be derived from the proto package. - An API version in the proto package **should** be prefixed with `api` e.g. the proto package segment `v1` becomes `apiv1`. - The terminal import path segment **should** be based on the product name found within the proto package and **must** be suffixed with `pb` e.g. `accessapproval` becomes `accessapprovalpb`. - This value **should** be left to the team owning the generated code to decide on. All packaging annotations **should** be specified in alphabetical order of name. Refer to the [Protobuf documentation][package docs] for more about language package options. **Important:** While languages other than Java have sensible defaults for APIs which don't include compound names, be aware that _adding_ this annotation (with a value not equivalent to the default) constitutes a breaking change in that language. When releasing protos, be sure that omissions are intentional. ## Rationale ### Java packaging options Set the option, `java_multiple_files`, to true to get a cleaner file structure. Doing so instructs `protoc` to create one output file per Protobuf type, which allows for more fine-grained imports. The option, `java_outer_classname`, is required in combination with `java_multiple_files`. It instructs `protoc` to wrap each compiled Protobuf type in a Java class whose name is the value of the option. This prevents potential naming collisions between generated types. ### Go packaging option The Go packaging option needs to be decided by the team that owns the generated code, because it is directly tied to the source code management practices of the team. Allowing every proto package to decide on their own Go package creates inconsistencies and friction in management of the code. Within that owning team, having a consistent structure in the Go package naming is critical to a consistent end user experience. ## Changelog - **2024-06-13**: Added guidance for Go packaging annotation. - **2024-06-05**: Added rationale for Java packaging options. - **2023-02-24**: Added guidance on protobuf syntax. - **2022-10-18**: Added guidance on Ruby/PHP/C# options. - **2019-11-18**: Added guidance on the packaging annotations. [package docs]: https://protobuf.dev/programming-guides/proto3/#packages --- id: 192 state: approved created: 2019-07-25 placement: category: polish order: 20 --- # Documentation Documentation is one of the most critical aspects of API design. Users of your API are unable to dig into the implementation to understand the API better; often, the API surface definition and its corresponding documentation will be the only things a user has. Therefore, it is important that documentation be as clear, complete, and unambiguous as possible. ## Guidance In APIs defined in protocol buffers, public comments **must** be included over every component (service, method, message, field, enum, and enum value) using the protocol buffers comment format. This is important even in cases where the comment is terse and uninteresting, as numerous tools read these comments and use them. Services, in particular, **should** have descriptive comments that explain what the service is and what users are able to do with it. **Note:** Many readers will not be native English speakers. Comments **should** avoid jargon, slang, complex metaphors, pop culture references, or anything else that will not easily translate. Additionally, many readers will have different backgrounds and viewpoints; if writing examples involving people, comments **should** use people who are non-controversial and no longer alive. ### Style Comments **should** be in grammatically correct American English. However, the first sentence of each comment **should** omit the subject and be in the third-person present tense: ```proto // Creates a book under the given publisher. rpc CreateBook(CreateBookRequest) returns (Book) { option (google.api.http) = { post: "/v1/{parent=publishers/*}/books" body: "book" }; } ``` ### Descriptions Descriptions of messages and fields **should** be brief but complete. Sometimes comments are necessarily perfunctory because there is little to be said; however, before jumping to that conclusion, consider whether some of the following questions are relevant: - What is it? - How do you use it? - What does it do if it succeeds? What does it do if it fails? - Is it idempotent? - What are the units? (Examples: meters, degrees, pixels) - What are the side effects? - What are common errors that may break it? - What is the expected input format? - What range of values does it accept? (Examples: `[0.0, 1.0)`, `[1, 10]`) - Is the range inclusive or exclusive? - For strings, what is the minimum and maximum length, and what characters are allowed? - If a value is above the maximum length, do you truncate or send an error? - Is it always present? (Example: "Container for voting information. Present only when voting information is recorded.") - Does it have a default setting? (Example: "If `page_size` is omitted, the default is 50.") ### Formatting Any formatting in comments **must** be in [CommonMark][]. Headings and tables **must not** be used, as these cause problems for several tools, and are unsuitable for client library reference documentation. Comments **should** use `code font` for field or method names and for literals (such as `true`). Raw HTML **must not** be used. "ASCII art" attempts to present a diagram within the protos **must not** be used. The Markdown within the protos is consumed by a large number of renderers, and any ASCII art is very unlikely to be well-presented by all of them. If a diagram is useful in order to understand the API, include a link to a documentation page containing the diagram as an image. ### Cross-references A comment can "link" to another component (service, method, message, field, enum, or enum value) as a Markdown reference link. The reference **must** be one of the following forms: - The fully-qualified name of the element e.g. `[Book][google.example.v1.Book]` - A scope-relative reference qualified e.g. `[Sci-Fi genre][Genre.GENRE_SCI_FI]` - An implied reference e.g. `[Book][]` which equates to `[Book][Book]` These references are resolved as per [name resolution][] rules. Containing fields names **must not** be used in references. They will not resolve. The original definition **must** be referenced instead. For example, `[author][Book.author.family_name]` where `author` is a field of `Book`, will not resolve, but `[author][Author.family_name]` will. [name resolution]: https://protobuf.dev/programming-guides/proto3/#name-resolution ### External links Comments **may** link to external pages to provide background information beyond what is described in the public comments themselves. External links **must** use absolute (rather than relative) URLs, including the protocol (usually `https`), and **should not** assume the documentation is located on any particular host. For example: `[Spanner Documentation](https://cloud.google.com/spanner/docs)` ### Trademarked names When referring to the proper, trademarked names of companies or products in comments, acronyms **should not** be used, unless the acronym is such dominant colloquial use that avoiding it would obscure the reference (example: [IBM][]). Comments **should** spell and capitalize trademarked names consistent with the trademark owner's current branding. ### Deprecations To deprecate a component (service, method, message, field, enum, or enum value), the `deprecated` [option](https://developers.google.com/protocol-buffers/docs/proto#options) **must** be set to `true`, and the first line of the respective comment **must** start with `"Deprecated: "` and provide alternative solutions for developers. If there is no alternative solution, a deprecation reason **must** be given. ### Internal comments Comments **may** be explicitly marked as internal by wrapping internal content in `(--` and `--)`. Non-public links, internal implementation notes (such as `TODO` and `FIXME` directives), and other such material **must** be marked as internal. **Note:** Comments **should** use only leading comments (not trailing comments or detached comments). In particular, comments **must not** use both a leading and trailing comment to describe any component, because this is a common source of inadvertent omissions of the internal content annotation. [commonmark]: https://commonmark.org/ [ibm]: https://en.wikipedia.org/wiki/IBM ## Changelog - **2024-10-29**: Include cross-reference resolution rules. - **2023-08-11**: Expand deprecated comment requirement to all components. - **2021-04-20**: Added guidance for deprecated services and RPCs. - **2020-04-01**: Added guidance requiring absolute URLs for external links. - **2020-02-14**: Added guidance around the use of trademarked names. - **2019-09-23**: Added guidance about not using both leading and trailing comments. - **2019-08-01**: Changed the examples from "shelves" to "publishers", to present a better example of resource ownership. --- id: 193 state: approved created: 2019-07-26 placement: category: polish order: 30 --- # Errors Effective error communication is an important part of designing simple and intuitive APIs. Services returning standardized error responses enable API clients to construct centralized common error handling logic. This common logic simplifies API client applications and eliminates the need for cumbersome custom error handling code. ## Guidance Services **must** return a [`google.rpc.Status`][Status] message when an API error occurs, and **must** use the canonical error codes defined in [`google.rpc.Code`][Code]. More information about the particular codes is available in the [gRPC status code documentation][]. Error messages **should** help a reasonably technical user *understand* and *resolve* the issue, and **should not** assume that the user is an expert in your particular API. Additionally, error messages **must not** assume that the user will know anything about its underlying implementation. Error messages **should** be brief but actionable. Any extra information **should** be provided in the `details` field. If even more information is necessary, you **should** provide a link where a reader can get more information or ask questions to help resolve the issue. It is also important to [set the right tone][writing-tone] when writing messages. The following sections describe the fields of `google.rpc.Status`. ### Status.message The `message` field is a developer-facing, human-readable "debug message" which **should** be in English. (Localized messages are expressed using a `LocalizedMessage` within the `details` field. See [`LocalizedMessage`](#localizedmessage) for more details.) Any dynamic aspects of the message **must** be included as metadata within the `ErrorInfo` that appears in [`details`](#status-details). The message is considered a problem description. It is intended for developers to understand the problem and is more detailed than [`ErrorInfo.reason`][ErrorInfo-reason], discussed [later](#errorinfo). Messages **should** use simple descriptive language that is easy to understand (without technical jargon) to clearly state the problem that results in an error, and offer an actionable resolution to it. For pre-existing (brownfield) APIs which have previously returned errors without machine-readable identifiers, the value of `message` **must** remain the same for any given error. For more information, see [Changing Error Messages](#changing-error-messages). ### Status.code The `code` field is the status code, which **must** be the numeric value of one of the elements of the [`google.rpc.Code`][Code] enum. For example, the value `5` is the numeric value of the `NOT_FOUND` enum element. ### Status.details The `details` field allows messages with additional error information to be included in the error response, each packed in a `google.protobuf.Any` message. Google defines a set of [standard detail payloads][details] for error details, which cover most common needs for API errors. Services **should** use these standard detail payloads when feasible. Each type of detail payload **must** be included at most once. For example, there **must not** be more than one [`BadRequest`][BadRequest] message in the `details`, but there **may** be a `BadRequest` and a [`PreconditionFailure`][PreconditionFailure]. All error responses **must** include an `ErrorInfo` within `details`. This provides machine-readable identifiers so that users can write code against specific aspects of the error. The following sections describe the most common standard detail payloads. #### ErrorInfo The [`ErrorInfo`][ErrorInfo] message is the primary way to send a machine-readable identifier. Contextual information **should** be included in `metadata` in `ErrorInfo` and **must** be included if it appears within an error message. The `reason` field is a short snake_case description of the cause of the error. Error reasons are unique within a particular domain of errors. The reason **must** be at most 63 characters and match a regular expression of `[A-Z][A-Z0-9_]+[A-Z0-9]`. (This is UPPER_SNAKE_CASE, without leading or trailing underscores, and without leading digits.) The reason **should** be terse, but meaningful enough for a human reader to understand what the reason refers to. Good examples: - `CPU_AVAILABILITY` - `NO_STOCK` - `CHECKED_OUT` - `AVAILABILITY_ERROR` Bad examples: - `THE_BOOK_YOU_WANT_IS_NOT_AVAILABLE` (overly verbose) - `ERROR` (too general) The `domain` field is the logical grouping to which the `reason` belongs. The domain **must** be a globally unique value, and is typically the name of the service that generated the error, e.g. `pubsub.googleapis.com`. The (reason, domain) pair form a machine-readable way of identifying a particular error. Services **must** use the same (reason, domain) pair for the same error, and **must not** use the same (reason, domain) pair for logically different errors. The decision about whether two errors are "the same" or not is not always clear, but **should** generally be considered in terms of the expected action a client might take to resolve them. The `metadata` field is a map of key/value pairs providing additional dynamic information as context. Each key within `metadata` **must** be at most 64 characters long, and conform to the regular expression `[a-z][a-zA-Z0-9-_]+`. Any request-specific information which contributes to the `Status.message` or `LocalizedMessage.message` messages **must** be represented within `metadata`. This practice is critical so that machine actors do not need to parse error messages to extract information. For example consider the following message: > An <e2-medium> VM instance with <local-ssd=3,nvidia-t4=2> is currently unavailable > in the <us-east1-a> zone. Consider trying your request in the <us-central1-f,us-central1-c> > zone(s), which currently has/have capacity to accommodate your request. Alternatively, > you can try your request again with a different VM hardware configuration > or at a later time. For more information, see the troubleshooting documentation. The `ErrorInfo.metadata` map for the same error could be: - `"zone": "us-east1-a"` - `"vmType": "e2-medium"` - `"attachment": "local-ssd=3,nvidia-t4=2"` - `"zonesWithCapacity": "us-central1-f,us-central1-c"` Additional contextual information that does not appear in an error message **may** also be included in `metadata` to allow programmatic use by the client. The metadata included for any given (reason,domain) pair can evolve over time: - New keys **may** be included - All keys that have been included **must** continue to be included (but may have empty values) In other words, once a user has observed a given key for a (reason, domain) pair, the service **must** allow them to rely on it continuing to be present in the future. The set of keys provided in each (reason, domain) pair is independent from other pairs, but services **should** aim for consistent key naming. For example, two error reasons within the same domain should not use metadata keys of `vmType` and `virtualMachineType`. #### LocalizedMessage [`google.rpc.LocalizedMessage`][LocalizedMessage] is used to provide an error message which **should** be localized to a user-specified locale where possible. If the [`Status.message`](#status-message) field has a sub-optimal value which cannot be changed due to the constraints in the [Changing Error Messages](#changing-error-messages) section, `LocalizedMessage` **may** be used to provide a better error message even when no user-specified locale is available. Regardless of how the locale for the message was determined, both the `locale` and `message` fields **must** be populated. The `locale` field specifies the locale of the message, following [IETF bcp47](https://www.rfc-editor.org/rfc/bcp/bcp47.txt) (Tags for Identifying Languages). Example values: `"en-US"`, `"fr-CH"`, `"es-MX"`. The `message` field contains the localized text itself. This **should** include a brief description of the error and a call to action to resolve the error. The message **should** include contextual information to make the message as specific as possible. Any contextual information in the message **must** be included in `ErrorInfo.metadata`. See [`ErrorInfo`](#errorinfo) for more details of how contextual information may be included in a message and the corresponding metadata. The `LocalizedMessage` payload **should** contain the complete resolution to the error. If more information is needed than can reasonably fit in this payload, then additional resolution information **must** be provided in a `Help` payload. See the [Help](#help) section for guidance. #### Help When other textual error messages (in `Status.message` or `LocalizedMessage.message`) don't provide the user sufficient context or actionable next steps, or if there are multiple points of failure that need to be considered in troubleshooting, a link to supplemental troubleshooting documentation **must** be provided in the `Help` payload. Provide this information in addition to a clear problem definition and actionable resolution, not as an alternative to them. The linked documentation **must** clearly relate to the error. If a single page contains information about multiple errors, the [`ErrorInfo.reason`](#errorinfo) value **must** be used to narrow down the relevant information. The `description` field is a textual description of the linked information. This **must** be suitable to display to a user as text for a hyperlink. This **must** be plain text (not HTML, Markdown etc). Example `description` value: `"Troubleshooting documentation for STOCKOUT errors"` The `url` field is the URL to link to. This **must** be an absolute URL, including scheme. Example `url` value: `"https://cloud.google.com/compute/docs/resource-error"` For publicly-documented services, even those with access controls on actual usage, the linked content **must** be accessible without authentication. For privately-documented services, the linked content **may** require authentication. ### Error messages Textual error messages can be present in both `Status.message` and `LocalizedMessage.message` fields. Messages **should** be succinct but actionable, with request-specific information (such as a resource name or region) providing precise details where appropriate. Any request-specific details **must** be present in [`ErrorInfo.metadata`](#errorinfo). #### Changing error messages Changing the content of `Status.message` over time must be done carefully, to avoid breaking clients who have previously had to rely on the message for all information. See [the rationale section](#updating-statusmessage) for more details. For a given RPC: - If the RPC has *always* returned `ErrorInfo` with machine-readable information, the content of `Status.message` **may** change over time. (For example, the API producer may provide a clearer explanation, or more request-specific information.) - Otherwise, the content of `Status.message` **must** be stable, providing the same text with the same request-specific information. Instead of changing `Status.message`, the API **should** include a [`LocalizedMessage`](#localizedmessage) within `Status.details`. Even if an RPC has always returned `ErrorInfo`, the API **may** keep the existing `Status.message` stable and add a [`LocalizedMessage`](#localizedmessage) within `Status.details`. The content of `LocalizedMessage.details` **may** change over time. ### Partial errors APIs **should not** support partial errors. Partial errors add significant complexity for users, because they usually sidestep the use of error codes, or move those error codes into the response message, where the user **must** write specialized error handling logic to address the problem. However, occasionally partial errors are necessary, particularly in bulk operations where it would be hostile to users to fail an entire large request because of a problem with a single entry. Methods that require partial errors **should** use [long-running operations][], and the method **should** put partial failure information in the metadata message. The errors themselves **must** still be represented with a [`google.rpc.Status`][Status] object. ### Permission Denied If the user does not have permission to access the resource or parent, regardless of whether or not it exists, the service **must** error with `PERMISSION_DENIED` (HTTP 403). Permission **must** be checked prior to checking if the resource or parent exists. If the user does have proper permission, but the requested resource or parent does not exist, the service **must** error with `NOT_FOUND` (HTTP 404). ## HTTP/1.1+JSON representation When clients use HTTP/1.1 as per [AIP-127](./0127.md), the error information is returned in the body of the response, as a JSON object. For backward compatibility reasons, this does not map precisely to `google.rpc.Status`, but contains the same core information. The schema is defined in the following proto: ```proto message Error { message Status { // The HTTP status code that corresponds to `google.rpc.Status.code`. int32 code = 1; // This corresponds to `google.rpc.Status.message`. string message = 2; // This is the enum version for `google.rpc.Status.code`. google.rpc.Code status = 4; // This corresponds to `google.rpc.Status.details`. repeated google.protobuf.Any details = 5; } Status error = 1; } ``` The most important difference is that the `code` field in the JSON is an HTTP status code, *not* the direct value of `google.rpc.Status.code`. For example, a `google.rpc.Status` message with a `code` value of 5 would be mapped to an object including the following code-related fields (as well as the message, details etc): ```json { "error": { "code": 404, // The HTTP status code for "not found" "status": "NOT_FOUND" // The name in google.rpc.Code for value 5 } } ``` The following JSON shows a fully populated HTTP/1.1+JSON representation of an error response. ```json { "error": { "code": 429, "message": "The zone 'us-east1-a' does not have enough resources available to fulfill the request. Try a different zone, or try again later.", "status": "RESOURCE_EXHAUSTED", "details": [ { "@type": "type.googleapis.com/google.rpc.ErrorInfo", "reason": "RESOURCE_AVAILABILITY", "domain": "compute.googleapis.com", "metadata": { "zone": "us-east1-a", "vmType": "e2-medium", "attachment": "local-ssd=3,nvidia-t4=2", "zonesWithCapacity": "us-central1-f,us-central1-c" } }, { "@type": "type.googleapis.com/google.rpc.LocalizedMessage", "locale": "en-US", "message": "An VM instance with is currently unavailable in the zone. Consider trying your request in the zone(s), which currently has/have capacity to accommodate your request. Alternatively, you can try your request again with a different VM hardware configuration or at a later time. For more information, see the troubleshooting documentation." }, { "@type": "type.googleapis.com/google.rpc.Help", "links": [ { "description": "Additional information on this error", "url": "https://cloud.google.com/compute/docs/resource-error" } ] } ] } } ``` ## Rationale ### Requiring ErrorInfo `ErrorInfo` is required because it further identifies an error. With only approximately twenty [available values][Code] for `Status.status`, it is difficult to disambiguate one error from another across an entire [API Service][API Service]. Also, error messages often contain dynamic segments that express variable information, so there needs to be machine-readable component of *every* error response that enables clients to use such information programmatically. ### Including LocalizedMessage `LocalizedMessage` was selected as the location to present alternate error messages. While `LocalizedMessage` **may** use a locale specified in the request, a service **may** provide a `LocalizedMessage` even without a user-specified locale, typically to provide a better error message in [situations where `Status.message` cannot be changed](#updating-statusmessage). Where the locale is not specified by the user, it **should** be `en-US` (US English). A service **may** include `LocalizedMessage` even when the same message is provided in `Status.message` and when localization into a user-specified locale is not supported. Reasons for this include: - An intention to support user-specified localization in the near future, allowing clients to consistently use `LocalizedMessage` and not change their error-reporting code when the functionality is introduced. - Consistency across all RPCs within a service: if some RPCs include `LocalizedMessage` and some only use `Status.message` for error messages, clients have to be aware of which RPCs will do what, or implement a fall-back mechanism. Providing `LocalizedMessage` on all RPCs allows simple and consistent client code to be written. ### Updating Status.message If a client has ever observed an error with `Status.message` populated (which it always will be) but without `ErrorInfo`, the developer of that client may well have had to resort to parsing `Status.message` in order to find out information beyond just what `Status.code` conveys. That information may be found by matching specific text (e.g. "Connection closed with unknown cause") or by parsing the message to find out metadata values (e.g. a region with insufficient resources). At that point, `Status.message` is implicitly part of the API contract, so **must not** be updated - that would be a breaking change. This is one reason for introducing `LocalizedMessage` into the `Status.details`. RPCs which have **always** included `ErrorInfo` are in a better position: the contract is then more about the stability of `ErrorInfo` for any given error. The reason and domain need to be consistent over time, and the metadata provided for any given (reason,domain) can only be expanded. It's still possible that clients could be parsing `Status.message` instead of using `ErrorInfo`, but they will always have had a more robust option available to them. ## Further reading - For which error codes to retry, see [AIP-194][aip-194]. - For how to retry errors in client libraries, see [AIP-4221][aip-4221]. ## Changelog - **2024-10-18**: Rewrite/restructure for clarity. - **2024-01-10**: Incorporate guidance for writing effective messages. - **2023-05-17**: Change the recommended language for `Status.message` to be the service's native language rather than English. - **2023-05-17**: Specify requirements for changing error messages. - **2023-05-10**: Require [`ErrorInfo`][ErrorInfo] for all error responses. - **2023-05-04**: Require uniqueness by message type for error details. - **2022-11-04**: Added guidance around PERMISSION_DENIED errors previously found in other AIPs. - **2022-08-12**: Reworded/Simplified intro to add clarity to the intent. - **2020-01-22**: Added a reference to the [`ErrorInfo`][ErrorInfo] message. - **2019-10-14**: Added guidance restricting error message mutability to if there is a machine-readable identifier present. - **2019-09-23**: Added guidance about error message strings being able to change. [aip-4221]: https://google.aip.dev/client-libraries/4221 [details]: https://github.com/googleapis/googleapis/blob/master/google/rpc/error_details.proto [ErrorInfo]: https://github.com/googleapis/googleapis/blob/6f3fcc058ff29989f6d3a71557a44b5e81b897bd/google/rpc/error_details.proto#L27-L76 [ErrorInfo-reason]: https://github.com/googleapis/googleapis/blob/master/google/rpc/error_details.proto#L57 [PreconditionFailure]: https://github.com/googleapis/googleapis/blob/6f3fcc058ff29989f6d3a71557a44b5e81b897bd/google/rpc/error_details.proto#L139-L166 [BadRequest]: https://github.com/googleapis/googleapis/blob/477a59d764428136ba1d857a9633c0d231de6efa/google/rpc/error_details.proto#L168-L218 [LocalizedMessage]: https://github.com/googleapis/googleapis/blob/e9897ed945336e2dc967b439ac7b4be6d2c62640/google/rpc/error_details.proto#L275-L285 [grpc status code documentation]: https://github.com/grpc/grpc/blob/master/doc/statuscodes.md [Code]: https://github.com/googleapis/googleapis/blob/master/google/rpc/code.proto [Status]: https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto [Status details]: https://github.com/googleapis/googleapis/blob/aeae5ea2b01ece6c0cee046ae84b881cdc62b95d/google/rpc/status.proto#L46-L48 [long-running operations]: ./0151.md [API Service]: https://cloud.google.com/apis/design/glossary#api_service [writing-tone]: https://developers.google.com/tech-writing/error-messages/set-tone [Error model]: https://cloud.google.com/apis/design/errors#error_model [Help]: https://github.com/googleapis/googleapis/blob/master/google/rpc/error_details.proto#L256 [resource-error]: https://cloud.google.com/compute/docs/resource-error [InvalidArgument]: https://github.com/googleapis/googleapis/blob/d4acb64370d333024a167551e7da854506109ba2/google/rpc/code.proto#L52-L58 --- id: 194 state: approved created: 2019-08-16 placement: category: polish order: 40 --- # Automatic retry configuration RPCs sometimes fail. When one does, the client performing the RPC needs to know whether it is safe to retry the operation. When status codes are used consistently across multiple APIs, clients can respond to failures appropriately. ## Guidance Clients **should** automatically retry requests for which repeated runs would not cause unintended state changes, which are non-transactional, and which are [unary](https://grpc.io/docs/guides/concepts/). Clients **should not** automatically retry transactional requests; instead these requests **should** have application-level retry logic that retries the entire transaction block from the start. Clients **should not** automatically retry requests in which repeated runs would cause unintended state changes. **Note:** This AIP does not cover client streaming or bi-directional streaming. **Note:** For client side retry behavior in the client libraries: see [AIP-4221](https://aip.dev/4221). ### Retryable codes For methods listed as retryable above, clients **should** retry the following error codes: - `UNAVAILABLE`: This code generally results from network hiccups, and is generally transient. It is retryable under the expectation that the connection will become available (soon). ### Non-retryable codes The following codes **should not** be automatically retried for any request: - `OK`: The request succeeded. - `CANCELLED`: An application can cancel a request, which **must** be honored. - `DEADLINE_EXCEEDED`: An application can set a deadline, which **must** be honored. - `INVALID_ARGUMENT`: Retrying a request with an invalid argument will never succeed. - `DATA_LOSS`: This is an unrecoverable error and **must** immediately be surfaced to the application. ### Generally non-retryable codes: The following codes generally **should not** be automatically retried for any request: - `RESOURCE_EXHAUSTED`: This code may be a signal that quota is exhausted. Retries therefore may not be expected to work for several hours; meanwhile the retries may have billing implications. If `RESOURCE_EXHAUSTED` is used for other reasons than quota and the expected time for the resource to become available is much shorter, it may be retryable. - `INTERNAL`: This code generally means that some internal part of the system has failed, and usually means a bug should be filed against the system. These **should** immediately be surfaced to the application. - `UNKNOWN`: Unlike `INTERNAL`, this code is reserved for truly unknown-to-the-system errors, and therefore may not be safe to retry. These **should** immediately be surfaced to the application. - `ABORTED`: This code typically means that the request failed due to a sequencer check failure or transaction abort. These **should not** be retried for an individual request; they **should** be retried at a level higher (the entire transaction, for example). Some codes **may** be automatically retried if a system is designed without synchronization or signaling between various components. For example, client might retry `NOT_FOUND` on a read operation, which is designed to hang forever until the resource is created. However, these types of systems are generally discouraged. Therefore, the following codes **should not** be automatically retried for any request: - `NOT_FOUND`: A client **should not** retry until a resource is created. - `ALREADY_EXISTS`: A client **should not** retry until a resource is deleted. - `PERMISSION_DENIED`: A client **should not** retry until it has permission. - `UNAUTHORIZED`: A client **should not** retry until it is authorized. - `UNAUTHENTICATED`: A client **should not** retry until it is authenticated. - `FAILED_PRECONDITION`: A client **should not** retry until system state changes. - `OUT_OF_RANGE`: A client **should not** retry until the range is extended. - `UNIMPLEMENTED`: A client **should not** retry until the RPC is implemented. ## Further reading - For parallel or retried request disambiguation, see [AIP-154](./0154.md). --- id: 200 state: approved created: 2018-06-28 placement: category: meta order: 30 redirect_from: /not-precedent --- # Precedent Many times, APIs are written in ways that do not match new guidance that is added to these standards after those APIs have already been released. Additionally, sometimes it can make sense to intentionally violate standards for particular reasons, such as maintaining consistency with established systems, meeting stringent performance requirements, or other practical concerns. Finally, as carefully as everyone reviews APIs before they are released, sometimes mistakes can slip through. Since it often is not feasible to fix past mistakes or make the standards serve every use case, APIs may be stuck with these exceptions for quite some time. Further, since new APIs often base their designs (names, types, structures, etc) on existing APIs, it is possible that a standards violation in one API could spill over into other APIs, even if original reason for the exception is not applicable to the other APIs. As a result of this problem, it is important to "stop the bleeding" of these standards exceptions into new APIs, and additionally document the reasons for each exception so that historical wisdom is not lost. ## Guidance If an API violates the AIP standards for any reason, there **must** be an internal comment linking to this document using its descriptive link ([aip.dev/not-precedent]()) to ensure others do not copy the violations or cite the errors as precedent of a "previously approved API". The comment should also include an explanation of what violates standards and why it is necessary. For example: ```proto message DailyMaintenanceWindow { // Time within the maintenance window to start the maintenance operations. // It must use the format "HH MM", where HH : [00-23] and MM : [00-59] GMT. // (-- aip.dev/not-precedent: This was designed for consistency with crontab, // and preceded the AIP standards. // Ordinarily, this type should be `google.type.TimeOfDay`. --) string start_time = 2; // Output only. Duration of the time window, automatically chosen to be // smallest possible in the given scenario. // (-- aip.dev/not-precedent: This preceded the AIP standards. // Ordinarily, this type should be `google.protobuf.Duration`. --) string duration = 3; } ``` **Important:** APIs should only be considered to be precedent-setting if they are in beta or GA. ### Local consistency If an API violates a standard throughout, it would be jarring and frustrating to users to break the existing pattern only for the sake of adhering to the global standard. For example, if all of an API's resources use `creation_time` (instead of the standard field `create_time` described in [AIP-142][]), a new resource in that API should continue to follow the local pattern. However, others who might otherwise copy that API should be made aware that this is contra-standard and not something to cite as precedent when launching new APIs. ```proto // ... message Book { // (-- aip.dev/not-precedent: This field was present before there was a // standard field. // Ordinarily, it should be spelled `create_time`. --) google.protobuf.Timestamp creation_time = 1; } // ... message Author { // (-- aip.dev/not-precedent: `Book` had `creation_time` before there was // a standard field, so we match that here for consistency. Ordinarily, // this would be spelled `create_time`. --) google.protobuf.Timestamp creation_time = 1; } ``` ### Pre-existing functionality Standards violations are sometimes overlooked before launching, resulting in APIs that become stable and therefore can not easily be modified. Additionally, a stable API may pre-date a standards requirement. In these scenarios, it is difficult to make the API fit the standard. However, the API should still cite that the functionality is contra-standard so that other APIs do not copy the mistake and cite the existing API as a reason why their design should be approved. ### Adherence to external spec Occasionally, APIs must violate standards because specific requests are implementations of an external specification (for example, OAuth), and their specification may be at odds with AIP guidelines. In this case, it is likely to be appropriate to follow the external specification. ### Adherence to existing systems Similar to the example of an external specification above, it may be proper for an API to violate AIP guidelines to fit in with an existing system in some way. This is a fundamentally similar case where it is wise to meet the customer where they are. A potential example of this might be integration with or similarity to a partner API. ### Expediency Sometimes there are users who need an API surface by a very hard deadline or money walks away. Since most APIs serve a business purpose, there will be times when an API could be better but cannot get it that way and into users' hands before the deadline. In those cases, API review councils **may** grant exceptions to ship APIs that violate guidelines due to time and business constraints. ### Technical concerns Internal systems sometimes have very specific implementation needs (e.g., they rely on operation transforms that speak UTF-16, not UTF-8) and adhering to AIP guidelines would require extra work that does not add significant value to API consumers. Future systems which are likely to expose an API at some point should bear this in mind to avoid building underlying infrastructure which makes it difficult to follow AIP guidelines. ## Changelog - **2020-03-27**: Reworded much of this AIP to follow [AIP-8][], and remove first and second person. No semantic changes. - **2019-05-04**: Changed to a public link ([aip.dev/not-precedent]()), and changed references to "the style guide" to use the more generic term "standards" (to account for a general shift to AIPs). [aip-8]: ./0008.md [aip-142]: ./0142.md --- id: 202 state: approved created: 2023-09-22 placement: category: fields order: 0 --- # Fields The [`google.api.FieldInfo`][field info proto] type, through its accompanying extension `google.api.field_info`, enriches a field's schema beyond the basic name and type information. ## Guidance Decorating a field with `google.api.field_info` is only necessary when explicitly stated in this AIP or another that leverages `google.api.FieldInfo` information. As such, the guidance herein applies to those scenarios as well. ### Format Fields with a primitive type can still have a specific format. To convey that type format, the `FieldInfo.Format` enumeration is used via the `(google.api.field_info).format` extension field. The following guidance conveys the meaning of and requirements for use of each `FieldInfo.Format` value. #### UUID4 The `UUID4` format represents a UUID version 4 value as governed by [RFC 4122][]. It **must** only be used on a field of type `string`. Such a value **may** be normalized by the service to entirely lowercase letters. For example, the value `F47AC10B-58CC-0372-8567-0E02B2C3D479` would be normalized to `f47ac10b-58cc-0372-8567-0e02b2c3d479`. As such, equivalence comparison **must not** be done via primitive text comparison. Instead, an [RFC 4122][] compliant implementation **must** be used. #### IPv4 The `IPV4` format represents an IP v4 address as governed by [RFC 791][]. It **must** only be used on a field of type `string`. Such a value **may** be condensed by the service, with leading zeros in each octet stripped. For example, `001.022.233.040` would be condensed to `1.22.233.40`. As such, equivalence comparison **must not** be done via primitive text comparison. Instead, an [RFC 791][] compliant implementation **must** be used. #### IPv6 The `IPV6` format represents an IP v6 address as governed by [RFC 4291][]. It **must** only be used on a field of type `string`. Such a value **may** be normalized by the service to entirely lowercase letters with zeros compressed, following [RFC 5952][]. For example, the value `2001:0DB8:0::0` would be normalized to `2001:db8::`. As such, equivalence comparison **must not** be done via primitive text comparison. Instead, an [RFC 4291][] compliant implementation **must** be used. #### IPv4 or IPv6 The `IPV4_OR_IPV6` value indicates that the field can be either an IP v4 or v6 address, as described in the [IPv4](#ipv4) and [IPv6](#ipv6) sections. #### Format Compatibility Adding a format specifier to an existing, unspecified field **is not** backwards compatible, *unless* the field in question has always conformed to the format being specified. Changing an existing format specifier to a different one in all cases **is not** backwards compatible. #### Extending Format Any new `FieldInfo.Format` value **must** be governed by an [IETF-approved RFC][ietf rfc] or a [Google-approved AIP](./0001.md). ## Rationale #### Why add a format specifier? The format of a primitive-typed field can be critical to its usability. Some programming languages may convey a specific type format as a standalone type, as Java does with [UUID][java uuid]. Most have specific structural requirements that are validated by the service, so conveying the format to the user ahead of time is critical to their experience. #### Why discourage primitive equality comparisons? The text representations of the supported formats have many nuances and transforming the value into a canonical representation is non-trivial. As such, aligning implementations between each consumer and each service without any issue is infeasible. #### Why document value normalizations? While primitive comparison is not recommended for any of the supported formats, uniform normalization of values is important to set consumer expectations, and create a user-friendly surface. #### Why require an RFC or AIP for new formats? Those formats which are sufficiently standardized to merit an RFC or AIP are stable enough and widely enough known to be incorporated as a supported value and see usage in Google APIs. Requiring such extra guidance means that governing the format specification is not the responsibility of the `FieldInfo.Format` enumeration itself. [field info proto]: https://github.com/googleapis/googleapis/blob/master/google/api/field_info.proto [rfc 4122]: https://datatracker.ietf.org/doc/html/rfc4122 [rfc 791]: https://datatracker.ietf.org/doc/html/rfc791 [rfc 4291]: https://datatracker.ietf.org/doc/html/rfc4291#section-2.2 [rfc 5952]: https://datatracker.ietf.org/doc/html/rfc5952 [ietf rfc]: https://www.ietf.org/standards/rfcs [java uuid]: https://docs.oracle.com/javase/8/docs/api/java/util/UUID.html --- id: 203 state: approved created: 2018-07-17 placement: category: fields order: 15 --- # Field behavior documentation When defining fields in protocol buffers, it is customary to explain to users certain aspects of the field's behavior (such as whether it is required or optional). Additionally, it can be useful for other tools to understand this behavior (for example, to optimize client library signatures). ## Guidance APIs use the `google.api.field_behavior` annotation to describe well-understood field behavior, such as a field being required or immutable. ```proto // The audio data to be recognized. RecognitionAudio audio = 2 [(google.api.field_behavior) = REQUIRED]; ``` - APIs **must** apply the `google.api.field_behavior` annotation on every field on a message or sub-message used in a request. - The annotation **must** include any [google.api.FieldBehavior][] values that accurately describe the behavior of the field. - `FIELD_BEHAVIOR_UNSPECIFIED` **must not** be used. - APIs **must** at minimum use one of `REQUIRED`, `OPTIONAL`, or `OUTPUT_ONLY`. **Warning:** Although `field_behavior` does not impact proto-level behavior, many clients (e.g. CLIs and SDKs) rely on them to generate code. Thoroughly review and consider which values are relevant when adding a new field. Fields with no annotation are interpreted as `OPTIONAL` for backwards-compatility. Nontheless, this annotation **must not** be omitted. **Note:** The vocabulary given in this document is for _descriptive_ purposes only, and does not itself add any validation. The purpose is to consistently document this behavior for clients. ### field behavior of nested messages `google.api.field_behavior` annotations on a nested message are independent of the annotations of the parent. For example, a nested message can have a field behavior of `REQUIRED` while the parent field can be `OPTIONAL`: ```proto message Title { string text = 1 [(google.api.field_behavior) = REQUIRED]; } message Slide { Title title = 1 [(google.api.field_behavior) = OPTIONAL]; } ``` In the case above, if a `title` is specified, the `text` field is required. ## Vocabulary ### Identifier The use of `IDENTIFIER` indicates that a field within a resource message is used to identify the resource. It **must** be attached to the `name` field and **must not** be attached to any other field (see [fields representing resource names]). The `IDENTIFIER` value conveys that the field is not accepted as input (i.e. `OUTPUT_ONLY`) in the context of a create method, while also being considered `IMMUTABLE` and accepted as input for mutation methods that accept the resource as the primary input e.g. [Standard Update][aip-134]. This annotation **must not** be applied to references to other resources within a message. ### Immutable The use of `IMMUTABLE` indicates that a field on a resource cannot be changed after its creation. This can apply to either fields that are input or outputs, required or optional. When a service receives an immutable field in an update request (or similar), even if included in the update mask, the service **should** ignore the field if the value matches, but **should** error with `INVALID_ARGUMENT` if a change is requested. Potential use cases for immutable fields (this is not an exhaustive list) are: - Attributes of resources that are not modifiable for the lifetime of the application (e.g. a disk type). **Note:** Fields which are "conditionally immutable" **must not** be given the immutable annotation. ### Input only The use of `INPUT_ONLY` indicates that the field is provided in requests and that the corresponding field will not be included in output. Additionally, a field **should** only be described as input only if it is a field in a resource message or a field of a message included within a resource message. Notably, fields in request messages (a message which only ever acts as an argument to an RPC, with a name usually ending in `Request`) **should not** be described as input only because this is already implied. Potential use cases for input only fields (this is not an exhaustive list) are: - The `ttl` field as described in AIP-214. **Warning:** Input only fields are rare and should be considered carefully before use. ### Optional The use of `OPTIONAL` indicates that a field is not required. A field **may** be described as optional if it is a field on a request message (a message that is an argument to an RPC, usually ending in `Request`), or a field on a submessage. ### Output only The use of `OUTPUT_ONLY` indicates that the field is provided in responses, but that including the field in a message in a request does nothing (the server **must** clear out any value in this field and **must not** throw an error as a result of the presence of a value in this field on input). Similarly, services **must** ignore the presence of output only fields in update field masks (see: AIP-161). Additionally, a field **should** only be described as output only if it is a field in a resource message, or a field of a message farther down the tree. Notably, fields in response messages (a message which only ever acts as a return value to an RPC, usually ending in `Response`) **should not** be described as output only because this is already implied. Output only fields **may** be set to empty values if appropriate to the API. Potential use cases for output only fields (this is not an exhaustive list) are: - Create or update timestamps. - Derived or structured information based on original user input. - Properties of a resource assigned by the service which can not be altered. ### Required The use of `REQUIRED` indicates that the field **must** be present (and set to a non-empty value) on the request or resource. A field **should** only be described as required if _either_: - It is a field on a resource that a user provides somewhere as input. In this case, the resource is only valid if a "truthy" value is _stored_. - When [creating][aip-133] the resource, a value **must** be provided for the field on the create request. - When [updating][aip-134] the resource, the user **may** omit the field provided that the field is also absent from the field mask, indicating no change to the field (otherwise it **must** be provided). - It is a field on a request message (a message that is an argument to an RPC, with a name usually ending in `Request`). In this case, a value **must** be provided as part of the request, and failure to do so **must** cause an error (usually `INVALID_ARGUMENT`). We define the term "truthy" above as follows: - For primitives, values other than `0`, `0.0`, empty string/bytes, and `false` - For repeated fields maps, values with at least one entry - For messages, any message with at least one "truthy" field. Fields **should not** be described as required in order to signify: - A field which will always be present in a response. - A field which is conditionally required in some situations. - A field on any message (including messages that are resources) which is never used as user input. **Note:** In most cases, empty values (such as `false` for booleans, `0` for integers, or the unspecified value for enums) are indistinguishable from unset values, and therefore setting a required field to a falsy value yields an error. A corollary to this is that a required boolean must be set to `true`. ### Unordered List The use of `UNORDERED_LIST` on a repeated field of a resource indicates that the service does not guarantee the order of the items in the list. A field **should** be described as an unordered list if the service does not guarantee that the order of the elements in the list will match the order that the user sent, including a situation where the service will sort the list on the user's behalf. A resource with an unordered list **may** return the list in a stable order, or **may** return the list in a randomized, unstable order. ## Backwards compatibility Adding or changing `google.api.field_behavior` values can represent a semantic change in the API that is perceived as incompatible for existing clients. The following are examples of backwards incompatible changes with `google.api.field_behavior`: * Adding `REQUIRED` to an existing field previously considered `OPTIONAL` (implicitly or otherwise) * Adding a new field annotated as `REQUIRED` to an existing request message * Adding `OUTPUT_ONLY` to an existing field previously accepted as input * Adding `INPUT_ONLY` to an existing field previously emitted as output * Adding `IMMUTABLE` to an existing field previously considered mutable * Removing `OUTPUT_ONLY` from an existing field previously ignored as input * Removing `IDENTIFIER` from an existing field. There are some changes that *are* backwards compatible, which are as follows: * Adding `OPTIONAL` to an existing field * Adding `IDENTIFIER` to an existing `name` field * Changing from `REQUIRED` to `OPTIONAL` on an existing field * Changing from `OUTPUT_ONLY` and/or `IMMUTABLE` to `IDENTIFIER` on an existing field * Removing `REQUIRED` from an existing field * Removing `INPUT_ONLY` from an existing field previously excluded in responses * Removing `IMMUTABLE` from an existing field previously considered immutable ## Rationale ### Identifier field behavior Resource names, the primary identifiers for any compliant resource, are never fully constructed by the user on create. Such fields are typically assigned `OUTPUT_ONLY` field behavior. They are, however, also often consumed as the primary identifier in scenarios where the resource itself is the primary request payload. Such fields could *not* be considered `OUTPUT_ONLY`. Furthermore, in mutation requests, like Standard Update, the resource name as the primary identifier cannot be changed in place. Such fields are typically assigned `IMMUTABLE` field behavior. These conflicting and context-dependent field behaviors meant that a new value was necessary to single out and convey the behavior of the resource name field. ### Required set of annotations A field used in a request message must be either an input or an output. In the case of an output, the `OUTPUT_ONLY` annotation is sufficient. In the case of an input, a field is either required or optional, and therefore should have at least the `REQUIRED` or `OPTIONAL` annotation, respectively. Only providing `INPUT_ONLY` does not convey the necessity of the field, so specifying either `REQUIRED` or `OPTIONAL` is still necessary. ### Requiring field behavior By including the field behavior annotation for each field, the overall behavior that the resource exhibits is more clearly defined. Clearly defined field behavior improves programmatic clients and user understanding. Requiring the annotation also forces the API author to explicitly consider the behavior when initially authoring of the API. Modifying field behavior after initial authoring can result in backwards-incompatible changes in clients. For example, making an optional field required results in backwards-incompatible changes in the method signature of an RPC or a resource in a [Declarative client][]. See the [Backwards compatibility](#backwards-compatibility) section for more detailed compatibility guidance. ## History In 2023-05 field_behavior was made mandatory. Prior to this change, the annotation was often omitted. Its values are relied upon to produce high quality clients. Furthermore, adding or changing some of the field_behavior values after the fact within a major version can be backwards-incompatible. See the [Backwards compatibility](#backwards-compatibility) section for more detailed compatibility guidance. The benefits of requiring field_behavior at the time that the API is authored surpass the costs to clients and API users of not doing so. [aip-133]: ./0133.md [aip-134]: ./0134.md [aip-180]: ./0180.md [google.api.FieldBehavior]: https://github.com/googleapis/googleapis/blob/master/google/api/field_behavior.proto#L49 [Declarative client]: ./0009.md#declarative-clients [fields representing resource names]: ./0122.md#fields-representing-resource-names ## Changelog - **2023-09-14**: Clarify that nested behavior and parent behavior are independent. - **2023-08-25**: Add guidance on `IDENTIFIER`. - **2023-07-20**: Describe compatibility guidance with new section. - **2023-05-24**: Clarify that `IMMUTABLE` does not imply input nor required. - **2023-05-10**: Added guidance to require the annotation. - **2020-12-15**: Added guidance for `UNORDERED_LIST`. - **2020-05-27**: Clarify behavior when receiving an immutable field in an update. - **2019-12-05**: Added guidance on output only fields in field masks. - **2019-06-18**: Use the machine-readable annotation, not comments. --- id: 205 state: approved created: 2019-02-18 placement: category: process order: 20 redirect_from: /beta-blocker --- # Beta-blocking changes APIs often release an Alpha version of their API in order to get early feedback from customers. This API is provisional and can change many times before the important feedback is incorporated and the API is made stable for Beta. Since the purpose of Alpha is to gather feedback, the API does not need to be perfect yet, and it's not strictly necessary for API authors to address every usability concern or address every point in the API standards. Often, API authors and API reviewers will not agree on the best design, and the best way to find out is by having users try out the API. However, once the feedback has been collected and the API is going to be promoted to Beta, usability concerns and style issues do need to be addressed. In order to ensure that these issues are not forgotten, they should be explicitly documented in the API. ## Guidance If an API has usability concerns or violates API standards, and the present design should receive additional scrutiny before being carried through to the Beta version, there **must** be an internal comment linking to this document using its descriptive link ([aip.dev/beta-blocker]()) to ensure that the design is corrected before the API is released to Beta. The comment **must** also indicate what kind of change should be made for Beta. For example: ```proto message InputConfig { // Parameters for input. // (-- aip.dev/beta-blocker: Convert well-known parameters into explicit // fields before the Beta launch. --) map parameters = 1; } ``` If an exception to API standards _does_ need to be carried through to Beta and GA, see [AIP-200][]. [aip-200]: ./0200.md --- id: 210 state: approved created: 2018-08-20 placement: category: design-patterns order: 110 --- # Unicode APIs should be consistent on how they explain, limit, and bill for string values and their encodings. This ranges from little ambiguities (like fields "limited to 1024 characters") all the way to billing confusion (are names and values of properties in Datastore billed based on characters or bytes?). In general, if we talk about limits measured in bytes, we are discriminating against non-ASCII text since it takes up more space. On the other hand, if we talk about "characters", we are ambiguous about whether those are Unicode "code points", "code units" for a particular encoding (e.g. UTF-8 or UTF-16), "graphemes", or "grapheme clusters". ## Unicode primer Character encoding tends to be an area we often gloss over, so a quick primer: - Strings are just bytes that represent numbers according to some encoding format. - When we talk about **characters**, we sometimes mean Unicode **code points**, which are numbers in the Unicode spec (up to 21 bits). - Other times we might mean **graphemes** or **grapheme clusters**, which may have multiple numeric representations and may be represented by more than one code point. For example, `á` may be represented as a composition of `U+0061 + U+0301` (the `a` + the accent combining mark) or as a single code point, `U+00E1`. - Protocol buffers uses **UTF-8** ("Unicode Transformation Format") which is a variable-length encoding scheme using up to 4 **code units** (8-bit bytes) per code point. ## Guidance ### Character definition **TL;DR:** In our APIs, "characters" means "Unicode code points". In API documentation (e.g., API reference documents, blog posts, marketing documentation, billing explanations, etc), "character" **must** be defined as a Unicode code point. ### Length units **TL;DR:** Set size limits in "characters" (as defined above). All string field length limits defined in API comments **must** be measured and enforced in characters as defined above. This means that there is an underlying maximum limit of (`4 * characters`) bytes, though this limit will only be hit when using exclusively characters that consist of 4 UTF-8 code units (32 bits). If you use a database system (e.g. Spanner) which allows you to define a limit in characters, it is safe to assume that this byte-defined requirement is handled by the underlying storage system. ### Billing units APIs **may** use either code points or bytes (using the UTF-8 encoding) as the unit for billing or quota measurement (e.g., Cloud Translation chooses to use characters). If an API does not define this, the assumption is that the unit of billing is characters (e.g., $0.01 _per character_, not $0.01 _per byte_). ### Unique identifiers **TL;DR:** Unique identifiers **should** limit to ASCII, generally only letters, numbers, hyphens, and underscores, and **should not** start with a number. Strings used as unique identifiers **should** limit inputs to ASCII characters, typically letters, numbers, hyphens, and underscores (`[a-zA-Z][a-zA-Z0-9_-]*`). This ensures that there are never accidental collisions due to normalization. If an API decides to allow all valid Unicode characters in unique identifiers, the API **must** reject any inputs that are not in Normalization Form C. Generally, unique identifiers **should not** start with a number as that prefix is reserved for Google-generated identifiers and gives us an easy way to check whether we generated a unique numeric ID for or whether the ID was chosen by a user. Unique identifiers **should** use a maximum length of 64 characters, though this limit may be expanded as necessary. 64 characters should be sufficient for most purposes as even UUIDs only require 36 characters. **Note:** See AIP-122 for recommendations about resource ID segments. ### Normalization **TL;DR:** Unicode values **should** be stored in [Normalization Form C][]. Values **should** always be normalized into Normalization Form C. Unique identifiers **must** always be stored in Normalization Form C (see the next section). Imagine we're dealing with Spanish input "estaré" (the accented part will be bolded throughout). This text has what we might visualize as 6 "characters" (in this case, they are grapheme clusters). It has two possible Unicode representations: - Using 6 code points: `U+0065` `U+0073` `U+0074` `U+0061` `U+0072` **`U+00E9`** - Using 7 code points: `U+0065` `U+0073` `U+0074` `U+0061` `U+0072` **`U+0065` `U+0301`** Further, when encoding to UTF-8, these code points have two different serialized representations: - Using 7 code-units (7 bytes): `0x65` `0x73` `0x74` `0x61` `0x72` **`0xC3` `0xA9`** - Using 8 code-units (8 bytes): `0x65` `0x73` `0x74` `0x61` `0x72` **`0x65` `0xCC` `0x81`** To avoid this discrepancy in size (both code units and code points), use [Normalization Form C][] which provides a canonical representation for strings. [normalization form c]: https://unicode.org/reports/tr15/ ### Uniqueness **TL;DR:** Unicode values **must** be normalized to [Normalization Form C][] before checking uniqueness. For the purposes of unique identification (e.g., `name`, `id`, or `parent`), the value **must** be normalized into [Normalization Form C][] (which happens to be the most compact). Otherwise we may have what is essentially "the same string" used to identify two entirely different resources. In our example above, there are two ways of representing what is essentially the same text. This raises the question about whether the two representations should be treated as equivalent or not. In other words, if someone were to use both of those byte sequences in a string field that acts as a unique identifier, would it violate a uniqueness constraint? The W3C recommends using Normalization Form C for all content moving across the internet. It is the most compact normalized form on Unicode text, and avoids most interoperability problems. If we were to treat two Unicode byte sequences as different when they have the same representation in NFC, we'd be required to reply to possible "Get" requests with content that is **not** in normalized form. Since that is definitely unacceptable, we **must** treat the two as identical by transforming any incoming string data into Normalized Form C or rejecting identifiers not in the normalized form. There is some debate about whether we should view strings as sequences of code points represented as bytes (leading to uniqueness determined based on the byte-representation of said string) or to interpret strings as a higher level abstraction having many different possible byte-representations. The stance taken here is that we already have a field type for handling that: `bytes`. Fields of type `string` already express an opinion of the validity of an input (it must be valid UTF-8). As a result, treating two inputs that have identical normalized forms as different due to their underlying byte representation seems to go against the original intent of the `string` type. This distinction typically doesn't matter for strings that are opaque to our services (e.g., `description` or `display_name`), however when we rely on strings to uniquely identify resources, we are forced to take a stance. Put differently, our goal is to allow someone with text in any encoding (ASCII, UTF-16, UTF-32, etc) to interact with our APIs without a lot of "gotchas". ## References - [Unicode normalization forms](https://unicode.org/reports/tr15/) - [Datastore pricing "name and value of each property"](https://cloud.google.com/datastore/pricing) doesn't clarify this. - [Natural Language pricing](https://cloud.google.com/natural-language/pricing) uses charges based on UTF-8 code points rather than code units. - [Text matching and normalization](https://sites.google.com/a/google.com/intl-eng/apis/matching?pli=1) --- id: 211 state: approved created: 2021-02-24 placement: category: design-patterns order: 115 --- # Authorization checks The majority of operations, whether reads or writes, require authorization: permission to do the thing the user is asking to do. Additionally, it is important to be careful how much information is provided to _unauthorized_ users, since leaking information can be a security concern. ## Guidance Services **must** check authorization before validating any request, to ensure both a secure API surface and a consistent user experience. An operation **may** require multiple permissions or preconditions in order to grant authorization. If a request can not pass the authorization check for any reason, the service **must** error with `PERMISSION_DENIED`, and the corresponding error message **should** look like: "Permission '`{p}`' denied on resource '`{r}`' (or it might not exist)." This avoids leaking resource existence. If it is not possible to determine authorization for a resource because the resource does not exist, the service **should** check authorization to read children on the parent resource, and return `NOT_FOUND` if the authorization check passes. ### Multiple operations A service could encounter a situation where it has two different operations with two different permissions, either of which would reveal the existence of a resource if called, but a user only has permission to call one of them. In this situation, the service **should** still only check for authorization applicable to the operation being called, and **should not** try to "help out" by checking for related authorization that would provide permission to reveal existence, because such algorithms are complicated to implement correctly and prone to accidental leaks. For example, posit a scenario where: - A resource exists within a given collection that a user is unable to read. - The user _does_ have the ability to create other resources, and the collection uses user-specified IDs (meaning that a failure because of a duplicate ID would reveal existence). In this situation, the get or create methods **should** still only check _their_ permissions when determining what error to return, and not one another's. ## Rationale [RFC 7231 §6.5.3][] states that services are permitted to use `404 Not Found` in lieu of `403 Forbidden` in situations where the service does not want to divulge existence, whereas this AIP argues for the use of `PERMISSION_DENIED` (which corresponds to `403 Forbidden` in HTTP) instead. We take this position for the following reasons: - The practice of "getting `404 Not Found` until you have enough permission to get `403 Forbidden`" is counter-intuitive and increases the difficulty of troubleshooting. - A service _could_ ameliorate this by sending information about missing permissions while still using the `404 Not Found` status code, but this constitutes a mixed message. - While `403 Forbidden` is essentially always an error requiring manual action, `404 Not Found` is often a valid response that the application can handle (e.g. "get or create"); overloading it for permission errors deprives applications of this benefit. - RFC 7231 §6.5.4 states that `404 Not Found` results are cacheable, but permission errors are not generally cacheable. Sending explicit cache controls on a conditional basis could ameliorate this, but would defeat the purpose. - The guidance here is more consistent with most other real-world authorization systems. [rfc 7231 §6.5.3]: https://tools.ietf.org/html/rfc7231#section-6.5.3 ## Changelog - **2021-05-11:** Added a more detailed "Rationale" section. --- id: 213 state: approved created: 2018-08-17 placement: category: protobuf order: 20 --- # Common components As specified in [AIP-215][], APIs must be self-contained except for the use of "common component" packages which are intended for use by multiple APIs. There are two kinds of common component packages: - Organization-specific common components, covering organization-specific concepts such as a "viewport" in Maps. - Global common components which are generic (i.e. not domain-specific), such as "timestamp" or "postal address". Where it is safe to share a single representation across multiple APIs, common components can make it easier for clients to interact with those APIs. Concept-specific client code can be written once, and messages can be used from the response of one API in the request of another without clunky copying, for example. This benefit comes with significant restrictions and limitations, however, and should not be attempted lightly. Note that even if the *domain* of a component is common, the requirements of a component may be organization-specific. For example, some organizations may have particular requirements of how financial values are represented, leading to multiple finance-oriented organization-specific common components - because any global common component would either not meet the organization-specific requirements, or be too complex for general use. ## Guidance - Organization-wide common component packages **must** end with `.type`, e.g. `google.geo.type` or `google.shopping.type`. - Organizations **must** consult the API design team before creating a new organization-wide common component package. - Organization-wide common component packages **must** be published in [the `googleapis` repository](https://github.com/googleapis/googleapis). - Organizations creating and publishing a new organization-wide common component package **must** update this AIP to include it in the [list below](#existing-organization-specific-common-component-packages). - Organizations **must not** define generic components in organization-specific common component packages, instead preferring global common components. - Common components **must not** be "moved" (that is, deleted from one common component package and added to a different one) from an organization-specific common component package to a global common component package or vice versa. - A common component **may** be copied from an organization-specific common component package to a global common component package (without deleting the original component) if it is found to be more widely-applicable than originally expected. - Fields **should not** be added to existing messages. - Values **should not** be added to existing enums. - Fields **must not** be removed from existing messages. - Values **must not** be removed from existing enums. - While documentation **may** be clarified, it **should not** change the meanings of existing values, including the validity of any given message or set of messages. - New proto messages and enums **may** be added to common component packages. - API teams **should** allow sufficient time for propagation to clients before using the new messages and enums in their APIs. Fields may take some time for any changes to propagate through publication to client libraries and other surfaces. - API teams **should** consult widely within their organization, and ideally with the API design team, before adding a new message or enum, due to the limitations listed above. ## Existing global common components The global common components, which public-facing protos for an API **may** safely import, are as follows: - [`google.api.*`](https://github.com/googleapis/googleapis/blob/master/google/api) (but *not* subpackages of `google.api`) - [`google.longrunning.Operation`](https://github.com/googleapis/googleapis/blob/master/google/longrunning/operations.proto) - [`google.protobuf.*`](https://github.com/protocolbuffers/protobuf/tree/master/src/google/protobuf) - [`google.rpc.*`](https://github.com/googleapis/googleapis/blob/master/google/rpc/) - [`google.type.*`][type] Note that some common components may have internal-only fields. APIs **should** generally only rely on fields which have been [released into open source](https://github.com/googleapis/googleapis). Google APIs **may** also import [`google.iam.v1.*`][iam], which provides the IAM messages used throughout Google. [iam]: https://github.com/googleapis/googleapis/tree/master/google/iam/v1 **Note:** Many APIs also import components from other packages for internal-only use (e.g. to apply visibility labels or provide instructions to internal infrastructure). This is acceptable provided that the _public_ components do not contain such references. ### Protobuf types The `google.protobuf` package is somewhat special in that it is shipped with protocol buffers itself, rather than with API tooling. (For most API designers, this should be an implementation detail). This package includes a small library of types useful for representing common programming language constructs: - [`google.protobuf.Duration`][duration]: Durations, with nanosecond-level precision. The protobuf runtime provides helper functions to convert to and from language-native duration objects where applicable (such as Python's [`timedelta`][timedelta]). - [`google.protobuf.Struct`][struct]: JSON-like structures (a dictionary of primitives, lists, and other dictionaries). The protobuf runtime provides helper functions in most languages to convert struct objects to and from JSON. - [`google.protobuf.Timestamp`][timestamp]: Timestamps, with nanosecond-level precision. The protobuf runtime provides helper functions in most languages to convert to and from language-native timestamp objects (such as Python's [`datetime`][datetime]). [datetime]: https://docs.python.org/3/library/datetime.html#datetime.datetime [duration]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/duration.proto [struct]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/struct.proto [timedelta]: https://docs.python.org/3/library/datetime.html#datetime.timedelta [timestamp]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/timestamp.proto ### API Types The `google.type` package provides a "standard library" of types useful for representing common concepts in APIs. While types are added from time to time and the definitive list is always [the code][type], several types deserve note: - [`google.type.Color`][color]: RGB or RGBA colors. - [`google.type.Date`][date]: Calendar dates, with no time or time zone component. - [`google.type.DayOfWeek`][day_of_week]: The day of the week, with no other date, time, or time zone component. - [`google.type.LatLng`][lat_lng]: Geographic coordinates. - [`google.type.Money`][money]: Currency. - [`google.type.PostalAddress`][postal_address]: Postal addresses in most countries. - [`google.type.TimeOfDay`][time_of_day]: Wall-clock time, with no date or time zone component. [type]: https://github.com/googleapis/googleapis/tree/master/google/type [color]: https://github.com/googleapis/googleapis/blob/master/google/type/color.proto [date]: https://github.com/googleapis/googleapis/blob/master/google/type/date.proto [day_of_week]: https://github.com/googleapis/googleapis/blob/master/google/type/dayofweek.proto [lat_lng]: https://github.com/googleapis/googleapis/blob/master/google/type/latlng.proto [money]: https://github.com/googleapis/googleapis/blob/master/google/type/money.proto [postal_address]: https://github.com/googleapis/googleapis/blob/master/google/type/postal_address.proto [time_of_day]: https://github.com/googleapis/googleapis/blob/master/google/type/timeofday.proto ### Adding to common protos Occasionally, it may be useful to add protos to these packages or to add to the list of commonly-available protos. In order to do this, [open an issue][] on the AIP repository in GitHub, noting the guidelines above. ## Existing organization-specific common component packages The following organization-specific common component packages exist and conform with the above guidance: - [`google.apps.script.type`](https://github.com/googleapis/googleapis/tree/master/google/apps/script/type) Common component package for Google Apps Script. - [`google.geo.type`](https://github.com/googleapis/googleapis/tree/master/google/geo/type) Common component package for Google Maps and the Geo organization. - [`google.actions.type`](https://github.com/googleapis/googleapis/tree/master/google/actions/type) Common component package for Actions on Google APIs. ## Non-conformant common component packages The following common component packages exist, but do not conform with the above guidance, and do not form a precedent for further such packages. - [`google.cloud.common`](https://github.com/googleapis/googleapis/tree/master/google/cloud/common): This does not conform to the requirement for the package name to end in `.type`. (This would otherwise be acceptable, and this package should be considered as the Cloud common component package.) - [`google.logging.type`](https://github.com/googleapis/googleapis/tree/master/google/logging/type): This appears to be API-specific, although it's used from multiple APIs; some aspects should probably be global or in a Cloud common component package. - [`google.cloud.workflows.type`](https://github.com/googleapis/googleapis/tree/master/google/cloud/workflows/type): API-specific types. - [`google.cloud.oslogin.common`](https://github.com/googleapis/googleapis/tree/master/google/cloud/oslogin/common): API-specific types, and a non-conformant name. - [`google.identity.accesscontextmanager.type`](https://github.com/googleapis/googleapis/tree/master/google/identity/accesscontextmanager/type): API-specific types. - [`google.networking.trafficdirector.type`](https://github.com/googleapis/googleapis/tree/master/google/networking/trafficdirector/type): API-specific types. ## Rationale Common components are effectively unversioned: APIs evolve independently of each other, both in terms of definition and implementation. A change such as adding a field is backward-compatible and predictable in specific APIs, and the API team can ensure that the server implementation is available before the API definition is published. By contrast, a change in a common component would effectively be universally available even if most API implementations did not take it into account. Adding a new message or enum is backward-compatible, as it does not affect existing APIs that may import other messages or enums from the same common component package. Consultation with the API design team is required for global common components and suggested for organization-specific common components as the border between "generic" and "organization-specific" is a gray area; some generic *concepts* have organization-specific use cases which surface through the components. ## Changelog - **2023-06-27**: Restructured AIPs 215 and 213 for clarity, and introduced the concept of organization-wide common protos more formally. - **2018-08-17**: Initial AIP written. [open an issue]: https://github.com/googleapis/aip/issues [aip-215]: ./0215.md --- id: 214 state: approved created: 2018-06-19 placement: category: design-patterns order: 120 --- # Resource expiration Customers often want to provide the time that a given resource or resource attribute is no longer useful or valid (e.g. a rotating security key). Currently we recommend that customers do this by specifying an exact "expiration time" into a `google.protobuf.Timestamp expire_time` field; however, this adds additional strain on the user when they want to specify a relative time offset until expiration rather than a specific time until expiration. Furthermore, the world understands the concept of a "time-to-live", often abbreviated to TTL, but the typical format of this field (an integer, measured in seconds) results in a sub-par experience when using an auto-generated client library. ## Guidance 1. APIs wishing to convey an expiration **must** rely on a [`google.protobuf.Timestamp`][timestamp] field called `expire_time`. 2. APIs wishing to allow a relative expiration time **must** define a `oneof` called `expiration` (or `{something}_expiration`) containing both the `expire_time` field and a separate [`google.protobuf.Duration`][duration] field called `ttl`, the latter marked as input only. 3. APIs **must** always return the expiration time in the `expire_time` field and leave the `ttl` field blank when retrieving the resource. 4. APIs that rely on the specific semantics of a "time to live" (e.g., DNS which must represent the TTL as an integer) **may** use an `int64 ttl` field (and **should** provide an [aip.dev/not-precedent](./0200.md) comment in this case). ### Example ```proto message ExpiringResource { // google.api.resource and other annotations and fields oneof expiration { // Timestamp in UTC of when this resource is considered expired. // This is *always* provided on output, and is calculated using // the value of [ttl][] if set when created. google.protobuf.Timestamp expire_time = 2; // Input only. The TTL for this resource. google.protobuf.Duration ttl = 3 [(google.api.field_behavior) = INPUT_ONLY]; } } ``` ## Rationale ### Alternatives considered #### A new standard field called `ttl` We considered allowing a standard field called `ttl` as an alternative way of defining the expiration, however doing so would require that API services continually update the field, like a clock counting down. This could potentially cause problems with the read-modify-write lifecycle where a resource is being processed for some time, and effectively has its life extended as a result of that processing time. #### Always use `expire_time` This is the current state of the world with a few exceptions. In this scenario, we could potentially push the computation of `now + ttl = expire_time` into client libraries; however, this leads to a somewhat frustrating experience in the command-line and using REST/JSON. Leaving things as they are is typically the default, but it seems many customers want the ability to define relative expiration times as it is quite a bit easier and removes questions of time zones, stale clocks, and other silly mistakes. [duration]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/duration.proto [timestamp]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/timestamp.proto --- id: 215 state: approved created: 2018-10-01 placement: category: protobuf order: 30 --- # API-specific protos APIs are mostly defined in terms of protos which are API-specific, with occasional dependencies on common components. Keeping APIs isolated from each other avoids versioning problems and client library packaging problems. ## Guidance - All protos specific to an API **must** be within a package with a major version (e.g., `google.library.v1`). - References to resources in other APIs **must** be expressed in terms of resource names ([AIP-122][]), rather than using the resource messages. - When two versions of an API use effectively the same (API-specific) proto that proto **must** be duplicated in each version. (In other words, APIs **must not** create their own "API-specific common component" packages.) - Organization-specific common components **may** be placed in a common package, as described in [AIP-213][], but **must not** be used by any API outside that organization. - Global common components (also described in AIP-213) **may** be freely used by any API. ## Rationale When one API depends on protos defined by another API, this introduces uncertainty in terms of customer-expected behavior and client library dependency management. Suppose `google.cloud.library.v1` depends on the protos (rather than abstract resources) in `google.cloud.movies.v2`. Any change to `google.cloud.movies.v2` can cause problems. For example: - If a field is added to a message in `google.cloud.movies.v2`, should customers using `google.cloud.library.v1` expect to see it? If so, how soon after the field has been added? What about other API changes? - If the whole major version `google.cloud.movies.v2` is deprecated (typically after v3 has been released), does that mean `google.cloud.library.v1` has to change to use `google.cloud.movies.v3`, and if so, does that require a new major version for the library API as well? - How should client library versioning reflect changes to dependent APIs? Keeping APIs isolated from each other, with a limited set of common components which are maintained in a highly disciplined way, reduces a lot of the issues with dependencies. API-specific common components shared across versions add complexity for client library generation and packaging, and are inflexible in terms of versioning. When protos are duplicated because they *start* off the same in multiple versions, they can still diverge over time as they are isolated from each other. ## Changelog - **2023-06-27**: Restructured AIPs 215 and 213 for clarity. - **2023-05-11**: Changed "PA" to "organization". - **2018-10-01**: Initial AIP written. [aip-213]: ./0213.md [aip-122]: ./0122.md --- id: 216 state: approved created: 2018-10-01 updated: 2022-06-02 placement: category: fields order: 110 --- # States Many API resources carry a concept of "state": ordinarily, the resource's place in its life cycle. For example, a virtual machine may be being provisioned, available for use, being spun down, or potentially be in one of several other situations. A job or query may be preparing to run, be actively running, have completed, and so on. ## Guidance Resources needing to communicate their state **should** use an enum, which **should** be called `State` (or, if more specificity is required, end in the word `State`). This enum **should** be nested within the message it describes when only used as a field within that message. **Important:** We use the term `State`, and _not_ `Status` (which is reserved for the HTTP and gRPC statuses). ### Enum values Ideally, Google APIs use the same terminology throughout when expressing the same semantic concepts. There are usually many words available to express a given state, but our customers often use multiple APIs together, and it is easier for them when our terms are consistent. At a high level: - Resources that are available for use are `ACTIVE` (preferred over terms such as "ready" or "available"). - Resources that have completed a (usually terminal) requested action use past participles (usually ending in `-ED`), such as `SUCCEEDED` (not "successful"), `FAILED` (not "failure"), `DELETED`, `SUSPENDED`, and so on. - Resources that are currently undergoing a state change use present participles (usually ending in `-ING`), such as `RUNNING`, `CREATING`, `DELETING`, and so on. In this case, it is expected that the state is temporary and will resolve to another state on its own, with no further user action. **Note:** Remember to only add states that are useful to customers. Exposing a large number of states simply because they exist in your internal system is unnecessary and adds confusion for customers. Each state must come with a use case for why it is necessary. ### Output only The field referencing the `State` enum in a resource **should** behave and be documented as "Output only", in accordance with [AIP-203](./0203.md). APIs **should not** allow a `State` enum to be directly updated through an "update" method (or directly set through the "create" method), and **should** instead use custom state transition methods. This is because update methods are generally not expected to have side effects, and also because updating state directly implies that it is possible to set the state to any available value, whereas states generally reflect a resource's progression through a lifecycle. ### State transition methods State transition methods are a special type of custom method that are responsible for transitioning a state field from one enum value to another. As part of the transition, other fields may also change, e.g. an `update_time` field. The method definition should look like the following: ```proto // Publishes a book. // The `state` of the book after publishing is `PUBLISHED`. // `PublishBook` can be called on Books in the state `DRAFT`; Books in a // different state (including `PUBLISHED`) returns an error. rpc PublishBook(PublishBookRequest) returns (Book) { option (google.api.http) = { post: "/v1/{name=publishers/*/books/*}:publish" body: "*" }; } ``` - The name of the method **should** be a verb followed by the singular form of the resource's message name. - The request message **must** match the RPC name, with a `Request` suffix. - The response message **should** be the resource itself. - If the RPC is [long-running][], the response message **should** be a `google.longrunning.Operation` which resolves to the resource itself. - The HTTP verb **must** be `POST`. - The HTTP URI **must** use a `:` character followed by the custom verb (`:publish` in the above example), and the verb in the URI **must** match the verb in the name of the RPC. - If word separation is required, `camelCase` **must** be used. - The `body` clause in the `google.api.http` annotation **must** be `"*"`. - The request message field receiving the resource name **should** map to the URI path. - This field **should** be called `name`. - The `name` field **should** be the only variable in the URI path. All remaining parameters **should** map to URI query parameters. - If the state transition is not allowed, the service **must** error with `FAILED_PRECONDITION` (HTTP 400). The request message should look like this: ```proto message PublishBookRequest { // The name of the book to publish. // Format: publishers/{publisher}/books/{book} string name = 1 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { type: "library.googleapis.com/Book" }]; } ``` - A resource name field **must** be included. It **should** be called `name`. - The comment for the field **should** document the resource pattern. - Other fields **may** be included. ## Additional Guidance ### Default value The zero value of each state enum **should** adhere to the following convention: ```proto enum State { // The default value. This value is used if the state is omitted. STATE_UNSPECIFIED = 0; // Other values... } ``` Resources **should not** provide an unspecified state to users, and this value **should not** actually be used. ### Value uniqueness Multiple top-level enums within the same package **must** not share the same values. This is because the C++ protoc code generator flattens top-level enum values into a single namespace. State enums **should** live inside the resource definition. ### Prefixes Using a `STATE_` prefix on every enum value is unnecessary. State enum values **should not** be prefixed with the enum name, except for the default value `STATE_UNSPECIFIED`. ### Breaking changes **TL;DR:** Clearly communicate to users that state enums may receive new values in the future, and be conscientious about adding states to an existing enum. Even though adding states to an existing states enum _can_ break existing user code, adding states is not considered a breaking change. Consider a state with only two values: `ACTIVE` and `DELETED`. A user may add code that checks `if state == ACTIVE`, and in the else cases simply assumes the resource is deleted. If the API later adds a new state for another purpose, that code will break. We ultimately can not control this behavior, but API documentation **should** actively encourage users to code against state enums with the expectation that they may receive new values in the future. APIs **may** add new states to an existing State enum when appropriate, and adding a new state is _not_ considered a breaking change. ### When to avoid states Sometimes, a `State` enum may not be what is best for your API, particularly in situations where a state has a very small number of potential values, or when states are not mutually exclusive. Consider the example of a state with only `ACTIVE` and `DELETED`, as discussed above. In this situation, the API may be better off exposing a `google.protobuf.Timestamp delete_time`, and instructing users to rely on whether it is set to determine deletion. ### Common states The following is a list of states in common use. APIs **should** consider prior art when determining state names, and **should** value local consistency above global consistency in the case of conflicting precedent. #### Resting states "Resting states" are lifecycle states that, absent user action, are expected to remain indefinitely. However, the user can initiate an action to move a resource in a resting state into certain other states (resting or active). - `ACCEPTED` - `ACTIVE` - `CANCELLED` - `DELETED` - `FAILED` - `SUCCEEDED` - `SUSPENDED` - `VERIFIED` #### Active states "Active states" are lifecycle states that typically resolve on their own into a single expected resting state. **Note:** Remember only to expose states that are useful to customers. Active states are valuable only if the resource will be in that state for a sufficient period of time. If state changes are immediate, active states are not necessary. - `CREATING` (usually becomes `ACTIVE`) - `DELETING` (usually becomes `DELETED`) - `PENDING` (usually becomes `RUNNING`) - `REPAIRING` (usually becomes `ACTIVE`) - `RUNNING` (usually becomes `SUCCEEDED`) - `SUSPENDING` (usually becomes `SUSPENDED`) ## Further reading - For information on enums generally, see [AIP-126][]. ## Changelog - **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-". - **2020-10-20**: Added guidance on prefixing enum values with enum name. - **2020-09-02**: Clarified that states are not directly set on create either. - **2019-12-05**: Changed guidance on state transition methods, downgrading **must** to **should** on the response type. - **2019-08-16**: Added guidance for state transition methods. - **2019-07-18**: Added explicit guidance on the unspecified value. [aip-126]: ./0126.md [long-running]: ./0151.md --- id: 217 state: approved created: 2019-08-26 placement: category: design-patterns order: 130 --- # Unreachable resources Occasionally, a user may ask for a list of resources, and some set of resources in the list are temporarily unavailable. The most typical use case is while supporting [Reading Across Collections][aip-159]. For example, a user may ask to list resources across multiple parent locations, but one of those locations is temporarily unreachable. In this situation, it is still desirable to provide the user with all the available resources, while indicating that something is missing. ## Guidance If a method to retrieve data is capable of partially failing due to one or more resources being temporarily unreachable, the response message **must** include a field to indicate this: ```proto message ListBooksResponse { // The books matching the request. repeated Book books = 1; // The next page token, if there are more books matching the // request. string next_page_token = 2; // Unreachable resources. repeated string unreachable = 3 [ (google.api.field_behavior) = UNORDERED_LIST ]; } ``` - The field **must** be a repeated string, and **should** be named `unreachable`. - The field **must** contain the resource names of the resources that are unreachable or those that impede reaching the requested collection, such as the parent resource of the collection that could not be reached. - For example, if an entire location is unreachable, preventing access to the localized collection of resources requested, the location resource is included. - The field **must** contain _service-relative_ resource names, and **must not** contain full resource names, resource URIs, or simple resource IDs. See [AIP-122][aip-122] for definitions. - For example, if a `Book` resource is unreachable, the _service-relative_ resource name `"shelves/scifi1/books/starwars4"` is included in `unreachable`, as opposed to the _full_ resource name `"//library.googleapis.com/shelves/scifi1/books/starwars4"`, the _parent-relative_ resource `"books/starwars4"`, the resource ID `"starwars4"`, or the resource URI. - The response **must not** provide any other information about the issue(s) that made the listed resources unreachable. - For example, the response cannot contain an extra field with error reasons for each `unreachable` entry. - The service **must** provide a way for the user to make a more specific request and receive an error with additional information e.g. via a Standard Get or a Standard List targeted at the unreachable collection parent. - The service **must** also allow the user to repeat the original call with more restrictive parameters. - The resource names that appear in `unreachable` **may** be heterogeneous. - The `unreachable` field definition **should** document what potential resources could be provided in this field, and note that it might expand later. - For example, if both an entire location and a specific resource in a different location are unreachable, the unreachable location's name e.g. `"projects/example123/locations/us-east1"` and the unreachable resource's name e.g. `"projects/example123/locations/europe-west2/instances/example456"` will both appear in `unreachable`. - The `unreachable` field **must not** have semantically meaningful ordering or structure within the list. Put differently, `unreachable` **must** be an unordered list. - As such, the `unreachable` field **must** be annotated with `UNORDERED_LIST` field behavior (see [AIP-203][unordered]). [aip-122]: ./0122.md [aip-160]: ./0160.md [unordered]: ./0203.md#unordered-list **Important:** If a single unreachable location or resource prevents returning any data by definition (for example, a list request for a single publisher where that publisher is unreachable), the service **must** fail the entire request with an error. ### Pagination While preparing a page of results to fulfill a page fetch RPC e.g. an [AIP-132][aip-132] Standard List call, if the service encounters any unreachable resources or collections they **must** do the following: - Include the resource name for the unreachable resource in the `unreachable` response field. - The resource name **must** be the most appropriately scoped for the unreachable resource or collection. - For example, if a specific zone within a region is unreachable, the unreachable resource name would be a zonal Location e.g. `projects/example/locations/us-west1-a`, but if an entire region is unreachable, the resource name would be a regional Location e.g. `projects/example/locations/us-west1`. - The resource name **must** be included, regardless of restrictive paging parameters e.g. `order_by`, when it is identified as unreachable. - Populate results that were previously considered unreachable on a following page if their availability is restored and the paging parameters allow for their inclusion. - Determining inclusion eligibility based on paging parameters also includes any documented default ordering behavior in the absence of user-specified ordering in the request. - For example, if region `projects/example/locations/us-west1` was unavailable in the first page of an ordered paging call, and including its resources would violate the ordering, those out-of-order resources are not included in the following page. - Similarly, if the same exact request is made, and resources previously considered unreachable are available again, they **must** be populated, within the constraints of the paging parameters. - Limit the number of unreachable resource names returned in a given response if, even after up-scoping the unreachable resource name, the number of unreachable resource names exceeds a documented maximum. - This maximum **must** be documented in the `unreachable` field comments directly. - This is independent of the `page_size` set by the caller. #### Retaining previous behavior Services **may** continue with previously implemented `unreachable` pagination behavior where changing it would induce an incompatible change as per [AIP-180][aip-180], but **must** document said behavior on the `unreachable` field(s) directly. [aip-180]: ./0180.md ### Adopting partial success In order for an existing API that has a default behavior *differing* from the aforementioned guidance i.e. the API call returns an error status instead of a partial result, to adopt the `unreachable` pattern the API **must** do the following: - The default behavior **must** be retained to avoid incompatible behavioral changes - For example, if the default behavior is to return an error if any location is unreachable, that default behavior **must** be retained. - The request message **must** have a `bool return_partial_success` field - The response message **must** have the standard `repeated string unreachable` field - The two aforementioned fields **must** be added simultaneously When the `bool return_partial_success` field is set to `true` in a request, the API **must** behave as described in the aforementioned guidance with regards to populating the `repeated string unreachable` response field. ```proto message ListBooksRequest { // Standard List request fields... // Setting this field to `true` will opt the request into returning the // resources that are reachable, and into including the names of those that // were unreachable in the [ListBooksResponse.unreachable] field. This can // only be `true` when reading across collections e.g. when `parent` is set to // `"projects/example/locations/-"`. bool return_partial_success = 4; } message ListBooksResponse { // Standard List Response fields... // Unreachable resources. Populated when the request opts into // `return_partial_success` and reading across collections e.g. when // attempting to list all resources across all supported locations. repeated string unreachable = 3 [ (google.api.field_behavior) = UNORDERED_LIST ]; } ``` #### Partial success granularity If the `bool return_partial_success` field is set to `true` in a request that is scoped beyond the supported granualirty of the API's ability to reasonably report unreachable resources, the API **should** return an `INVALID_ARGUMENT` error with details explaining the issue. For example, if the API only supports `return_partial_success` when [Reading Across Collections][aip-159], it returns an `INVALID_ARGUMENT` error when given a request scoped to a specific parent resource collection. The supported granularity **must** be documented on the `return_partial_success` field. ## Rationale ### Using service-relative resource names In general, relative resource names, as defined in AIP-122, are the best practice for referring to resources by name _within_ a service and in other services when that other service is obvious. The full resource name format is strictly less consumable (e.g., requires extra parsing client side), and over-specified for the uses of `unreachable`. Resource URIs are not transport agnostic, as they are unusable in standard methods for gRPC users, and simple resource IDs do not provide enough information about exactly which resource was unreachable in a heterogenous list of resources. ### Minimizing extra error details in response The context in which an unreachable resource is discovered may be sensitive and the state of the system fluid between calls. As such, it is preferred to defer to the service by making a more specific RPC to get more details about a specific resource or parent. This allows the parent to handle all necessary RPC checks and system state resolution on at time of request, rather than by shoehorning potentially privileged or stale information into the broader list call it was unreachable for. ### Unordered `unreachable` contents It is important for broad API consistency that the contents of `unreachable` not have a specific or order semantic structure. If each API baked a specific ordering into a standard field, no single implementation, client or server side, would be correct. ### Per page `unreachable` resources Populating `unreachable` resources on a per page basis allows end users to identify immediately when a page is incomplete, rather than _after_ paging through all results. Paging to completion is not guaranteed, so it is important to communicate as soon as possible when there are unreachable resource missing from a given page. Furthermore, it allows users to identify when there is a potential issue that they need to account for in subsequent calls. Finally, retaining unreachable resources until the end of paging results requires services to retain the state for what should be indepedent and fully isolated API calls. ### Using request field to opt-in Introducing a new request field as means of opting into the partial success behavior is the best way to communicate user intent while keeping the default behavior backwards compatible. The alternative, changing the default behavior with the introduction of the `unreachable` response field, presents a backwards incompatible change. Users that previously expected failure when any resource was unreachable, assume the successful response means all resources are accounted for in the response. ### Introducing fields simultaneously Introducing the request and response fields simultaneously is to prevent an invalid intermediate state that is presented by only adding one or the other. If only `unreachable` is added, then it could be assumed that it being empty means all resources were returned when that may not be true. If only `return_partial_success` is added, then the user wouldn't have a means of knowing which resources were unreachable. ### Partial success granularity limitations At a certain level of request scope granularity, an API is simply unable to enumerate the resources that are unreachable. For example, global-only APIs may be unable to provide granularity at a localized collection level. In such a case, preemptively returning an error when `return_partial_success=true` protects the user from the risks of the alternative - expecting unreachable resources if there was an issue, but not getting any, thus falsely assuming everything was retrieved. This aligns with guidance herein that suggests failing requests that cannot be fulfilled preemptively. ## History ### Pagination guidance The original guidance for how to populate the `unreachable` field revolved around consuming the contents as if they were the paged results. This meant that paged resources and unreachable resources couldn't be returned in the same response i.e. page, and users needed to completely page through all results in order to see if any were unreachable. See the Rationale section for the reasoning around the changes. ## Further reading - For listing across collections, see [AIP-159][]. ## Changelog - **2024-07-29**: Reformat guidance, add explicit resource name format - **2024-07-26**: Change pagination guidance. requirement. - **2024-07-19**: Add guidance for brownfield adoption of partial success. [aip-159]: ./0159.md --- id: 231 state: approved created: 2019-06-18 updated: 2022-06-02 placement: category: operations order: 200 --- # Batch methods: Get Some APIs need to allow users to get a specific set of resources at a consistent time point (e.g. using a read transaction). A batch get method provides this functionality. ## Guidance APIs **may** support Batch Get using the following pattern: ```proto rpc BatchGetBooks(BatchGetBooksRequest) returns (BatchGetBooksResponse) { option (google.api.http) = { get: "/v1/{parent=publishers/*}/books:batchGet" }; } ``` - The RPC's name **must** begin with `BatchGet`. The remainder of the RPC name **should** be the plural form of the resource being retrieved. - The request and response messages **must** match the RPC name, with `Request` and `Response` suffixes. - The HTTP verb **must** be `GET`. - The HTTP URI **must** end with `:batchGet`. - The URI path **should** represent the collection for the resource, matching the collection used for simple CRUD operations. If the operation spans parents, a dash (`-`) **may** be accepted as a wildcard. - There **must not** be a body key in the `google.api.http` annotation. - The operation **must** be atomic: it **must** fail for all resources or succeed for all resources (no partial success). For situations requiring partial failures, `List` ([AIP-132][]) methods **should** be used. - If the operation covers multiple locations and at least one location is down, the operation **must** fail. ### Request message The request for a batch get method **should** be specified with the following pattern: ```proto message BatchGetBooksRequest { // The parent resource shared by all books being retrieved. // Format: publishers/{publisher} // If this is set, the parent of all of the books specified in `names` // must match this field. string parent = 1 [ (google.api.resource_reference) = { child_type: "library.googleapis.com/Book" }]; // The names of the books to retrieve. // A maximum of 1000 books can be retrieved in a batch. // Format: publishers/{publisher}/books/{book} repeated string names = 2 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { type: "library.googleapis.com/Book" }]; } ``` - A `parent` field **should** be included, unless the resource being retrieved is a top-level resource, to facilitate inclusion in the URI as well to permit a single permissions check. If a caller sets this field, and the parent collection in the name of any resource being retrieved does not match, the request **must** fail. - This field **should** be required if only 1 parent per request is allowed. - The field **should** identify the [resource type][aip-122-parent] that it references. - The comment for the field **should** document the resource pattern. - The request message **must** include a repeated field which accepts the resource names specifying the resources to retrieve. The field **should** be named `names`. - If no resource names are provided, the API **should** error with `INVALID_ARGUMENT`. - The field **should** be required. - The field **should** identify the [resource type][aip-122-names] that it references. - The comment for the field **should** document the resource pattern. - Other fields besides `name` **may** be "hoisted" from the [standard Get request][request-message]. There is no way to allow for these fields to accept different values for different resources; if this is needed, use the [alternative request message form](#nested-request-objects). - Batch get **should not** support pagination because transactionality across API calls would be extremely difficult to implement or enforce, and the request defines the exact scope of the response anyway. - The request message **must not** contain any other required fields, and **should not** contain other optional fields except those described in this or another AIP. - The comment above the `names` field **should** document the maximum number of requests allowed. ### Response message The response for a batch get method **should** be specified with the following pattern: ```proto message BatchGetBooksResponse { // Books requested. repeated Book books = 1; } ``` - The response message **must** include one repeated field corresponding to the resources being retrieved. - The order of books in the response **must** be the same as the names in the request. [aip-122-names]: ./0122.md#fields-representing-resource-names [aip-122-parent]: ./0122.md#fields-representing-a-resources-parent [request-message]: ./0131.md#request-message ### Nested request objects If the [standard Get request message][request-message] contains a field besides the resource name that needs to be different between different resources being requested, the batch message **may** alternatively hold a `repeated` field of the [standard Get request message][request-message]. This is generally discouraged unless your use case really requires it. The request for a batch get method using this approach **should** be specified with the following pattern: ```proto message BatchGetBooksRequest { // The parent resource shared by all books being retrieved. // Format: publishers/{publisher} // If this is set, the parent field in the GetBookRequest messages // must either be empty or match this field. string parent = 1 [ (google.api.resource_reference) = { child_type: "library.googleapis.com/Book" }]; // The requests specifying the books to retrieve. // A maximum of 1000 books can be retrieved in a batch. repeated GetBookRequest requests = 2 [(google.api.field_behavior) = REQUIRED]; } ``` - A `parent` field **should** be included. If a caller sets this field, and the parent collection in the name of any resource being retrieved does not match, the request **must** fail. - This field **should** be required if only 1 parent per request is allowed. - The field **should** identify the [resource type][aip-122-parent] that it references. - The comment for the field **should** document the resource pattern. - The request message **must** include a repeated field which accepts the request messages specifying the resources to retrieve, as specified for [standard Get methods][request-message]. The field **should** be named `requests`. - The field **should** be required. - Other fields **may** be "hoisted" from the [standard Get request][request-message], which means that the field can be set at either the batch level or child request level. Similar to `parent`, if both the batch level and child request level are set for the same field, the values **must** match. - Batch get **should not** support pagination because transactionality across API calls would be extremely difficult to implement or enforce, and the request defines the exact scope of the response anyway. - The request message **must not** contain any other required fields, and **should not** contain other optional fields except those described in this or another AIP. - The comment above the `requests` field **should** document the maximum number of requests allowed. [aip-132]: https://aip.dev/132 ## Changelog - **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-". - **2020-09-16**: Suggested annotating `parent`, `names`, and `requests` fields. - **2020-08-27**: Removed parent recommendations for top-level resources. - **2020-03-24**: Clarified behavior if no resource names are sent. - **2019-09-11**: Changed the primary recommendation to specify a repeated string instead of a repeated standard Get request message. Moved the original recommendation into its own section. - **2019-08-01**: Changed the examples from "shelves" to "publishers", to present a better example of resource ownership. --- id: 233 state: approved created: 2019-06-18 updated: 2022-06-02 placement: category: operations order: 210 --- # Batch methods: Create Some APIs need to allow users to create multiple resources in a single transaction. A batch create method provides this functionality. ## Guidance APIs **may** support Batch Create using the following two patterns: Returning the response synchronously ```proto rpc BatchCreateBooks(BatchCreateBooksRequest) returns (BatchCreateBooksResponse) { option (google.api.http) = { post: "/v1/{parent=publishers/*}/books:batchCreate" body: "*" }; } ``` Returning an Operation which resolves to the response asynchronously ```proto rpc BatchCreateBooks(BatchCreateBooksRequest) returns (google.longrunning.Operation) { option (google.api.http) = { post: "/v1/{parent=publishers/*}/books:batchCreate" body: "*" }; option (google.longrunning.operation_info) = { response_type: "BatchCreateBooksResponse" metadata_type: "BatchCreateBooksOperationMetadata" }; } ``` - The RPC's name **must** begin with `BatchCreate`. The remainder of the RPC name **should** be the plural form of the resource being created. - The request and response messages **must** match the RPC name, with `Request` and `Response` suffixes. - If the batch method returns an `google.longrunning.Operation`, both the `response_type` and `metadata_type` fields **must** be specified. - The HTTP verb **must** be `POST`. - The HTTP URI **must** end with `:batchCreate`. - The URI path **should** represent the collection for the resource, matching the collection used for simple CRUD operations. If the operation spans parents, a dash (`-`) **may** be accepted as a wildcard. - The body clause in the `google.api.http` annotation **should** be `"*"`. ### Atomic vs. Partial Success - The batch create method **may** support atomic (all resources created or none are) or partial success behavior. To make a choice, consider the following factors: - **Complexity of Ensuring Atomicity:** Operations that are simple passthrough database transactions **should** use an atomic operation, while operations that manage complex resources **should** use partial success operations. - **End-User Experience:** Consider the perspective of the API consumer. Would atomic behavior be preferable for the given use case, even if it means that a large batch could fail due to issues with a single or a few entries? - Synchronous batch create **must** be atomic. - Asynchronous batch create **may** support atomic or partial success. - If supporting partial success, see [Operation metadata message](#operation-metadata-message) requirements. ### Request message The request for a batch create method **should** be specified with the following pattern: ```proto message BatchCreateBooksRequest { // The parent resource shared by all books being created. // Format: publishers/{publisher} // If this is set, the parent field in the CreateBookRequest messages // must either be empty or match this field. string parent = 1 [ (google.api.resource_reference) = { child_type: "library.googleapis.com/Book" }]; // The request message specifying the resources to create. // A maximum of 1000 books can be created in a batch. repeated CreateBookRequest requests = 2 [(google.api.field_behavior) = REQUIRED]; } ``` - A `parent` field **should** be included, unless the resource being created is a top-level resource. If a caller sets this field, and the `parent` field of any child request message does not match, the request **must** fail. The `parent` field of child request messages can be omitted if the `parent` field in this request is set. - This field **should** be required if only 1 parent per request is allowed. - The field **should** identify the [resource type][aip-122-parent] that it references. - The comment for the field **should** document the resource pattern. - The request message **must** include a repeated field which accepts the request messages specifying the resources to create, as specified for [standard Create methods][request-message]. The field **should** be named `requests`. - The field **should** be required. - Other fields **may** be "hoisted" from the [standard Create request][request-message], which means that the field can be set at either the batch level or child request level. Similar to `parent`, if both the batch level and child request level are set for the same field, the values **must** match. - Fields which must be unique cannot be hoisted (e.g. Customer-provided id fields). - The request message **must not** contain any other required fields, and **should not** contain other optional fields except those described in this or another AIP. - The comment above the `requests` field **should** document the maximum number of requests allowed. ### Response message The response for a batch create method **should** be specified with the following pattern: ```proto message BatchCreateBooksResponse { // Books created. repeated Book books = 1; } ``` - The response message **must** include one repeated field corresponding to the resources that were created. ### Operation metadata message - The `metadata_type` message **must** either match the RPC name with `OperationMetadata` suffix, or be named with `Batch` prefix and `OperationMetadata` suffix if the type is shared by multiple Batch methods. - If batch create method supports partial success, the metadata message **must** include a `map failed_requests` field to communicate the partial failures. - The key in this map is the index of the request in the `requests` field in the batch request. - The value in each map entry **must** mirror the error(s) that would normally be returned by the singular Standard Create method. - If a failed request can eventually succeed due to server side retries, such transient errors **must not** be communicated using `failed_requests`. - When all requests in the batch fail, `Operation.error` **must** be set with `code = google.rpc.Code.Aborted` and `message = "None of the requests succeeded, refer to the BatchCreateBooksOperationMetadata.failed_requests for individual error details"` - The metadata message **may** include other fields to communicate the operation progress. ### Adopting Partial Success In order for an existing Batch API to adopt the partial success pattern, the API must do the following: - The default behavior must be retained to avoid incompatible behavioral changes. - If the API returns an Operation: - The request message **must** have a `bool return_partial_success` field. - The Operation `metadata_type` **must** include a `map failed_requests` field. - When the `bool return_partial_success` field is set to true in a request, the API should allow partial success behavior, otherwise it should continue with atomic behavior as default. - If the API returns a direct response synchronously: - Since the existing clients will treat a success response as an atomic operation, the existing version of the API **must not** adopt the partial success pattern. - A new version **must** be created instead that returns an Operation and follows the partial success pattern described in this AIP. ## Rationale ### Restricting synchronous batch methods to be atomic The restriction that synchronous batch methods must be atomic is a result of the following considerations. The previous iteration of this AIP recommended batch methods must be atomic. There is no clear way to convey partial failure in a sync response status code because an OK implies it all worked. Therefore, adding a new field to the response to indicate partial failure would be a breaking change because the existing clients would interpret an OK response as all resources created. On the other hand, as described in [AIP-193](https://aip.dev/193), Operations are more capable of presenting partial states. The response status code for an Operation does not convey anything about the outcome of the underlying operation and a client has to check the response body to determine if the operation was successful. ### Communicating partial failures The AIP recommends using a `map failed_requests` field to communicate partial failures, where the key is the index of the failed request in the original batch request. The other options considered were: - A `repeated google.rpc.Status` field. This was rejected because it is not clear which entry corresponds to which request. - A `map` field, where the key is the request id of the failed request. This was rejected because: - Client will need to maintain a map of request_id -> request in order to use the partial success response. - Populating a request id for the purpose of communicating errors could conflict with [AIP-155](https://aip.dev/155) if the service can not guarantee idempotency for an individual request across multiple batch requests. - A `repeated FailedRequest` field, where FailedRequest contains the individual create request and the `google.rpc.Status`. This was rejected because echoing the request payload back in response is discouraged due to additional challenges around user data sensitivity. [aip-122-parent]: ./0122.md#fields-representing-a-resources-parent [request-message]: ./0133.md#request-message ## Changelog - **2025-03-06**: Added detailed guidance for partial success behavior, and decision framework for choosing between atomic and partial success - **2023-04-18**: Changed the recommendation to allow returning partial successes. - **2022-06-02**: Changed suffix descriptions to eliminate superfluous "-". - **2020-09-16**: Suggested annotating `parent` and `requests` fields. - **2020-08-27**: Removed parent recommendations for top-level resources. - **2019-08-01**: Changed the examples from "shelves" to "publishers", to present a better example of resource ownership. --- id: 234 state: approved created: 2019-06-18 updated: 2022-06-02 placement: category: operations order: 220 --- # Batch methods: Update Some APIs need to allow users to modify a set of resources in a single transaction. A batch update method provides this functionality. ## Guidance APIs **may** support Batch Update using the following two patterns: Returning the response synchronously ```proto rpc BatchUpdateBooks(BatchUpdateBooksRequest) returns (BatchUpdateBooksResponse) { option (google.api.http) = { post: "/v1/{parent=publishers/*}/books:batchUpdate" body: "*" }; } ``` Returning an Operation which resolves to the response asynchronously ```proto rpc BatchUpdateBooks(BatchUpdateBooksRequest) returns (google.longrunning.Operation) { option (google.api.http) = { post: "/v1/{parent=publishers/*}/books:batchUpdate" body: "*" }; option (google.longrunning.operation_info) = { response_type: "BatchUpdateBooksResponse" metadata_type: "BatchUpdateBooksOperationMetadata" }; } ``` - The RPC's name **must** begin with `BatchUpdate`. The remainder of the RPC name **should** be the plural form of the resource being updated. - The request and response messages **must** match the RPC name, with `Request` and `Response` suffixes. - If the batch method returns an `google.longrunning.Operation`, both the `response_type` and `metadata_type` fields **must** be specified. - The HTTP verb **must** be `POST`. - The HTTP URI **must** end with `:batchUpdate`. - The URI path **should** represent the collection for the resource, matching the collection used for simple CRUD operations. If the operation spans parents, a dash (`-`) **may** be accepted as a wildcard. - The body clause in the `google.api.http` annotation **should** be `"*"`. ### Atomic vs. Partial Success - The batch update method **may** support atomic (all resources updated or none are) or partial success behavior. To make a choice, consider the following factors: - **Complexity of Ensuring Atomicity:** Operations that are simple passthrough database transactions **should** use an atomic operation, while operations that manage complex resources **should** use partial success operations. - **End-User Experience:** Consider the perspective of the API consumer. Would atomic behavior be preferable for the given use case, even if it means that a large batch could fail due to issues with a single or a few entries? - Synchronous batch update **must** be atomic. - Asynchronous batch update **may** support atomic or partial success. - If supporting partial success, see [Operation metadata message](#operation-metadata-message) requirements. ### Request message The request for a batch update method **should** be specified with the following pattern: ```proto message BatchUpdateBooksRequest { // The parent resource shared by all books being updated. // Format: publishers/{publisher} // If this is set, the parent field in the UpdateBookRequest messages // must either be empty or match this field. string parent = 1 [ (google.api.resource_reference) = { child_type: "library.googleapis.com/Book" }]; // The request message specifying the resources to update. // A maximum of 1000 books can be modified in a batch. repeated UpdateBookRequest requests = 2 [(google.api.field_behavior) = REQUIRED]; } ``` - A `parent` field **should** be included, unless the resource being updated is a top-level resource. If a caller sets this field, and the parent collection in the name of any resource being updated does not match, the request **must** fail. - This field **should** be required if only 1 parent per request is allowed. - The field **should** identify the [resource type][aip-122-parent] that it references. - The comment for the field **should** document the resource pattern. - The request message **must** include a repeated field which accepts the request messages specifying the resources to update, as specified for [standard Update methods][request-message]. The field **should** be named `requests`. - The field **should** be required. - Other fields **may** be "hoisted" from the [standard Update request][request-message], which means that the field can be set at either the batch level or child request level. Similar to `parent`, if both the batch level and child request level are set for the same field, the values **must** match. - The `update_mask` field is a good candidate for hoisting. - The request message **must not** contain any other required fields, and **should not** contain other optional fields except those described in this or another AIP. - The comment above the `requests` field **should** document the maximum number of requests allowed. ### Response message The response for a batch update method **should** be specified with the following pattern: ```proto message BatchUpdateBooksResponse { // Books updated. repeated Book books = 1; } ``` - The response message **must** include one repeated field corresponding to the resources that were updated. ### Operation metadata message - The `metadata_type` message **must** either match the RPC name with `OperationMetadata` suffix, or be named with `Batch` prefix and `OperationMetadata` suffix if the type is shared by multiple Batch methods. - If batch update method supports partial success, the metadata message **must** include a `map failed_requests` field to communicate the partial failures. - The key in this map is the index of the request in the `requests` field in the batch request. - The value in each map entry **must** mirror the error(s) that would normally be returned by the singular Standard Update method. - If a failed request can eventually succeed due to server side retries, such transient errors **must not** be communicated using `failed_requests`. - When all requests in the batch fail, `Operation.error` **must** be set with `code = google.rpc.Code.Aborted` and `message = "None of the requests succeeded, refer to the BatchUpdateBooksOperationMetadata.failed_requests for individual error details"` - The metadata message **may** include other fields to communicate the operation progress. ### Adopting Partial Success In order for an existing Batch API to adopt the partial success pattern, the API must do the following: - The default behavior must be retained to avoid incompatible behavioral changes. - If the API returns an Operation: - The request message **must** have a `bool return_partial_success` field. - The Operation `metadata_type` **must** include a `map failed_requests` field. - When the `bool return_partial_success` field is set to true in a request, the API should allow partial success behavior, otherwise it should continue with atomic behavior as default. - If the API returns a direct response synchronously: - Since the existing clients will treat a success response as an atomic operation, the existing version of the API **must not** adopt the partial success pattern. - A new version **must** be created instead that returns an Operation and follows the partial success pattern described in this AIP. ## Rationale ### Restricting synchronous batch methods to be atomic The restriction that synchronous batch methods must be atomic is a result of the following considerations. The previous iteration of this AIP recommended batch methods must be atomic. There is no clear way to convey partial failure in a sync response status code because an OK implies it all worked. Therefore, adding a new field to the response to indicate partial failure would be a breaking change because the existing clients would interpret an OK response as all resources updated. On the other hand, as described in [AIP-193](https://aip.dev/193), Operations are more capable of presenting partial states. The response status code for an Operation does not convey anything about the outcome of the underlying operation and a client has to check the response body to determine if the operation was successful. ### Communicating partial failures The AIP recommends using a `map failed_requests` field to communicate partial failures, where the key is the index of the failed request in the original batch request. The other options considered were: - A `repeated google.rpc.Status` field. This was rejected because it is not clear which entry corresponds to which request. - A `map` field, where the key is the request id of the failed request. This was rejected because: - Client will need to maintain a map of request_id -> request in order to use the partial success response. - Populating a request id for the purpose of communicating errors could conflict with [AIP-155](https://aip.dev/155) if the service can not guarantee idempotency for an individual request across multiple batch requests. - A `repeated FailedRequest` field, where FailedRequest contains the individual update request and the `google.rpc.Status`. This was rejected because echoing the request payload back in response is discouraged due to additional challenges around user data sensitivity. [aip-122-parent]: ./0122.md#fields-representing-a-resources-parent [request-message]: ./0134.md#request-message ## Changelog - **2025-03-06**: Changed recommendation to allow partial success, along with detailed guidance - **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-". - **2020-09-16**: Suggested annotating `parent` and `requests` fields. - **2020-08-27**: Removed parent recommendations for top-level resources. - **2019-09-11**: Fixed the wording about which child field the `parent` field should match. - **2019-08-01**: Changed the examples from "shelves" to "publishers", to present a better example of resource ownership. --- id: 235 state: approved created: 2019-06-18 updated: 2022-06-02 placement: category: operations order: 230 --- # Batch methods: Delete Some APIs need to allow users to delete a set of resources in a single transaction. A batch delete method provides this functionality. ## Guidance APIs **may** support Batch Delete using the following two patterns: Returning the response synchronously ```proto rpc BatchDeleteBooks(BatchDeleteBooksRequest) returns (google.protobuf.Empty) { option (google.api.http) = { post: "/v1/{parent=publishers/*}/books:batchDelete" body: "*" }; } ``` Returning an Operation which resolves to the response asynchronously ```proto rpc BatchDeleteBooks(BatchDeleteBooksRequest) returns (google.longrunning.Operation) { option (google.api.http) = { post: "/v1/{parent=publishers/*}/books:batchDelete" body: "*" }; option (google.longrunning.operation_info) = { response_type: "google.protobuf.Empty" metadata_type: "BatchDeleteBooksOperationMetadata" }; } ``` - The RPC's name **must** begin with `BatchDelete`. The remainder of the RPC name **should** be the plural form of the resource being deleted. - The request message **must** match the RPC name, with a `Request` suffix. - The response message **should** be `google.protobuf.Empty`. - If the resource is [soft deleted][soft-delete], the response message **should** be a response message containing the updated resources. - If the batch method returns an `google.longrunning.Operation`, both the `response_type` and `metadata_type` fields **must** be specified. - If the resource is [soft deleted][soft-delete], the `response_type` **should** be a response message containing the updated resources. - The HTTP verb **must** be `POST` (not `DELETE`). - The HTTP URI **must** end with `:batchDelete`. - The URI path **should** represent the collection for the resource, matching the collection used for simple CRUD operations. If the operation spans parents, a dash (`-`) **may** be accepted as a wildcard. - The body clause in the `google.api.http` annotation **should** be `"*"`. ### Atomic vs. Partial Success - The batch delete method **may** support atomic (all resources deleted or none are) or partial success behavior. To make a choice, consider the following factors: - **Complexity of Ensuring Atomicity:** Operations that are simple passthrough database transactions **should** use an atomic operation, while operations that manage complex resources **should** use partial success operations. - **End-User Experience:** Consider the perspective of the API consumer. Would atomic behavior be preferable for the given use case, even if it means that a large batch could fail due to issues with a single or a few entries? - Synchronous batch delete **must** be atomic. - Asynchronous batch delete **may** support atomic or partial success. - If supporting partial success, see [Operation metadata message](#operation-metadata-message) requirements. ### Request message The request for a batch delete method **should** be specified with the following pattern: ```proto message BatchDeleteBooksRequest { // The parent resource shared by all books being deleted. // Format: publishers/{publisher} // If this is set, the parent of all of the books specified in `names` // must match this field. string parent = 1 [ (google.api.resource_reference) = { child_type: "library.googleapis.com/Book" }]; // The names of the books to delete. // A maximum of 1000 books can be deleted in a batch. // format: publishers/{publisher}/books/{book} repeated string names = 2 [ (google.api.field_behavior) = REQUIRED, (google.api.resource_reference) = { type: "library.googleapis.com/Book" }]; } ``` - A `parent` field **should** be included, unless the resource being deleted is a top-level resource. If a caller sets this field, and the parent collection in the name of any resource being deleted does not match, the request **must** fail. - This field **should** be required if only 1 parent per request is allowed. - The field **should** identify the [resource type][aip-122-parent] that it references. - The comment for the field **should** document the resource pattern. - The request message **must** include a repeated field which accepts the resource names specifying the resources to delete. The field **should** be named `names`. - The field **should** be required. - The field **should** identify the [resource type][aip-122-names] that it references. - The comment for the field **should** document the resource pattern. - Other fields besides `name` **may** be "hoisted" from the [standard Delete request][request-message]. There is no way to allow for these fields to accept different values for different resources; if this is needed, use the [alternative request message form](#request-message-containing-standard-delete-request-messages). - The request message **must not** contain any other required fields, and **should not** contain other optional fields except those described in this or another AIP. - The comment above the `names` field **should** document the maximum number of requests allowed. - Filter-based matching **must not** be supported. ### Request message containing standard delete request messages If the [standard Delete request message][request-message] contains a field besides the resource name that needs to be different between different resources being requested, the batch message **may** alternatively hold a `repeated` field of the [standard Delete request message][request-message]. This is generally discouraged unless your use case really requires it. The request for a batch delete method **should** be specified with the following pattern: ```proto message BatchDeleteBooksRequest { // The parent resource shared by all books being deleted. // Format: publishers/{publisher} // If this is set, the parent of all of the books specified in the // DeleteBookRequest messages must match this field. string parent = 1 [ (google.api.resource_reference) = { child_type: "library.googleapis.com/Book" }]; // The requests specifying the books to delete. // A maximum of 1000 books can be deleted in a batch. repeated DeleteBookRequest requests = 2 [(google.api.field_behavior) = REQUIRED]; } ``` - A `parent` field **should** be included. If a caller sets this field, and the parent collection in the name of any resource being deleted does not match, the request **must** fail. - This field **should** be required if only 1 parent per request is allowed. - The field **should** identify the [resource type][aip-122-parent] that it references. - The comment for the field **should** document the resource pattern. - The request message **must** include a repeated field which accepts the request messages specifying the resources to delete, as specified for [standard Delete methods][request-message]. The field **should** be named `requests`. - The field **should** be required. - Other fields **may** be "hoisted" from the [standard Delete request][request-message], which means that the field can be set at either the batch level or child request level. Similar to `parent`, if both the batch level and child request level are set for the same field, the values **must** match. - Fields which must be unique cannot be hoisted (e.g. `etag`). - The request message **must not** contain any other required fields, and **should not** contain other optional fields except those described in this or another AIP. - The comment above the `requests` field **should** document the maximum number of requests allowed. - Filter-based matching **must not** be supported unless it is infeasible to support critical use cases without it, because it makes it too easy for users to accidentally delete important data. If it is unavoidable, see [AIP-165][]. ### Response message (soft-delete only) In the case where a response message is necessary because the resource is soft-deleted, the response **should** be specified with the following pattern: ```proto message BatchDeleteBooksResponse { // Books deleted. repeated Book books = 1; } ``` - The response message **must** include one repeated field corresponding to the resources that were soft-deleted. ### Operation metadata message - The `metadata_type` message **must** either match the RPC name with `OperationMetadata` suffix, or be named with `Batch` prefix and `OperationMetadata` suffix if the type is shared by multiple Batch methods. - If batch delete method supports partial success, the metadata message **must** include a `map failed_requests` field to communicate the partial failures. - The key in this map is the index of the request in the `requests` field in the batch request. - The value in each map entry **must** mirror the error(s) that would normally be returned by the singular Standard Delete method. - If a failed request can eventually succeed due to server side retries, such transient errors **must not** be communicated using `failed_requests`. - When all requests in the batch fail, `Operation.error` **must** be set with `code = google.rpc.Code.Aborted` and `message = "None of the requests succeeded, refer to the BatchDeleteBooksOperationMetadata.failed_requests for individual error details"` - The metadata message **may** include other fields to communicate the operation progress. ### Adopting Partial Success In order for an existing Batch API to adopt the partial success pattern, the API must do the following: - The default behavior must be retained to avoid incompatible behavioral changes. - If the API returns an Operation: - The request message **must** have a `bool return_partial_success` field. - The Operation `metadata_type` **must** include a `map failed_requests` field. - When the `bool return_partial_success` field is set to true in a request, the API should allow partial success behavior, otherwise it should continue with atomic behavior as default. - If the API returns a direct response synchronously: - Since the existing clients will treat a success response as an atomic operation, the existing version of the API **must not** adopt the partial success pattern. - A new version **must** be created instead that returns an Operation and follows the partial success pattern described in this AIP. ## Rationale ### Restricting synchronous batch methods to be atomic The restriction that synchronous batch methods must be atomic is a result of the following considerations. The previous iteration of this AIP recommended batch methods must be atomic. There is no clear way to convey partial failure in a sync response status code because an OK implies it all worked. Therefore, adding a new field to the response to indicate partial failure would be a breaking change because the existing clients would interpret an OK response as all resources created. On the other hand, as described in [AIP-193](https://aip.dev/193), Operations are more capable of presenting partial states. The response status code for an Operation does not convey anything about the outcome of the underlying operation and a client has to check the response body to determine if the operation was successful. ### Communicating partial failures The AIP recommends using a `map failed_requests` field to communicate partial failures, where the key is the index of the failed request in the original batch request. The other options considered were: - A `repeated google.rpc.Status` field. This was rejected because it is not clear which entry corresponds to which request. - A `map` field, where the key is the request id of the failed request. This was rejected because: - Client will need to maintain a map of request_id -> request in order to use the partial success response. - Populating a request id for the purpose of communicating errors could conflict with [AIP-155](https://aip.dev/155) if the service can not guarantee idempotency for an individual request across multiple batch requests. - A `repeated FailedRequest` field, where FailedRequest contains the individual create request and the `google.rpc.Status`. This was rejected because echoing the request payload back in response is discouraged due to additional challenges around user data sensitivity. [aip-122-names]: ./0122.md#fields-representing-resource-names [aip-122-parent]: ./0122.md#fields-representing-a-resources-parent [aip-165]: ./0165.md [request-message]: ./0135.md#request-message [soft-delete]: ./0135.md#soft-delete ## Changelog - **2025-03-06**: Changed recommendation to allow partial success, along with detailed guidance - **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-". - **2020-09-16**: Suggested annotating `parent`, `names`, and `requests` fields. - **2020-08-27**: Removed parent recommendations for top-level resources. - **2020-03-27**: Added reference to AIP-165 for criteria-based deletion. - **2019-10-11**: Changed the primary recommendation to specify a repeated string instead of a repeated standard Delete message. Moved the original recommendation into its own section. - **2019-09-11**: Fixed the wording about which child field the `parent` field should match. - **2019-08-01**: Changed the examples from "shelves" to "publishers", to present a better example of resource ownership. --- id: 236 state: approved created: 2023-03-30 updated: 2023-03-30 placement: category: resource-design order: 240 --- # Policy preview A policy is a resource that provides rules that admit or deny access to other resources. Generally, the outcome of a policy can be evaluated to a specific set of outcomes. Changes to policies without proper validation may have unintended consequences that can severely impact a customer’s overall infrastructure setup. To safely update resources, it is beneficial to test these changes via policy rollout APIs. Preview is a rollout safety mechanism for policy resources, which gives the customer the ability to validate the effect of their proposed changes against production traffic prior to the changes going live. The result of the policy evaluation against traffic is logged in order to give the customer the data required to test the correctness of the change. Firewall policies exemplify a case that is suitable for previewing. A new configuration can be evaluated against traffic to observe which IPs would be allowed or denied. This gives the customer the data to guide a decision on whether to promote the proposed changes to live. The expected flow for previewing a policy is as follows: 1. The user creates an experiment containing a new policy configuration intended to replace the live policy. 2. The user uses the "startPreview" method to start generating logs which compare the live and experiment policy evaluations against live traffic. 3. The user inspects the logs to determine whether the experiment has the intended result. 4. The user uses the "commit" method to promote the experiment to live. ## Guidance ### Non-goals This proposal is for a safety mechanism for policy rollouts only. Safe rollouts for non-policy resources are not in scope. ### Experiments A new configuration of a policy to be previewed is stored as a nested collection under the policy. These nested collections are known as experiments. A hypothetical policy resource called, `Policy`, is used throughout. It has the following resource name pattern: `projects/{project}/locations/{location}/policies/{policy}` The experimental versions of the resource used for previewing or other safe rollout practices are represented as a nested collection under `Policy` using a new resource type. The resource type **must** follow the naming convention *RegularResourceType*`Experiment`. The following pattern is used for the experiment collection: `projects/{project}/locations/{location}/policies/{policy}/experiments/{experiment}` A proto used to represent an experiment **must** contain the following: 1. The required top-level fields for a resource, like `name` and `etag` 2. The policy message that is being tested itself 3. The field, `preview_metadata`, which contains metadata specific to previewing the experiment of a specific resource type. ```proto message PolicyExperiment { // google.api.resource, name, and other annotations and fields // The policy experiment. This Policy will be used to preview the effects of // the change but will not affect live traffic. Policy policy = 2; // The metadata associated with this policy experiment. PolicyPreviewMetadata preview_metadata = 3 [(google.api.field_behavior) = OUTPUT_ONLY]; // Allows clients to store small amounts of arbitrary data. map annotations = 4; } ``` - The experiment proto **must** have a top-level field with the same type as the live policy. - It **must** be named as the live resource type. For example, if the experiment is for FirewallPolicy, then this field **must** be named `firewall_policy`. - The name inside the embedded `policy` message **must** be the name of the live policy. - When the user is ready to promote an experiment, they **must** copy the `policy` message into the live policy and delete the experiment. This can be done manually or via a "commit" custom method. - A product **may** support multiple experiments concurrently being previewed for a single live policy. - Each experiment must generate logs having each entry preceded by log_prefix so that the user can compare the results of the experiment with the behavior of the live policy. - The number of experimental configurations for a given live policy **may** be capped at a certain number and the cap **must** be documented. - Cascading deletes **must** occur: if the live policy is deleted, all experiments **must** also be deleted. - `map` [annotations][aip-128-annotations] **must** allow clients to store small amounts of arbitrary data. ### Metadata `preview_metadata` tracks all metadata of previewing the experiment. The messages **must** follow the convention: *RegularResourceType*`PreviewMetadata`. This is so the proto can be defined uniquely for each resource type in the same service with experiments. ```proto message PolicyPreviewMetadata { // Possible values of the state of previewing the experiment. enum State { // Default value. This value is unused. STATE_UNDEFINED = 0; // The experiment is actively previewing. ACTIVE = 1; // The previewing of the experiment has been stopped. SUSPENDED = 2; } // The state of previewing the experiment. State state = 1; // An identifying string common to all logs generated when previewing the // experiment. Searching all logs for this string will isolate the results. string log_prefix = 2; // The most recent time at which this experiment started previewing. google.protobuf.Timestamp start_time = 3; // The most recent time at which this experiment stopped previewing. google.protobuf.Timestamp stop_time = 4; } ``` - `PolicyPreviewMetadata` **must** have the fields defined in the proto above. - It **may** have additional fields if the service or resource requires it. - When an experiment is first previewed, `preview_metadata` **must** be absent. - It is present on the experiment once the "startPreview" method is used. - All `preview_metadata` fields **must** be output only. - `state` changes between `ACTIVE` and `SUSPENDED` when previewing is started or stopped. This happens when the "startPreview" or "stopPreview custom methods are invoked, respectively. - The first time the "startPreview" custom method is used, the system **must** create `preview_metadata` and do the following: - It **must** set the `state` to `ACTIVE` - It **must** populate `start_time` with the current time. - `start_time` **must** be updated every time `state` is changed to `ACTIVE`. - It **must** set a system generated `log_prefix` string, which is a predefined constant hard coded by the system developers. - The same value is used for previewing experiments for the given resource type. For example, "FirewallPolicyPreviewLog" for FirewallPolicy. - When the "stopPreview" custom method is used, the system **must** do the following: - It **must** set the `state` to `SUSPENDED` - It **must** populate the `stop_time` with the current time. ### Methods #### create - The resource **must** be created using long-running [Create][aip-133-long-running] and `google.longrunning.operation_info.response_type` **must** be `PolicyExperiment`. - Creating a new experiment to preview **must** support the following use cases: - Preview a new policy. - Preview an update to an already live policy. - Preview a deletion of a current policy. - For the update and delete use cases, the `policy` field in the experiment **must** have the full payload of the live policy copied into it, including the name. - The user **must** set the rules to the new intended state to preview an update. - The user **must** set set the rules to represent a no-op to preview a delete. - To preview a new policy, the system must do the following: - If the system does not support a nested collection without a live policy, the user **must** create a live policy and set the rules to represent a no-op. For example, the rules of a no-op policy **may** be empty. - An experiment is created as a child of the no-op policy. - If the system supports previewing multiple experiments for a live policy, calling "create" more than once **must** create multiple experiments. #### update - The resource **must** be updated using long-running [Update][aip-134-long-running] and `google.longrunning.operation_info.response_type` **must** be `PolicyExperiment`. - The name inside `policy` **must not** change but the other fields can in order to change the experiment being previewed because this `policy` is intended to replace the live policy, and the name of the live policy **must not** change. - The system **must** set the `state` to `SUSPENDED` if the `state` was `ACTIVE` at the time of an update. - This is so the user can easily distinguish between different versions of the experiment being previewed. #### get - The standard method, [Get][aip-131], **must** be included for `PolicyExperiment` resource types. #### list - The standard method, [List][aip-132], **must** be included for `PolicyExperiment` resource types. - Filtering on `PolicyPreviewMetadata` indicates which experiments are actively previewed. - For example, the following filter string returns a List response with experiments being previewed: preview_metadata.state = ACTIVE. #### delete - The resource **must** be deleted using long-running [Delete][aip-135-long-running] and `google.longrunning.operation_info.response_type` **must** be `PolicyExperiment`. #### startPreview ```proto // Starts previewing a PolicyExperiment. This triggers the system to start // generating logs to evaluate the PolicyExperiment. rpc StartPreviewPolicyExperiment(StartPreviewPolicyExperimentRequest) returns (google.longrunning.Operation) { option (google.api.http) = { post: "/v1/{name=policies/*/experiments/*}:startPreview" body: "*" }; option (google.longrunning.operation_info) = { response_type: "PolicyExperiment" metadata_type: "StartPreviewPolicyExperimentMetadata" }; } // The request message for the startPreview custom method. message StartPreviewPolicyExperimentRequest { // The name of the PolicyExperiment. string name = 1; } ``` - This custom method is required. - `google.longrunning.Operation.metadata_type` **must** follow guidance on [Long-running operations][aip-151] - This method **must** trigger the system to start generating logs to preview the experiment. - Whenever the method is called successfully, the system **must** set the following values in the `PolicyPreviewMetadata`: - `log_prefix` to the predefined constant. - `start_time` to the current time - `state` to `ACTIVE`. - If the method is called on an experiment with the rules representing a no-op, then the system **must** preview the deletion of the live policy. #### stopPreview ```proto // Stops previewing a PolicyExperiment. This triggers the system to stop // generating logs to evaluate the PolicyExperiment. rpc StopPreviewPolicyExperiment(StopPreviewPolicyExperimentRequest) returns (google.longrunning.Operation) { option (google.api.http) = { post: "/v1/{name=policies/*/experiments/*}:stopPreview" body: "*" }; option (google.longrunning.operation_info) = { response_type: "PolicyExperiment" metadata_type: "StopPreviewPolicyExperimentMetadata" }; } // The request message for the stopPreview custom method. message StopPreviewPolicyExperimentRequest { // The name of the PolicyExperiment. string name = 1; } ``` - This custom method is required. - `google.longrunning.Operation.metadata_type` **must** follow guidance on [Long-running operations][aip-151] - This method **must** trigger the system to stop generating logs to preview the experiment. - Whenever the method is called successfully, the system **must** set the following values in the `PolicyPreviewMetadata`: - `stop_time` to the current time - `state` to `SUSPENDED` #### commit The resource **may** expose a new custom method called "commit" to promote an experiment. The system copies `policy` from the experiment into the live policy and then deletes the experiment. Declarative clients **may** manually copy fields from an experiment into the live policy and then delete the experiment rather than calling "commit" if preferable. ```proto // Commits a PolicyExperiment. This copies the PolicyExperiment's policy message // to the live policy then deletes the PolicyExperiment. rpc CommitPolicyExperiment(CommitPolicyExperimentRequest) returns (google.longrunning.Operation) { option (google.api.http) = { post: "/v1/{name=policies/*/experiments/*}:commit" body: "*" }; option (google.longrunning.operation_info) = { response_type: "google.protobuf.Empty" metadata_type: "CommitPolicyExperimentMetadata" }; } // The request message for the commit custom method. message CommitPolicyExperimentRequest { string name = 1; string etag = 2; string parent_etag = 3; } ``` - `google.longrunning.Operation.metadata_type` **must** follow guidance on [Long-running operations][aip-151] - The method **must** atomically copy `policy` from the experiment into the live policy, and then delete the experiment. - If any experiment fails "commit", previewing it **must not** stop, and the live policy **must not** be updated. - The method can be called on an experiment in any state. - The `etag` **must** match that of the experiment in order for commit to be successful. This is so the user does not commit an unintended version of the experiment. - If no `etag` is provided, the API **must not** succeed to prevent the user from unintentionally committing a different version of the experiment as intended. - A `parent_etag` **may** be provided to guarantee that the experiment overwrites a specific version of the live policy. - The method is not idempotent and calling it twice on the same experiment **must** return a 404 NOT_FOUND as the experiment is deleted as part of the first call. ### Changes to live policy API methods #### delete - A delete of the live policy **must** delete all experiments. - To maintain the experiments while negating the effect of the live policy, the live policy **must** be changed to a no-op policy instead of using this method. ### Logging Logging is crucial for the user to evaluate whether an experiment should be promoted to live. Logs **must** contain the results of the evaluated experiment, the `etag` associated with that experiment alongside that of the live policy, and be preceded by the value of `log_prefix`. - The `etag` fields help the user identify which configurations of the live and experiment are evaluated in the log. - `log_prefix` helps the user separate logs specifically generated for previewing the experiment from other use cases. Overall, these logs help the user make a decision about whether to promote the experiment to live. ## Changelog - **2023-04-27:** Methods for start and stop renamed. State to enum. Annotations added. - **2023-03-30:** Initial AIP written. [aip-128-annotations]: https://aip.dev/128#annotations [aip-131]: https://aip.dev/131 [aip-132]: https://aip.dev/132 [aip-133-long-running]: https://aip.dev/133#long-running-create [aip-134-long-running]: https://aip.dev/134#long-running-update [aip-135-long-running]: https://aip.dev/135#long-running-delete [aip-151]: https://google.aip.dev/151 --- title: General order: 0 categories: - code: meta - code: process - code: api-concepts title: API Concepts - code: resource-design title: Resource Design - code: operations - code: fields - code: design-patterns title: Design Patterns - code: compatibility title: Compatibility and Versioning - code: polish - code: protobuf title: Protocol buffers - code: misc title: Miscellaneous default: true