charles/webstory
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
4d4be92df7adc931e66d9331bd8a3049572754a7
webstory/aip.md
T
charles 4e111c7f6d add: aip, a story resource
2026-03-24 22:49:44 -07:00

12672 lines
505 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
id: 1
state: approved
created: 2018-08-20
placement:
category: meta
order: 10
js_scripts:
- /assets/js/graphviz/viz.js
- /assets/js/graphviz/lite.render.js
- /assets/js/aip/aip-graphviz.js
---
# AIP Purpose and Guidelines
As the corpus of Google APIs has grown and the API Governance team has grown to
meet the demand of supporting them, it is increasingly necessary to have a
corpus of documentation for API producers, reviewers, and other interested
parties to reference. The API style guide and introductory One Platform
documentation are intentionally terse and high-level. The AIP collection
offers a method to provide consistent documentation for API design guidance.
## What is an AIP?
AIP stands for **API Improvement Proposal**, which is a design document
providing high-level, concise documentation for API development. They are to
serve as the source of truth for API-related documentation at Google and the
means by which API teams discuss and come to consensus on API guidance. AIPs
are maintained as Markdown files in the [AIP GitHub repository][].
## Types of AIPs
There are several different types of AIPs, described below. The list of AIP
types may evolve over time as necessary.
### Guidance
These AIPs describe guidance on API design. These are provided as instruction
for API producers to help write simple, intuitive, and consistent APIs, and are
used by API reviewers as a basis for review comments.
### Process
These AIPs describe a process surrounding API design. They often affect the AIP
process itself and are used to enhance the way in which AIPs are handled.
## Stakeholders
As with any process there are many different stakeholders when it comes to
reviewing and working with AIPs. Below is a summary of the escalation path
starting with the API producer.
```graphviz
digraph d_front_back {
rankdir=BT;
ranksep=0.3;
node [ style="filled,solid" shape=box fontname="Roboto" ];
producer [ label="API Producer" ];
editors [ label="AIP Editors" ];
tl_infra [ label="Infrastructure TL" ];
tl_design [ label="Design TL" ];
tl [ label="TL" ];
producer -> editors;
editors -> tl_infra -> tl;
editors -> tl_design -> tl;
}
```
As noted in the diagram above, the TL is the final decision-maker on the AIP
process and the final point of escalation if necessary.
### Editors
The editors are the set of people who make decisions on AIPs. The general goal
is that the AIP process is collaborative and that we largely work on the basis
of consensus. However, a limited number of designated approvers is necessary,
and these Googlers will be approvers for each AIP in the general scope.
The list of AIP editors is currently:
- Angie Lin ([@alin04][])
- Jon Skeet ([@jskeet][])
- Jose Juan Zavala Iglesias ([@itsStrobe][])
- Louis Dejardin ([@loudej][])
- Noah Dietz ([@noahdietz][])
- Sam Levenick ([@slevenick][])
- Sam Woodard ([@shwoodard][])
The editors are also responsible for the administrative and editorial aspects
of shepherding AIPs and managing the AIP pipeline and workflow. They approve
PRs to AIPs, assign proposal numbers, manage the agenda, set AIP states, and so
forth. They also ensure that AIPs are readable (proper spelling, grammar,
sentence structure, markup, etc.).
AIP editorship is by invitation of the current editors.
## Domain-specific AIPs
Some AIPs may be specific to a particular domain (for example, only to APIs
within a certain PA, or even a certain team). In this situation, the group will
be given a particular block of AIPs to use in accordance with AIP-2, and the
applicable AIPs will clearly indicate their scope.
## States
At any given time, AIPs may exist in a variety of states as they work their way
through the process. The following is a summary of each state.
### Draft
The initial state for an AIP is the "Draft" state. This means that the AIP is
being discussed and iterated upon, primarily by the original authors. While the
editors _may_ get involved at this stage, it is not necessary.
**Note:** If significant, high-level iteration is required, it is recommended
to draft AIPs in a Google doc instead of a PR. AIPs that are migrated into the
AIP system from Google Docs **may** skip the draft state and go directly to
reviewing provided there is sufficient approval.
### Reviewing
Once discussion on an AIP has generally concluded, but before it is formally
accepted it moves to the "Reviewing" state. This means that the authors have
reached a general consensus on the proposal and the editors are now involved.
At this stage the editors may request changes or suggest alternatives to the
proposal before moving forward.
**Note:** As a formal matter, one AIP approver (other than the author) **must**
provide formal signoff to advance an AIP to the reviewing state. Additionally,
there **must not** be formal objections ("changes requested" on the GitHub PR)
from other approvers.
### Approved
Once an approved AIP has been agreed upon, it enters "approved" state and is
considered "best current practice".
**Note:** As a formal matter, two AIP approvers (other than the author)
**must** provide formal signoff to advance an AIP to the approved state.
Additionally, there **must not** be formal objections ("changes requested" on
the GitHub PR) from other approvers.
### Withdrawn
If an AIP is withdrawn by the author or champion, it enters "withdrawn" state.
AIPs that are withdrawn may be taken up by another champion.
### Rejected
If an AIP is rejected by the AIP editors, it enters "rejected" state. AIPs that
are rejected remain, and provide documentation and reference to inform future
discussions.
### Deferred
If an AIP has not been acted upon for a significant period of time, the editors
may mark it as "deferred".
### Replaced
If an AIP has been replaced by another AIP, it enters "replaced" state. AIP
editors are responsible to provide a notice explaining the replacement and
rationale (the replacement AIP should also clearly explain the rationale).
In general, API producers should rely primarily on AIPs in the "approved"
state.
## Workflow
The following workflow describes the process for proposing an AIP, and moving
an AIP from proposal to implementation to final acceptance.
### Overview
```graphviz
digraph d_front_back {
rankdir=LR;
node [ style="filled,solid" shape=box fontname="Roboto" ];
draft [ label="Draft" fillcolor="orange" ];
reviewing [ label="Reviewing" fillcolor="lightskyblue" ];
approved [ label="Approved" fillcolor="palegreen" ];
withdrawn [ label="Withdrawn" fillcolor="mistyrose" ];
rejected [ label="Rejected" fillcolor="mistyrose" ];
deferred [ label="Deferred" fillcolor="lightsteelblue" ];
replaced [ label="Replaced" fillcolor="lightsteelblue" ];
draft -> reviewing;
draft -> withdrawn [ style=dashed, color=mistyrose3 ];
draft -> rejected [ style=dashed, color=mistyrose3 ];
reviewing -> approved;
reviewing -> withdrawn [ style=dashed, color=mistyrose3 ];
reviewing -> rejected [ style=dashed, color=mistyrose3 ];
draft -> deferred [ style=dashed, color=lightsteelblue3 ];
reviewing -> deferred [ style=dashed, color=lightsteelblue3 ];
approved -> replaced [ style=dashed, color=lightsteelblue3 ];
reviewing -> replaced [ style=dashed, color=lightsteelblue3 ];
}
```
### Proposing an AIP
In order to propose an AIP, first [open an issue][] to circulate the
fundamental idea for initial feedback. It should generally be possible to
describe the idea in a couple of pages.
When proposing a new AIP or changes to an existing one, it is best to reference
prior art and/or example use cases that the proposal will impact, so as to
ensure that the proposal is grounded in a realistic problem space. So, proposals
**should** provide concrete references and/or well-defined examples. Appropriate
material includes, but is not limited to, the following:
- Existing external RFCs or standards
- A corpus of APIs that have aligned on a similar pattern e.g. `Search` methods
- A concrete use case that has yet to be solved that exists or could exist in
one or more APIs e.g. adding an AIP-202 Format for AIP-143 Unicode CLDR
region codes
Once ready, create a PR with a new file in the AIP directory using a file
titled `aip/new.md`. Ensure that the PR is editable by maintainers.
In most circumstances, the editors will assign the proposal an AIP number and
submit the PR with the AIP in the "Reviewing" state. The editors may reject an
AIP outright if they have an obvious reason to do so (e.g. the proposal was
already discussed and rejected in another AIP or is fundamentally unsound), in
which case the PR is not merged.
### Discussing an AIP
Once the PR is merged, the AIP author is responsible for championing the AIP on
a follow-up approval pull request. This means that the author is responsible
for pushing towards consensus around the proposal. This may involve a
discussion at the regularly scheduled meetings for the API Governance team.
The AIP author may modify the AIP over the course of discussion by submitting
follow-up commits to the PR.
### Accepting an AIP
The editors will work together to ensure that qualified proposals do not linger
in review.
To gain final approval, an AIP **must** be approved by, at minimum, the TL with
responsibility over the domain covered by the AIP (either design or
infrastructure) and at least one other editor, with no editors actively
requesting changes.
**Note:** If an AIP editor is the primary author of an AIP, then at least two
_other_ editors must approve it.
Once the AIP is approved, the editors will update the state of the AIP to
reflect this and submit the PR.
### Withdrawing or Rejecting an AIP
The author of an AIP may decide, after further consideration, that an AIP
should not advance. If so, the author may withdraw the AIP by updating the PR
adding a notice of withdrawal with an explanation of the rationale.
Additionally, the author may be unable to get consensus among the group and the
AIP editors may elect to reject the AIP. In this situation, the AIP editors
shall amend the PR adding a notice of rejection with an explanation of the
rationale. In both cases, the AIP editors update the state accordingly and
submit the PR.
### Replacing an AIP
In rare cases, it may be necessary to replace an AIP with another one. This is
not general practice: minor edits to approved AIPs are acceptable, and will be
the common way to tweak guidance. However, if new guidance fundamentally alters
the old guidance in some way, then the AIP editors shall create a new AIP that,
once approved, will replace the old one. The old one then enters "Replaced"
state, and will link to the new, current AIP.
## Changelog
- **2025-01-09**: Add requirement to include references/examples in proposals.
- **2024-09-04**: Updated names of current editors and remove TLs.
- **2023-05-10**: Updated names of current and editors and TLs.
- **2019-07-30**: Further clarified AIP quorum requirements.
- **2019-05-12**: Collapsed AIP approvers and editors into a single position,
relaxed approval rules from full quorum.
- **2019-05-04**: Updated the AIP to refer to GitHub processes, rather than
internal processes.
[aip github repository]: https://github.com/googleapis/aip
[open an issue]: https://github.com/googleapis/aip/issues
[@alin04]:https://github.com/alin04
[@jskeet]: https://github.com/jskeet
[@loudej]: https://github.com/loudej
[@noahdietz]: https://github.com/noahdietz
[@slevenick]: https://github.com/slevenick
[@shwoodard]: https://github.com/shwoodard
[@itsStrobe]: https://github.com/itsStrobe
---
id: 2
state: approved
created: 2018-08-23
placement:
category: meta
order: 20
---
# AIP Numbering
The AIP system provides a mechanism to index and have a single source of truth
for API Improvement Proposals, as well as iterate on them collaboratively and
transparently.
This document describes the AIP numbering system.
## Assigning AIP Numbers
The AIP editors (see [AIP-1](./0001.md)) are responsible for assigning a number
to each AIP when it is accepted as a draft for review. Importantly, _all_ AIPs
have numbers, not just approved ones. The [AIP Index](/) clearly delineates
which AIPs are approved and binding and which are under discussion.
The editors may decide to reserve a specific block of numbers for groups of
AIPs that are related in some way (for example, that are only scoped to a
specific subset of APIs).
Beyond this, AIP numbers are assigned arbitrarily. In general, the editors will
take the next AIP number off of the stack to assign to a draft AIP, but
occasionally may use a special/joke number if useful for mnemonic or other
reasons.
## AIP Blocks
Currently recognized blocks of AIP numbers are:
### Generally Applicable
- **1-99:** Reserved for meta-AIPs (generally process-related).
- **100-999:** General API design guidance
### Google Product Areas
- **2700-2799:** Apps (Google Workspace)
- **2500-2599:** Cloud
- **3000-3099:** Actions on Google
- **3200-3299:** Firebase
- **4100-4199:** Auth libraries
- **4200-4299:** Client libraries
- **4600-4699:** Geo
To request a block for a specific team that is publishing API guidance or
documentation germane to that specific team, reach out to api-editors@.
## Changelog
- **2019-10-03:** The 3000-3099 block was assigned to Actions on Google.
- **2019-01-26:** The general API design guidance block was expanded to include
100-199.
- **2018-10-24:** The 4600-4699 block was assigned to Google Geo.
- **2018-10-02:** The 2500-2599 block was assigned to Google Cloud Platform.
- **2018-10-02:** The 2700-2799 block was assigned to Google Workspace.
---
id: 3
state: approved
created: 2023-03-28
placement:
category: meta
order: 30
---
# AIP Versioning
This AIP defines the versioning scheme of the AIPs.
## Guidance
- The AIPs **must** be versioned by date, using the [ISO-8601 format of
`YYYY-MM-DD`](https://www.w3.org/TR/NOTE-datetime), that corresponds to the
date the version was added.
- AIPs versions **must** be available as a tag on the source control system used
to store the AIPs, of the format `v{version}`. Example: `v2023-03-28`.
- The AIPs **must** have a new version when there is a significant change to one
or more AIPs.
- Each AIP **must** include a changelog section, with the date the change was
made and a short description.
## Rationale
Versions serve as reference points to AIPs at a specific point in time. They are
crucial since guidance on an AIP can be reversed, or include significant changes
such that they are no longer similar to the original design. APIs using AIPs may
need to reference older AIP guidance to justify their design choice.
### Rationale for date-based versioning
With date-based versioning, a client can easily find the AIP guidance at the
time an API was authored.
### Why not individually versioned AIPs
An alternative to a universal version is to have specific versions attached to
each AIP.
AIPs often cross-reference one another. If each AIP had a specific version, then
cross-references would also have to specify specific versions of those
referenced AIPs to provide complete guidance.---
id: 8
state: approved
created: 2019-05-28
placement:
category: meta
order: 40
---
# AIP Style and Guidance
AIP stands for **API Improvement Proposal**, which is a design document
providing high-level, concise documentation for API design and development. The
goal is for these documents to serve as the source of truth for API-related
documentation at Google and the way API teams discuss and come to consensus on
API guidance.
AIPs are most useful when they are clear and concise, and cover a single topic
or inquiry well. In the same way that AIPs describe consistent patterns and
style for use in APIs, they also _follow_ consistent patterns and style.
## Guidance
- AIPs **must** cover a single, discrete topic, and provide clear, actionable
guidance.
- AIPs **must not** duplicate or contradict guidance in another AIP.
- AIPs **may** also cover what _not_ to do, but **should not** cover _only_
anti-patterns.
- If AIP guidance is conditional (e.g. a design pattern such as Jobs), the
guidance **must** clearly explain under what conditions the guidance should be
followed.
Guidance contained within an AIP **must** be beneficial to one or more
types of clients or their authors, including but not limited to:
- Asset inventories which can be used to audit and analyze resources.
- Command line interfaces for exploration and simple automation.
- Custom controllers (e.g. auto-scalers) which poll live state and adjust
resource configuration accordingly.
- [IaC][] clients for orchestration and automation of multiple resources.
- Recommendation tools which provide guidance on which APIs are useful for
specific use cases, and how to use them.
- SDKs to interact with an API from a programming language, often used heavily
for data-plane operations.
- Security orchestration, automation, and remediation tools.
- Simple scripts to automate or orchestrate tasks.
- Test frameworks.
- Tools that operate on resource data at rest.
- Visual User Interfaces for visualization and one-off manual actions.
- Users.
Examples of improvements include:
- Requiring new proto annotations that enable more descriptive interfaces on
clients (e.g. requiring `singular` and `plural` on a `google.api.resource`
annotation).
AIP guidance **must not** be a significant detriment to a client's usability or
implementation difficulty, or maintenance difficulty.
Examples of detriments include:
- Introduction of a non-uniform pattern in a standard method such that all
clients must introduce additional code without sufficient benefit (e.g. List
behaves like this *except* for resources that start with the name Foo).
- Renames of well-established fields for minor improvements in readability (e.g.
rename `expire_time` to `lapse_time` since `lapse` is a common term in my
service).
While the length of AIPs will necessarily vary based on the complexity of the
question, most AIPs **should** be able to cover their content in roughly two
printed pages.
### File structure
AIPs **must** be written in Markdown, and **must** be named using their
four-digit number (example: `0008.md`). AIPs that serve a specific scope
**must** be in the subdirectory for that scope.
AIPs **must** have appropriate front matter.
```yaml
---
id: 8
state: reviewing
created: 2019-05-28
permalink: /8
redirect_from:
- /08
- /008
- /0008
---
```
Front matter for AIPs **must** include:
- The `aip` key:
- `id`: Required. The ID for the given AIP, as an integer.
- `state`: Required. The current state of the AIP, in all lower-case. The
valid states are listed in [AIP-1][], and common states are `draft`,
`reviewing`, and `approved`.
- `created`: Required. The ISO-8601 date (`yyyy-mm-dd`) when the AIP was
originally drafted, with no quotes.
- `updated`: The ISO-8601 date (`yyyy-mm-dd`) when the AIP was last revised.
- `scope`: The scope for the AIP. This **must** match the directory name for
that scope. Required for AIPs with IDs >= 1000, prohibited otherwise.
- The `permalink` key (required): This **must** be set to
`/{aip.scope}/{aip.id}`. If there is no scope, use `/{aip.id}` instead.
- The `redirect_from` key: This should include a list of any `/{aip.id}`
permutations that a reader would be likely to enter, including:
- `/{aip.id}` (for AIPs where the permalink includes the scope)
- AIP IDs with zero-padding, for each level of zero-padding up to four digits
(for example: `/08`, `/008`, `/0008`).
### Document structure
AIPs **must** begin with a top-level heading with the AIP's title (`# Title`).
The title **should** be a noun (not an imperative). For example, "Bad API
precedents" not "Avoid breaking API precedent".
AIPs **should** then begin with an introduction (with no additional heading),
followed by a `## Guidance` heading. If necessary, the AIP **may** include any
of the following after the guidance, in the following order:
- "Further reading" is a bulleted list of links to other AIPs that are useful to
fully understand the current AIP.
- "Appendices" covering further explanation in the same AIP. These are
relatively rare but are important in cases where an AIP requires a lot of
justification for the decision. Often this is primarily an explanation of
alternatives considered to help explain the guidance.
- "Changelog" is a bulleted list of changes made to the AIP since the first
writing.
The guidance section **may** include subsections that elaborate further on
details. Subsections will automatically create an entry in the table of
contents, and an anchor for citations.
Below is an example AIP shell that uses each major section:
```md
# AIP title
The introductory text explains the background and reason why the AIP exists. It
lays out the basic question, but does not tell the reader what to do.
## Guidance
The "guidance" section helps the reader know what to do. A common format for
the guidance section is a high-level imperative, followed by an example,
followed by a bulleted list explaining the example.
### Subsection
Individual subsections can be cited individually, and further elaborate
details.
## Rationale
The "rationale" section is optional, and helps the reader understand the
motivation behind specific guidance within the AIP.
Deeper explanations of design justification and tradeoffs **must** be in the
rationale instead of other sections, to ensure the rest of the document acts as
an easily actionable reference.
## History
The "history" section is optional, and documents events and context around a
significant edit to an AIP. For example, explanation of rewrite would be
included in this section
While the changelog is a dotted list of one-line summaries of changes to an AIP,
the history section should elaborate on significant events in a descriptive
format.
The section **must not** be used to exhaustively enumerate all changes. This
is what the changelog provides.
## Further reading
A bulleted list of (usually) other AIPs, in the following format:
- [AIP-1](./0001.md): AIP purpose and guidelines
## Changelog
A bulleted list of changes in reverse chronological order, using the following
format:
- **2020-02-18**: Specified ordering.
- **2019-07-01**: Added a subsection clarifying XYZ.
```
AIPs **should** attempt to follow this overall format if possible, but AIPs
**may** deviate from it if necessary (in particular, if the AIP would be more
difficult to understand, even for a reader already accustomed to reading AIPs in
the usual format).
**Note:** Except for the title, AIPs **must** only use the second heading level
(`##`) and above. AIPs **should** only use the second and third heading levels
(`##`, `###`).
### Requirement keywords
AIPs **should** use the following requirement level keywords: "MUST", "MUST
NOT", "SHOULD", "SHOULD NOT", and "MAY", which are to be interpreted as
described in [RFC 2119][].
When using these terms in AIPs, they **must** be lower-case and **bold**. These
terms **should not** be used in other ways.
If "SHOULD" or "SHOULD NOT" are used, they **must** include valid examples of
where other concerns may override the guidance.
**Important:** If rationale is used, it exists to provide background and a more
complete understanding, but **must not** contain guidance (and RFC-2119 terms
**must not** be used).
### Code examples
API design examples in AIPs **should** use [protocol buffers][]. Examples
**should** cover only enough syntax to explain the concept. When using RPCs in
examples, a `google.api.http` annotation **should** be included.
### Referencing AIPs
When AIPs reference other AIPs, the prosaic text **must** use the format
`AIP-XXXX` without zero-padding (e.g., `AIP-8`, not `AIP-0008`), and **must**
link to the relevant AIP. AIP links **may** point to a particular section of the
AIP if appropriate.
**Important:** AIP links **must** use the relative path to the file in the
repository (such as `./0008.md` for core AIPs, or `../0008.md` for AIPs in a
subdirectory); this ensures that the link works both on the AIP site, when
viewing the Markdown file on GitHub, using the local development server, or a
branch.
## Rationale
### Designing for a broad set of clients
API guidance, similar to any software, is most beneficial when there is a clear
purpose and target beneficiary.
The beneficiaries of improved API design are users. These users interact with
APIs via a variety of clients, depending on their use case as enumerated above.
API guidance must in turn consider the impact broadly across these clients.
[aip-1]: ./0001.md
[Infrastructure as Code]: ./0009.md#iac
[IaC]: ./0009.md#iac
[protocol buffers]: https://developers.google.com/protocol-buffers/
[rfc 2119]: https://www.ietf.org/rfc/rfc2119.txt
## Changelog
- **2023-05-20**: Increase API guidance scope to include broad set of clients.
- **2023-03-30**: Removed appendix, added rationale and history to the template.
- **2020-02-18**: Specified reverse chronological ordering for changelog items.
- **2019-08-23**: Added guidance for internal AIP links.---
id: 9
state: approved
created: 2019-08-01
placement:
category: meta
order: 100
---
# Glossary
In the name of brevity, this AIP defines some common terminology here rather
than in each AIP individually.
## Guidance
The following terminology **should** be used consistently throughout AIPs.
### API
Application programming interface. This can be a local interface (such as a
client library) or a Network API (defined below).
### API backend
A set of servers and related infrastructure that implements the business logic
for an API service. An individual API backend server is often called an API
server.
### API consumer
The entity that consumes an API service. For Google APIs, it typically is a
Google project that owns the client application or the server resource.
### API definition
The definition of an API, usually defined in a Protocol Buffer service. An API
definition can be implemented by any number of API services.
### API frontend
A set of servers plus related infrastructure that provides common functionality
across API services, such as load balancing and authentication. An individual
API frontend server is often called an API proxy.
**Note:** the API frontend and the API backend may run next to each other or far
away from each other. In some cases, they can be compiled into a single
application binary and run inside a single process.
### API interface
The element of an API specification IDL that groups API methods, such as a
Protocol Buffers `service` definition. It is typically mapped to a similar high
level grouping mechanism in most programming languages, like a `class` or
`interface`.
### API method
An individual operation within an API. It is typically represented in Protocol
Buffers by an `rpc` definition, and is mapped to a function in the API in most
programming languages.
### API producer
The entity that produces an API service. For Google APIs, it typically is a
Google team responsible for the API service.
### API product
An API service and its related components, such as Terms of Service,
documentation, client libraries, and service support, are collectively presented
to customers as a API product. For example, Google Calendar API.
**Note:** people sometimes refer to an API product simply as an API.
### API service
A deployed implementation of one or more APIs, exposed on one or more network
addresses, such as the Cloud Pub/Sub API.
### API service definition
The combination of API definitions (`.proto` files) and API service
configurations (`.yaml` files) used to define an API service. The schema for
Google API service definition is `google.api.Service`.
### API service endpoint
Refers to a network address that an API service uses to handle incoming API
requests. One API service may have multiple API service endpoints, such as
`https://pubsub.googleapis.com` and `https://content-pubsub.googleapis.com`.
### API service name
Refers to the logical identifier of an API service. Google APIs use RFC 1035 DNS
compatible names as their API service names, such as `pubsub.googleapis.com`.
### API title
Refers to the user-facing product title of an API service, such as "Cloud Pub/Sub
API".
### API request
A single invocation of an API method. It is often used as the unit for billing,
logging, monitoring, and rate limiting.
### API version
The version of an API or a group of APIs if they are defined together. An API
version is often represented by a string, such as "v1", and presents in API
requests and Protocol Buffers package names.
### Client
Clients are programs that perform a specific tasks by calling an API or generic
tools, such as CLIs, that expose the API in a user-accessible fashion or operate
on resource data at rest.
Examples of clients include the following:
- Command line interfaces
- Libraries, such as an SDK for a particular programming language
- Scripts that operates on a JSON representation of a resource after reading it
from an API
- Tools, such as a [Declarative clients][]
- Visual UIs, such as a web application
### Google API
A Network API exposed by a Google service. Most of these are hosted on the
`googleapis.com` domain. It does not include other types of APIs, such as client
libraries and SDKs.
### Declarative Clients
Declarative Clients, also known as Infrastructure as Code (IaC), describes a
category of clients that consumes a markup language or code that represents
resources exposed by an API, and executes the appropriate imperative actions to
drive the resource to that desired state. To determine what changes to make and
if a set of updates was successful a declarative client compares server side
resource attributes with client defined values. The comparison feature ensures
accuracy of a creation or an update but it requires services to treat the client
set fields as read-only and diligently preserve those values.
Examples of complexities that declarative clients abstract away include:
- Determining the appropriate imperative action (create / update / delete) to
achieve desired state.
- Ordering of these imperative actions.
[Terraform][] is an example of such a client.
### User
A human being which is using an API directly, such as with cURL. This term is
defined to differentiate usage in the AIPs between a human *user* and a
programmatic *client*.
### Network API
An API that operates across a network of computers. Network APIs communicate
using network protocols including HTTP, and are frequently produced by
organizations separate from those that consume them.
[Declarative clients]: #declarative-clients
[Terraform]: https://www.terraform.io/
## Changelog
- **2025-08-13**: Add API inteface entry
- **2024-12-18**: Downcase headings and terms as per dev docs style
- **2024-10-23**: Add API Title entry
- **2023-07-24**: Rename IaC to Declarative Clients
- **2023-04-01**: Adding definition of IaC
- **2023-03-24**: Reformatting content to include anchor links.
---
id: 100
state: approved
created: 2018-08-27
placement:
category: process
order: 10
js_scripts:
- /assets/js/graphviz/viz.js
- /assets/js/graphviz/lite.render.js
- /assets/js/aip/aip-graphviz.js
---
# API Design Review FAQ
API design review exists to ensure a simple, intuitive, and consistent API
experience throughout our API corpus.
## Do I need API design approval?
**TL;DR:** You usually need API design approval if you are launching an API
that users can code against (either now or in the future) at the beta or GA
quality level.
API design review is fundamentally about ensuring we provide a simple and
consistent experience for our users, and therefore is only expected for APIs
that users code directly against.
The following flowchart illustrates whether or not your API needs to go through
the design review process:
```graphviz
digraph {
node [ style="filled,solid" shape=box fontname="Roboto" ];
graph [ splines=ortho, nodesep=0.2 ];
audience [ label="Who should code directly\nagainst this API?"
shape=diamond fillcolor=bisque ];
subgraph audience_responses {
rank = "same";
node [ shape=oval fillcolor=orange ];
googlers [ label="Googlers\nOnly" fillcolor=lightcoral ];
public [ label="Anyone" fillcolor=limegreen ];
partners [ label="Partners\nOnly" fillcolor=deepskyblue ];
partners -> public -> googlers [ style=invisible arrowhead=none ];
}
subgraph and_ever {
rank = "same";
node [ shape=diamond fillcolor=bisque ];
forever_partners [ label="Forever?" shape=diamond fillcolor=bisque ];
forever_googlers [ label="Forever?" shape=diamond fillcolor=bisque ];
}
subgraph forever_responses {
rank = "same";
node [ shape=oval fillcolor=orange ];
forever_partners_yes [ label="Yes" fillcolor=deepskyblue ];
forever_no [ label="No, anyone\neventually" fillcolor=limegreen ];
forever_googlers_yes [ label="Yes" fillcolor=lightcoral ];
}
release_level [ label="What release\nlevel?" shape=diamond fillcolor=bisque ];
# Not required is not in the subgraph with the other outcomes because
# it makes the graph much smaller if it can be higher in the image.
subgraph release_levels {
rank = "same";
node [ shape=oval ];
alpha [ label="Alpha" fillcolor=darkorange ];
beta [ label="Beta" fillcolor=goldenrod1 ];
ga [ label="GA" fillcolor=limegreen ];
alpha -> beta -> ga [ style=invisible arrowhead=none ];
}
changes [ label="Any changes\nfrom beta?" shape=diamond fillcolor=bisque ];
subgraph changes_bool {
rank = "same";
node [ shape=oval ];
changes_yes [ label="Yes" fillcolor=goldenrod1 ];
changes_no [ label="No" fillcolor=limegreen ];
}
subgraph outcome {
rank = "same";
node [ style="rounded,filled" ];
fyi [ label="FYI" fillcolor=lightblue ];
recommended [ label="Recommended" fillcolor=limegreen ];
required [ label="⚠ Required" fillcolor=goldenrod1 ];
not_required
[ label="Not Required" fillcolor=lightgrey style="rounded,filled" ];
recommended -> required -> fyi -> not_required
[style=invisible arrowhead=none];
}
audience -> googlers [ arrowhead=none style=dashed color=grey ];
audience -> partners [ arrowhead=none ];
audience -> public [ arrowhead=none ];
partners -> forever_partners;
googlers -> forever_googlers [ style=dashed color=grey ];
public -> release_level;
forever_partners -> forever_partners_yes [ arrowhead=none ];
forever_googlers -> forever_googlers_yes
[ arrowhead=none style=dashed color=grey ];
forever_partners -> forever_no [ arrowhead=none ];
forever_googlers -> forever_no [ arrowhead=none ];
forever_partners_yes -> recommended;
forever_googlers_yes -> not_required [ style=dashed color=grey ];
forever_no -> release_level;
release_level -> alpha [ arrowhead=none ];
release_level -> beta [ arrowhead=none ];
release_level -> ga [ arrowhead=none ];
alpha -> recommended;
beta -> required;
ga -> changes;
changes -> changes_yes [ arrowhead=none ];
changes -> changes_no [ arrowhead=none ];
changes_yes -> required;
changes_no -> fyi;
}
```
### Who should code directly against it?
One of the more complex questions is, "Who should code directly against this
API?" API design review is primarily concerned about the API's audience. This
means we care about who is permitted to write their own HTTP/gRPC calls against
the service, and who is able to see the documentation. (We do _not_ care about
questions such as whether the service is exposed on the public network.)
Design review is expected if the general public is intended to read
documentation and write code that interacts with the service.
The following situations **do not** require design review:
- An API which will only ever be used by Googlers, or internal tools (for
example, Pantheon).
- An API which will only ever be called by an executable program released by
Google (even if the API could be reverse-engineered from the executable).
- An API which will only ever be called by a single customer or small set of
customers under contract, and which will _never_ be made more widely
available. (Design review is still recommended in this case, but not
required.)
### Alpha
For alpha, API design review is optional but recommended. It may often make
sense to endeavor to get initial feedback from customers quickly, and launching
an alpha can be a way of gaining data to determine the best answer to some
usability questions; therefore, bypassing review may be expedient. On the other
hand, launching an alpha requires building an implementation which then takes
engineering effort to update if the API design review at the beta stage raises
concerns. Because API design review can precede implementation work, we
recommend a design review for alpha.
## Why is design review important?
**TL;DR:** Product excellence.
Our design review process exists to ensure that the APIs that we present to
customers are **simple**, **intuitive**, and **consistent**. Your reviewer
approaches your API from the standpoint of a naïve user, thinks through the
resources and actions that your API provides, and attempts to make the surface
as accessible and extensible as possible.
Your design reviewer is not only evaluating your API, but also checking to
ensure that your API is consistent with Google's existing corpus of APIs. Many
customers use multiple APIs, and therefore it is important that our conventions
and naming choices line up with customer expectations.
## What should I expect?
### How long does the review process take?
Reviewers make an effort to keep up with their assigned reviews and offer
feedback frequently, so as not to cause unnecessary delay, but it's generally
best to begin the review process early in case there are delays.
The design review process varies based on the size and complexity of the
underlying API surface:
- Incremental changes to existing APIs generally take a few days.
- Small APIs usually take around a week.
- Entirely new APIs with large surfaces tend to take no less than a week. In
cases with extraordinarily large surfaces (e.g., Cloud AutoML), reviews may
take a month or more to go through design review.
### How do reviewers approach my API?
API reviewers seek to approach your API the same way that your users will, by
focusing primarily on the API surface and its user-facing documentation. In an
ideal world, your API reviewer will ask the types of design questions that
users will ask (and nudge the API toward raising fewer of those questions in
the first place).
### What is precedent?
In general, we want Google APIs to be as consistent as possible. Once customers
learn their first Google API, it should be easier to learn the second (and then
the third, and so on) because we are using the same patterns consistently.
We refer to **precedent** to mean decisions that have already been made by
previous APIs, which generally should be binding upon newer APIs in similar
situations. The most common example of this is naming: we have a list of
[standard fields][] that dictate how we use common terms like `name`,
`create_time`, and so forth, and which also dictates that we always attach _the
same_ name to the same concept.
Precedent also applies to _patterns_. All APIs should implement pagination the
same way. Ditto for long-running operations, import and export, and so on. Once
a pattern has been established, we seek to implement that pattern the same way
wherever it is germane.
## What should I do?
### ...if I have a launch on a tight deadline?
The best thing that you can do is to engage design review as early as possible.
Additionally, make your reviewers aware of your timeline so that they are
aware, and can endeavor to provide you the best possible service. We _want_ you
to make your deadline if at all possible.
For time-sensitive _alpha_ launches, an API **may** launch without receiving
design review approval. Such launches **must** be limited to a known set of
users. In this case, the reviewers will provide notes for the API team to take
under consideration for subsequent stages.
**Warning:** Launching an API in alpha with an incomplete design review **does
not** enshrine that API's decisions. Design review will be required to promote
the API to beta, and API reviewers will block your beta launch if there are
issues.
For launch stages after alpha, the API design review is mandatory due to its
impact on user experience across the board. Your team's inconsistencies affect
more than just your team.
In some cases, there is a difficult choice to be made between product
excellence and either engineering effort or deadlines. These are difficult
business decisions and we understand that they are sometimes necessary;
however, a director or VP must make an explicit choice to put these other
concerns ahead of product excellence when choosing to bypass design review or
disregard reviewers' feedback.
### ...to make my review go faster?
A few tips:
- Begin API review as early as possible, and follow up frequently.
- Run the [API linter][] beforehand. (If you are disabling the linter at any
point, explain why. Reviewers often find that the linter is disabled because
it did its job.)
- Ensure that every message, RPC, and field is _usefully_ commented. Comments
should be in valid American English and say something meaningful.
- If your API reviewer asks you to explain something, add the explanation _in
the proto comments_, rather than the code review conversation. This will very
often save you a round trip.
### ...if one of my API reviewers is unresponsive?
Reach out to the reviewer on Chat. If that fails, reach out to the other
reviewer, who will coordinate accordingly. If that fails also, escalate
according to [AIP-1][].
### ...if I have a design question?
The first places to look at the [API style guide][], the [AIP index][], and
other public APIs within Google. Other public APIs are particularly valuable;
it is common that someone has encountered a situation that is germane to your
question.
### ...if I have a question not covered there?
Reach out to api-design@google.com with your question.
This generally works best when you are seeking guidance on a specific question
related to API design, and when you clearly explain your use case and provide
examples.
**Note:** The membership of this list comprises almost exclusively volunteers,
who spend the majority of their time doing something else. We do our best to be
responsive, but please be patient with us.
### ...if a question is complex and languishing in a CL?
While the code review interface is the best way to resolve questions when
practical, sometimes there are issues that are sufficiently complicated that
working them out in the code review tool is not feasible. In this situation,
reach out to your reviewers and ask to schedule a meeting. In general, most
issues can be discussed in 30 minutes.
When this happens, make sure that someone documents what is discussed in the
CL, so that the history is preserved.
### ...if my API needs to violate a standard?
Clearly document (using an internal comment in the proto) that you are
violating an API design guideline and your rationale for doing so. This comment
**must** be prefixed with `aip.dev/not-precedent`.
In general, your rationale for the design guideline violation **should** be in
accordance with one of the enumerated reasons listed in [AIP-200][]. If it is
not, work together with your API reviewer to determine the right thing to do.
### ...if a reviewer is bringing up a previously-settled issue?
If you have a different reviewer from your API's previous stages, this might
happen. In general, the best approach is simply to reference the code review
where the issue was decided. Reviewers want to avoid causing you churn, and
therefore usually give deference to previous reviews. This is usually
sufficient to resolve the question promptly.
Occasionally, the reviewer may believe that the previous reviewer made a
significant mistake, and that correcting it is important. In this case, you and
your reviewer should work together to determine the best course of action.
### ...if the team and the reviewers strongly disagree?
Escalate according to [AIP-1][].
## Does my PA or team have any particular guidelines?
The Cloud PA has specific guidelines to ensure additional uniformity across
Cloud, and Cloud APIs have their own reviewer pool. Other teams may adopt
similar (but not necessarily identical) rules and systems. Some teams that
produce multiple APIs (for example, machine learning) may also have guidelines
that apply to that group of APIs.
In all cases, we endeavor to make these guidelines available as AIPs; the
higher AIP numbers are reserved for specific PA and team use (see [AIP-2][]),
and these AIPs are listed in the [AIP index][].
[aip-1]: ./0001.md
[aip-2]: ./0002.md
[aip-200]: ./0200.md
[aip index]: /
[api linter]: https://github.com/googleapis/api-linter
[api style guide]: https://cloud.google.com/apis/design/
[standard fields]: https://cloud.google.com/apis/design/standard_fields
---
id: 111
state: approved
created: 2023-06-17
placement:
category: api-concepts
order: 50
---
# Planes
Resources and methods on an API can be divided into the *plane* that they reside
or perform operations upon. For the context of APIs, the following planes are
defined:
- Management plane: a uniform, resource-oriented API that primarily
configures and allows retrieval of resources.
- Data plane: a heterogenous API (ideally resource-oriented) that reads and
write user data. Often connects to entities provisioned by the management
plane, such as virtual machines.
The term "plane" was originally used in networking architecture. Although system
and network architecture often defines additional planes (e.g. control plane or
power planes), as the AIPs are focused on the interface, they are not defined in
this AIP.
## Guidance
### Management Plane
Management resources and methods exist primarily to provision, configure, and
audit the resources that the data plane interfaces with.
As an example, the following are considered management resources for a cloud
provider:
- virtual machines
- virtual private networks
- virtual disks
- a blob store instance
- a project or account
### Data Plane
Methods on the data plane operate on user data in a variety of data formats, and
generally interface with a resource provisioned via a management plane API.
Examples of data plane methods include:
- writing and reading rows in a table
- pushing to or pulling from a message queue
- uploading blobs to or downloading blobs from a blob store instance
Data plane APIs **may** be heterogenous across a larger API surface, due to
requirements including high throughput, low latency, or the need to adhere to an
existing interface specification (e.g. ANSI SQL).
- For convenience, resources and methods that operate on the data plane **may**
expose themselves via resource-oriented management APIs. If so, those resources
and methods **must** adhere to the requirements of the management plane as
specified in the other AIPs ([AIP-131][] through [AIP-135][]).
### Major distinctions between management and data plane
- [Declarative clients][] operate on the management plane exclusively.
- Data planes are often on the critical path of user-facing functionality, and
therefore:
- Have higher availability requirements than management planes.
- Are more performance-sensitive than management planes.
- Require higher-throughput than management planes.
[AIP-131]: ./0131.md
[AIP-135]: ./0135.md
[Declarative clients]: ./0009.md#declarative-clients
## Changelog
- **2023-06-10**: Introduction of this AIP.
---
id: 121
state: approved
created: 2019-01-26
placement:
category: resource-design
order: 10
---
# Resource-oriented design
Resource-oriented design is a pattern for specifying [RPC][] APIs, based on
several high-level design principles (most of which are common to recent public
HTTP APIs):
- The fundamental building blocks of an API are individually-named _resources_
(nouns) and the relationships and hierarchy that exist between them.
- A small number of standard _methods_ (verbs) provide the semantics for most
common operations. However, custom methods are available in situations where
the standard methods do not fit.
- Stateless protocol: Each interaction between the client and the server is
independent, and both the client and server have clear roles.
Readers might notice similarities between these principles and some principles
of [REST][]; resource-oriented design borrows many principles from REST, while
also defining its own patterns where appropriate.
## Guidance
When designing an API, consider the following (roughly in logical order):
- The resources (nouns) the API will provide
- The relationships and hierarchies between those resources
- The schema of each resource
- The methods (verbs) each resource provides, relying as much as possible on
the standard verbs.
### Resources
A resource-oriented API **should** generally be modeled as a resource
hierarchy, where each node is either a simple resource or a collection of
resources.
A _collection_ contains resources of _the same type_. For example, a publisher
has the collection of books that it publishes. A resource usually has fields,
and resources may have any number of sub-resources (usually collections).
**Note:** While there is some conceptual alignment between storage systems and
APIs, a service with a resource-oriented API is not necessarily a database, and
has enormous flexibility in how it interprets resources and methods. API
designers **should not** expect that their API will be reflective of their
database schema. In fact, having an API that is identical to the underlying
database schema is actually an anti-pattern, as it tightly couples the surface
to the underlying system.
### Methods
Resource-oriented APIs emphasize resources (data model) over the methods
performed on those resources (functionality). A typical resource-oriented API
exposes a large number of resources with a small number of methods on each
resource. The methods can be either the standard methods ([Get][], [List][],
[Create][], [Update][], [Delete][]), or [custom methods][].
If the request to or the response from a standard method (or a custom method in
the same *service*) **is** the resource or **contains** the resource, the
resource schema for that resource across all methods **must** be the same.
| Standard method | Request | Response |
| --------------- | --------------------- | ---------------------- |
| Create | Contains the resource | Is the resource |
| Get | None | Is the resource |
| Update | Contains the resource | Is the resource |
| Delete | None | None |
| List | None | Contains the resources |
*The table above describes each standard method's relationship to the resource,
where "None" indicates that the resource neither **is** nor **is contained** in
the request or the response*
A resource **must** support at minimum [Get][]: clients must be
able to validate the state of resources after performing a mutation such
as [Create][], [Update][], or [Delete][].
A resource **must** also support [List][], except for [singleton resources][]
where more than one resource is not possible.
**Note:** A custom method in resource-oriented design does _not_ entail
defining a new or custom HTTP verb. Custom methods use traditional HTTP verbs
(usually `POST`) and define the custom verb in the URI.
APIs **should** prefer standard methods over custom methods; the purpose of
custom methods is to define functionality that does not cleanly map to any of
the standard methods. Custom methods offer the same design freedom as
traditional RPC APIs, which can be used to implement common programming
patterns, such as database transactions, import and export, or data analysis.
### Strong Consistency
For methods that operate on the [management plane][], the completion of those
operations (either successful or with an error, long-running operation, or
synchronous) **must** mean that the state of the resource's existence and all
user-settable values have reached a steady-state.
[Output only][] values unrelated to the resource [state][] **should** also have
reached a steady-state for values that are related to the resource [state][].
Examples include:
- Following a successful create that is the latest mutation on a resource, a get
request for a resource **must** return the resource.
- Following a successful update that is the latest mutation on a resource, a get
request for a resource **must** return the final values from the update
request.
- Following a successful delete that is the latest mutation on a resource, a get
request for a resource **must** return `NOT_FOUND` (or the resource with the
`DELETED` state value in the case of [soft delete][])
Clients of resource-oriented APIs often need to orchestrate multiple operations
in sequence (e.g. create resource A, create resource B which depends on A), and
ensuring that resources immediately reflect steady user state after an operation
is complete ensures clients can rely on method completion as a signal to begin
the next operation.
[Output only][] fields ideally would follow the same guidelines, but as
these fields can often represent a resource's live state, it's sometimes
necessary for these values to change after a successful mutation operation to
reflect a state change.
### Stateless protocol
As with most public APIs available today, resource-oriented APIs **must**
operate over a [stateless protocol][]: The fundamental behavior of any
individual request is independent of other requests made by the caller.
This is to say, each request happens in isolation of other requests made by that
client or another, and resources exposed by an API are directly addressable
without needing to apply a series of specific requests to "reach" the desired
resource.
In an API with a stateless protocol, the server has the responsibility for
persisting data, which may be shared between multiple clients, while clients
have sole responsibility and authority for maintaining the application state.
### Cyclic References
The relationship between resources, such as with [resource references][],
**must** be representable via a [directed acyclic graph][]. The parent-child
relationship also **must** be acyclic, and as per [AIP-124][] a given resource
instance will only have one canonical parent resource.
A cyclic relationship between resources increases the complexity of managing
resources. Consider resources A and B that refer to
each other. The process to create said resources are:
1. create resource A without a reference to B. Retrieve id for resource A.
2. create resource B with a reference to A. Retrieve id for resource B.
3. update resource A with the reference to B.
The delete operation may also become more complex, due to reasoning about which
resource must be dereferenced first for a successful deletion.
This requirement does not apply to relationships that are expressed via
[output only][] fields, as they do not require the user to specify the values
and in turn do not increase resource management complexity.
[AIP-124]: ./0124.md
[create]: ./0133.md
[custom methods]: ./0136.md
[delete]: ./0135.md
[directed acyclic graph]: https://en.wikipedia.org/wiki/Directed_acyclic_graph
[get]: ./0131.md
[list]: ./0132.md
[management plane]: ./0111.md#management-plane
[output only]: ./0203.md#output-only
[rest]: https://en.wikipedia.org/wiki/Representational_state_transfer
[resource references]: ./0122.md#fields-representing-another-resource
[rpc]: https://en.wikipedia.org/wiki/Remote_procedure_call
[singleton resources]: ./0156.md
[soft delete]: ./0164.md
[state]: ./0216.md
[stateless protocol]: https://en.wikipedia.org/wiki/Stateless_protocol
[update]: ./0134.md
## Changelog
- **2024-07-08**: Clarify acyclic nature of parent-child relationship.
- **2023-08-24**: Added guidance on consistency guarantees of methods.
- **2023-07-23**: Clarify stateless protocol definition.
- **2023-01-21**: Explicitly require matching schema across standard methods.
- **2022-12-19**: Added a section requiring Get and List.
- **2022-11-02**: Added a section restricting resource references.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
---
id: 122
state: approved
created: 2019-01-26
placement:
category: resource-design
order: 20
---
# Resource names
Most APIs expose _resources_ (their primary nouns) which users are able to
create, retrieve, and manipulate. Additionally, resources are _named_: each
resource has a unique identifier that users use to reference that resource, and
these names are what users should _store_ as the canonical names for the
resources.
## Guidance
All resource names defined by an API **must** be unique within that API. (See
the section on [full resource names](#full-resource-names) below for more
information on referring to resources across APIs.)
Resource names are formatted according to the [URI path schema][], but without
the leading slash:
publishers/123/books/les-miserables
users/vhugo1802
- Resource name components **should** usually alternate between collection
identifiers (example: `publishers`, `books`, `users`) and resource IDs
(example: `123`, `les-miserables`, `vhugo1802`).
- Resource names **must** use the `/` character to separate individual segments
of the resource name.
- Non-terminal segments of a resource name **must not** contain a `/`
character.
- The terminal segment of a resource name **should not** contain a `/`
character.
- Resource names **should** only use characters available in DNS names, as
defined by [RFC-1123](https://tools.ietf.org/html/rfc1123).
- Additionally, resource IDs **should not** use upper-case letters.
- If additional characters are necessary, resource names **should not** use
characters that require URL-escaping, or characters outside of ASCII.
- If Unicode characters can not be avoided, resource names **must** be stored
in Normalization Form C (see [AIP-210][]).
- Resources **must** expose a `name` field that contains its resource name.
- Resources **may** provide the resource ID as a separate field (e.g.
`book_id`). This field **must** apply the
[`OUTPUT_ONLY`](./0203.md#output-only) field behavior classification.
- Resources **may** expose a separate, system-generated unique ID field
[(`uid`)](./0148.md#uid). This field **must** apply the
[`OUTPUT_ONLY`](./0203.md#output-only) field behavior classification.
- Resources **must not** expose tuples, self-links, or other forms of
resource identification.
- All ID fields **should** be strings.
**Note:** Resource names as described here are used within the scope of a
single API (or else in situations where the owning API is clear from the
context), and are only required to be unique within that scope. For this
reason, they are sometimes called _relative resource names_ to distinguish them
from _full resource names_ (discussed below).
[aip-210]: ./0210.md
[uri path schema]: https://datatracker.ietf.org/doc/html/rfc3986#appendix-A
### Collection identifiers
The collection identifier segments in a resource name **must** be the plural
form of the noun used for the resource. (For example, a collection of
`Publisher` resources is called `publishers` in the resource name.)
- Collection identifiers **must** be concise American English terms.
- Collection identifiers **must** be in `camelCase`.
- Collection identifiers **must** begin with a lower-cased letter and contain
only ASCII letters and numbers (`/[a-z][a-zA-Z0-9]*/`).
- Collection identifiers **must** be plural.
- In situations where there is no plural word ("info"), or where the singular
and plural terms are the same ("moose"), the non-pluralized (singular) form
is correct. Collection segments **must not** "coin" words by adding "s" in
such cases (e.g, avoid "infos").
- Within any given single resource name, collection identifiers **must** be
unique. (e.g. `people/xyz/people/abc` is invalid)
#### Nested collections
If a resource name contains multiple levels of a hierarchy, and a parent
collection's name is used as a prefix for the child resource's name, the child
collection's name **may** omit the prefix. For example, given a collection of
`UserEvent` resources that would normally be nested underneath `users`:
```
users/vhugo1802/userEvents/birthday-dinner-226
```
An API **should** use the less-redundant form:
```
users/vhugo1802/events/birthday-dinner-226
```
In this situation, the _message_ and _resource type_ are still called
`UserEvent`; only the collection and resource identifiers in the pattern(s) are
shortened. Since the _resource type_ is not shortened, the `singular` and
`plural` are similarly _not shortened_.
```
message UserEvent {
option (google.api.resource) = {
type: "example.googleapis.com/UserEvent"
// Only the collection & resource identfiers in the `pattern` are shortened.
pattern: "projects/{project}/users/{user}/events/{event}"
singular: "userEvent"
plural: "userEvents"
};
string name = 1;
}
```
**Note:** APIs wishing to do this **must** follow this format consistently
throughout all of its `pattern` entries defined and anywhere else the
resource is referenced in the API, or else not at all.
### Resource ID segments
A resource ID segment identifies the resource within its parent collection. In
the resource name `publishers/123/books/les-miserables`, `123` is the resource
ID for the publisher, and `les-miserables` is the resource ID for the book.
- If resource IDs are user-specified, the API **must** document allowed
formats. User-specified resource IDs **should** conform to [RFC-1034][];
which restricts to letters, numbers, and hyphen, with the first character
a letter, the last a letter or a number, and a 63 character maximum.
- Additionally, user-specified resource IDs **should** restrict letters to
lower-case (`^[a-z]([a-z0-9-]{0,61}[a-z0-9])?$`).
- Characters outside of ASCII **should not** be permitted; however, if
Unicode characters are necessary, APIs **must** follow guidance in
[AIP-210][].
- If resource IDs are not user-settable, the API **should** document the
basic format, and any upper boundaries (for example, "at most 63
characters").
- For more information, see the [create][] standard method.
[create]: ./0133.md#user-specified-ids
[rfc-1034]: https://tools.ietf.org/html/rfc1034
### Resource ID aliases
It is sometimes valuable to provide an alias for common lookup patterns for
resource IDs. For example, an API with `users` at the top of its resource
hierarchy may wish to provide `users/me` as a shortcut for retrieving
information for the authenticated user.
APIs **may** provide programmatic aliases for common lookup patterns. However,
all data returned from the API **must** use the canonical resource name.
### Full resource names
In most cases, resource names are used within a single API only, or else they
are used in contexts where the owning API is clear (for example,
`string pubsub_topic`).
However, sometimes it is necessary for services to refer to resources in an
arbitrary API. In this situation, the service **should** use the _full resource
name_, a schemeless URI with the owning [API's service name](./0009.md#api-service-name),
followed by the relative resource name:
```
//library.googleapis.com/publishers/123/books/les-miserables
//calendar.googleapis.com/users/vhugo1802
```
**Note:** The full resource name **should not** be used for cross-API
references where the owning API is clear; it is only used if a field refers to
resources in multiple APIs where ambiguity is possible.
### Resource URIs
The full resource name is a schemeless URI, but slightly distinct from the full
URIs we use to access a resource. The latter includes the protocol
(HTTPS), the API version, and the specific [service endpoint](./0009.md#api-service-endpoint)
to target:
```
https://library.googleapis.com/v1/publishers/123/books/les-miserables
https://calendar.googleapis.com/v3/users/vhugo1802
```
The version is not included in the full resource name because the full resource
name is expected to persist from version to version. Even though the API
surface may change between major versions, multiple major versions of the same
API are expected to use the same underlying data.
**Note:** The correlation between the full resource name and the service's
endpoint is by convention. In particular, one service is able to have multiple
endpoints (example use cases include regionalization, MTLS, and private access),
and the full resource name does not change between these.
### Fields representing resource names
When defining a resource, the first field **should** be the resource name,
which **must** be of type `string` and **must** be called `name` for the
resource name. The message **should** include a `google.api.resource`
annotation declaring the type (see [AIP-123][] for more on this).
```proto
// A representation of a book in the library.
message Book {
option (google.api.resource) = {
type: "library.googleapis.com/Book"
pattern: "publishers/{publisher}/books/{book}"
};
// The resource name of the book.
// Format: publishers/{publisher}/books/{book}
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// Other fields...
}
```
When defining a method that retrieves or acts on an already-existing resource
(such as `GetBook` or `ArchiveBook`), the first field of the request message
**should** be the resource name, which **must** be of type `string` and
**must** be called `name` for the resource name. The field **should** also be
annotated with the `google.api.resource_reference` annotation, referencing the
resource type ([AIP-123][]).
```proto
// Request message for ArchiveBook
message ArchiveBookRequest {
// The book to archive.
// Format: publishers/{publisher}/books/{book}
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
// Other fields...
}
```
**Note:** Fields **must not** be called `name` except for this purpose. For
other use cases, either use a different term or prepend an adjective (for
example: `display_name`).
[aip-123]: ./0123.md
### Fields representing a resource's parent
When defining a method that retrieves resources from a collection or adds a new
resource to a collection (such as `ListBooks` or `CreateBook`), the first field
of the request message **should** be of type `string` and **should** be called
`parent` for the resource name of the collection. The `parent` field **should**
also be annotated with the `google.api.resource_reference` annotation,
referencing the parent's resource type ([AIP-123][]).
```proto
// Request message for ListBooks.
message ListBooksRequest {
// The publisher to list books from.
// Format: publishers/{publisher_id}
string parent = 1 [(google.api.resource_reference) = {
type: "library.googleapis.com/Publisher"
}];
// Other fields (e.g. page_size, page_token, filter, etc.)...
}
```
If there is more than one possible parent type, the `parent` field **should**
be annotated with the `child_type` key on `google.api.resource_reference`
instead:
```proto
// Request message for ListBooks.
message ListBooksRequest {
// The parent to list books from.
// Format:
// - publishers/{publisher_id}
// - authors/{author_id}
string parent = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// Other fields (e.g. page_size, page_token, filter, etc.)...
}
```
**Note:** Fields **should not** be called `parent` except for this purpose. For
other use cases, use a synonymous term if possible.
### Fields representing another resource
When a field represents another resource, the field **should** be of type
`string` and accept the resource name of the other resource. The field name
**should** be equivalent to the corresponding message's name in snake case.
- Field names **may** include a leading adjective if appropriate (such as
`string dusty_book`).
- Field names **should not** use the `_name` suffix unless the field would be
ambiguous without it (e.g., `crypto_key_name`)
- Fields representing another resource **should** provide the
`google.api.resource_reference` annotation with the resource type being
referenced.
- If using the resource name is not possible and using the ID component alone is
strictly necessary, the field **should** use an `_id` suffix (e.g. `shelf_id`).
The field **should not** be of type `message` using the `message` that
implements the resource, __except__ for one of following conditions:
- The API is internal-only, has tight lifecycle relationships, and has a
permission model that enables inherited access to embedded resources.
- The embedding of the resource is done as part of the AIP-162 revisions
pattern.
Example of a resource reference:
```proto
// A representation of a book in a library.
message Book {
option (google.api.resource) = {
type: "library.googleapis.com/Book"
pattern: "publishers/{publisher}/books/{book}"
};
// Name of the book.
// Format is `publishers/{publisher}/books/{book}`
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// The shelf where the book currently sits.
// Format is `shelves/{shelf}`.
string shelf = 2 [(google.api.resource_reference) = {
type: "library.googleapis.com/Shelf"
}];
// Other fields...
}
```
## Further reading
- For evolving resource names over time, see
[AIP-180](./0180.md#changing-resource-names).
- For resource types, see [AIP-123][].
## Rationale
### Using names instead of IDs
For any large system, there are many kinds of resources. To use simple resource
IDs to identify a resource, we'd actually need use a resource-specific tuple to
reliably identify it, such as `(bucket, object)` or `(user, album, photo)`. This
creates several issues:
* Developers have to understand and remember such anonymous tuples.
* Passing tuples is generally harder than passing strings.
* Centralized infrastructures, such as logging and access control systems,
don't understand specialized tuples.
* Specialized tuples limit API design flexibility, such as providing
reusable API interfaces. For example,
[Long Running Operations][aip-151] can work with many other API interfaces
because they use flexible resource names.
[aip-151]: ./0151.md
### Standardizing on `name`
The concept of resource names is not a new one, and is formalized in Uniform
Resource Names (URN) in conjunction with Uniform Resource Identifiers (URI) and
Uniform Resource Locators (URL). Considering that the term "name" is so heavily
overloaded in general, usage outside of a very well-defined meaning would be
confusing for developers. So, the field name `name` is reserved in the context
of AIP-compliant APIs so as to eliminate any confusion with resource names, and
force other would be "name" fields to use a more specific field name.
### Disallow embedding of resources
Using a resource message directly as the type of a field within another resource
is problematic for a number of reasons, which are as follows:
* Complicates the resource lifecycle: If the dependency resource is
deleted, what happens to the embedded reference in the dependent resource?
Data retention and clean up operations will be significantly complicated.
* Bypasses permissions: If every resource has its own set of permissions, a user
with read permission on the dependent resource that doesn't have the same
permission on the dependency resource suddenly cannot see the full resource.
* Tightly couples resources in all aspects: Changing the requirements in the
schema, permissions, or otherwise for either resource impacts the other,
significantly increasing complexity of roll outs.
Referencing by name, as is recommended, eliminates all of this complexity by
preventing resource data duplication, and forcing the owning service to be
involved in the resolution of the reference (via Standard Methods), guaranteeing
isolation of logical concerns per-resource.
## History
### Disallowing UUIDs in user-specified IDs
As part of an effort to make APIs more declarative-friendly, APIs were required
to allow user-specified resource ID on creation. As part of this, a supporting
field `string uid` containing a UUID was also encouraged. Guidance was then
added to discourage the _user-specified resource ID_ from being a UUID or
even resembling one.
Ostensibly, the fact that `uid` contained a service-generated UUID motivated
this restriction on the user-specified resource ID, forcing the user-specified
resource ID to be a more "meaningful" value. However, we no longer saw value in
this requirement, hence its removal.
## Changelog
- **2025-03-10**: Drop guidance disallowing UUID in user-specified ID.
- **2024-10-15**: Add some rationale we found for use of `name` as a field and
instead of IDs as an identifier.
- **2024-06-14**: Clarify resource annotation shortening rules for nested
collections.
- **2023-09-19**: Prohibit duplicate collection identifiers.
- **2023-09-01**: Add a clause that allows embedding for revision resource
messages.
- **2023-08-10**: Explicitly disallow embedding resource messages in a resource.
- **2023-03-24**: Correction: full resource name contains the service name rather
than the service endpoint
- **2023-03-17**: Add `OUTPUT_ONLY` guidance for resource ID fields.
- **2020-10-06**: Added declarative-friendly guidance, and tightened character
set restrictions.
- **2020-10-05**: Clarified when full resource names are used.
- **2020-05-19**: Clarified that resource _IDs_ avoid capital characters, not
the entire resource _name_.
- **2020-04-27**: Tighten the restriction on valid characters.
- **2019-12-05**: Added guidance for resource annotations.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership. Also changed the final
example from a Pub/Sub example to the usual Book example.
- **2019-07-30**: Changed the nested collection brevity suggestion from "may"
to "should"
---
id: 123
state: approved
created: 2019-05-12
placement:
category: resource-design
order: 30
---
# Resource types
Most APIs expose _resources_ (their primary nouns) which users are able to
create, retrieve, and manipulate. APIs are allowed to name their resource types
reasonably freely (within the requirements of this AIP), and are only required
to ensure uniqueness within that API. This means that it is possible (and often
desirable) for different APIs to use the same type name. For example, a Memcache
and Redis API would both want to use `Instance` as a type name.
When mapping the relationships between APIs and their resources, however, it
becomes important to have a single, globally-unique type name. Additionally,
tools such as Kubernetes or GraphQL interact with APIs from multiple providers.
## Terminology
In the guidance below, we use the following terms:
- **Service Name:** This is the name defined in the [service configuration][].
This usually (but not necessarily) matches the hostname that users use to
call the service. Example: `pubsub.googleapis.com`. This is equivalent to an
[API Group][] in Kubernetes.
- **Type:** This is the name used for the type within the API e.g. the name of
the Protobuf `message`. This is equivalent to an [Object][] in Kubernetes.
## Guidance
APIs **must** define a resource type for each resource in the API, according to
the following pattern: `{Service Name}/{Type}`. The type name **must**:
- Match the containing API type's name.
- Start with an uppercase letter.
- Only contain alphanumeric characters.
- Be of the singular form of the noun.
- Use PascalCase (UpperCamelCase).
### Examples
Examples of resource types include:
- `pubsub.googleapis.com/Topic`
- `pubsub.googleapis.com/Subscription`
- `spanner.googleapis.com/Database`
- `spanner.googleapis.com/Instance`
- `networking.istio.io/Instance`
### Annotating resource types
APIs **should** annotate the resource types for each resource in the API using
the [`google.api.resource`][resource] annotation:
```proto
// A representation of a Pub/Sub topic.
message Topic {
option (google.api.resource) = {
type: "pubsub.googleapis.com/Topic"
pattern: "projects/{project}/topics/{topic}"
singular: "topic"
plural: "topics"
};
// Name and other fields...
}
```
- Patterns **must** correspond to the [resource name][aip-122].
- Pattern variables (the segments within braces) **must** use `snake_case`, and
**must not** use an `_id` suffix.
- Pattern variables **must** conform to the format `[a-z][_a-z0-9]*[a-z0-9]`.
- Pattern variables **must** be unique within any given pattern. (e.g.
`projects/{abc}/topics/{abc}` is invalid; this is usually a natural
corollary of collection identifiers being unique within a pattern.)
- Resources with [multiple patterns][multi-pattern-resources] **must**
preserve ordering: new patterns **must** be added at the end of the list, and
existing patterns **must not** be removed or re-ordered, as this breaks client
library backward compatibility.
- Singular **must** be the lower camel case of the type.
- Pattern variables **must** be the singular form of the resource type e.g.
a pattern variable representing a `Topic` resource ID is named `{topic}`.
- Plural **must** be the lower camel case plural of the singular.
- Pattern collection identifier segments **must** match the plural of the
resources, except in the case of [nested collections][].
#### Pattern uniqueness
When multiple patterns are defined within a resource, these patterns **must**
be mutually unique, where uniqueness is defined as being by-character identical
once all resource ID path segments have been removed, leaving all `/`
separators.
Therefore the following two patterns **must not** be defined within the same
resource:
- `user/{user}`
- `user/{user_part_1}~{user_part_2}`
## Rationale
### Type and message name alignment
In addition to simple schema-resource coherence and alignment, a number of
consumers benefit from the `{Type}` and `message` names matching. Consumers have
simpler lookups, client libraries get the same in addition to aligned user
experience where resource-oriented code has naming aligned with the generated
`message` code, generated reference documentation aligns resources with
`message` docs, etc.
### Singular and Plural
Well-defined singular and plurals of a resource enable clients to determine the
proper name to use in code and documentation.
lowerCamelCase can be translated into other common forms of a resource name
such as UpperCamelCase and snake_case.
<!-- prettier-ignore-start -->
[aip-122]: ./0122.md
[multi-pattern-resources]: https://google.aip.dev/client-libraries/4231#multi-pattern-resources
[API Group]: https://kubernetes.io/docs/concepts/overview/kubernetes-api/#api-groups
[nested collections]: ./0122.md#collection-identifiers
[Object]: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#types-kinds
[resource]: https://github.com/googleapis/googleapis/blob/master/google/api/resource.proto
[service configuration]: https://github.com/googleapis/googleapis/blob/master/google/api/service.proto
<!-- prettier-ignore-end -->
## Changelog
- **2025-01-09**: Strongly align resource type and message naming.
- **2024-08-07**: Added multi-pattern ordering compatibility requirements.
- **2023-09-19**: Prohibited duplicate pattern variables.
- **2023-05-06**: Added requirement of singular and plural.
- **2023-01-28**: Clarified guidance for the resource type name.
- **2022-10-28**: Added pattern variable format guidance.
- **2020-05-14**: Added pattern uniqueness.
- **2019-12-05**: Added guidance on patterns.
- **2019-07-17**: Fleshed out the annotation example somewhat.
---
id: 124
state: approved
created: 2020-03-20
placement:
category: resource-design
order: 40
---
# Resource association
APIs sometimes have resource hierarchies that can not be cleanly expressed in
the usual tree structure. For example, a resource may have a many-to-one
relationship with two other resource types instead of just one. Alternatively,
a resource may have a many-to-many relationship with another resource type.
## Guidance
A resource **must** have at most one canonical parent, and `List` requests
**must not** require two distinct "parents" to work.
### Multiple many-to-one associations
If a resource has a many-to-one relationship with multiple resource types, it
**must** choose at most one of them to be the canonical parent. The resource
**may** be associated with other resources through other fields on the
resource.
```proto
message Book {
// The resource name pattern for Book indicates that Publisher is the
// canonical parent.
option (google.api.resource) = {
type: "library.googleapis.com/Book"
pattern: "publishers/{publisher}/books/{book}"
};
// The resource name for the book.
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// The resource name for the book's author.
string author = 2 [(google.api.resource_reference) = {
type: "library.googleapis.com/Author"
}];
}
```
When listing resources with multiple associations in this way, the RPC **must**
treat the `string parent` field as required as discussed in [AIP-132][], and
**must not** add additional required arguments. The RPC **should** include a
`string filter` field that allows users to filter by other resource
associations as discussed in [AIP-160][].
**Note:** Resource reference fields **must** accept the same resource name
format that is used in the `name` field of the referenced resource.
### Many-to-many associations
Many-to-many associations are less common in APIs than they are in relational
databases, in part because they are more difficult to model and present over
network interfaces.
An API **may** contain many-to-many relationships, and **should** use a
repeated field containing a list of resource names, following the principles
described for repeated fields in [AIP-144][].
```proto
message Book {
option (google.api.resource) = {
type: "library.googleapis.com/Book"
pattern: "publishers/{publisher}/books/{book}"
};
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// The resource names for the book's authors.
repeated string authors = 2 [(google.api.resource_reference) = {
type: "library.googleapis.com/Author"
}];
}
```
**Note:** See [AIP-144][] for more information on repeated fields, including
how to handle common issues such as atomic changes.
If the use of a repeated field is too restrictive, or if more metadata is
required along with the association, an API **may** model a many-to-many
relationship using a sub-resource with two one-to-many associations.
```proto
message BookAuthor {
// The resource pattern for BookAuthor indicates that Book is the
// canonical parent.
option (google.api.resource) = {
type: "library.googleapis.com/BookAuthor"
pattern: "publishers/{publisher}/books/{book}/authors/{book_author}"
};
// The resource name for the book-author association.
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// The resource name for the author.
string author = 2 [(google.api.resource_reference) = {
type: "library.googleapis.com/Author"
}];
// Other fields...
}
```
**Note:** Using subresources to model an association between resources is only
recommended if additional metadata is required in the relationship, or if the
restrictions around the use of a repeated field preclude the use of that
approach.
[aip-132]: ./0132.md
[aip-144]: ./0144.md
[aip-160]: ./0160.md
## Changelog
- **2021-04-07**: Clarified that resource reference fields accept resource
names with the same format as the `name` field of the resource.
---
id: 126
state: approved
created: 2019-07-24
placement:
category: resource-design
order: 60
---
# Enumerations
It is common for a field to only accept or provide a discrete and limited set
of values. In these cases, it can be useful to use enumerations (generally
abbreviated "enums") in order to clearly communicate what the set of allowed
values are.
## Guidance
APIs **may** expose enum objects for sets of values that are expected to change
infrequently:
```proto
// A representation of a book.
message Book {
// Other fields...
// Possible formats in which the book may be published.
enum Format {
// Default value. This value is unused.
FORMAT_UNSPECIFIED = 0;
// The printed format, in hardback.
HARDBACK = 1;
// The printed format, in paperback.
PAPERBACK = 2;
// An electronic book format.
EBOOK = 3;
// An audio recording.
AUDIOBOOK = 4;
}
// The format of the book.
Format format = 99;
// Other fields...
}
```
- All enum values **must** use `UPPER_SNAKE_CASE`.
- The first value of the enum **should** be the name of the enum itself
followed by the suffix `_UNSPECIFIED`.
- An exception to this rule is if there is a clearly useful zero value. In
particular, if an enum needs to present an `UNKNOWN`, it is usually clearer
and more useful for it to be a zero value rather than having both. The
`UNKNOWN` value **may** be prefixed by the enum name as is typical for
avoiding enum value name collisions.
- Enums which will only be used in a single message **should** be nested within that message.
In this case, the enum **should** be declared immediately before it is used.
- The non-zero values of such a nested enum definition **should not** be prefixed by the name
of the enum itself. This generally requires users to write `MyState.MYSTATE_ACTIVE` in their
code, which is unnecessarily verbose.
- Enums which will be used by multiple messages **should** be defined at the package level and
**should** be defined at the bottom of the proto file (see [AIP-191][]).
- Some languages (including C++) hoist enum values into the parent namespace, which can result
in conflicts for enums with the same values in the same proto package. To avoid sharing values,
APIs **should** prefix package-level enum values with the name of the enum.
- Enums **should** document whether the enum is frozen or they expect to add
values in the future.
### When to use enums
Enums can be more accessible and readable than strings or booleans in many
cases, but they do add overhead when they change. Therefore, enums **should**
receive new values infrequently. While the definition of "infrequently" may
change based on individual use cases, a good rule of thumb is no more than once
a year. For enums that change frequently, the API **should** use a string and
document the format.
Additionally, enums **should not** be used when there is a competing,
widely-adopted standard representation (such as with [language codes][bcp-47]
or [media types][]).
**Note:** If an enumerated value needs to be shared across APIs, an enum
**may** be used, but the assignment between enum values and their corresponding
integers **must** match.
### Alternatives
For enumerated values where the set of allowed values changes frequently, APIs
**should** use a `string` field instead, and **must** document the allowed
values. String fields with enumerated values **should** use `kebab-case` for
their values.
For enumerated values where there is a competing, widely-adopted standard
representation (generally, but not necessarily, a string), that standard
representation **should** be used. This is true even if only a small subset of
values are permitted, because using enums in this situation often leads to
frustrating lookup tables when trying to use multiple APIs together.
Boolean fields **may** be used in situations where it is clear that no further
flexibility will be needed. The default value **must** be `false`.
**Note:** When using protocol buffers, it is impossible to distinguish between
`false` and unset. If this is a requirement, an enum **may** be a better design
choice (although `google.protobuf.BoolValue` is also available).
## Further reading
- For states, a special type of enum, see [AIP-216][].
[aip-191]: ./0191.md
[aip-216]: ./0216.md
[bcp-47]: https://en.wikipedia.org/wiki/IETF_language_tag
[media types]: https://en.wikipedia.org/wiki/Media_type
## Changelog
- **2025-10-03**: Added prefixing guidance for `UNKNOWN` value exception.
---
id: 127
state: approved
created: 2019-08-22
placement:
category: protobuf
order: 10
---
# HTTP and gRPC Transcoding
APIs that follow [resource-oriented design][aip-121] are defined using
[RPCs][rpc], but the resource-oriented design framework allows them to also be
presented as APIs that largely follow REST/JSON conventions. This is important
in order to help developers use their existing knowledge: over 80% of the
public APIs available follow most REST conventions, and developers are
accustomed to that pattern.
## Guidance
APIs **must** provide HTTP definitions for each RPC that they define, except
for bi-directional streaming RPCs, which can not be natively supported using
HTTP/1.1. When providing a bi-directional streaming method, an API **should**
also offer an alternative method that does not rely on bi-directional
streaming.
### HTTP method and path
When using protocol buffers, each RPC **must** define the HTTP method and path
using the `google.api.http` annotation:
```proto
rpc CreateBook(CreateBookRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books"
body: "book"
};
}
message CreateBookRequest {
// The publisher who will publish this book.
// When using HTTP/JSON, this field is automatically populated based
// on the URI, because of the `{parent=publishers/*}` syntax.
string parent = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The book to create.
// When using HTTP/JSON, this field is populated based on the HTTP body,
// because of the `body: "book"` syntax.
Book book = 2 [(google.api.field_behavior) = REQUIRED];
// The user-specified ID for the book.
// When using HTTP/JSON, this field is populated based on a query string
// argument, such as `?bookId=foo`. This is the fallback for fields that
// are not included in either the URI or the body.
// Note that clients use camelCase format to communicate the field names
// to the service.
string book_id = 3;
}
```
- The first key (`post` in this example) corresponds to the HTTP method. RPCs
**may** use `get`, `post`, `patch`, or `delete`.
- RPCs **must** use the prescribed HTTP verb for each standard method, as
discussed in [AIP-131][], [AIP-132][], [AIP-133][], [AIP-134][], and
[AIP-135][]
- RPCs **should** use the prescribed HTTP verb for custom methods, as
discussed in [AIP-136][].
- RPCs **should not** use `put` or `custom`.
- The corresponding value represents the URI.
- URIs **must** use the `{foo=bar/*}` syntax to represent a variable that
should be populated in the request proto. When extracting a [resource
name][aip-122], the variable **must** include the entire resource name, not
just the ID component.
- URIs **may** use nested fields for their variable names. (Additionally,
AIP-134 mandates this for `Update` requests.)
- URIs **must** use the `*` character to represent ID components, which
matches all URI-safe characters except for `/`. URIs **may** use `**` as
the final segment of a URI if matching `/` is required.
- The `body` key defines which single top-level field in the request will be
sent as the HTTP body. If the body is `*`, then this indicates that the
request object itself is the HTTP body. The request body is encoded as JSON
as defined by protocol buffers' canonical [JSON encoding][].
- RPCs **must not** define a `body` at all for RPCs that use the `GET` or
`DELETE` HTTP verbs.
- RPCs **must** use the prescribed `body` for Create ([AIP-133][]) and Update
([AIP-134][]) requests.
- RPCs **should** use the prescribed `body` for custom methods ([AIP-136][]).
- The `body` **must not** contain a nested field (or use the `.` character),
- The `body` **must not** be the same as a URI parameter.
- The `body` **must not** be a `repeated` field.
- Fields **should not** use the `json_name` annotation to alter the field
name in JSON, unless doing so for backwards-compatibility reasons.
**Note:** Bi-directional streaming RPCs should not include a `google.api.http`
annotation at all. If feasible, the service **should** provide non-streaming
equivalent RPCs.
### Multiple URI bindings
Occasionally, an RPC needs to correspond to more than one URI:
```proto
rpc CreateBook(CreateBookRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books"
body: "book"
additional_bindings: {
post: "/v1/{parent=authors/*}/books"
body: "book"
}
additional_bindings: {
post: "/v1/books"
body: "book"
}
};
}
```
- RPCs **may** define any number of additional bindings. The structure is
identical to the `google.api.http` annotation (in fact, it is a recursive
reference).
- RPCs **must not** define an additional binding within an additional binding.
- The `body` clause **must** be identical in the top-level annotation and each
additional binding.
## Changelog
- **2022-08-18**: Added the comment that query string parameter names are
in camelCase.
- **2021-01-06**: Added clarification around `body` and nested fields.
- **2019-09-23**: Added a statement about request body encoding, and guidance
discouraging `json_name`.
<!-- prettier-ignore-start -->
[aip-121]: ./0121.md
[aip-122]: ./0122.md
[aip-131]: ./0131.md
[aip-132]: ./0132.md
[aip-133]: ./0133.md
[aip-134]: ./0134.md
[aip-135]: ./0135.md
[aip-136]: ./0136.md
[json encoding]: https://developers.google.com/protocol-buffers/docs/proto3#json
[rpc]: https://en.wikipedia.org/wiki/Remote_procedure_call
<!-- prettier-ignore-end -->
---
id: 128
state: approved
created: 2020-10-06
placement:
category: resource-design
order: 65
---
# Declarative-friendly interfaces
Many services need to interact with common DevOps tools, particularly those
that create and manage network-addressable resources (such as virtual machines,
load balancers, database instances, and so on). These tools revolve around the
principle of "configuration as code": the user specifies the complete intended
landscape, and tooling is responsible for making whatever changes are necessary
to achieve the user's specification.
These tools are **declarative**: rather than specifying specific _actions_ to
take, they specify the desired _outcome_, with the actions being derived based
on the differences between the current landscape and the intended one.
Furthermore, there are numerous popular DevOps tools, with more being
introduced each year. Integrating hundreds of resource types with multiple
tools requires uniformity, so that integration can be automated.
## Guidance
### Resources
Resources that are declarative-friendly **must** use only strongly-consistent
standard methods for managing resource lifecycle, which allows tools to support
these resources generically, as well as conforming to other
declarative-friendly guidance (see [further reading](#further-reading)).
Declarative-friendly resources **should** designate that they follow the
declarative-friendly style:
```proto
message Book {
option (google.api.resource) = {
type: "library.googleapis.com/Book"
pattern: "publishers/{publisher}/books/{book}"
style: DECLARATIVE_FRIENDLY
};
// Name and other fields...
}
```
### Reconciliation
If a resource takes time (more than a few seconds) for updates to be realized,
the resource **should** include a `bool reconciling` field to disclose that
changes are in flight. This field **must** be output only.
A resource **must** set the `reconciling` field to `true` if the current state
of the resource does not match the user's intended state, and the system is
working to reconcile them. This is regardless of whether the root cause of
going into reconciliation was user or system action.
**Note:** Services responding to a `GET` request **must** return the resource's
current state (not the intended state).
## Further reading
A significant amount of guidance is more strict for declarative-friendly
interfaces, due to the focus on automation on top of these resources. This list
is a comprehensive reference to declarative-friendly guidance in other AIPs:
- Resources **should not** employ custom methods: see AIP-136.
- Resources **must** use the `Update` method for repeated fields: see AIP-144.
- Resources **must** include certain standard fields: see AIP-148.
- Resources **must** have an `etag` field: see AIP-154.
- Resources **should** provide change validation: see AIP-163.
- Resources **should not** implement soft-delete. If the id cannot be re-used,
the resource **must** implement soft-delete and the undelete RPC: see AIP-164
### Annotations
See [AIP-148 annotations][].
## Changelog
- **2023-07-13**: Move `annotations` from AIP-148.
- **2023-06-17**: Definition of plane was removed and incorporated into AIP-111.
- **2023-05-11**: Removed must on resource_id, which was upstreamed to a general
must.
[AIP-148 annotations]: ./0148.md#annotations
---
id: 129
state: approved
created: 2023-09-20
updated: 2023-10-31
placement:
category: resource-design
order: 65
---
# Server-Modified Values and Defaults
Services often provide default values for resource fields, and occasionally
normalize the user input before returning it in the response. The guidance
herein describes how services document such behavior for the benefit of
consumers.
## Guidance
### Single Owner Fields
Fields **must** have a single owner, whether that is the client or the server.
Server owned fields **must** be indicated with the `OUTPUT_ONLY` field_behavior.
All other types of fields **must** be considered to be owned by the client. The
server **must** respect the value (or lack thereof) for all client owned fields
and not modify them.
### Effective Values
There are instances where a service will allocate, generate, or calculate a
value if the client chooses not to specify one. For example: a client creates a
virtual machine without specifying a static IP address for the virtual machine
to be available on. Such a scenario is opting into dynamic IP address
allocation.
Some examples of these types of fields are ones that are:
* generated (UUID)
* allocated (dynamic IP address)
* assigned (most recent software package version)
An attribute with an effective value ***must*** be expressed as two fields in
the API:
* a mutable field that can be optionally set by the user and **must not** be
modified by the service
* an `OUTPUT_ONLY` field that records the effective value decided on by the
service
Example:
```proto
message VirtualMachine {
…
string ip_address = 4;
string effective_ip_address = 5 [
(google.api.field_behavior) = OUTPUT_ONLY
];
}
```
#### Naming
Effective values **must** be named by prefixing `effective_` to the mutable
field's name.
### User-Specified Fields
For user-specified fields, the value in response from the service **must** be
the same as provided by the create or update request. For string fields this
means returning the value unchanged, with one exception:
* When a string field has a data type annotation, a normalized string that
represents the given value **may** be returned.
### Normalizations
A field that is normalized by the service **must** be annotated with the
`google.api.field_info` extension. See ([AIP-202][]) for guidance on using
this extension The allowed set of normalizations includes the following formats:
* uuid
* ipv4
* ipv6
* email
Normalizations on fields **must** be described using the `google.api.field_info`
annotation.
## Rationale
Server-modified and default values often make it harder to implement
[declarative clients][]. These clients are often unable to
tell when their desired state matches the current state for these fields, as the
rules by which a server may modify and return values are complex, not public,
and not repeatable.
### Rationale for Single Owner Fields
When fields do not have a single owner they can cause issues for
[declarative clients][]. These clients may attempt to set
values for fields that are overwritten by server set values, leading to the
client entering an infinite loop to correct the change.
### Rationale for Naming
Consistent naming is important for identifying standard behavior across APIs
and fields. Programmatic association between user-specified and effective values
depends on consistent naming.
### Rationale for Normalizations
Normalizations are important to allow services to store and return values in a
standard way while communicating to clients what changes are semantically
identical. Normalizing a value on the service side allows the service to accept
a wider range of semantically identical inputs without needing to maintain every
value as a raw string. Surfacing the normalization that is being applied to
clients allows for client side comparison of sent and retrieved values to check
for differences.
For example, in a resource that accepts an email address on a particular field
a client may specify a given email address in a variety of ways. For the email
`ada@example.com` a client may choose to specify `ADA@example.com`,
`aDa@example.com`, or `AdA@example.com`. These are semantically identical and
*should* all be accepted by the service. The service then may choose to
normalize the email address for storage and retrieval through downcasing or
canonicalization. Importantly, the information surfaced to clients on the
normalization of a field will not describe the normalization algorithm itself,
but instead the comparison method used to accurately compute if two values
should be considered equal.
### Rationale for Field Value Handling
For fields not using an allowed normalization,
[Declarative clients][] will not be able to identify which changes are
semantically meaningful. When a [Declarative client][Declarative clients]
sends a particular value it will ensure that the value is being returned by the
service to validate it was set correctly.
## Changelog
- **2023-10-31:** Update to approved.
[Declarative clients]: ./0009.md#declarative-clients
[aip-202]: ./0202.md
<!-- prettier-ignore-start -->
[aip-180]: ./0180.md
<!-- prettier-ignore-end -->
---
id: 130
state: approved
created: 2023-03-13
updated: 2023-09-05
placement:
category: operations
order: 9
---
# Methods
An API is composed of one or more methods, which represent a specific operation
that a service can perform on behalf of the consumer.
## Guidance
### Categories of Methods
The following enumerates multiple categories of methods that exist, often
grouped up under some object (e.g. collection or resource) that the method
operates upon.
| Category Name | Related AIPs | [Declarative client][] integration | CLI / UI integration | SDK integration |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------- | ---------------------------------- | -------------------- | --------------- |
| *Standard Methods* | | | | |
| **Standard collection methods**: operate on a collection of resources (List or Create). | [AIP-121][], [AIP-132][], [AIP-133][] | automatable | automatable | automatable |
| **Standard resource methods**: fetch or mutate a single resource (Get, Update, Delete). | [AIP-121][], [AIP-131][], [AIP-134][], [AIP-135][] | automatable | automatable | automatable |
| **Batch resource methods**: fetch or mutate multiple resources in a collection by name. | [AIP-231][], [AIP-233][], [AIP-234][], [AIP-235][] | may be used to optimize queries | automatable | automatable |
| **Aggregated list methods**: fetch or mutate multiple resources of the same type across multiple collections. | [AIP-159][] | not useful nor automatable | automatable | automatable |
| *Custom Fetch Methods* | | | | |
| **Custom collection fetch methods**: fetch information across a collection that cannot be expressed via a standard method. | [AIP-136][] | handwritten | automatable | automatable |
| **Custom resource fetch methods**: fetch information for a single resource that cannot be expressed via a standard method. | [AIP-136][] | handwritten | automatable | automatable |
| *Custom Mutation Methods* | | | | |
| **Backing up a resource**: storing a copy of a resource at a particular point in time. | [AIP-162][] | unused or handwritten | automatable | automatable |
| **Restoring a resource**: setting a resource to a version from a particular point in time. | [AIP-162][] | unused or handwritten | automatable | automatable |
| **Renaming a resource**: modify the resource's name or id while preserving configuration and data. | [AIP-136][] | unused or handwritten | automatable | automatable |
| **Custom collection mutation methods**: perform an imperative operation referencing a collection that may mutate one or more resources within that collection in fashion that cannot be easily achieved by standard methods (e.g. state transitions). | [AIP-136][] | unused or handwritten | automatable | automatable |
| **Custom resource mutation methods**: perform an imperative operation on a resource that may mutate it in a way a standard method cannot (e.g. state transitions). | [AIP-136][] | unused or handwritten | automatable | automatable |
| *Misc Custom Methods* | | | |
| **Stateless Methods**: a method that has no permanent effect on any data within the API (e.g. translating text) | [AIP-136][] | unused or handwritten | automatable | automatable |
| *None of the above* | | | | |
| **Streaming methods**: methods that communicate via client, server, or bi-directional streams. | | handwritten | handwritten | automatable |
### Choosing a method category
While designing a method, API authors should choose from the defined categories
in the following order:
1. Standard methods (on collections and resources)
1. Standard batch or aggregate methods
1. Custom methods (on collections, resources, or stateless)
1. Streaming methods
## Rationale
Resource-oriented standard and custom methods are recommended first, as they can
be expressed in the widest variety of clients (Declarative clients, CLIs, UIs, and so on), and
offer the most uniform experience that allows users to apply their knowledge of
one API to another.
If a standard method is unsuitable, then custom methods (that are mounted to a
resource or collection) offer a lesser, but still valuable level of consistency,
helping the user reason about the scope of the action and the object whose
configuration is read to inform that action. Although mutative custom methods
are not uniform enough to have a automated integration with exclusively
resource-oriented clients such as [Declarative client][]s, they are still a pattern that
can easily recognized by CLIs, UIs, and SDKs.
If one cannot express their APIs in a resource-oriented fashion at all, then the
operation falls in a category where the lack of uniformity makes it difficult
for any client aside from SDKs to model the operation. This category is
preferred last due to the fact that a user cannot rely on their knowledge of
similar APIs, as well as the issue that integration with many clients will
likely have to be hand-written.
[AIP-121]: ./0121.md
[AIP-131]: ./0131.md
[AIP-132]: ./0132.md
[AIP-133]: ./0133.md
[AIP-134]: ./0134.md
[AIP-135]: ./0135.md
[AIP-136]: ./0136.md
[AIP-159]: ./0159.md
[AIP-162]: ./0162.md
[AIP-231]: ./0231.md
[AIP-233]: ./0233.md
[AIP-234]: ./0234.md
[AIP-235]: ./0235.md
[Declarative client]: ./0009.md#declarative-clients
## Changelog
- **2023-09-05**: Update to approved.---
id: 131
state: approved
created: 2019-01-22
updated: 2022-06-02
placement:
category: operations
order: 10
---
# Standard methods: Get
In REST APIs, it is customary to make a `GET` request to a resource's URI (for
example, `/v1/publishers/{publisher}/books/{book}`) in order to retrieve that
resource.
Resource-oriented design ([AIP-121][]) honors this pattern through the `Get`
method. These RPCs accept the URI representing that resource and return the
resource.
## Guidance
APIs **must** provide a get method for resources. The purpose of the get method
is to return data from a single resource.
Get methods are specified using the following pattern:
```proto
rpc GetBook(GetBookRequest) returns (Book) {
option (google.api.http) = {
get: "/v1/{name=publishers/*/books/*}"
};
option (google.api.method_signature) = "name";
}
```
- The RPC's name **must** begin with the word `Get`. The remainder of the RPC
name **should** be the singular form of the resource's message name.
- The request message **must** match the RPC name, with a `Request` suffix.
- The response message **must** be the resource itself. (There is no
`GetBookResponse`.)
- The response **should** usually include the fully-populated resource unless
there is a reason to return a partial response (see [AIP-157][]).
- The HTTP verb **must** be `GET`.
- The URI **should** contain a single variable field corresponding to the
resource name.
- This field **should** be called `name`.
- The URI **should** have a variable corresponding to this field.
- The `name` field **should** be the only variable in the URI path. All
remaining parameters **should** map to URI query parameters.
- There **must not** be a `body` key in the `google.api.http` annotation.
- There **should** be exactly one `google.api.method_signature` annotation,
with a value of `"name"`.
### Request message
Get methods implement a common request message pattern:
```proto
message GetBookRequest {
// The name of the book to retrieve.
// Format: publishers/{publisher}/books/{book}
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
}
```
- A resource name field **must** be included. It **should** be called `name`.
- The field **should** be [annotated as required][aip-203].
- The field **must** identify the [resource type][aip-123] that it
references.
- The comment for the `name` field **should** document the resource pattern.
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in
another AIP.
**Note:** The `name` field in the request object corresponds to the `name`
variable in the `google.api.http` annotation on the RPC. This causes the `name`
field in the request to be populated based on the value in the URL when the
REST/JSON interface is used.
### Errors
See [errors][], in particular [when to use PERMISSION_DENIED and
NOT_FOUND errors][permission-denied].
[aip-121]: ./0121.md
[aip-123]: ./0123.md
[aip-157]: ./0157.md
[aip-203]: ./0203.md
[errors]: ./0193.md
[permission-denied]: ./0193.md#permission-denied
## Changelog
- **2023-03-17**: Align with AIP-122 and make Get a must.
- **2022-11-04**: Aggregated error guidance to AIP-193.
- **2022-06-02**: Changed suffix descriptions to eliminate superfluous "-".
- **2020-06-08**: Added guidance on returning the full resource.
- **2019-10-18**: Added guidance on annotations.
- **2019-08-12**: Added guidance for error cases.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
- **2019-05-29**: Added an explicit prohibition on arbitrary fields in standard
methods.
---
id: 132
state: approved
created: 2019-01-21
updated: 2022-06-02
placement:
category: operations
order: 20
---
# Standard methods: List
In many APIs, it is customary to make a `GET` request to a collection's URI
(for example, `/v1/publishers/1/books`) in order to retrieve a list of
resources, each of which lives within that collection.
Resource-oriented design ([AIP-121][]) honors this pattern through the `List`
method. These RPCs accept the parent collection (and potentially some other
parameters), and return a list of responses matching that input.
## Guidance
APIs **must** provide a `List` method for resources unless the resource is a
[singleton][]. The purpose of the `List` method is to return data from a finite
collection (generally singular unless the operation supports [reading across
collections][]).
List methods are specified using the following pattern:
```proto
rpc ListBooks(ListBooksRequest) returns (ListBooksResponse) {
option (google.api.http) = {
get: "/v1/{parent=publishers/*}/books"
};
option (google.api.method_signature) = "parent";
}
```
- The RPC's name **must** begin with the word `List`. The remainder of the RPC
name **should** be the plural form of the resource being listed.
- The request and response messages **must** match the RPC name, with
`Request` and `Response` suffixes.
- The HTTP verb **must** be `GET`.
- The collection whose resources are being listed **should** map to the URI
path.
- The collection's parent resource **should** be called `parent`, and
**should** be the only variable in the URI path. All remaining parameters
**should** map to URI query parameters.
- The collection identifier (`books` in the above example) **must** be a
literal string.
- The `body` key in the `google.api.http` annotation **must** be omitted.
- If the resource being listed is not a top-level resource, there **should**
be exactly one `google.api.method_signature` annotation, with a value of
`"parent"`. If the resource being listed is a top-level resource, there
**should** be either no `google.api.method_signature` annotation, or exactly
one `google.api.method_signature` annotation, with a value of `""`.
### Request message
List methods implement a common request message pattern:
```proto
message ListBooksRequest {
// The parent, which owns this collection of books.
// Format: publishers/{publisher}
string parent = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The maximum number of books to return. The service may return fewer than
// this value.
// If unspecified, at most 50 books will be returned.
// The maximum value is 1000; values above 1000 will be coerced to 1000.
int32 page_size = 2;
// A page token, received from a previous `ListBooks` call.
// Provide this to retrieve the subsequent page.
//
// When paginating, all other parameters provided to `ListBooks` must match
// the call that provided the page token.
string page_token = 3;
}
```
- A `parent` field **must** be included unless the resource being listed is a
top-level resource. It **should** be called `parent`.
- The field **should** be [annotated as required][aip-203].
- The field **must** identify the [resource type][aip-123] of the resource
being listed.
- The `page_size` and `page_token` fields, which support pagination, **must**
be specified on all list request messages. For more information, see
[AIP-158][].
- The comment above the `page_size` field **should** document the maximum
allowed value, as well as the default value if the field is omitted (or set
to `0`). If preferred, the API **may** state that the server will use a
sensible default. This default **may** change over time.
- If a user provides a value greater than the maximum allowed value, the API
**should** coerce the value to the maximum allowed.
- If a user provides a negative or other invalid value, the API **must** send
an `INVALID_ARGUMENT` error.
- The `page_token` field **must** be included on all list request messages.
- The request message **may** include fields for common design patterns
relevant to list methods, such as `string filter` and `string order_by`.
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
**Note:** List methods **should** return the same results for any user that has
permission to make a successful List request on the collection. Search methods
are more relaxed on this.
### Response message
List methods implement a common response message pattern:
```proto
message ListBooksResponse {
// The books from the specified publisher.
repeated Book books = 1;
// A token, which can be sent as `page_token` to retrieve the next page.
// If this field is omitted, there are no subsequent pages.
string next_page_token = 2;
}
```
- The response message **must** include one repeated field corresponding to the
resources being returned, and **should not** include any other repeated
fields unless described in another AIP (for example, AIP-217).
- The response **should** usually include fully-populated resources unless
there is a reason to return a partial response (see AIP-157).
- The `next_page_token` field, which supports pagination, **must** be included
on all list response messages. It **must** be set if there are subsequent
pages, and **must not** be set if the response represents the final page. For
more information, see AIP-158.
- The message **may** include a `int32 total_size` (or `int64 total_size`)
field with the number of items in the collection.
- The value **may** be an estimate (the field **should** clearly document
this if so).
- If filtering is used, the `total_size` field **should** reflect the size of
the collection _after_ the filter is applied.
### Ordering
`List` methods **may** allow clients to specify sorting order; if they do, the
request message **should** contain a `string order_by` field.
- Values **should** be a comma separated list of fields. For example:
`"foo,bar"`.
- The default sorting order is ascending. To specify descending order for a
field, users append a `" desc"` suffix; for example: `"foo desc, bar"`.
- Redundant space characters in the syntax are insignificant.
`"foo, bar desc"`, `" foo , bar desc "`, and `"foo,bar desc"` are all
equivalent.
- Subfields are specified with a `.` character, such as `foo.bar` or
`address.street`.
- The resulting list order **should** be based on the field type's natural
comparator e.g. numerics ordered numerically, strings ordered
lexicographically, etc. However, APIs **may** choose to use a different
ordering; if so, it **must** be documented in the `order_by` definition.
- Furthermore, [well-known][] types, like `Timestamp` and `Duration` are
compared as their representative type; `Timestamp` is compared as time e.g.
before or after, `Duration` is compared as a quantity e.g. more or less.
<!-- TODO(#220): Add a reference to AIP-161 once it is written. -->
**Note:** Only include ordering if there is an established need to do so. It is
always possible to add ordering later, but removing it is a breaking change.
### Filtering
List methods **may** allow clients to specify filters; if they do, the request
message **should** contain a `string filter` field. Filtering is described in
more detail in AIP-160.
**Note:** Only include filtering if there is an established need to do so. It
is always possible to add filtering later, but removing it is a breaking
change.
### Soft-deleted resources
Some APIs need to "[soft delete][]" resources, marking them as deleted or
pending deletion (and optionally purging them later).
APIs that do this **should not** include deleted resources by default in list
requests. APIs with soft deletion of a resource **should** include a
`bool show_deleted` field in the list request that, if set, will cause
soft-deleted resources to be included.
### Errors
See [errors][], in particular [when to use PERMISSION_DENIED and
NOT_FOUND errors][permission-denied].
## Further reading
- For details on pagination, see [AIP-158](./0158.md).
- For listing across multiple parent collections, see [AIP-159](./0159.md).
[aip-121]: ./0121.md
[aip-123]: ./0123.md
[aip-157]: ./0157.md
[aip-158]: ./0158.md
[aip-203]: ./0203.md
[errors]: ./0193.md
[permission-denied]: ./0193.md#permission-denied
[reading across collections]: ./0159.md
[singleton]: ./0156.md
[soft delete]: ./0135.md#soft-delete
[well-known]: https://protobuf.dev/reference/protobuf/google.protobuf
## Changelog
- **2025-02-25**: Require documentation for ordering not matching field type
with clarification on ordering of well-known types.
- **2023-03-22**: Fix guidance wording to mention AIP-159.
- **2023-03-17**: Align with AIP-122 and make Get a must.
- **2022-11-04**: Aggregated error guidance to AIP-193.
- **2022-06-02**: Changed suffix descriptions to eliminate superfluous "-".
- **2020-09-02**: Add link to the filtering AIP.
- **2020-08-14**: Added error guidance for permission denied cases.
- **2020-06-08**: Added guidance on returning the full resource.
- **2020-05-19**: Removed requirement to document ordering behavior.
- **2020-04-15**: Added guidance on List permissions.
- **2019-10-18**: Added guidance on annotations.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
- **2019-07-30**: Added guidance about documenting the ordering behavior.
- **2019-05-29**: Added an explicit prohibition on arbitrary fields in standard
methods.
---
id: 133
state: approved
created: 2019-01-23
updated: 2022-06-02
placement:
category: operations
order: 30
---
# Standard methods: Create
In REST APIs, it is customary to make a `POST` request to a collection's URI
(for example, `/v1/publishers/{publisher}/books`) in order to create a new
resource within that collection.
Resource-oriented design ([AIP-121][]) honors this pattern through the `Create`
method. These RPCs accept the parent collection and the resource to create (and
potentially some other parameters), and return the created resource.
## Guidance
APIs **should** generally provide a create method for resources unless it is
not valuable for users to do so. The purpose of the create method is to create
a new resource in an already-existing collection.
Create methods are specified using the following pattern:
```proto
rpc CreateBook(CreateBookRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books"
body: "book"
};
option (google.api.method_signature) = "parent,book";
}
```
- The RPC's name **must** begin with the word `Create`. The remainder of the
RPC name **should** be the singular form of the resource being created.
- The request message **must** match the RPC name, with a `Request` suffix.
- The response message **must** be the resource itself. There is no
`CreateBookResponse`.
- The response **should** include the fully-populated resource, and **must**
include any fields that were provided unless they are input only (see
[AIP-203][]) or there is a reason to return a partial response (see
[AIP-157][]).
- If the create RPC is [long-running](#long-running-create), the response
message **must** be a `google.longrunning.Operation` which resolves to the
resource itself.
- The HTTP verb **must** be `POST`.
- The collection where the resource is being added **should** map to the URI
path.
- The collection's parent resource **should** be called `parent`, and
**should** be the only variable in the URI path.
- The collection identifier (`books` in the above example) **must** be
a literal string.
- There **must** be a `body` key in the `google.api.http` annotation, and it
**must** map to the resource field in the request message.
- All remaining fields **should** map to URI query parameters.
- There **should** be exactly one `google.api.method_signature` annotation,
with a value of `"parent,{resource},{resource}_id"`, or "`"parent,{resource}"`
if the resource ID is not required.
- If the API is operating on the [management plane][], the operation should have
[strong consistency][]: the completion of a create operation **must** mean
that all user-settable values and the existence of the resource have reached a
steady-state and reading resource state returns a consistent response.
### Request message
Create methods implement a common request message pattern:
```proto
message CreateBookRequest {
// The parent resource where this book will be created.
// Format: publishers/{publisher}
string parent = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The ID to use for the book, which will become the final component of
// the book's resource name.
//
// This value should be 4-63 characters, and valid characters
// are /[a-z][0-9]-/.
string book_id = 2 [(google.api.field_behavior) = REQUIRED];
// The book to create.
Book book = 3 [(google.api.field_behavior) = REQUIRED];
}
```
- A `parent` field **must** be included unless the resource being created is a
top-level resource. It **should** be called `parent`.
- The field **should** be [annotated as required][aip-203].
- The field **must** identify the [resource type][aip-123] of the resource
being created.
- A `{resource}_id` field **must** be included for management plane resources,
and **should** be included for data plane resources.
- The resource field **must** be included and **must** map to the POST body.
- The request message **must not** contain any other required fields and
**should not** contain other optional fields except those described in this
or another AIP.
### Long-running create
Some resources take longer to create a resource than is reasonable for a
regular API request. In this situation, the API **should** use a long-running
operation (AIP-151) instead:
```proto
rpc CreateBook(CreateBookRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books"
body: "book"
};
option (google.longrunning.operation_info) = {
response_type: "Book"
metadata_type: "OperationMetadata"
};
}
```
- The response type **must** be set to the resource (what the return type would
be if the RPC was not long-running).
- Both the `response_type` and `metadata_type` fields **must** be specified.
**Important:** Declarative-friendly resources (AIP-128) **should** use
long-running operations. The service **may** return an LRO that is already set
to done if the request is effectively immediate.
### User-specified IDs
An API **must** allow a user to specify the ID component of a resource (the last
segment of the resource name) on creation if the API is operating on the
[management plane][].
On the [data plane][], an API **should** allow a user to specify the ID.
Exceptional cases should have the following behavior:
- The data plane resource allows identical records without a need to
disambiguate between the two (e.g. rows in a table with no primary key).
- The data plane resource will not be exposed in [Declarative clients][].
An API
**may** allow the `{resource}_id` field have the [field_behavior][] `OPTIONAL`,
and generate a system-generated ID if one is not specified.
For example:
```
// Using user-specified IDs.
publishers/lacroix/books/les-miserables
// Using system-generated IDs.
publishers/012345678-abcd-cdef/books/12341234-5678-abcd
```
- The `{resource}_id` field **must** exist on the request message, not the
resource itself.
- The field **may** be required or optional. If it is required, it **should**
include the corresponding annotation.
- The `name` field on the resource **must** be ignored.
- There **should** be exactly one `google.api.method_signature` annotation on
the RPC, with a value of `"parent,{resource},{resource}_id"` if the resource
being created is not a top-level resource, or with a value of
`"{resource},{resource}_id"` if the resource being created is a top-level
resource.
- The documentation **should** explain what the acceptable format is, and the
format **should** follow the guidance for resource name formatting in
[AIP-122][].
- If a user tries to create a resource with an ID that would result in a
duplicate resource name, the service **must** error with `ALREADY_EXISTS`.
- However, if the user making the call does not have permission to see the
duplicate resource, the service **must** error with `PERMISSION_DENIED`
instead.
**Note:** For REST APIs, the user-specified ID field, `{resource}_id`,
is provided as a query parameters on the request URI.
### Errors
See [errors][], in particular [when to use PERMISSION_DENIED and
NOT_FOUND errors][permission-denied].
## Further reading
- For ensuring idempotency in `Create` methods, see [AIP-155][].
- For naming resources involving Unicode, see [AIP-210][].
## Rationale
### Requiring user-specified ids
[Declarative clients][] use the resource ID as a way to identify a resource for
applying updates and for conflict resolution. The lack of a user-specified ID
means a client is unable to find the resource unless they store the identifier
locally, and can result in re-creating the resource. This in turn has a
downstream effect on all resources that reference it, forcing them to update to
the ID of the newly-created resource.
Having a user-specified ID also means the client can precalculate the resource
name and use it in references from other resources.
[aip-121]: ./0121.md
[aip-122]: ./0122.md
[aip-123]: ./0123.md
[aip-155]: ./0155.md
[aip-157]: ./0157.md
[aip-203]: ./0203.md
[aip-210]: ./0210.md
[data plane]: ./0111.md#data-plane
[management plane]: ./0111.md#management-plane
[errors]: ./0193.md
[field_behavior]: ./0203.md
[Declarative clients]: ./0009.md#declarative-clients
[permission-denied]: ./0193.md#permission-denied
[strong consistency]: ./0121.md#strong-consistency
## Changelog
- **2025-10-03**: Allow use of partial responses via AIP-157.
- **2023-10-20**: Clarify that {resource}_id is only required for management plane resources.
- **2023-08-24**: Adding consistency requirement.
- **2023-05-11**: Changing guidance around resource_id to a must.
- **2022-11-04**: Referencing aggregated error guidance in AIP-193, similar to
other CRUDL AIPs.
- **2022-06-02**: Changed suffix descriptions to eliminate superfluous "-".
- **2020-10-06**: Added declarative-friendly guidance.
- **2020-08-14**: Updated error guidance to use permission denied over
forbidden.
- **2020-06-08**: Added guidance on returning the full resource.
- **2019-11-22**: Added clarification on what error to use if a duplicate name
is sent.
- **2019-10-18**: Added guidance on annotations.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
- **2019-06-10**: Added guidance for long-running create.
- **2019-05-29**: Added an explicit prohibition on arbitrary fields in standard
methods.
---
id: 134
state: approved
created: 2019-01-24
updated: 2022-06-02
placement:
category: operations
order: 40
---
# Standard methods: Update
In REST APIs, it is customary to make a `PATCH` or `PUT` request to a
resource's URI (for example, `/v1/publishers/{publisher}/books/{book}`) in
order to update that resource.
Resource-oriented design ([AIP-121][]) honors this pattern through the `Update`
method (which mirrors the REST `PATCH` behavior). These RPCs accept the URI
representing that resource and return the resource.
## Guidance
APIs **should** generally provide an update method for resources unless it is
not valuable for users to do so. The purpose of the update method is to make
changes to the resources without causing side effects.
Update methods are specified using the following pattern:
```proto
rpc UpdateBook(UpdateBookRequest) returns (Book) {
option (google.api.http) = {
patch: "/v1/{book.name=publishers/*/books/*}"
body: "book"
};
option (google.api.method_signature) = "book,update_mask";
}
```
- The RPC's name **must** begin with the word `Update`. The remainder of the
RPC name **should** be the singular form of the resource's message name.
- The request message **must** match the RPC name, with a `Request` suffix.
- The response message **must** be the resource itself. (There is no
`UpdateBookResponse`.)
- The response **should** include the fully-populated resource, and **must**
include any fields that were sent and included in the update mask unless
they are input only (see AIP-203) or there is a reason to return a partial
response (see [AIP-157][]).
- If the update RPC is [long-running](#long-running-update), the response
message **must** be a `google.longrunning.Operation` which resolves to the
resource itself.
- The method **should** support partial resource update, and the HTTP verb
**should** be `PATCH`.
- If the method will only ever support full resource replacement, then the
HTTP verb **may** be `PUT`. However, this is strongly discouraged because
it becomes a backwards-incompatible change to add fields to the resource.
- The resource's `name` field **should** map to the URI path.
- The `{resource}.name` field **should** be the only variable in the URI
path.
- There **must** be a `body` key in the `google.api.http` annotation, and it
**must** map to the resource field in the request message.
- All remaining fields **should** map to URI query parameters.
- There **should** be exactly one `google.api.method_signature` annotation, with
a value of `"{resource},update_mask"`.
- If the API is operating on the [management plane][], the operation should have
[strong consistency][]: the completion of an update operation **must** mean
that all user-settable values and the existence of the resource have reached a
steady-state and reading resource state returns a consistent response.
**Note:** Unlike the other four standard methods, the URI path here references
a nested field (`book.name`) in the example. If the resource field has a word
separator, `snake_case` is used.
### Request message
Update methods implement a common request message pattern:
```proto
message UpdateBookRequest {
// The book to update.
//
// The book's `name` field is used to identify the book to update.
// Format: publishers/{publisher}/books/{book}
Book book = 1 [(google.api.field_behavior) = REQUIRED];
// The list of fields to update.
google.protobuf.FieldMask update_mask = 2;
}
```
- The request message **must** contain a field for the resource.
- The field **must** map to the `PATCH` body.
- The field **should** be [annotated as required][aip-203].
- A `name` field **must** be included in the resource message. It **should**
be called `name`.
- The field **must** identify the [resource type][aip-123] of the resource
being updated.
- If partial resource update is supported, a field mask **must** be included.
It **must** be of type `google.protobuf.FieldMask`, and it **must** be
called `update_mask`.
- The fields used in the field mask correspond to the resource being updated
(not the request message).
- The field **must** be optional, and the service **must** treat an omitted
field mask as an implied field mask equivalent to all fields that are
populated (have a non-empty value).
- Update masks **must** support a special value `*`, meaning full replacement
(the equivalent of `PUT`).
- API producers need to be conscious of how adding new, mutable fields to a
resource will be handled when consumers use `*` without knowledge of said
new, mutable fields. Likewise consumers need to use `*` only when the
risks of doing so are acceptable. In general, it is safest to explicitly
specify the fields to update rather than use `*`.
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
### Side effects
In general, update methods are intended to update the data within the resource.
Update methods **should not** trigger other side effects. Instead, side effects
**should** be triggered by custom methods.
In particular, this entails that [state fields][] **must not** be directly
writable in update methods.
### PATCH and PUT
**TL;DR:** Google APIs generally use the `PATCH` HTTP verb only, and do not
support `PUT` requests.
We standardize on `PATCH` because Google updates stable APIs in place with
backwards-compatible improvements. It is often necessary to add a new field to
an existing resource, but this becomes a breaking change when using `PUT`.
To illustrate this, consider a `PUT` request to a `Book` resource:
PUT /v1/publishers/123/books/456
{"title": "Mary Poppins", "author": "P.L. Travers"}
Next consider that the resource is later augmented with a new field (here we
add `rating`):
```proto
message Book {
string title = 1;
string author = 2;
// Subsequently added to v1 in place...
int32 rating = 3;
}
```
If a rating is set on a book and the existing `PUT` request was executed, it
would wipe out the book's rating. In essence, a `PUT` request unintentionally
wiped out data because the previous version did not know about it.
### Long-running update
Some resources take longer to update a resource than is reasonable for a
regular API request. In this situation, the API **should** use a long-running
operation ([AIP-151][]) instead:
```proto
rpc UpdateBook(UpdateBookRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
patch: "/v1/{book.name=publishers/*/books/*}"
body: "book"
};
option (google.longrunning.operation_info) = {
response_type: "Book"
metadata_type: "OperationMetadata"
};
}
```
- The response type **must** be set to the resource (what the return type would
be if the RPC was not long-running).
- Both the `response_type` and `metadata_type` fields **must** be specified.
**Note:** Declarative-friendly resources ([AIP-128][]) **should** use long-running
update.
### Create or update
If the service uses client-assigned resource names, `Update` methods **may**
expose a `bool allow_missing` field, which will cause the method to succeed in
the event that the user attempts to update a resource that is not present (and
will create the resource in the process):
```proto
message UpdateBookRequest {
// The book to update.
//
// The book's `name` field is used to identify the book to be updated.
// Format: publishers/{publisher}/books/{book}
Book book = 1 [(google.api.field_behavior) = REQUIRED];
// The list of fields to be updated.
google.protobuf.FieldMask update_mask = 2;
// If set to true, and the book is not found, a new book will be created.
// In this situation, `update_mask` is ignored.
bool allow_missing = 3;
}
```
More specifically, the `allow_missing` flag triggers the following behavior:
- If the method call is on a resource that does not exist, the resource is
created. All fields are applied regardless of any provided field mask.
- However, if any required fields are missing or fields have invalid values,
an `INVALID_ARGUMENT` error is returned.
- If the method call is on a resource that already exists, and all fields
match, the existing resource is returned unchanged.
- If the method call is on a resource that already exists, only fields declared
in the field mask are updated.
The user **must** have the update permissions to call `Update` even with
`allow_missing` set to `true`. For customers that want to prevent users from
creating resources using the update method, IAM conditions **should** be used.
### Etags
An API may sometimes need to allow users to send update requests which are
guaranteed to be made against the most current data (a common use case for this
is to detect and avoid race conditions). Resources which need to enable this do
so by including a `string etag` field, which contains an opaque,
server-computed value representing the content of the resource.
In this situation, the resource **should** contain a `string etag` field:
```proto
message Book {
option (google.api.resource) = {
type: "library.googleapis.com/Book"
pattern: "publishers/{publisher}/books/{book}"
};
// The resource name of the book.
// Format: publishers/{publisher}/books/{book}
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// The title of the book.
// Example: "Mary Poppins"
string title = 2;
// The author of the book.
// Example: "P.L. Travers"
string author = 3;
// The etag for this book.
// If this is provided on update, it must match the server's etag.
string etag = 4;
}
```
The `etag` field **may** be either [required][] or [optional][]. If it is set,
then the request **must** succeed if and only if the provided etag matches the
server-computed value, and **must** fail with an `ABORTED` error otherwise. The
`update_mask` field in the request does not affect the behavior of the `etag`
field, as it is not a field _being_ updated.
### Expensive fields
APIs sometimes encounter situations where some fields on a resource are
expensive or impossible to reliably return.
This can happen in a few situations:
- A resource may have some fields that are very expensive to compute, and that
are generally not useful to the customer on update requests.
- A single resource sometimes represents an amalgamation of data from multiple
underlying (and eventually consistent) data sources. In these situations, it
is impossible to return authoritative information on the fields that were not
changed.
In this situation, an API **may** return back only the fields that were
updated, and omit the rest, and **should** document this behavior if they do
so.
### Errors
See [errors][], in particular [when to use PERMISSION_DENIED and
NOT_FOUND errors][permission-denied].
In addition, if the user does have proper permission, but the requested resource
does not exist, the service **must** error with `NOT_FOUND` (HTTP 404) unless
`allow_missing` is set to `true`.
[aip-121]: ./0121.md
[aip-128]: ./0128.md
[aip-151]: ./0151.md
[aip-157]: ./0157.md
[aip-203]: ./0203.md
[errors]: ./0193.md
[management plane]: ./0111.md#management-plane
[permission-denied]: ./0193.md#permission-denied
[state fields]: ./0216.md
[strong consistency]: ./0121.md#strong-consistency
[required]: ./0203.md#required
[optional]: ./0203.md#optional
## Changelog
- **2025-10-03**: Allow use of partial responses via AIP-157.
- **2024-12-03**: Add caveats to usage of full replacement `update_mask`.
- **2024-03-14**: Make `update_mask` optional field_behaviour guidance a **must**.
- **2023-08-26**: Adding consistency requirement.
- **2023-07-17**: Make `update_mask` name guidance a **must**.
- **2022-11-04**: Aggregated error guidance to AIP-193.
- **2022-06-02**: Changed suffix descriptions to eliminate superfluous "-".
- **2021-11-04**: Changed the permission check if `allow_missing` is set.
- **2021-07-08**: Added error guidance for resource not found case.
- **2021-03-05**: Changed the etag error from `FAILED_PRECONDITION` (which
becomes HTTP 400) to `ABORTED` (409).
- **2020-10-06**: Added guidance for declarative-friendly resources.
- **2020-10-06**: Added guidance for `allow_missing`.
- **2020-08-14**: Added error guidance for permission denied cases.
- **2020-06-08**: Added guidance on returning the full resource.
- **2019-10-18**: Added guidance on annotations.
- **2019-09-10**: Added a link to the long-running operations AIP
([AIP-151][]).
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
- **2019-06-10**: Added guidance for long-running update.
- **2019-05-29**: Added an explicit prohibition on arbitrary fields in standard
methods.
---
id: 135
state: approved
created: 2019-01-24
updated: 2022-06-02
placement:
category: operations
order: 50
---
# Standard methods: Delete
In REST APIs, it is customary to make a `DELETE` request to a resource's URI
(for example, `/v1/publishers/{publisher}/books/{book}`) in order to delete
that resource.
Resource-oriented design (AIP-121) honors this pattern through the `Delete`
method. These RPCs accept the URI representing that resource and usually return
an empty response.
## Guidance
APIs **should** generally provide a delete method for resources unless it is
not valuable for users to do so.
Delete methods are specified using the following pattern:
```proto
rpc DeleteBook(DeleteBookRequest) returns (google.protobuf.Empty) {
option (google.api.http) = {
delete: "/v1/{name=publishers/*/books/*}"
};
option (google.api.method_signature) = "name";
}
```
- The RPC's name **must** begin with the word `Delete`. The remainder of the
RPC name **should** be the singular form of the resource's message name.
- The request message **must** match the RPC name, with a `Request` suffix.
- The response message **should** be `google.protobuf.Empty`.
- If the resource is [soft deleted](#soft-delete), the response message
**should** be the resource itself.
- If the delete RPC is [long-running](#long-running-delete), the response
message **must** be a `google.longrunning.Operation` which resolves to the
correct response.
- The HTTP verb **must** be `DELETE`.
- The request message field receiving the resource name **should** map to the
URI path.
- This field **should** be called `name`.
- The `name` field **should** be the only variable in the URI path. All
remaining parameters **should** map to URI query parameters.
- There **must not** be a `body` key in the `google.api.http` annotation.
- There **should** be exactly one `google.api.method_signature` annotation,
with a value of `"name"`. If an etag or force field are used, they **may** be
included in the signature.
- If the API is operating on the [Management Plane][], the operation should have
[strong consistency][]: the completion of a delete operation **must** mean
that the existence of the resource has reached a steady-state and reading
resource state returns a consistent response.
- The API **must** fail with a `FAILED_PRECONDITION` error if child resources
are present. See guidance on [Cascading Delete](#cascading-delete) if forcing
deletion of parent and child resources is necessary.
- If the only child resource type is a [Singleton][aip-156], deletion **must**
be allowed, because the lifecycle of a Singleton is tied to that of its
parent resource. This applies even if there are multiple different Singleton
resource types for the same parent resource.
The Delete method **should** succeed if and only if a resource was present and
was successfully deleted. If the resource did not exist, the method **should**
send a `NOT_FOUND` error.
### Request message
Delete methods implement a common request message pattern:
```proto
message DeleteBookRequest {
// The name of the book to delete.
// Format: publishers/{publisher}/books/{book}
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
}
```
- A `name` field **must** be included. It **should** be called `name`.
- The field **should** be [annotated as required][aip-203].
- The field **must** identify the [resource type][aip-123] that it
references.
- The comment for the field **should** document the resource pattern.
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
### Soft delete
**Note:** This material was moved into its own document to provide a more
comprehensive treatment: AIP-164.
### Long-running delete
Some resources take longer to delete a resource than is reasonable for a
regular API request. In this situation, the API **should** use a long-running
operation instead:
```proto
rpc DeleteBook(DeleteBookRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
delete: "/v1/{name=publishers/*/books/*}"
};
option (google.longrunning.operation_info) = {
response_type: "google.protobuf.Empty"
metadata_type: "OperationMetadata"
};
}
```
- The response type **must** be set to the appropriate return type if the RPC
was not long-running: `google.protobuf.Empty` for most Delete RPCs, or the
resource itself for soft delete (AIP-164).
- Both the `response_type` and `metadata_type` fields **must** be specified
(even if they are `google.protobuf.Empty`).
### Cascading delete
Sometimes, it may be necessary for users to be able to delete a resource as
well as all applicable child resources. However, since deletion is usually
permanent, it is also important that users not do so accidentally, as
reconstructing wiped-out child resources may be quite difficult.
If an API allows deletion of a resource that may have child resources, the API
**should** provide a `bool force` field on the request, which the user sets to
explicitly opt in to a cascading delete.
```proto
message DeletePublisherRequest {
// The name of the publisher to delete.
// Format: publishers/{publisher}
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/Publisher"
}];
// If set to true, any books from this publisher will also be deleted.
// (Otherwise, the request will only work if the publisher has no books.)
bool force = 2;
}
```
The API **must** fail with a `FAILED_PRECONDITION` error if the `force` field
is `false` (or unset) and child resources are present.
### Protected delete
Sometimes, it may be necessary for users to ensure that no changes have been
made to a resource that is being deleted. If a resource provides an [etag][],
the delete request **may** accept the etag (as either required or optional):
```proto
message DeleteBookRequest {
// The name of the book to delete.
// Format: publishers/{publisher}/books/{book}
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
// Optional. The etag of the book.
// If this is provided, it must match the server's etag.
string etag = 2;
}
```
If the etag is provided and does not match the server-computed etag, the
request **must** fail with a `ABORTED` error code.
**Note:** Declarative-friendly resources (AIP-128) **must** provide the `etag`
field for Delete requests.
### Delete if existing
If the service uses client-assigned resource names, `Delete` methods **may**
expose a `bool allow_missing` field, which will cause the method to succeed in
the event that the user attempts to delete a resource that is not present (in
which case the request is a no-op):
```proto
message DeleteBookRequest {
// The book to delete.
// Format: publishers/{publisher}/books/{book}
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference).type = "library.googleapis.com/Book"
];
// If set to true, and the book is not found, the request will succeed
// but no action will be taken on the server
bool allow_missing = 2;
}
```
More specifically, the `allow_missing` flag triggers the following behavior:
- If the method call is on a resource that does not exist, the request is a
no-op.
- The `etag` field is ignored.
- If the method call is on a resource that already exists, the resource is
deleted (subject to other checks).
**Note:** Declarative-friendly resources (AIP-128) **should** expose the
`bool allow_missing` field.
### Errors
If the user does not have permission to access the resource, regardless of
whether or not it exists, the service **must** error with `PERMISSION_DENIED`
(HTTP 403). Permission **must** be checked prior to checking if the resource
exists.
If the user does have proper permission, but the requested resource does not
exist, the service **must** error with `NOT_FOUND` (HTTP 404) unless
`allow_missing` is set to `true`.
[aip-123]: ./0123.md
[aip-131]: ./0131.md
[aip-132]: ./0132.md
[aip-136]: ./0136.md
[aip-154]: ./0154.md
[aip-156]: ./0156.md
[aip-203]: ./0203.md
[aip-214]: ./0214.md
[aip-216]: ./0216.md
[management plane]: ./0111.md#management-plane
[strong consistency]: ./0121.md#strong-consistency
[etag]: ./0134.md#etags
## Further reading
- For soft delete and undelete, see AIP-164.
- For bulk deleting large numbers of resources based on a filter, see AIP-165.
## Changelog
- **2024-06-11**: Add deletion behavior for parent resource deletion requests without a `force` field.
- **2023-08-24**: Adding consistency requirement.
- **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-".
- **2022-02-02**: Changed eTag error from `FAILED_PRECONDITION` to `ABORTED` making it consistent with change to [AIP-154][] & [AIP-134][etag] on 2021-03-05.
- **2020-10-06**: Added guidance for declarative-friendly resources.
- **2020-10-06**: Added guidance for allowing no-op delete for missing
resources.
- **2020-10-06**: Moved soft delete and undelete guidance into a new AIP-164.
- **2020-06-08**: Added guidance for `Get` of soft-deleted resources.
- **2020-02-03**: Added guidance for error cases.
- **2019-10-18**: Added guidance on annotations.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
- **2019-06-10**: Added guidance for long-running delete.
- **2019-05-29**: Added an explicit prohibition on arbitrary fields in standard
methods.
---
id: 136
state: approved
created: 2019-01-25
updated: 2023-03-02
placement:
category: operations
order: 100
---
# Custom methods
Resource-oriented design (AIP-121) uses custom methods to provide a means to
express arbitrary actions that are difficult to model using only the standard
methods. Custom methods are important because they provide a means for an API's
vocabulary to adhere to user intent.
## Guidance
Custom methods **should** only be used for functionality that can not be easily
expressed via standard methods; prefer standard methods if possible, due to
their consistent semantics. (Of course, this only applies if the functionality
in question actually conforms to the normal semantics; it is _not_ a good idea
to contort things to endeavor to make the standard methods "sort of work".)
While custom methods vary widely in how they are designed, many principles
apply consistently:
```proto
// Archives the given book.
rpc ArchiveBook(ArchiveBookRequest) returns (ArchiveBookResponse) {
option (google.api.http) = {
post: "/v1/{name=publishers/*/books/*}:archive"
body: "*"
};
}
```
**Note:** The pattern above shows a custom method that operates on a specific
resource. Custom methods can be associated with resources, collections, or
services. The bullets below apply in all three cases.
- The name of the method **should** be a verb followed by a noun.
- The name **must not** contain prepositions ("for", "with", etc.).
- The verb in the name **should not** contain any of the standard method verbs ([Get][],
[List][], [Create][], [Update][], [Delete][]).
- The name **must not** include the term `Async`. Instead, if the intention is
to differentiate between immediate and long-running RPCs, the suffix `LongRunning`
**may** be used for this purpose. For example, to create a long-running book creation
RPC (if the standard `CreateBook` method was designed before long-running aspects were
considered), a custom `CreateBookLongRunning` method could be introduced.
- The HTTP method **must** be `GET` or `POST`:
- `GET` **must** be used for methods retrieving data or resource state.
- `POST` **must** be used if the method has side effects or mutates resources
or data.
- The HTTP URI **must** use a `:` character followed by the custom verb
(`:archive` in the above example), and the verb in the URI **must** match the
verb in the name of the RPC.
- If word separation is required, `camelCase` **must** be used.
- The `body` clause in the `google.api.http` annotation **should** be `"*"`.
- See [HTTP and gRPC Transcoding][transcoding] for more information.
- Custom methods **should** take a request message matching the RPC
name, with a `Request` suffix.
- Custom methods **should** return a response message matching the RPC
name, with a `Response` suffix.
- When operating on a specific resource, a custom method **may** return the
resource itself.
### Resource-based custom methods
Custom methods **must** operate on a resource if the API can be modeled
as such:
```proto
// Archives the given book.
rpc ArchiveBook(ArchiveBookRequest) returns (ArchiveBookResponse) {
option (google.api.http) = {
post: "/v1/{name=publishers/*/books/*}:archive"
body: "*"
};
}
```
- The parameter for the resource's name **must** be called `name`, and
be the only variable in the URI path.
### Collection-based custom methods
While most custom methods operate on a single resource, some custom methods
**may** operate on a collection instead:
```proto
// Sorts the books from this publisher.
rpc SortBooks(SortBooksRequest) returns (SortBooksResponse) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:sort"
body: "*"
};
}
```
- If the collection's resource has a parent, that resource **must** be called
`parent` and be the only variable in the URI path.
- The collection key (`books` in the above example) **must** be literal.
### Stateless methods
Some custom methods are not attached to resources at all. These methods are
generally _stateless_: they accept a request and return a response, and have no
permanent effect on data within the API.
```proto
// Translates the provided text from one language to another.
rpc TranslateText(TranslateTextRequest) returns (TranslateTextResponse) {
option (google.api.http) = {
post: "/v1/{project=projects/*}:translateText"
body: "*"
};
}
```
- If the method runs in a particular scope (such as a project, as in the above
example), the field name in the request message **should** be the name of the
scope resource. If word separators are necessary, `snake_case` **must** be
used.
- The URI **should** place both the verb and noun after the `:` separator
(avoid a "faux collection key" in the URI in this case, as there is no
collection). For example, `:translateText` is preferable to `text:translate`.
- Stateless methods **must** use `POST` if they involve billing.
### Declarative-friendly resources
Declarative-friendly resources usually **should not** employ custom methods
(except specific declarative-friendly custom methods discussed in other AIPs),
because declarative-friendly tools are unable to automatically determine what
to do with them.
An exception to this is for rarely-used, fundamentally imperative operations,
such as a `Move` or `Rename` operation, for which there would not be an
expectation of declarative support.
## Rationale
### HTTP path
Similar to standard methods, a custom method that operates on a resource or
collection needs a `name` or `parent` parameter to indicate the resource that it
operates on. This convention allows clients to map custom methods to the
appropriate resource.
### HTTP methods
Allowing both `GET` and `POST` HTTP verbs allows a clear distinction for
which methods do not mutate data, and which ones do. Methods that only
read data have first-class concepts in some clients (DataSources in
Terraform) and clearly indicate to a user which methods can be called
without risk of runtime impact.
### Disallowing prepositions
Generally, method names with prepositions indicate that a new method is being
used where a field should instead be added to an existing method, or the method
should use a distinct verb. For example, if a `CreateBook` message already
exists and you are considering adding `CreateBookFromDictation`, consider a
`TranscribeBook` method instead. Similarly, if there is desire for a
property-specific look-up method, instead of `GetBookByAuthor` consider a
`SearchBooks` with an `author` field as a search dimension. This helps prevent
an explosion of hyper-focused methods that bloat API and client surfaces, and
add complexity to both managing and consuming the API.
[get]: ./0131.md
[list]: ./0132.md
[create]: ./0133.md
[update]: ./0134.md
[delete]: ./0135.md
[transcoding]: ./0127.md
### RPC name
The term "async" is commonly used in programming languages to indicate whether
a specific method call is synchronous or asynchronous, including for making RPCs.
That sync/async aspect is at a different abstraction level to whether the RPC
itself is intended to start a long-running operation. Using "async" within the
RPC name itself causes confusion, and can even cause issues for client libraries
which generate both synchronous and asynchronous methods to call the RPC in some
languages.
## Changelog
- **2025-05-12:** Extend disallowing prepositions rationale.
- **2025-01-09:** Add original rationale for disallowing prepositions in names.
- **2023-11-16:** Included link to AIP-127 "HTTP and gRPC Transcoding" for guidance on body definition.
- **2023-05-16:** Added prohibition of the term "async" within RPC names.
- **2023-05-09:** Adding guidance for POST and GET, require parent instead of
the resource singular.
- **2023-03-02:** Explicitly discourage use of standard method verbs.
- **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-".
- **2020-10-06:** Added declarative-friendly guidance.
- **2019-08-01:** Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
---
id: 140
state: approved
created: 2019-07-22
placement:
category: fields
order: 0
---
# Field names
Naming fields in a way that is intuitive to users can often be one of the most
challenging aspects of designing an API. This is true for many reasons; often a
field name that seems entirely intuitive to the author can baffle a reader.
Additionally, users rarely use only one API; they use many APIs together. As a
result, a single company using the same name to mean different things (or
different names to mean the same thing) can often cause unnecessary confusion,
because users can no longer take what they've already learned from one API and
apply that to another.
In short, APIs are easiest to understand when field names are simple,
intuitive, and consistent with one another.
## Guidance
Field names **should** be in correct American English.
Field names **should** clearly and precisely communicate the concept being
presented and avoid overly general names that are ambiguous. That said, field
names **should** avoid including unnecessary words. In particular, avoid
including adjectives that always apply and add little cognitive value. For
example, a `proxy_settings` field might be as helpful as
`shared_proxy_settings` if there is no unshared variant.
**Important:** Field names often appear in generated client surfaces. Ensure they
are appropriately descriptive and of suitable length.
### Case
Field definitions in protobuf files **must** use `lower_snake_case` names.
These names are mapped to an appropriate naming convention in JSON and in
generated code.
Additionally, each word in the field **must not** begin with a number, because
it creates ambiguity when converting between snake case and camel case.
Similarly, fields **must not** contain leading, trailing, or adjacent
underscores.
### Uniformity
APIs **should** endeavor to use the same name for the same concept and
different names for different concepts wherever possible. This includes names
across multiple APIs, in particular if those APIs are likely to be used
together.
### Repeated fields
Repeated fields **must** use the proper plural form, such as `books` or
`authors`. On the other hand, non-repeated fields **should** use the singular
form such as `book` or `author`. This implies that resource names **should**
use the singular form as well, since the field name should follow the resource
name (e.g., use `repeated Book books`, **not** `Books books = 1`).
### Prepositions
Field names **should not** include prepositions (such as "with", "for", "at",
"by", etc). For example:
- `error_reason` (**not** `reason_for_error`)
- `author` (**not** `written_by`)
It is easier for field names to match more often when following this
convention. Additionally, prepositions in field names may also indicate a
design concern, such as an overly-restrictive field or a sub-optimal data type.
This is particularly true regarding "with": a field named `book_with_publisher`
likely indicates that the book resource may be improperly structured and worth
redesigning.
**Note:** The word "per" is an exception to this rule. See [AIP-141][] for
guidance in the case where "per" is used as part of a unit (e.g. "miles per hour").
### Abbreviations
For well known name abbreviations among software developers, such as "config"
and "spec", the abbreviations **should** be used in API definitions instead of
the full spelling. This will make the source code easy to read and write.
Examples:
- `config` (**not** `configuration`)
- `id` (**not** `identifier`)
- `info` (**not** `information`)
- `spec` (**not** `specification`)
- `stats` (**not** `statistics`)
Furthermore, well known abbreviations for units **should** be used in field names.
See [AIP-141][] for more guidance on fields that represent quantities. Examples:
- `distance_km` (**not** `distance_kilometers`)
- `width_px` (**not** `width_pixels`)
### Adjectives
For uniformity, field names that contain both a noun and an adjective
**should** place the adjective _before_ the noun. For example:
- `collected_items` (**not** `items_collected`)
- `imported_objects` (**not** `objects_imported`)
### Verbs
Field names **must not** be named to reflect an intent or action. They
**must not** be verbs. Rather, because the field defines the *desired
value* for mutations, e.g. Create and Update, and the *current value*
for reads, e.g. Get and List, the name **must** be a noun. It defines
what is so, not what to do.
- `collected_items` (**not** `collect_items`)
- `disabled` (**not** `disable`)
In contrast, method names, whether standard or custom, change facets of
resources and are named as verbs.
### Booleans
Boolean fields **should** omit the prefix "is". For example:
- `disabled` (**not** `is_disabled`)
- `required` (**not** `is_required`)
**Note:** Field names that would otherwise be [reserved words](#reserved-words)
are an exception to this rule. For example, `is_new` (**not** `new`).
### String vs. bytes
When using `bytes`, the contents of the field are base64-encoded when using
JSON on the wire. Services **should** use `bytes` when there is a need to send
binary contents over the wire, and **should not** ask the user to manually
base64-encode a field into a `string` field. The one exception is when the
API is designed to handle data that is meant to be base64-encoded at rest and
the double base64-encoding as a result of using `bytes` is undesirable - in
this case, services **may** use a `string`.
### URIs
Field names representing arbitrary URIs **should** use `uri`. In particular,
note that URLs are URIs but not all URIs are URLs.
Field names **may** use a prefix in front of `uri` as appropriate.
Field names that can only represent a URL **should** use `url`.
```proto
message Book {
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// A URL pointing to an image of the book.
string image_url = 2;
// A URI identifying the book.
// This could be an ISBN or a URL.
string uri = 3;
}
```
**Note:** APIs that have previously used `uri` for URL fields may continue to
do so to avoid unnecessary API changes and to preserve local consistency.
### Reserved words
Field names **should** avoid using names that are likely to conflict with
keywords in common programming languages, such as `new`, `class`, `function`,
`import`, etc. Reserved keywords can cause hardship for developers using the
API in that language.
### Conflicts
Messages **should not** include a field with the same name as the enclosing
message (ignoring case transformations). This causes conflicts when generating
code in some languages.
### Display names
Many resources have a human-readable name, often used for display in UI. This
field **should** be called `display_name`, and **should not** have a uniqueness
requirement.
If an entity has an official, formal name (such as a company name or the title
of a book), an API **may** use `title` as the field name instead. The `title`
field **should not** have a uniqueness requirement.
## Further reading
- For naming resource fields, see [AIP-122][].
- For naming fields representing quantities, see [AIP-141][].
- For naming fields representing time, see [AIP-142][].
## Rationale
### URI vs URL
The guidance itself mentions that all URLs are URIs, but not all URIs are URLs.
The goal of aligning on URI is to enable a more generalizable field, that can
handle a variety of use cases, as well as drive standardization across APIs,
creating a more coherent surface. At the same time, the requirement being a
**should** allows for more specific terms when it truly merits it.
## Changelog
- **2025-10-23**: Call out AIP-141 unit abbreviations as preferred.
- **2025-03-10**: Add rationale for URI guidance.
- **2024-12-20**: Copy over abbreviations guidance from old design site.
- **2024-08-26**: Codify exception to string and base64 guidance
- **2024-05-18**: Documented the effect of field names on client surfaces.
- **2023-04-25**: Field names **must not** be expressed as verbs.
- **2021-07-12**: Normalized display name guidance to "should".
- **2021-04-07**: Added base64 and bytes guidance.
- **2021-03-05**: Added prohibition on leading, trailing, or adjacent
underscores.
- **2020-06-10**: Added prohibition on starting any word with a number.
- **2020-05-29**: Added guidance around URIs.
- **2020-03-24**: Added guidance around conflicting field and message names.
- **2020-01-30**: Added guidance around `display_name` and `title`.
[aip-122]: ./0122.md
[aip-141]: ./0141.md
[aip-142]: ./0142.md
---
id: 141
state: approved
created: 2019-07-18
placement:
category: fields
order: 20
---
# Quantities
Many services need to represent a discrete quantity of items (number of bytes,
number of miles, number of nodes, etc.).
## Guidance
Quantities with a clear unit of measurement (such as bytes, miles, and so on)
**must** include the unit of measurement as the suffix. When appropriate, units
**should** use generally accepted abbreviations, and abbreviations **should not**
be pluralized (for example, `distance_km` rather than `distance_kilometers`).
```proto
// A representation of a non-stop air route.
message Route {
// The airport where the route begins.
string origin = 1;
// The destination airport.
string destination = 2;
// The distance between the origin and destination airports.
// This value is also used to determine the credited frequent flyer miles.
int32 distance_miles = 3;
}
```
If the quantity is a number of items (for example, the number of nodes in a
cluster), then the field **should** use the suffix `_count` (**not** the prefix
`num_`):
```proto
// A cluster of individual nodes.
message Cluster {
// The number of nodes in the cluster.
int32 node_count = 1;
}
```
**Note:** Fields **must not** use unsigned integer types, because many
programming languages and systems do not support them well.
### Compound units
Quantities with compound units of measurement **may** use separating underscores
between units as needed for clarity. Unabbreviated units **must** be separated.
Abbreviated units **should not** be separated unless otherwise ambiguous.
Compound units **should** be in plural form, with all component units in
singular form except for the final component unit, which should be in plural
form unless abbreviated.
- `energy_kwh` (**not** `energy_kw_h`)
- `energy_kw_fortnights` (**not** `energy_kwfortnight` or `energy_kw_fortnight`)
**Note:** Metric prefixes **must not** be separated from their base unit.
### Inverse units
Quantities with units of measurement that are or include inverse units
**should** indicate all inverse units as a compound unit after a compound
of any non-inverse units, separated by the word "per".
The inverse compound unit **should** be in singular form.
- `speed_miles_per_hour` (**not** `speed_mph`)
- `speed_meters_per_second` (**not** `speed_meters_per_seconds` or `speed_meter_per_second`)
- `event_count_per_hour` (**not** `events_per_hour`, `event_counts_per_hour`, or `hourly_events`)
- `price_per_kwh` (using [`google.type.Money`][money])
**Note:** This guidance does not apply in cases where generally accepted derived
units with special names and symbols exist for inverse quantities. For example,
the derived unit 'hertz' **should** be used when appropriate for reciprocal time.
### Specialized messages
It is sometimes useful to create a message that represents a particular
quantity. This is particularly valuable in two situations:
- Grouping two or more individual quantities together. Example:
[`google.protobuf.Duration`][duration]
- Representing a common concept where the unit of measurement may itself vary.
Example: [`google.type.Money`][money]
APIs **may** create messages to represent quantities when appropriate. When
using these messages as fields, APIs **should** use the name of the message as
the suffix for the field name if it makes intuitive sense to do so.
## Changelog
- **2025-07-09**: Added guidance for compound units.
- **2025-07-09**: Clarified guidance to not pluralize abbreviated units.
- **2025-07-09**: Clarified guidance to use per_ prefix to represent inverse units.
- **2019-09-13**: Added the prohibition on uint and fixed types.
<!-- prettier-ignore-start -->
[duration]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/duration.proto
[money]: https://github.com/googleapis/googleapis/blob/master/google/type/money.proto
<!-- prettier-ignore-end -->
---
id: 142
state: approved
created: 2019-07-16
placement:
category: fields
order: 30
---
# Time and duration
Many services need to represent the concepts surrounding time. Representing
time can be challenging due to the intricacies of calendars and time zones, as
well as the fact that common exchange formats (such as JSON) lack a native
concept of time.
## Guidance
Fields representing time **should** use the common, generally used components
(such as [`google.protobuf.Timestamp`][timestamp] or
[`google.type.Date`][date]) for representing time or duration types. These
types are [common components][aip-213], and using them consistently allows
infrastructure and tooling to provide a better experience when interacting with
time values.
### Timestamps
Fields that represent an absolute point in time (independent of any time zone
or calendar) **should** use the [`google.protobuf.Timestamp`][timestamp] type,
(which uses UNIX timestamps under the hood and hold nanosecond precision).
These fields **should** have names ending in `_time`, such as `create_time` or
`update_time`. For repeated fields, the names **should** end in `_times`
instead.
Many timestamp fields refer to an activity (for example, `create_time` refers
to when the applicable resource was created). For these, the field **should**
be named with the `{imperative}_time` form. For example, if a book is being
published, the field storing the time when this happens would use the
imperative form of the verb "to publish" ("publish") resulting in a field
called `publish_time`. Fields **should not** be named using the past tense
(such as `published_time`, `created_time` or `last_updated_time`).
### Durations
Fields that represent a span between two points in time (independent of any
time zone or calendar) **should** use the
[`google.protobuf.Duration`][duration] type.
To illustrate the distinction between timestamps and durations, consider a
flight record:
```proto
// A representation of a (very incomplete) flight log.
message FlightRecord {
// The absolute point in time when the plane took off.
google.protobuf.Timestamp takeoff_time = 1;
// The length (duration) of the flight, from takeoff to landing.
google.protobuf.Duration flight_duration = 2;
}
```
**Note:** Observant readers may notice that the [timestamp][] and [duration][]
messages have the same structure (`int64 seconds` and `int32 nanos`). However,
the distinction between these is important, because they have different
semantic meaning. Additionally, tooling is able to base behavior off of which
message is used. For example, a Python-based tool could convert timestamps to
[datetime][py_datetime] objects and durations to [timedelta][py_timedelta]
objects.
### Relative time segments
In some cases, it may be necessary to represent a time segment inside a stream.
In these cases, the [`google.protobuf.Duration`][duration] type **should** be
used, and the field name **should** end with `_offset`. To ensure that the
meaning is clear, the field **must** have a comment noting the point that the
offset is relative to.
To illustrate this, consider a resource representing a segment of an audio
stream:
```proto
message AudioSegment {
// The duration relative to the start of the stream representing the
// beginning of the segment.
google.protobuf.Duration start_offset = 1;
// The total length of the segment.
google.protobuf.Duration segment_duration = 2;
}
```
### Civil dates and times
Fields that represent a calendar date or wall-clock time **should** use the
appropriate common components:
- Civil date: [`google.type.Date`][date]
- Wall-clock time: [`google.type.TimeOfDay`][time_of_day]
Fields representing civil dates **should** have names ending in `_date`, while
fields representing civil times or datetimes **should** have names ending in
`_time`.
**Note:** Both the `Date` and `TimeOfDay` components are timezone-naïve. Fields
that require timezone-awareness **should** use `DateTime` (see below).
### Civil timestamps
Fields that represent a civil timestamp (date and time, optionally with a time
zone) **should** use the [`google.type.DateTime`][datetime] component, and the
field name **should** end in `_time`.
### Compatibility
Occasionally, APIs are unable to use the common structures for legacy or
compatibility reasons. For example, an API may conform to a separate
specification that mandates that timestamps be integers or [ISO-8601][]
strings.
In these situations, fields **may** use other types. If possible, the following
naming conventions apply:
- For integers, include the meaning (examples: `time`, `duration`, `delay`,
`latency`) **and** the unit of measurement (valid values: `seconds`,
`millis`, `micros`, `nanos`) as a final suffix. For example,
`send_time_millis`.
- For strings, include the meaning (examples: `time`, `duration`, `delay`,
`latency`) but no unit suffix.
In all cases, clearly document the expected format, and the rationale for its
use.
<!-- prettier-ignore-start -->
[aip-213]: ./0213.md
[date]: https://github.com/googleapis/googleapis/blob/master/google/type/date.proto
[datetime]: https://github.com/googleapis/googleapis/blob/master/google/type/datetime.proto
[duration]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/duration.proto
[iso-8601]: https://www.iso.org/iso-8601-date-and-time-format.html
[py_datetime]: https://docs.python.org/3/library/datetime.html#datetime.datetime
[py_timedelta]: https://docs.python.org/3/library/datetime.html#datetime.timedelta
[time_of_day]: https://github.com/googleapis/googleapis/blob/master/google/type/timeofday.proto
[timestamp]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/timestamp.proto
<!-- prettier-ignore-end -->
---
id: 143
state: approved
created: 2019-07-24
placement:
category: fields
order: 40
---
# Standardized codes
Many common concepts, such as spoken languages, countries, currency, and so on,
have common codes (usually formalized by the [International Organization for
Standardization][iso]) that are used in data communication and processing.
These codes address the issue that there are often different ways to express
the same concept in written language (for example, "United States" and "USA",
or "Español" and "Spanish").
## Guidance
For concepts where a standardized code exists and is in common use, fields
representing these concepts **should** use the standardized code for both input
and output.
```proto
// A message representing a book.
message Book {
// Other fields...
// The IETF BCP-47 language code representing the language in which
// the book was originally written.
// https://en.wikipedia.org/wiki/IETF_language_tag
string language_code = 99;
}
```
- Fields representing standardized concepts **must** use the appropriate data
type for the standard code (usually `string`).
- Fields representing standardized concepts **should not** use enums, even if
they only allow a small subset of possible values. Using enums in this
situation often leads to frustrating lookup tables when using multiple APIs
together.
- Fields representing standardized concepts **must** indicate which standard
they follow, preferably with a link (either to the standard itself, the
Wikipedia description, or something similar).
- The field name **should** end in `_code` or `_type` unless the concept has an
obviously clearer suffix.
- When accepting values provided by users, validation **should** be
case-insensitive unless this would introduce ambiguity (for example, accept
both `en-gb` and `en-GB`). When providing values to users, APIs **should**
use the canonical case (in the example above, `en-GB`).
### Content types
Fields representing a content or media type **must** use [IANA media types][].
For legacy reasons, the field **should** be called `mime_type`.
### Countries and regions
Fields representing individual countries or nations **must** use the [Unicode
CLDR region codes][cldr] ([list][]), such as `US` or `CH`, and the field
**must** be called `region_code`.
**Important:** Please read the [rationale](#countryregion-field-naming) for this
requirement.
### Currency
Fields representing currency **must** use [ISO-4217 currency codes][iso-4217],
such as `USD` or `CHF`, and the field **must** be called `currency_code`.
**Note:** For representing an amount of money in a particular currency, rather
than the currency code itself, use [`google.type.Money`][money].
### Language
Fields representing spoken languages **must** use [IETF BCP-47 language
codes][bcp-47] ([list][]), such as `en-US` or `de-CH`, and the field **must**
be called `language_code`.
### Time zones
Fields representing a time zone **should** use the [IANA TZ][] codes, and the
field **must** be called `time_zone`.
Fields also **may** represent a UTC offset rather than a time zone (note that
these are subtly different). In this case, the field **must** use the [ISO-8601
format][] to represent this, and the field **must** be named `utc_offset`.
## Rationale
### Country/region field naming
The use of `region_code` instead of `country_code` is critical to being able to
convey regions that are distinct from any country and to avoid any political
disputes associated with said region regarding their sovereignty or affiliation.
Google and many other companies are supporters of Unicode CLDR and standardize
their product internationalization efforts on Unicode CLDR, and APIs are no
different here. Furthermore, many of the values supported by Unicode CLDR are
not countries on their own, so using a more generic name is actually more
compatible with the specification.
## Changelog
- **2024-12-03**: Strengthen rationale of country/region field naming
- **2024-11-12**: Change country/region code list to CLDR list from IANA list
- **2020-05-12**: Replaced `country_code` guidance with `region_code`,
correcting an original error.
<!-- prettier-ignore-start -->
[bcp-47]: https://en.wikipedia.org/wiki/IETF_language_tag
[cldr]: http://cldr.unicode.org/
[iana media types]: https://www.iana.org/assignments/media-types/media-types.xhtml
[iana tz]: http://www.iana.org/time-zones
[iso]: https://www.iso.org/
[iso-4217]: https://en.wikipedia.org/wiki/ISO_4217
[iso-8601 format]: https://en.wikipedia.org/wiki/ISO_8601#Time_offsets_from_UTC
[list]: https://www.unicode.org/cldr/charts/46/supplemental/territory_information.html
[money]: https://github.com/googleapis/googleapis/blob/master/google/type/money.proto
<!-- prettier-ignore-end -->
---
id: 144
state: approved
created: 2020-03-19
updated: 2022-06-02
placement:
category: fields
order: 50
---
# Repeated fields
Representing lists of data in an API is trickier than it often appears. Users
often need to modify lists in place, and longer data series within a single
resource pose a challenge for pagination.
## Guidance
Resources **may** use repeated fields where appropriate.
```proto
message Book {
option (google.api.resource) = {
type: "library.googleapis.com/Book"
pattern: "publishers/{publisher}/books/{book}"
};
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
repeated string authors = 2;
}
```
- Repeated fields **must** use a plural field name.
- If the English singular and plural words are identical ("moose", "info"),
the dictionary word **must** be used rather than attempting to coin a new
plural form.
- Repeated fields **should** have an enforced upper bound that will not cause a
single resource payload to become too large. A good rule of thumb is 100
elements.
- If repeated data has the chance of being too large, the API **should** use
a sub-resource instead.
- Repeated fields **must not** represent the body of another resource inline.
Instead, the message **should** provide the resource names of the associated
resources.
### Scalars and messages
Repeated fields **should** use a scalar type (such as `string`) if they are
certain that additional data will not be needed in the future, as using a
message type adds significant cognitive overhead and leads to more complicated
code.
However, if additional data is likely to be needed in the future, repeated
fields **should** use a message instead of a scalar proactively, to avoid
parallel repeated fields.
### Update strategies
A resource **may** use two strategies to enable updating a repeated field:
direct update using the [standard `Update`][aip-134] method, or custom `Add`
and `Remove` methods.
A standard `Update` method has one key limitation: the user is only able to
update _the entire_ list. Field masks are unable to address individual entries
in a repeated field. This means that the user must read the resource, make
modifications to the repeated field value as needed, and send it back. This is
fine for many situations, particularly when the repeated field is expected to
have a small size (fewer than 10 or so) and race conditions are not an issue,
or can be guarded against with [ETags][aip-154].
**Note:** Declarative-friendly resources **must** use the standard `Update`
method, and not introduce `Add` and `Remove` methods. If declarative tools need
to reason about particular relationships while ignoring others, consider using
a subresource instead.
If atomic modifications are required, the API **should** define custom methods
using the verbs `Add` and `Remove`:
**Note:** If both of these strategies are too restrictive, consider using a
subresource instead.
```proto
rpc AddAuthor(AddAuthorRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{book=publishers/*/books/*}:addAuthor"
body: "*"
};
}
rpc RemoveAuthor(RemoveAuthorRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{book=publishers/*/books/*}:removeAuthor"
body: "*"
};
}
```
- The data being added or removed **should** be a primitive (usually a
`string`).
- For more complex data structures with a primary key, the API **should** use
a map with the `Update` method instead.
- The RPC's name **must** begin with the word `Add` or `Remove`. The remainder
of the RPC name **should** be the singular form of the field being added.
- The request message **must** match the RPC name, with a `Request` suffix.
- The response message **should** be the resource itself, unless there is useful
context to provide in the response, in which case the response message must
match the RPC name, with a `Response` suffix.
- When the response is the resource itself, it **should** include the
fully-populated resource.
- The HTTP verb **must** be `POST`, as is usual for [custom methods][aip-136].
- The HTTP URI **must** end with `:add*` or `:remove*`, where `*` is the
snake-case singular name of the field being added or removed.
- The request message field receiving the resource name **should** map to the
URI path.
- The HTTP variable **should** be the name of the resource (such as `book`)
rather than `name` or `parent`.
- That variable **should** be the only variable in the URI path.
- The body clause in the `google.api.http` annotation **should** be `"*"`.
- If the data being added in an `Add` RPC is already present, the method
**must** error with `ALREADY_EXISTS`.
- If the data being removed in a `Remove` RPC is not present, the method
**must** error with `NOT_FOUND`.
#### Request Message
```proto
message AddAuthorRequest {
// The name of the book to add an author to.
string book = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference).type = "library.googleapis.com/Book"
];
string author = 2 [(google.api.field_behavior) = REQUIRED];
}
message RemoveAuthorRequest {
// The name of the book to remove an author from.
string book = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference).type = "library.googleapis.com/Book"
];
string author = 2 [(google.api.field_behavior) = REQUIRED];
}
```
- A resource field **must** be included. It **should** be the name of the
resource (such as `book`) rather than `name` or `parent`.
- The field **should** be [annotated as required][aip-203].
- The field **should** identify the [resource type][aip-123] that it
references.
- A field for the value being added or removed **must** be included. It
**should** be the singular name of the field.
- The field **should** be [annotated as required][aip-203].
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
## Changelog
- **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-".
- **2020-10-17**: Recommended returning the resource itself in Add and Remove
RPCs over separate response types.
- **2020-10-17**: Added guidance for Add and Remove RPCs and requests.
[aip-123]: ./0123.md
[aip-134]: ./0134.md
[aip-136]: ./0136.md
[aip-154]: ./0154.md
[aip-203]: ./0203.md
---
id: 145
state: approved
created: 2020-05-28
placement:
category: fields
order: 60
---
# Ranges
Services often need to represent ranges of discrete or continuous values. These
have wide differences in meaning, and come in many types: integers, floats, and
timestamps, just to name a few, and the expected meaning of a range can vary in
subtle ways depending on the type of range being discussed.
## Guidance
A resource or message representing a range **should** ordinarily use two
separate fields of the same type, with prefixes `start_` and `end_`:
```proto
// A representation of a chapter in a book.
message Chapter {
string title = 1;
// The page where this chapter begins.
int32 start_page = 2;
// The page where the next chapter or section begins.
int32 end_page = 3;
}
```
### Inclusive or exclusive ranges
Fields representing ranges **should** use inclusive start values and exclusive
end values (half-closed intervals) in most situations; in interval notation:
`[start_xxx, end_xxx)`.
Exclusive end values are preferable for the following reasons:
- It conforms to user expectations, particularly for continuous values such as
timestamps, and avoids the need to express imprecise "limit values" (e.g.
`2012-04-20T23:59:59`).
- It is consistent with most common programming languages, including C++, Java,
Python, and Go.
- It is easier to reason about abutting ranges: `[0, x), [x, y), [y, z)`, where
values are chainable from one range to the next.
### Timestamp intervals
The following section describes the use of the [google.type.Interval][interval]
type, found amongst the common protos that are described in [AIP-213][].
This type represents a range between two timestamps, with an inclusive start
value and exclusive end value.
Ranges between two timestamps which conform to the expectations of the
`Interval` message **should** use this rather than having separate start and
end fields. This allows client code to be written against the `Interval`
message (such as checking whether a given timestamp occurs within the interval)
and reused across multiple intervals in the same API, or even across multiple
APIs.
APIs **may** use start and end timestamp fields instead. In particular, if a
message within an API is inherently describing an interval with extra
information about that interval, the additional level of nesting introduced
by using the `Interval` message may be undesirable.
### Exceptions
In some cases, there is significant colloquial precedent for inclusive start
and end values (closed intervals), to the point that using an exclusive end
value would be confusing even for people accustomed to them.
For example, when discussing dates (not to be confused with timestamps), most
people use inclusive end: a conference with dates "April 21-23" is expected to
run for three days: April 21, April 22, and April 23. This is also true for
days of the week: a business that is open "Monday through Friday" is open, not
closed, on Fridays.
In this situation, the prefixes `first` and `last` **should** be used instead:
```proto
// A representation of a chapter in a book.
message Chapter {
string title = 1;
// The first page of the chapter.
int32 first_page = 2;
// The last page of the chapter.
int32 last_page = 3;
}
```
Fields representing ranges with significant colloquial precedent for inclusive
start and end values **should** use inclusive end values with `first_` and
`last_` prefixes for those ranges only. The service **should** still use
exclusive end values for other ranges where this does not apply, and **must**
clearly document each range as inclusive or exclusive.
[aip-213]: ./0213.md
[interval]: https://github.com/googleapis/googleapis/blob/master/google/type/interval.proto
---
id: 146
state: approved
created: 2019-05-28
placement:
category: fields
order: 70
---
# Generic fields
Most fields in any API, whether in a request, a resource, or a custom response,
have a specific type or schema. This schema is part of the contract that
developers write their code against.
However, occasionally it is appropriate to have a generic or polymorphic field
of some kind that can conform to multiple schemata, or even be entirely
free-form.
## Guidance
While generic fields are generally rare, a service **may** introduce generic
field where necessary. There are several approaches to this depending on how
generic the field needs to be; in general, services **should** attempt to
introduce the "least generic" approach that is able to satisfy the use case.
### Oneof
A `oneof` **may** be used to introduce a type union: the user or service is
able to specify one of the fields inside the `oneof`. Additionally, a `oneof`
**may** be used with the same type (usually strings) to represent a semantic
difference between the options.
Because the individual fields in the `oneof` have different keys, a developer
can programmatically determine which (if any) of the fields is populated.
A `oneof` preserves the largest degree of type safety and semantic meaning for
each option, and services **should** generally prefer them over other generic
or polymorphic options when feasible. However, the `oneof` construct is
ill-suited when there is a large (or unlimited) number of potential options, or
when there is a large resource structure that would require a long series of
"cascading oneofs".
**Note:** Adding additional possible fields to an existing `oneof` is a
non-breaking change, but moving existing fields into or out of a `oneof` is
breaking (it creates a backwards-incompatible change in Go protobuf stubs).
### Maps
Maps **may** be used in situations where many values _of the
same type_ are needed, but the keys are unknown or user-determined.
Maps are usually not appropriate for generic fields because the map values all
share a type, but occasionally they are useful. In particular, a map can
sometimes be suited to a situation where many objects of the same type are
needed, with different behavior based on the names of their keys (for example,
using keys as environment names).
### Struct
The [`google.protobuf.Struct`][struct] object **may** be used to represent arbitrary
nested JSON. Keys can be strings, and values can be floats, strings, booleans,
arrays, or additional nested structs, allowing for an arbitrarily nested
structure that can be represented as JSON (and is automatically represented as
JSON when using REST/JSON).
A `Struct` is most useful when the service does not know the schema in advance,
or when a service needs to store and retrieve arbitrary but structured user
data. Using a `Struct` is convenient for users in this case because they can
easily get JSON objects that can be natively manipulated in their environment
of choice.
If a service needs to reason about the _schema_ of a `Struct`, it **should**
use [JSONSchema][] for this purpose. Because JSONSchema is itself JSON, a valid
JSONSchema document can itself be stored in a `Struct`.
### Any
The [`google.protobuf.Any`][any] object can be used to send an arbitrary
serialized protocol buffer and a type definition.
However, this introduces complexity, because an `Any` becomes useless for any
task other than blind data propagation if the consumer does not have access to
the proto. Additionally, even if the consumer _does_ have the proto, the
consumer has to ensure the type is registered and then deserialize manually,
which is an often-unfamiliar process.
Because of this, `Any` **should not** be used unless other options are
infeasible.
<!-- prettier-ignore-start -->
[any]: https://github.com/protocolbuffers/protobuf/tree/master/src/google/protobuf/any.proto
[struct]: https://github.com/protocolbuffers/protobuf/tree/master/src/google/protobuf/struct.proto
[JSONSchema]: https://json-schema.org/
<!-- prettier-ignore-end -->
---
id: 147
state: approved
created: 2020-07-24
placement:
category: fields
order: 80
---
# Sensitive fields
Sometimes APIs need to collect sensitive information such as private encryption
keys meant to be _stored_ by the underlying service but not intended to be
_read_ after writing due to the sensitive nature of the data. For this type of
data, extra consideration is required for the representation of the sensitive
data in API requests and responses.
## Guidance
If the sensitive information is _required_ for the resource as a whole to
exist, the data **should** be accepted as an [input-only field][input-only]
with no corresponding output field. Because the sensitive data must be present
for the resource to exist, users of the API may assume that existence of the
resource implies storage of the sensitive data. For example:
```proto
message SelfManagedKeypair {
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// The public key data in PEM-encoded form.
bytes public_key = 2;
// The private key data in PEM-encoded form.
bytes private_key = 3 [
(google.api.field_behavior) = INPUT_ONLY];
}
```
If the sensitive information is _optional_ within the containing resource, an
[output-only][] boolean field with a postfix of `_set` **should** be used to
indicate whether or not the sensitive information is present. For example:
```proto
message Integration {
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
string uri = 2;
// A secret to be passed in the `Authorization` header of the webhook.
string shared_secret = 3 [
(google.api.field_behavior) = INPUT_ONLY];
// True if a `shared_secret` has been set for this Integration.
bool shared_secret_set = 4 [
(google.api.field_behavior) = OUTPUT_ONLY];
}
```
If it is important to be able to identify the sensitive information without
allowing it to be read back entirely, a field of the same type with an
`obfuscated_` prefix **may** be used instead of the boolean `_set` field to
provide contextual information about the sensitive information. The specific
nature of the obfuscation is outside the scope of this AIP. For example:
```proto
message AccountRecoverySettings {
// An email to use for account recovery.
string email = 1 [
(google.api.field_behavior) = INPUT_ONLY];
// An obfuscated representation of the recovery email. For example,
// `ada@example.com` might be represented as `a**@e*****e.com`.
string obfuscated_email = 2 [
(google.api.field_behavior) = OUTPUT_ONLY];
}
```
<!-- prettier-ignore-start -->
[input-only]: ./0203.md#input-only
[output-only]: ./0203.md#output-only
<!-- prettier-ignore-end -->
---
id: 148
state: approved
created: 2020-10-06
placement:
category: fields
order: 90
---
# Standard fields
Certain concepts are common throughout any corpus of APIs. In these situations,
it is useful to have a standard field name and behavior that is used
consistently to communicate that concept.
## Guidance
Standard fields **should** be used to describe their corresponding concept, and
**should not** be used for any other purpose.
### Resource names and IDs
#### name
Every resource **must** have a `string name` field, used for the resource name
(AIP-122), which **should** be the first field in the resource.
**Note:** The `_name` suffix **should not** be used to describe other types of
names unless otherwise covered in this AIP.
#### parent
The `string parent` field refers to the resource name of the parent of a
collection, and **should** be used in most `List` (AIP-132) and `Create`
(AIP-133) requests.
### Other names
#### display_name
The `string display_name` field **must** be a mutable, user-settable field
where the user can provide a human-readable name to be used in user interfaces.
[Declarative-friendly resources][] **should** include this field.
Display names **should not** have uniqueness requirements, and **should** be
limited to <= 63 characters.
#### title
The `string title` field **should** be the official name of an entity, such as
a company's name. This is a more formal variant of `string display_name`.
#### given_name
The `string given_name` field **must** refer to a human or animal's given name.
Resources **must not** use `first_name` for this concept, because the given
name is not placed first in many cultures.
#### family_name
The `string family_name` field **must** refer to a human or animal's family
name. Resources **must not** use `last_name` for this concept, because the
family name is not placed last in many cultures.
### Timestamps
#### create_time
The output only `google.protobuf.Timestamp create_time` field **must**
represent the timestamp when the resource was created. This **may** be either
the time creation was initiated or the time it was completed.
[Declarative-friendly resources][] **should** include this field.
#### update_time
The output only `google.protobuf.Timestamp update_time` field **must**
represent the timestamp when the resource was most recently updated. Any change
to the resource made by users **must** refresh this value; changes to a
resource made internally by the service **may** refresh this value.
[Declarative-friendly resources][] **should** include this field.
#### delete_time
The output only `google.protobuf.Timestamp delete_time` field **must** represent
the timestamp that a resource was soft deleted. This **may** correspond to either
the time when the user requested deletion, or when the service successfully
soft deleted the resource. If a resource is not soft deleted, the `delete_time`
field **must** be empty.
Resources that support soft delete (AIP-164) **should** provide this field.
#### expire_time
The `google.protobuf.Timestamp expire_time` field **should** represent the time
that a given resource or resource attribute is no longer useful or valid (e.g. a
rotating security key). It **may** be used for similar forms of expiration as
described in AIP-214.
Services **may** provide an `expire_time` value that is inexact, but the
resource **must not** expire before that time.
#### purge_time
The `google.protobuf.Timestamp purge_time` field **should** represent the time
when a soft deleted resource will be purged from the system (see AIP-164).
It **may** be used for similar forms of expiration as described in AIP-214.
Resources that support soft delete **should** include this field.
Services **may** provide a `purge_time` value that is inexact, but the resource
**must not** be purged from the system before that time.
### Annotations
To store small amounts of arbitrary data, a `map<string, string> annotations`
field **may** be added.
The `annotations` field **must** use the [Kubernetes limits][] to maintain wire
compatibility, and **should** require dot-namespaced annotation keys to prevent
tools from trampling over one another.
Examples of information that might be valuable to store in annotations include:
- For CI/CD, an identifier of the pipeline run or version control identifier
used to propagate.
**Note:** Annotations are distinct from various forms of labels. Labels can be
used by server-side policies, such as IAM conditions. Annotations exist to
allow client tools to store their own state information without requiring a
database.
### Well known string fields
#### IP address
A field that represents an IP address **must** comply with the following:
* use type `string`
* use the name `ip_address` or end with the suffix `_ip_address` e.g.
`resolved_ip_address`
* specify the IP address version format via one of the supported formats `IPV4`,
`IPV6`, or if it can be either, `IPV4_OR_IPV6` (see [AIP-202][aip-202])
#### uid
The output only `string uid` field refers to a system-assigned unique
identifier for a resource. When provided, this field **must** be a [UUID4][]
and **must** specify this format via the `UUID4` format extension (see
[AIP-202][aip-202]). [Declarative-friendly resources][] **should** include this
field.
## Further reading
- For standardized codes, see AIP-143.
- For the `etag` field, see AIP-154.
- For the `request_id` field, see AIP-155.
- For the `filter` field, see AIP-160.
- For fields related to resource revisions, see AIP-162.
- For the `validate_only` field, see AIP-163.
- For fields related to soft delete and undelete, see AIP-164.
## Rationale
### Well known string fields
Some fields represent very well defined concepts or artifacts that sometimes
also have strict governance of their semantics. For such fields, presenting an
equally standardized API surface is important. This enables development of
improved API consumer tools and documentation, as well as a more unified user
experience across the platform.
## History
Before 2023-07, `purge_time` for soft-deleted resources was also called
`expire_time`. `purge_time` was introduced to reduce user confusion.
## Changelog
- **2023-10-05**: Introduce well known string fields with IP Address and `uid`.
- **2023-08-14**: Introduce the term `annotations` from AIP-128.
- **2023-07-13**: Introduce the term `purge_time`.
- **2021-04-06**: Require output only field behavior for `uid` and `delete_time`
fields.
<!-- prettier-ignore -->
[aip-202]: ./0202.md
[declarative-friendly resources]: ./0128.md#resources
[kubernetes limits]: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/#syntax-and-character-set
[uuid4]: https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random)---
id: 149
state: approved
created: 2021-02-12
placement:
category: fields
order: 100
---
# Unset field values
In many messages, many fields are optional: the user is not required to provide
them, or for output fields, the service might not populate the field.
In most cases, there is no meaningful difference between setting it to a
default value (such as `0`) as opposed to not setting it at all; however,
occasionally this distinction is meaningful.
## Guidance
Services defined in protocol buffers **should** use the `optional` keyword for
primitives if and only if it is necessary to distinguish setting the field to
its default value (`0`, `false`, or empty string) from not setting it at all:
```proto
// A representation of a book in a library.
message Book {
option (google.api.resource) = {
type: "library.googleapis.com/Book"
pattern: "publishers/{publisher}/books/{book}"
};
// The name of the book.
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// The rating for the book, from 0 to 5.
// 0 is distinct from no rating.
optional int32 rating = 2;
}
```
**Important:** Services **should not** need to distinguish between the default
value and unset most of the time; if an alternative design does not require
such a distinction, it is usually preferred. In practice, this means `optional`
**should** only ever be used for integers and floats.
**Important:** Tracking field presence is *not* the same as documenting API
field behavior as defined in [AIP-203][]. For example, a field labeled with
`optional` for presence tracking **may** also be annotated as
`google.api.field_behavior = REQUIRED` if the field must be set. If you only
want to document the server perceived behavior of a field, read [AIP-203][].
### Backwards compatibility
It is a backwards incompatible change to add or remove the `optional` qualifier
to an existing field. This is because the compiled src API is changed (in some
languages). For example, in Golang, adding `optional` changes the field type of
primitives to be the pointer variant of their original type, e.g. a field
formerly of type `string` becomes `*string`, etc. Accordingly, this change
requires that both clients and servers update their usage of the changed field
in unison, which is risky and error prone. Additional information is
[documented by Protobuf][proto docs].
## Rationale
### field behavior and `optional`
The field behavior annotation and `optional` label are not mutually exclusive,
because they address different problems. The former,
`google.api.field_behavior`, focuses on communicating the server's perception of
a field within the API e.g. if it is required or not, if it is immutable, etc.
The latter, proto3's `optional`, is a wire format and code generation option
that is strictly for toggling field presence tracking. While it might be
confusing for a field to be simultaneously annotated with
`google.api.field_behavior = REQUIRED` and labeled as `optional`, they are
unrelated in practice and can reasonably be used together.
## Changelog
- **2024-06-05**: Add backwards compatibility considerations
- **2023-06-20**: Differentiate from field behavior documentation
[AIP-203]: ./0203.md
[proto docs]: https://protobuf.dev/programming-guides/field_presence/#considerations-for-change-compatibility
---
id: 151
state: approved
created: 2019-07-25
placement:
category: operations
order: 110
---
# Long-running operations
Occasionally, an API may need to expose a method that takes a significant
amount of time to complete. In these situations, it is often a poor user
experience to simply block while the task runs; rather, it is better to return
some kind of promise to the user and allow the user to check back in later.
The long-running operations pattern is roughly analogous to a [Python
Future][], or a [Node.js Promise][]. Essentially, the user is given a token
that can be used to track progress and retrieve the result.
## Guidance
Individual API methods that might take a significant amount of time to complete
**should** return a [`google.longrunning.Operation`][lro] object instead of the
ultimate response message.
```proto
// Create a book.
rpc CreateBook(CreateBookRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books"
body: "book"
};
option (google.longrunning.operation_info) = {
response_type: "Book"
metadata_type: "OperationMetadata"
};
}
```
- The response type **must** be `google.longrunning.Operation`. The `Operation`
proto definition **must not** be copied into individual APIs.
- The response **must not** be a streaming response.
- The method **must** include a `google.longrunning.operation_info` annotation,
which **must** define both response and metadata types.
- The response and metadata types **must** be defined in the file where the
RPC appears, or a file imported by that file.
- If the response and metadata types are defined in another package, the
fully-qualified message name **must** be used.
- The response type **should not** be `google.protobuf.Empty` (except for
[`Delete`][aip-135] methods), unless it is certain that response data will
_never_ be needed. If response data might be added in the future, define an
empty message for the RPC response and use that.
- The metadata type is used to provide information such as progress, partial
failures, and similar information on each `GetOperation` call. The metadata
type **should not** be `google.protobuf.Empty`, unless it is certain that
metadata will _never_ be needed. If metadata might be added in the future,
define an empty message for the RPC metadata and use that.
- APIs with messages that return `Operation` **must** implement the
[`Operations`][lro] service. Individual APIs **must not** define their own
interfaces for long-running operations to avoid non-uniformity.
- If an RPC supports a [validate-only mode][aip-163], the response to a
validation request **must** be one of the following:
- A successful response with an `Operation` which is already complete, with
the `done` field set to `true`, and a valid (but potentially empty) response
message in the `response` field, wrapped in a `google.protobuf.Any` message.
The `name` field **may** be empty, to avoid the service having to maintain
state for successful validation.
- An immediate error response (typically "bad request")
- An `Operation` with the `done` field set to `false`, to indicate
long-running validation. In this case, the `name` field **must** be set,
to allow clients to poll the long-running validation operation until it
has completed. Successful validation **must** eventually be represented by
an operation with `done=true` and a valid (but potentially empty) wrapped
response message in the `response` field. Unsuccessful validation **must**
eventually be represented by an operation with `done=true` and the error
details provided in the `error` field.
**Note:** User expectations can vary on what is considered "a significant
amount of time" depending on what work is being done. A good rule of thumb is
10 seconds.
### Standard methods
APIs **may** return an `Operation` from the [`Create`][aip-133],
[`Update`][aip-134], or [`Delete`][aip-135] standard methods if appropriate. In
this case, the response type in the `operation_info` annotation **must** be the
standard and expected response type for that standard method.
When creating or deleting a resource with a long-running operation, the
resource **should** be included in [`List`][aip-132] and [`Get`][aip-131]
calls; however, the resource **should** indicate that it is not usable,
generally with a [state enum][aip-216].
### Parallel operations
A resource **may** accept multiple operations that will work on it in parallel,
but is not obligated to do so:
- Resources that accept multiple parallel operations **may** place them in a
queue rather than work on the operations simultaneously.
- Resources that do not permit multiple operations in parallel (denying any
new operation until the one that is in progress finishes) **must** return
`ABORTED` if a user attempts a parallel operation, and include an error
message explaining the situation.
- Resources with [declarative-friendly APIs][aip-128] **may** allow subsequent
updates to preempt existing operations. In this case, the latest update
begins processing and previous operations are marked as `ABORTED` with an
error message explaining the situation.
### Expiration
APIs **may** allow their operation resources to expire after sufficient time
has elapsed after the operation completed.
**Note:** A good rule of thumb for operation expiry is 30 days.
### Errors
Errors that prevent a long-running operation from _starting_ **must** return an
error response ([AIP-193][]), similar to any other method.
Operations that fail during their execution phase **must** return an
error response ([AIP-193][]), placed in the `Operation.error` [google.rpc.Status][]
field.
Non-terminal errors that occur over the course of an operation **may** be placed
in the metadata message and the field(s) **must** be [AIP-193][] compliant
[google.rpc.Status][].
### Backwards compatibility
Changing either the `response_type` or `metadata_type` of a long-running operation
is a breaking change.
## Rationale
### Validate-only behavior
The guidance for validate-only responses comes from a tension
between clients, which benefit from "fully formed" operations that can
be treated uniformly, and servers, which don't wish to maintain
additional state for trivial operations. It seems counterintuitive
that just validating a request should generate more state, but a
full operation response that can be fetched later would either
require that or "special" singleton operation IDs. The guidance
provided is a compromise: by returning a "done" operation, clients
can use existing logic to check that the operation has completed
successfully (and therefore doesn't need to be fetched for an
updated status) but server don't need to maintain any additional state.
<!-- prettier-ignore-start -->
[aip-128]: ./0128.md
[aip-131]: ./0131.md
[aip-132]: ./0132.md
[aip-133]: ./0133.md
[aip-134]: ./0134.md
[aip-135]: ./0135.md
[aip-163]: ./0163.md
[aip-193]: ./0193.md
[aip-216]: ./0216.md
[google.rpc.Status]: https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto
[lro]: https://github.com/googleapis/googleapis/blob/master/google/longrunning/operations.proto
[node.js promise]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Using_promises
[python future]: https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Future
<!-- prettier-ignore-end -->
## Changelog
- **2025-02-04**: Clarified error propagation behavior for failures
that occur during long-running operations.
- **2024-04-23**: Provided pattern for validation on RPCs returning
long-running operations.
- **2022-05-31**: Added compatibility section.
- **2020-08-24**: Clarified that responses are not streaming responses.
- **2020-06-24**: Added guidance for parallel operations.
- **2020-03-20**: Clarified that both `response_type` and `metadata_type` are
required.
- **2019-11-22**: Added a short explanation of what `metadata_type` is for.
- **2019-09-23**: Added guidance on errors.
- **2019-08-23**: Added guidance about fully-qualified message names when the
message name is in another package.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
---
id: 152
state: approved
created: 2020-04-27
updated: 2022-06-02
placement:
category: design-patterns
order: 10
---
# Jobs
Occasionally, APIs may need to expose a task that takes significant time to
complete, and where a transient [long-running operation][aip-151] is not
appropriate. For example, a task could need to run repeatedly, or have separate
permissions for configuring the task as opposed to running it.
## Guidance
An API **may** define a `Job` resource to represent a particular task with
distinct setup, configuration, and execution:
```proto
message WriteBookJob {
option (google.api.resource) = {
type: "library.googleapis.com/WriteBookJob"
pattern: "publishers/{publisher}/writeBookJobs/{write_book_job}"
};
// Name and other fields...
}
```
- The name of the resource **must** end with the word "Job".
- The prefix **should** be a valid RPC name, with a verb and a noun.
- The service **should** define all five of the standard methods (AIP-131,
AIP-132, AIP-133, AIP-134, AIP-135), and use them as the
primary way to configure the job.
### Run method
The service **should** define a `Run` custom method that executes the job
immediately:
```proto
rpc RunWriteBookJob(RunWriteBookJobRequest)
returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{name=publishers/*/writeBookJobs/*}:run"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "RunWriteBookJobResponse"
metadata_type: "RunWriteBookJobMetadata"
};
}
```
- The RPC's name **must** begin with the word `Run`. The remainder of the
RPC name **should** be the singular form of the job resource being run.
- The request message **must** match the RPC name, with a `Request` suffix.
- The method **should** return a [long-running operation][aip-151], which
**must** resolve to a response message that includes the result of running
the job.
- The response message name must match the RPC name, with a `Response`
suffix.
- The method **may** use any metadata message it wishes.
- The HTTP verb **must** be `POST`, as is usual for [custom methods][aip-136].
- The body clause in the `google.api.http` annotation **should** be `"*"`.
- The URI path **should** contain a single `name` variable corresponding to the
name of the job resource being run.
- The URI path **must** end with `:run`.
- Errors that prevent execution of the job from _starting_ **must** return an
error response (AIP-193), similar to any other method. Errors that occur
over the course of the job execution **may** be placed in the metadata
message. The errors themselves **must** still be represented with a
[`google.rpc.Status`][status] object.
### Run request message
Run methods implement a common request message pattern:
```proto
message RunWriteBookJobRequest {
// The name of the job to run.
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/WriteBookJob"
}];
}
```
- A singular `string name` field **must** be included.
- The field **should** be [annotated as required][aip-203].
- The field **should** identify the [resource type][aip-123] that it
references.
### Executions and results
Ordinarily, the API **should** provide results to the user as the final
response of the `Run` method. However, this is sometimes insufficient; for
example, a job that runs on a recurring schedule in the background can not
deliver results to the user in this way.
The service **may** store resources representing individual executions along
with their result as a sub-collection of resources under the job, which allows
the user to list *past* job executions. A service that does this **should**
define the `Get`, `List`, and `Delete` methods for the execution resources:
```proto
message WriteBookJobExecution {
option (google.api.resource) = {
type: "library.googleapis.com/WriteBookJobExecution"
pattern: "publishers/{publisher}/writeBookJobs/{write_book_job}/executions/{execution}"
};
// Name and other information about the execution, such as metadata, the
// result, error information, etc.
}
```
In this case, the operation returned by job's `Run` method **should** refer to
the child resource.
## Changelog
- **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-".
- **2020-11-02**: Expanded guidance on HTTP, field behavior, and resource
reference annotations and request format.
<!-- prettier-ignore-start -->
[aip-123]: ./0123.md
[aip-136]: ./0136.md
[aip-151]: ./0151.md
[aip-203]: ./0203.md
[status]: https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto
<!-- prettier-ignore-end -->
---
id: 153
state: approved
created: 2019-12-16
placement:
category: design-patterns
order: 20
---
# Import and export
Many users want to be able to load data into an API, or get their existing data
out of an API. This is particularly important for enterprise users, who are
often concerned about vendor lock-in.
## Guidance
APIs **may** support import and export operations, which **may** create
multiple new resources, or they **may** populate data into a single resource.
### Multiple resources
Services **may** support importing and exporting multiple resources into or out
of an API, and **should** implement a common pattern to do so:
```proto
rpc ImportBooks(ImportBooksRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:import"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "ImportBooksResponse"
metadata_type: "ImportBooksMetadata"
};
}
rpc ExportBooks(ExportBooksRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:export"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "ExportBooksResponse"
metadata_type: "ExportBooksMetadata"
};
}
```
- The method **must** return a long-running operation (see [AIP-151][]) unless
the service can guarantee that it will _never_ need more than a few seconds
to complete.
- The HTTP verb **must** be `POST`, and the `body` **must** be `"*"`.
- A `parent` field **should** be included as part of the URI.
- If importing into or exporting from multiple resources is required, the API
**should** keep the `parent` field and allow the user to use the `-`
character to indicate multiple parents (see [AIP-159][]).
- On import, if the user provides a specific parent, the API **must** reject
any imported resources that would be added to a different parent.
- The URI suffix **should** be `:import` or `:export`.
### Data for a single resource
Services **may** support importing and exporting data into or out of a single
resource, and **should** implement a common pattern to do so:
```proto
rpc ImportPages(ImportPagesRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{book=publishers/*/books/*}:importPages"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "ImportPagesResponse"
metadata_type: "ImportPagesMetadata"
};
}
rpc ExportPages(ExportPagesRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{book=publishers/*/books/*}:exportPages"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "ExportPagesResponse"
metadata_type: "ExportPagesMetadata"
};
}
```
- The method **must** return a long-running operation (see [AIP-151][]) unless
the service can guarantee that it will _never_ need more than a few seconds
to complete.
- The HTTP verb **must** be `POST`, and the `body` **must** be `"*"`.
- A field representing the resource that data is being imported into **should**
be included as part of the URI. The field **should** be named after the
resource (and **should not** be called `name`).
- The URI suffix should include both the verb and a noun for the data itself,
such as `:importPages` or `:exportPages`.
### Request object
Imports and exports often require two fundamentally different types of
configuration:
1. Configuration specific to the source or destination.
2. Configuration regarding the imported or exported data itself.
Source or destination configuration should be grouped into a single message and
placed inside a oneof:
```proto
message ImportBooksRequest {
string parent = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
oneof source {
AuthorSource author_source = 2;
TranslatorSource translator_source = 3;
}
string isbn_prefix = 4;
}
message ExportBooksRequest {
string parent = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
oneof destination {
PrinterDestination printer_destination = 2;
TranslatorDestination translator_destination = 3;
}
string filter = 4;
}
```
- The source configuration messages **must** be placed within a `oneof source`
(for import) or `oneof destination` (for export), even if there is only one.
(This maintains flexibility to add more later.)
- Configuration related to the data itself (and therefore common across all
sources) **must** be placed at the top-level of the request message.
**Note:** The configuration for import and export **may** be different from one
another. (For example, it would be sensible to import from a file but export to
a directory.)
### Inline sources
APIs **may** also permit import and export "inline", where the contents to be
imported or exported are provided in the request or response.
```proto
message InlineSource {
repeated Book books = 1;
}
```
- The source or destination **should** be named `InlineSource` or
`InlineDestination`.
- The message **should** include a repeated field representing the resource.
However, if the resource structure is complex, the API **may** use a separate
inline representation. In this situation, the same format **must** be used
for both import and export.
### Partial failures
While partial failures are normally discouraged, import and export RPCs
**should** include partial failure information in the metadata object. Each
individual error **should** be a `google.rpc.Status` object describing the
error. For more on errors, see [AIP-193][].
[aip-151]: ./0151.md
[aip-159]: ./0159.md
[aip-193]: ./0193.md
[google bigquery]: https://cloud.google.com/bigquery
[google storage]: https://cloud.google.com/storage
---
id: 154
state: approved
created: 2019-07-24
placement:
category: design-patterns
order: 30
---
# Resource freshness validation
APIs often need to validate that a client and server agree on the current state
of a resource before taking some kind of action on that resource. For example,
two processes updating the same resource in parallel could create a race
condition, where the latter process "stomps over" the effort of the former one.
ETags provide a way to deal with this, by allowing the server to send a
checksum based on the current content of a resource; when the client sends that
checksum back, the server can ensure that the checksums match before acting on
the request.
## Guidance
A resource **may** include an `etag` field on any resource where it is
important to ensure that the client has an up to date resource before acting on
certain requests:
```proto
// A representation of a book.
message Book {
// Other fields...
// This checksum is computed by the server based on the value of other
// fields, and may be sent on update and delete requests to ensure the
// client has an up-to-date value before proceeding.
string etag = 99;
}
```
- The etag field **must** be a string, and **must** be named `etag`.
- The etag field on the _resource_ **should not** be given any behavior
annotations.
- The etag field **must** be provided by the server on output, and values
**should** conform to [RFC 7232][].
- If a user sends back an etag which matches the current etag value, the
service **must** permit the request (unless there is some other reason for
failure).
- If a user sends back an etag which does not match the current etag value, the
service **must** send an `ABORTED` error response (unless another error takes
precedence, such as `PERMISSION_DENIED` if the user is not authorized).
- If the user does not send an etag value at all, the service **should** permit
the request. However, services with strong consistency or parallelism
requirements **may** require users to send etags all the time and reject the
request with an `INVALID_ARGUMENT` error in this case.
**Note:** ETag values **should** include quotes as described in [RFC 7232][].
For example, a valid etag is `"foo"`, not `foo`.
### Declarative-friendly resources
A resource that is declarative-friendly (AIP-128) **must** include an `etag`
field.
### Etags on request methods
In some situations, the etag needs to belong on a request message rather than
the resource itself. For example, an `Update` standard method can "piggyback"
off the `etag` field on the resource, but the `Delete` standard method can not:
```proto
message DeleteBookRequest {
// The name of the book.
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
// The current etag of the book.
// If an etag is provided and does not match the current etag of the book,
// deletion will be blocked and an ABORTED error will be returned.
string etag = 2 [(google.api.field_behavior) = OPTIONAL];
}
```
On a request message, the `etag` field **should** be given a behavior annotation
- either `REQUIRED` or `OPTIONAL`. See AIP-203 for more information.
An `etag` field **may** also be used on custom methods, similar to the example
above.
### Strong and weak etags
ETags can be either "strongly validated" or "weakly validated":
- A strongly validated etag means that two resources bearing the same etag are
byte-for-byte identical.
- A weakly validated etag means that two resources bearing the same etag are
equivalent, but may differ in ways that the service does not consider to be
important.
Resources **may** use either strong or weak etags, as it sees fit, but
**should** document the behavior. Additionally, weak etags **must** have a `W/`
prefix as mandated by [RFC 7232][].
[rfc 7232]: https://tools.ietf.org/html/rfc7232#section-2.3
## Further reading
- For how to retry on errors in client libraries, see AIP-194.
## Changelog
- **2021-04-01**: Updated an additional reference to `FAILED_PRECONDITION`
to `ABORTED`.
- **2021-03-05**: Changed the etag error from `FAILED_PRECONDITION` (which
becomes HTTP 400) to `ABORTED` (409).
- **2020-10-06**: Added declarative-friendly resource requirement.
- **2020-09-02**: Clarified that other errors may take precedence over
`FAILED_PRECONDITION` for etag mismatches.
- **2020-09-02**: Add guidance for etags on request messages.
- **2019-09-23**: Changed the title to "resource freshness validation".
---
id: 155
state: approved
created: 2019-05-06
placement:
category: design-patterns
order: 40
---
# Request identification
It is sometimes useful for an API to have a unique, customer-provided
identifier for particular requests. This can be useful for several purposes,
such as de-duplicating requests from parallel processes, ensuring the safety of
retries, or auditing.
The most important purpose for request IDs is to provide idempotency
guarantees: allowing the same request to be issued more than once without
subsequent calls having any effect. In the event of a network failure, the
client can retry the request, and the server can detect duplication and ensure
that the request is only processed once.
## Guidance
APIs **may** add a `string request_id` parameter to request messages (including
those of standard methods) in order to uniquely identify particular requests.
```proto
message CreateBookRequest {
// The parent resource where this book will be created.
// Format: publishers/{publisher}
string parent = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The ID to use for the book, which will become the final component of
// the book's resource name.
//
// This value should be 4-63 characters, and valid characters
// are /[a-z][0-9]-/.
string book_id = 2 [(google.api.field_behavior) = REQUIRED];
// The book to create.
Book book = 3 [(google.api.field_behavior) = REQUIRED];
// A unique identifier for this request. Restricted to 36 ASCII characters.
// A random UUID is recommended.
// This request is only idempotent if a `request_id` is provided.
string request_id = 4 [(google.api.field_info).format = UUID4];
}
```
- Providing a request ID **must** guarantee idempotency.
- If a duplicate request is detected, the server **should** return the
response for the previously successful request, because the client most
likely did not receive the previous response.
- APIs **may** choose any reasonable timeframe for honoring request IDs.
- The `request_id` field **must** be provided on the request message to which
it applies (and it **must not** be a field on resources themselves).
- Request IDs **should** be optional.
- Request IDs **should** be able to be UUIDs, and **may** allow UUIDs to be the
only valid format. The format restrictions for request IDs **must** be
documented.
- Request IDs that are UUIDs **must** be annotated with the
`google.api.FieldInfo.Format` value `UUID4` using the extension
`(google.api.field_info).format = UUID4`. See [AIP-202](./0202.md) for more.
### Stale success responses
In some unusual situations, it may not be possible to return an identical
success response. For example, a duplicate request to create a resource may
arrive after the resource has not only been created, but subsequently updated;
because the service has no other need to retain the historical data, it is no
longer feasible to return an identical success response.
In this situation, the method **may** return the current state of the resource
instead. In other words, it is permissible to substitute the historical success
response with a similar response that reflects more current data.
## Further reading
- For which codes to retry, see [AIP-194](https://aip.dev/194).
- For how to retry errors in client libraries, see
[AIP-4221](https://aip.dev/4221).
## Rationale
### Using UUIDs for request identification
When a value is required to be unique, leaving the format open-ended can lead to
API consumers incorrectly providing a duplicate identifier. As such,
standardizing on a universally unique identifier drastically reduces the chance
for collisions when done correctly.
## Changelog
- **2024-01-08**: Add book_id to request message.
- **2023-10-02**: Add UUID format extension guidance.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
---
id: 156
state: approved
created: 2019-05-12
placement:
category: resource-design
order: 70
---
# Singleton resources
APIs sometimes need to represent a resource where exactly one instance of the
resource always exists within any given parent. A common use case for this is
for a config object.
## Guidance
An API **may** define _singleton resources_. A singleton resource **must**
always exist by virtue of the existence of its parent, with one and exactly one
per parent.
For example:
```proto
message Config {
option (google.api.resource) = {
type: "api.googleapis.com/Config"
pattern: "users/{user}/config"
singular: "config"
plural: "configs"
};
// additional fields including name
}
```
The `Config` singleton would have the following RPCs:
```proto
rpc GetConfig(GetConfigRequest) returns (Config) {
option (google.api.http) = {
get: "/v1/{name=users/*/config}"
};
}
rpc UpdateConfig(UpdateConfigRequest) returns (Config) {
option (google.api.http) = {
patch: "/v1/{config.name=users/*/config}"
body: "config"
};
}
```
- Singleton resources **must not** have a user-provided or system-generated ID;
their [resource name][aip-122] includes the name of their parent followed by
one static-segment.
- Example: `users/1234/config`
- Singleton resources are always singular.
- Example: `users/1234/thing`
- Singleton resource definitions **must** provide both the `singular` and
`plural` fields (see above example).
- Singleton resources **may** parent other resources.
- Singleton resources **must not** define the [`Create`][aip-133] or
[`Delete`][aip-135] standard methods. The singleton is implicitly created or
deleted when its parent is created or deleted.
- Singleton resources **should** define the [`Get`][aip-131] and
[`Update`][aip-134] methods, and **may** define custom methods as
appropriate.
- However, singleton resources **must not** define the [`Update`][aip-134]
method if all fields on the resource are [output only][aip-203].
- Singleton resources **may** define the [`List`][aip-132] method, but **must**
implement it according to [AIP-159][aip-159]. See the example below.
- The trailing segment in the path pattern that typically represents the
collection **should** be the `plural` form of the Singleton resource e.g.
`/v1/{parent=users/*}/configs`.
- If a parent resource ID is provided instead of the hyphen `-` as per
AIP-159, then the service **should** return a collection of one Singleton
resource corresponding to the specified parent resource.
```proto
rpc ListConfigs(ListConfigsRequest) returns (ListConfigsResponse) {
option (google.api.http) = {
get: "/v1/{parent=users/*}/configs"
};
}
message ListConfigsRequest {
// To list all configs, use `-` as the user id.
// Formats:
// * `users/-`
// * `users/{user}`
//
// Note: Specifying an actual user id will return a collection of one config.
// Use GetConfig instead.
string parent = 1 [
(google.api.resource_reference).child_type = "api.googleapis.com/Config"];
// other standard pagination fields...
}
```
## Rationale
### Support for Standard List
While Singleton resources are not directly part of a collection themselves, they
can be viewed as part of their parent's collection. The one-to-one relationship
of parent-to-singleton means that for every one parent there is one singleton
instance, naturally enabling some collection-based methods when combined with
the pattern of [Reading Across Collections][aip-159]. The Singleton can present
as a collection to the API consumer as it is indirectly one based on its parent.
Furthermore, presenting the Singleton resource as a pseudo-collection in such
methods enables future expansion to a real collection, should a Singleton be
found lacking.
### Including `plural` definition
While a Singleton is by definition singular, there are certain cases where
a Singleton resource may appear in a plural form e.g., if the service supports
Standard List (as defined here). As such, it is better to forward declare the
plural form of the Singleton resource type than to not have it when needed.
## Changelog
- **2024-04-15:** Singletons must specify `singular` and `plural` in resource.
- **2023-08-10:** Add Standard `List` support.
- **2023-07-26:** Clarified that read-only singletons should not have `Update`.
- **2021-11-02:** Added an example message and state parent eligibility.
- **2021-01-14:** Changed example from `settings` to `config` for clarity.
[aip-122]: ./0122.md
[aip-131]: ./0131.md
[aip-132]: ./0132.md
[aip-133]: ./0133.md
[aip-134]: ./0134.md
[aip-135]: ./0135.md
[aip-159]: ./0159.md
[aip-203]: ./0203.md#output-only
---
id: 157
state: approved
created: 2019-01-26
placement:
category: design-patterns
order: 50
---
# Partial responses
Sometimes, a resource can be either large or expensive to compute, and the API
needs to give the user control over which fields it sends back.
## Guidance
APIs **may** support partial responses in one of two ways:
### Field masks parameter
Field masks (`google.protobuf.FieldMask`) can be used for granting the user
fine-grained control over what fields are returned. An API **should** support the mask in a side channel.
For example, the parameter can be specified either using an HTTP query
parameter, an HTTP header, or a [gRPC metadata entry][1]. Google Cloud APIs specify field masks as a [system parameter][0].
Field masks **should not** be specified in the [request](./0157.md#read-masks-as-a-request-field).
- The value of the field mask parameter **must** be a `google.protobuf.FieldMask`.
- The field mask parameter **must** be optional:
- An explicit value of `"*"` **should** be supported, and **must** return all
fields.
- If the field mask parameter is omitted, it **must** default to `"*"`, unless otherwise documented.
- An API **may** allow read masks with non-terminal repeated fields (unlike
update masks), but is not obligated to do so.
**Note:** Changing the default value of the field mask parameter is a [breaking change](../0180.md#semantic-changes).
### View enumeration
Alternatively, an API **may** support partial responses with view enums.
View enums are useful for situations where an API only wants to expose a small
number of permutations to the user:
```proto
enum BookView {
// The default / unset value.
// The API will default to the BASIC view.
BOOK_VIEW_UNSPECIFIED = 0;
// Include basic metadata about the book, but not the full contents.
// This is the default value (for both ListBooks and GetBook).
BOOK_VIEW_BASIC = 1;
// Include everything.
BOOK_VIEW_FULL = 2;
}
```
- The enum **should** be specified as a `view` field on the request message.
- The enum **should** be named something ending in `-View`
- The enum **should** at minimum have values named `BASIC` and `FULL` (although
it **may** have values other than these).
- The `UNSPECIFIED` value **must** be valid (not an error), and the API
**must** document what the unspecified value will do.
- For List RPCs, the effective default value **should** be `BASIC`.
- For the following RPC types, the effective default value **should** be
either `BASIC` or `FULL`:
- Get
- Create
- Update
- Soft Delete
- Custom Method
- The enum **should** be defined at the top level of the proto file (as it is
likely to be needed in multiple requests, e.g. both `Get` and `List`). See
[AIP-126][] for more guidance on top-level enumerations.
- APIs **may** add fields to a given view over time. APIs **must not** remove a
field from a given view (this is a breaking change).
**Note:** If a service requires (or might require) multiple views with
overlapping but distinct values, there is a potential for a namespace
conflict. In this situation, the service **should** nest the view enum within
the individual resource.
**Note:** Having a partial response be the default of standard methods can
degrade the effectiveness of declarative clients. Providing a mechanism to
request the full resource be populated in the response, like this View
pattern, is preferred if partial responses are deemed necessary.
### Read masks as a request field
**Warning:** Read mask as a single, explicit field on the request message is
**DEPRECATED** for Google APIs. The [system parameter](#field-masks-parameter)
**must** be used instead. The following guidance is for the benefit of existing
legacy Google and external non-Google usage.
An API **may** support read masks as a single field on the request message:
`google.protobuf.FieldMask read_mask`.
- The read mask **must** be a `google.protobuf.FieldMask` and **should** be
named `read_mask`.
- The field mask **should** be optional:
- An explicit value of `"*"` **should** be supported, and **must** return all
fields.
- If the field mask parameter is not provided, all fields **must** be
returned.
- An API **may** allow read masks with non-terminal repeated fields (unlike
update masks), but is not obligated to do so.
## Rationale
### Deprecating `read_mask` in request messages
As mentioned, Google API infrastructure implements a service-wide response
field filtering mechanism, so there is no need for individual API methods to
specify a `read_mask` in their request schema. Doing so is both redundant and a
potential point of conflict for the client or service.
## Changelog
- **2025-10-03**: Added default view guidance for other RPC types, and
declarative client warning.
- **2025-06-16**: Reinstate read mask guidance as historical/external reference.
- **2023-05-09**: Fix top-level enum example and link to AIP-126.
- **2022-03-14:** Updated guidance on default value and how to specify a read mask.
- **2021-10-06:** Updated the guidance with system parameters.
- **2021-03-04:** Added guidance for conflicting view enums.
[0]: https://cloud.google.com/apis/docs/system-parameters
[1]: https://grpc.io/docs/what-is-grpc/core-concepts/#metadata
[AIP-126]: ./0126.md---
id: 158
state: approved
created: 2019-02-18
placement:
category: design-patterns
order: 60
---
# Pagination
APIs often need to provide collections of data, most commonly in the [List][]
standard method. However, collections can often be arbitrarily sized, and also
often grow over time, increasing lookup time as well as the size of the
responses being sent over the wire. Therefore, it is important that collections
be paginated.
## Guidance
RPCs returning collections of data **must** provide pagination _at the outset_,
as it is a [backwards-incompatible change](#backwards-compatibility) to add
pagination to an existing method.
```proto
// The request structure for listing books.
message ListBooksRequest {
// The parent, which owns this collection of books.
// Format: publishers/{publisher}
string parent = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The maximum number of books to return. The service may return fewer than
// this value.
// If unspecified, at most 50 books will be returned.
// The maximum value is 1000; values above 1000 will be coerced to 1000.
int32 page_size = 2;
// A page token, received from a previous `ListBooks` call.
// Provide this to retrieve the subsequent page.
//
// When paginating, all other parameters provided to `ListBooks` must match
// the call that provided the page token.
string page_token = 3;
}
// The response structure from listing books.
message ListBooksResponse {
// The books from the specified publisher.
repeated Book books = 1;
// A token that can be sent as `page_token` to retrieve the next page.
// If this field is omitted, there are no subsequent pages.
string next_page_token = 2;
}
```
- Request messages for collections **should** define an `int32 page_size`
field, allowing users to specify the maximum number of results to return.
- The `page_size` field **must not** be required.
- If the user does not specify `page_size` (or specifies `0`), the API
chooses an appropriate default, which the API **should** document. The API
**must not** return an error.
- If the user specifies `page_size` greater than the maximum permitted by the
API, the API **should** coerce down to the maximum permitted page size.
- If the user specifies a negative value for `page_size`, the API **must**
send an `INVALID_ARGUMENT` error.
- The API **may** return fewer results than the number requested (including
zero results), even if not at the end of the collection.
- Request messages for collections **should** define a `string page_token`
field, allowing users to advance to the next page in the collection.
- The `page_token` field **must not** be required.
- If the user changes the `page_size` in a request for subsequent pages, the
service **must** honor the new page size.
- The user is expected to keep all other arguments to the RPC the same; if
any arguments are different, the API **should** send an `INVALID_ARGUMENT`
error.
- The response **must not** be a streaming response.
- Response messages for collections **should** define a
`string next_page_token` field, providing the user with a page token that may
be used to retrieve the next page.
- The field containing pagination results **should** be the first field in
the message and have a field number of `1`. It **should** be a repeated
field containing a list of resources constituting a single page of results.
- If the end of the collection has been reached, the `next_page_token` field
**must** be empty. This is the _only_ way to communicate
"end-of-collection" to users.
- If the end of the collection has not been reached (or if the API can not
determine in time), the API **must** provide a `next_page_token`.
- Response messages for collections **may** provide an `int32 total_size`
field, providing the user with the total number of items in the list.
- This total **may** be an estimate (but the API **should** explicitly
document that).
### Skipping results
The request definition for a paginated operation **may** define an `int32 skip`
field to allow the user to skip results.
The `skip` value **must** refer to the number of individual resources to skip,
not the number of pages.
For example:
- A request with no page token and a `skip` value of `30` returns a single page
of results starting with the 31st result.
- A request with a page token corresponding to the 51st result (because the
first 50 results were returned on the first page) and a `skip` value of `30`
returns a single page of results starting with the 81st result.
If a `skip` value is provided that cannot be fulfilled e.g. due to latency of
querying a massive data set, the response **must** be `200 OK` with an empty
result set. If it is _known_ to put the cursor beyond the total size of the
collection, the response **must not** include a `next_page_token`.
### Opacity
Page tokens provided by APIs **must** be opaque (but URL-safe) strings, and
**must not** be user-parseable. This is because if users are able to
deconstruct these, _they will do so_. This effectively makes the implementation
details of your API's pagination become part of the API surface, and it becomes
impossible to update those details without breaking users.
**Warning:** Base-64 encoding an otherwise-transparent page token is **not** a
sufficient obfuscation mechanism.
For page tokens which do not need to be stored in a database, and which do not
contain sensitive data, an API **may** obfuscate the page token by defining an
internal protocol buffer message with any data needed, and send the serialized
proto, base-64 encoded.
Page tokens **must** be limited to providing an indication of where to continue
the pagination process only. They **must not** provide any form of
authorization to the underlying resources, and authorization **must** be
performed on the request as with any other regardless of the presence of a page
token.
### Expiring page tokens
Many APIs store page tokens in a database internally. In this situation, APIs
**may** expire page tokens a reasonable time after they have been sent, in
order not to needlessly store large amounts of data that is unlikely to be
used. It is not necessary to document this behavior.
**Note:** While a reasonable time may vary between APIs, a good rule of thumb
is three days.
## Backwards compatibility
Adding pagination to an existing RPC is a backwards-incompatible change. This
may seem strange; adding fields to proto messages is generally backwards
compatible. However, this change is _behaviorally_ incompatible.
Consider a user whose collection has 75 resources, and who has already written
and deployed code. If the API later adds pagination fields, and sets the
default to 50, then that user's code breaks; it was getting all resources, and
now is only getting the first 50 (and does not know to advance pagination).
Even if the API set a higher default limit, such as 100, the user's collection
could grow, and _then_ the code would break.
Additionally, [client libraries implement automatic
pagination](https://google.aip.dev/client-libraries/4233), typically representing paginated
RPCs using different method signatures to unpaginated ones. This means that
adding pagination to a previously-unpaginated method causes a breaking change
in those libraries.
For this reason, it is important to always add pagination to RPCs returning
collections _up front_; they are consistently important, and they can not be
added later without causing problems for existing users.
**Warning:** This also entails that, in addition to presenting the pagination
fields, they **must** be _actually implemented_ with a non-infinite default
value. Implementing an in-memory version (which might fetch everything then
paginate) is reasonable for initially-small collections.
[list]: ./0132.md
## Rationale
### Degraded `skip` response
Large collections, complex queries, and globally distributed data can all
contribute to a paginated method being unable to quickly or confidently fulfill
a given `skip` request. Backend queries can timeout, data collation can take
time, and the end user experience need not suffer as a result. In such cases,
the pagination interface can be leveraged to keep the client engaged by
providing a `next_page_token`, while the service collects an appropriate result.
When the service has definitively determined that the requested `skip` exceeds
the available results, the pagination interface is again applied and
`next_page_token` is omitted to signal the end of results.
## Changelog
- **2025-07-08**: Clarify degraded skip response guidance
- **2020-05-24**: Clarified that adding pagination breaks client
libraries.
- **2020-05-13**: Added guidance for skipping results.
- **2020-08-24**: Clarified that responses are not streaming responses.
- **2020-06-24**: Clarified that page size is always optional for users.
- **2019-02-12**: Added guidance on the field being paginated over.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
- **2019-07-19**: Update the opacity requirement from "should" to "must".
---
id: 159
state: approved
created: 2019-07-26
placement:
category: design-patterns
order: 70
---
# Reading across collections
Sometimes, it is useful for a user to be able to retrieve resources across
multiple collections, or retrieve a single resource without needing to know
what collection it is in.
## Guidance
APIs **may** support reading resources across multiple collections by allowing
users to specify a `-` (the hyphen or dash character) as a wildcard character
in a standard [`List`][aip-132] method:
```
GET /v1/publishers/-/books?filter=...
```
- The URI pattern **must** still be specified with `*` and permit the
collection to be specified; a URI pattern **must not** hard-code the `-`
character.
- The method **must** explicitly document that this behavior is supported.
- The resources provided in the response **must** use the canonical name of the
resource, with the actual parent collection identifiers (instead of `-`).
- Services **may** support reading across collections on `List` requests
regardless of whether the identifiers of the child resources are guaranteed
to be unique. However, services **must not** support reading across
collections on `Get` requests if the child resources might have a collision.
- Cross-parent requests **should not** support `order_by`. If they do, the
field **must** document that it is best effort. This is because cross-parent
requests introduce ambiguity around ordering, especially if there is
difficulty reaching a parent (see [AIP-217][]).
**Important:** If listing across multiple collections introduces the
possibility of partial failures due to unreachable parents (such as when
listing across locations), the method **must** indicate this following the
guidance in [AIP-217][].
### Unique resource lookup
Sometimes, a resource within a sub-collection has an identifier that is unique
across parent collections. In this case, it may be useful to allow a
[`Get`][aip-131] method to retrieve that resource without knowing which parent
collection contains it. In such cases, APIs **may** allow users to specify the
wildcard collection ID `-` (the hyphen or dash character) to represent any
parent collection:
```
GET https://example.googleapis.com/v1/publishers/-/books/{book}
```
- The URI pattern **must** still be specified with `*` and permit the
collection to be specified; a URI pattern **must not** hard-code the `-`
character.
- The method **must** explicitly document that this behavior is supported.
- The resource name in the response **must** use the canonical name of the
resource, with actual parent collection identifiers (instead of `-`). For
example, the request above returns a resource with a name like
`publishers/123/books/456`, _not_ `publishers/-/books/456`.
- The resource ID **must** be unique within parent collections.
## Further reading
- For partial failures due to unreachable resources, see [AIP-217][].
[aip-131]: ./0131.md
[aip-132]: ./0132.md
[aip-217]: ./0217.md
## Changelog
- **2019-08-26**: Added a reference to guidance for unreachable resources.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
---
id: 160
state: approved
created: 2020-02-24
placement:
category: design-patterns
order: 80
---
# Filtering
Often, when listing resources (using a list method as defined in [AIP-132][] or
something reasonably similar), it is desirable to filter over the collection
and only return results that the user is interested in.
It is tempting to define a structure to handle the precise filtering needs for
each API. However, filtering requirements evolve frequently, and therefore it
is prudent to use a string field with a structured syntax accessible to a
non-technical audience. This allows updates to be able to be made
transparently, without waiting for UI or client updates.
**Note:** Because list filters are intended for a potentially non-technical
audience, they sometimes borrow from patterns of colloquial speech rather than
common patterns found in code.
## Guidance
APIs **may** provide filtering to users on `List` methods (or similar methods
to query a collection, such as `Search`). If they choose to do so, they
**should** follow the common specification for filters discussed here. The
syntax is formally defined in the [EBNF grammar][].
When employing filtering, a request message **should** have exactly one
filtering field, `string filter`. Filtering of related objects is handled
through [traversal](#traversal-operator) or [functions](#functions).
**Note:** List Filters have fuzzy matching characteristics with support for
result ranking and scoring. For developers interested in deterministic
evaluation of list filters, see [CEL][].
### Literals
A bare literal value (examples: "42", "Hugo") is a value to be matched against.
Literals appearing alone (with no specified field) **should** usually be
matched anywhere it may appear in an object's field values.
However, a service **may** choose to only consider certain fields; if so, it
**must** document which fields it considers. A service **may** include new
fields over time, but **should** do so judiciously and consider impact on
existing users.
**Note:** Literals separated by whitespace are considered to have a fuzzy
variant of `AND`. Therefore, `Victor Hugo` is roughly equivalent to
`Victor AND Hugo`.
### Logical Operators
Filtering implementations **should** provide the binary operators:
| Operator | Example | Meaning |
| -------- | ------------- | -------------------------------------- |
| `AND` | `a AND b` | True if `a` and `b` are true. |
| `OR` | `a OR b OR c` | True if any of `a`, `b`, `c` are true. |
**Note:** To match common patterns of speech, the `OR` operator has higher
precedence than `AND`, unlike what is found in most programming languages. The
expression `a AND b OR c` evaluates: `a AND (b OR c)`. API documentation and
examples **should** encourage the use of explicit parentheses to avoid
confusion, but **should not** require explicit parentheses.
### Negation Operators
Filtering implementations **should** provide the unary operators `NOT` and `-`.
These are used interchangeably, and a service that supports negation **must**
support both formats.
| Operator | Example | Meaning |
| -------- | ------- | ------------------------ |
| `NOT` | `NOT a` | True if `a` is not true. |
| `-` | `-a` | True if `a` is not true. |
### Comparison Operators
Filtering implementations **should** provide the binary comparison operators
`=`, `!=`, `<`, `>`, `<=`, and `>=` for string, numeric, timestamp, and
duration fields (but **should not** provide them for booleans or enums).
| Operator | Example | Meaning |
| -------- | ------------ | ----------------------------------------------- |
| `=` | `a = true` | True if `a` is true. |
| `!=` | `a != 42` | True unless `a` equals 42. |
| `<` | `a < 42` | True if `a` is a numeric value below 42. |
| `>` | `a > "foo"` | True if `a` is lexically ordered after "foo". |
| `<=` | `a <= "foo"` | True if `a` is "foo" or lexically before it. |
| `>=` | `a >= 42` | True if `a` is a numeric value of 42 or higher. |
**Note:** Unlike in most programming languages, field names **must** appear on
the left-hand side of a comparison operator; the right-hand side only accepts
literals and logical operators.
Because filters are accepted as query strings, type conversion takes place to
translate the string to the appropriate strongly-typed value:
- Enums expect the enum's string representation (case-sensitive).
- Booleans expect `true` and `false` literal values.
- Numbers expect the standard integer or float representations. For floats,
exponents are supported (e.g. `2.997e9`).
- [Durations][] expect a numeric representation followed by an `s` suffix (for
seconds). Examples: `20s`, `1.2s`.
- [Timestamps][] expect an [RFC-3339][] formatted string (e.g.
`2012-04-21T11:30:00-04:00`). UTC offsets are supported.
**Warning:** The identifiers `true`, `false`, and `null` only carry intrinsic
meaning when used in the context of a typed field reference.
Additionally, when comparing strings for equality, services **should** support
wildcards using the `*` character; for example, `a = "*.foo"` is true if `a`
_ends with_ ".foo".
### Traversal operator
Filtering implementations **should** provide the `.` operator, which indicates
traversal through a message, map, or struct.
| Example | Meaning |
| --------------- | ----------------------------------------------------- |
| `a.b = true` | True if `a` has a boolean `b` field that is true. |
| `a.b > 42` | True if `a` has a numeric `b` field that is above 42. |
| `a.b.c = "foo"` | True if `a.b` has a string `c` field that is "foo". |
Traversal **must** be written using the field names from the resource. If a
service wishes to support "implicit fields" of some kind, they **must** do so
through well-documented [functions](#functions). A service **may** specify a
subset of fields that are supported for traversal.
If a user attempts to traverse to a field that is not defined on the message,
the service **should** return an error with `INVALID_ARGUMENT`. A service
**may** permit traversal to undefined keys on maps and structs, and **should**
document how it behaves in this situation.
When evaluating an expression involving a traversal, if any non-primitive field
in the chain is not set on the entry being evaluated, the entry **should** be
skipped i.e. not match the filter expression. This applies even when the
comparison is a `!=`, which would imply matching on empty values. In the
examples above, if resource field `a` is not set on the resource instance, that
instance is skipped as a non-match.
**Important:** The `.` operator **must not** be used to traverse through a
repeated field or list, except for specific use with the `:` operator.
### Has Operator
Filtering implementations **must** provide the `:` operator, which means "has".
It is usable with collections (repeated fields or maps) as well as messages,
and behaves slightly differently in each case.
Repeated fields query to see if the repeated structure contains a matching
element:
| Example | Meaning |
| ---------- | ----------------------------------------------------------- |
| `r:42` | True if `r` contains 42. |
| `r.foo:42` | True if `r` contains an element `e` such that `e.foo = 42`. |
**Important:** Filters can not query a _specific_ element on a repeated field
for a value. For example, `e.0.foo = 42` and `e[0].foo = 42` are **not** valid
filters.
Maps, structs, messages can query either for the presence of a field in the map
or a specific value:
| Example | Meaning |
| ---------- | ----------------------------------- |
| `m:foo` | True if `m` contains the key "foo". |
| `m.foo:*` | True if `m` contains the key "foo". |
| `m.foo:42` | True if `m.foo` is 42. |
There are two slight distinctions when parsing messages:
- When traversing messages, a field is only considered to be present if it has
a non-default value.
- When traversing messages, field names are snake case, although
implementations **may** choose to support automatic conversion between camel
case and snake case.
For all aforementioned types, simply checking for the presence of a top-level
resource field is possible with the `*` value:
| Example | Meaning |
| ---------- | ---------------------------------------- |
| `r:*` | True if `repeated` field `r` is present. |
| `p:*` | True if `map` field `p` is present. |
| `m:*` | True if `message` field `m` is present. |
**Note:** For `map` and `repeated` fields, there is no semantic difference
between an unset field and "set with empty value" - they both resolve to "not
present".
### Functions
The filtering language supports a function call syntax in order to support
API-specific extensions. An API **may** define a function using the
`call(arg...)` syntax, and **must** document any specific functions it
supports.
### Limitations
A service **may** specify further structure or limitations for filter queries,
above what is defined here. For example, a service may support the logical
operators but only permit a certain number of them (to avoid "queries of death"
or other performance concerns).
Further structure or limitations **must** be clearly documented, and
**must not** violate requirements set forth in this document.
### Validation
If a non-compliant or schematically invalid `filter` string is specified,
the API **should** error with `INVALID_ARGUMENT`. Wherever validation is relaxed
for `filter`, the API **must** document the difference.
Schematic validation refers, but is not limited to, the following:
- Fields referenced in the `filter` **must** exist on the filtered schema
- Field values provided in the `filter` **must** align to the type of the field
- For example, for a field `int32 age` a `filter` like `"age=hello"` is
invalid
- Field values for bounded data types e.g. `enum` provided in the `filter`
**must** be a valid value in the set
- Field values for standardized types e.g. `Timestamp` **must** conform to the
documented standard (see [Comparison Operators](#comparison-operators) for a
list of such types)
<!-- prettier-ignore-start -->
[aip-132]: ./0132.md
[cel]: https://github.com/google/cel-spec
[cel-cpp]: https://github.com/google/cel-cpp
[cel-go]: https://github.com/google/cel-go
[durations]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/duration.proto
[ebnf grammar]: /assets/misc/ebnf-filtering.txt
[rfc-3339]: https://tools.ietf.org/html/rfc3339
[timestamps]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/timestamp.proto
<!-- prettier-ignore-end -->
## Changelog
- **2025-10-03**: Clarify top-level field has operator behavior.
- **2025-01-07**: Clarify behavior for unset field in traversal operator chain.
- **2024-12-11**: Move non-compliant filter guidance to Validation section.
---
id: 161
state: approved
created: 2021-03-01
placement:
category: design-patterns
order: 85
---
# Field masks
Often, when updating resources (using an update method as defined in AIP-134 or
something reasonably similar), it is desirable to specify exactly which fields
are being updated, so that the service can ignore the rest, even if the user
sends new values.
It is tempting to define a mask format to handle the precise needs for each
API. However, masking requirements evolve, and therefore it is prudent to use a
structured syntax. This allows updates to be able to be made transparently,
without waiting for UI or client updates.
## Guidance
These masks of field names are called "field masks". Fields representing a field
mask **must** use the `google.protobuf.FieldMask` type. Field masks are most
common on Update requests (AIP-134).
Field masks **must** always be relative to the resource:
**Warning:** Read masks as a single field on the request message, for
example: `google.protobuf.FieldMask read_mask` are **DEPRECATED**. Instead, see
AIP-157.
```proto
message UpdateBookRequest {
// The book to update.
//
// The book's `name` field is used to identify the book to update.
// Format: publishers/{publisher}/books/{book}
Book book = 1 [(google.api.field_behavior) = REQUIRED];
// The list of fields to update.
// Fields are specified relative to the book
// (e.g. `title`, `rating`; *not* `book.title` or `book.rating`).
google.protobuf.FieldMask update_mask = 2;
}
```
### Read-write consistency
Read and write behavior for field masks **must** be self-consistent if a mask
is present:
- If a user updates a resource with a given mask, and then reads the same
resource with the same mask, the exact same data **must** be returned.
- Exception: [Output only fields](#output-only-fields).
- Similarly, reading a resource with a given mask and then updating the
resource with the returned data and the same mask **must** be a no-op.
**Note:** This implies that any mask that is valid for either read or write
**must** be valid for both.
### Specifying specific fields
Field masks **must** permit the specification of specific fields in a defined
struct, using the `.` character for traversal.
Because field masks are always relative to the resource, direct fields on the
resource require no traversal (examples: `title`, `rating`). Traversal is used
when resources contain messages (example: `author.given_name`).
**Note:** A user **must** be able to specify either a field as a whole, or one
of its subfields: `author` and `author.given_name` are both valid.
### Map fields
Field masks **may** permit the specification of specific fields in a map, if
and only if the map's keys are either strings or integers, using the `.`
character for traversal.
Field masks **should** support string keys that contain characters that are
problematic for the field mask syntax, using the backtick character.
```proto
message Book {
// The name of the book.
// Format: publishers/{publisher}/books/{book}
string name = 1;
// Reviews for the back cover. The key is the author of the review,
// and the value is the text of the review.
//
// Valid field masks: reviews, reviews.smith, reviews.`John Smith`
map<string, string> reviews = 2;
}
```
### Wildcards
Field masks **may** permit the use of the `*` character on a repeated field or
map to indicate the specification of particular sub-fields in the collection:
```proto
message Book {
option (google.api.resource) = {
type: "library.googleapis.com/Book"
pattern: "publishers/{publisher}/books/{book}"
};
// The name of the book.
// Format: publishers/{publisher}/books/{book}
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// The author or authors of the book.
// Valid field masks: authors, authors.*.given_name, authors.*.family_name
// Invalid field masks: authors.0, authors.0.given_name
repeated Author authors = 2;
}
message Author {
// The author's given name.
string given_name = 1;
// The author's family name.
string family_name = 2;
}
```
**Note:** Field masks **must not** permit accessing a particular element of a
repeated field by index, and **must** return an `INVALID_ARGUMENT` error if
this is attempted.
### Output only fields
If a user includes an output only field in an update mask indirectly (by using
a wildcard or specifying an overall message that includes an output-only
subfield), the service **must** ignore any output only fields provided as
input, even if they are cleared or modified.
If a user directly specifies an output only field in an update mask, the
service **must** ignore the output only fields provided as input, even if
they are cleared or modified, to permit the same field mask to be used for
input and output.
### Invalid field mask entries
When reading data, field masks **may** ignore entries that point to a value
that can not exist (either a field that does not exist, or a map key that the
service considers invalid).
When writing data, field masks **should** return an `INVALID_ARGUMENT` error if
an entry points to a value that can not exist; however, the service **may**
permit deletions.
## Changelog
- **2023-10-18**: Update guidance for presence of output only fields in update
mask.
- **2023-07-17**: Move `update_mask` guidance to AIP-134.
---
id: 162
state: draft
created: 2019-09-17
updated: 2023-09-01
placement:
category: design-patterns
order: 88
---
# Resource Revisions
Some APIs need to have resources with a revision history, where users can
reason about the state of the resource over time. There are several reasons for
this:
- Users may want to be able to roll back to a previous revision, or diff
against a previous revision.
- An API may create data which is derived in some way from a resource at a
given point in time. In these cases, it may be desirable to snapshot the
resource for reference later.
**Note:** We use the word _revision_ to refer to a historical reference for a
particular resource, and intentionally avoid the term _version_, which refers
to the version of an API as a whole.
## Guidance
APIs **may** store a revision history for a resource. Examples of when it is
useful include:
- When it is valuable to expose older versions of a resource via an API. This
can avoid the overhead of the customers having to write their own API to store
and enable retrieval of revisions.
- Other resources depend on different revisions of a resource.
- There is a need to represent the change of a resource over time.
APIs implementing resources with a revision history **should** abstract resource
revisions as nested collection of the resource. Sometimes, the revisions
collection can be a top level collection, exceptions include:
- If resource revisions are meant to have longer lifespan than the parent
resource. In other words, resource revisions exist after resource deletion.
```proto
message BookRevision {
// The name of the book revision.
string name = 1;
// The snapshot of the book
Book snapshot = 2
[(google.api.field_behavior) = OUTPUT_ONLY];
// The timestamp that the revision was created.
google.protobuf.Timestamp create_time = 3
[(google.api.field_behavior) = OUTPUT_ONLY];
// Other revision IDs that share the same snapshot.
repeated string alternate_ids = 4
[(google.api.field_behavior) = OUTPUT_ONLY];
}
```
- The `message` **must** be annotated as a resource (AIP-123).
- The `message` name **must** be named `{ResourceType}Revision`.
- The resource revision **must** contain a field with a message type of the
parent resource, with a field name of `snapshot`.
- The value of `snapshot` **must** be the configuration of the parent
at the point in time the revision was created.
- The resource revision **must** contain a `create_time` field (see [AIP-142][]).
- The resource revision **may** contain a repeated field `alternate_ids`, which would
contain a list of resource IDs that the revision is also known by (e.g. `latest`)
### Creating Revisions
Depending on the resource, different APIs may have different strategies for
- Create a new revision any time that there is a change to the parent resource
- Create a new revision when important system state changes
- Create a new revision when specifically requested
APIs **may** use any of these strategies. APIs **must**
document their revision creation strategy.
### Resource names for revisions
When referring to specific revision of a resource, the subcollection name
**must** be named `revisions`. Resource revisions have names with the format
`{resource_name}/revisions/{revision_id}`. For example:
```
publishers/123/books/les-miserables/revisions/c7cfa2a8
```
### Server-specified Aliases
Services **may** reserve specific IDs to be [aliases][alias] (e.g.
`latest`). These are read-only and managed by the service.
```
GET /v1/publishers/{publisher}/books/{book}/revisions/{revision_id}
```
- If a `latest` ID exists, it **must** represent the most recently created
revision. The content of `publishers/{publisher}/books/{book}/revisions/latest`
and `publishers/{publisher}/books/{book}` can differ, as the latest revision may
be different from the current state of the resource.
### User-Specified Aliases
APIs **may** provide a mechanism for users to assign an [alias][] ID to an
existing revision with a custom method "alias":
```proto
rpc AliasBookRevision(AliasBookRevisionRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{name=publishers/*/books/*/revisions/*}:alias"
body: "*"
};
}
```
```proto
message AliasBookRevisionRequest {
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/BookRevision"
}];
// The ID of the revision to alias to, e.g. `CURRENT` or a semantic
// version.
string alias_id = 2 [(google.api.field_behavior) = REQUIRED];
}
```
- The request message **must** have a `name` field:
- The field **must** be [annotated as required][aip-203].
- The field **must** identify the [resource type][aip-123] that it
references.
- The request message **must** have a `alias_id` field:
- The field **must** be [annotated as required][aip-203].
- If the user calls the method with an existing `alias_id`, the request **must**
succeed and the alias will be updated to refer to the provided revision. This
allows users to write code against a specific alias (e.g. `published`) and the
revision can change with no code change.
### Rollback
A common use case for a resource with a revision history is the ability to roll
back to a given revision. APIs **should** handle this with a `Rollback` custom
method:
```proto
rpc RollbackBook(RollbackBookRequest) returns (BookRevision) {
option (google.api.http) = {
post: "/v1/{name=publishers/*/books/*/revisions/*}:rollback"
body: "*"
};
}
```
- The method **must** use the `POST` HTTP verb.
- The method **should** return a resource revision.
```proto
message RollbackBookRequest {
// The revision that the book should be rolled back to.
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/BookRevision"
}];
}
```
- The request message **must** have a `name` field, referring to the resource
revision whose configuration the resource should be rolled back to.
- The field **must** be [annotated as required][aip-203].
- The field **must** identify the [resource type][aip-123] that it
references.
### Child resources
Resources with a revision history **may** have child resources. If they do,
there are two potential variants:
- Child resources where each child resource is a child of the parent resource
as a whole.
- Child resources where each child resource is a child of _a single revision
of_ the parent resource.
APIs **should not** include multiple levels of resources with revisions, as
this quickly becomes difficult to reason about.
### Standard methods
Any standard methods **must** implement the corresponding AIPs (AIP-131,
AIP-132, AIP-133, AIP-134, AIP-135), with the following additional behaviors:
- List methods: By default, revisions in the list response **must** be ordered
in reverse chronological order. User can supply `order_by` to override the
default behavior.
- If the revision supports aliasing, a delete method with the resource name
of the alias (e.g. `revisions/1.0.2`) **must** remove the alias instead of
deleting the resource.
As revisions are nested under the resource, also see [cascading delete][].
## Rationale
### Abstract revisions as nested collection
Revisions being resources under nested collection make revisions a first class
citizen.
- Revisions can offer standard get, list, and delete methods.
- It retains the flexibility of extending new fields to revision in addition to
the resource message.
### Tagging to Aliases
Previously, a concept of `tag` existed. This concept was redundant with that of
an [alias][], and the terms were consolidated to reduce complexity in the AIPs.
### Output only resource configuration
Although it was an option to have the revision take in the resource
configuration as part of the create method, doing so would have allowed users to
submit resource configuration for a revision that the resource was never in.
`OUTPUT_ONLY` and requiring that a created revision represents the resource at
current point in time eliminates that issue.
## History
### Switching from a collection extension to a subcollection
In 2023-09, revisions are abstracted as a nested resource collection. Prior to
this, revisions are more like extension of an existing resource by using `@`
symbol. List and delete revisions were custom methods on the resource
collection. A single Get method was used to retrieve either the resource
revision, or the resource.
Its primary advantage was allowing a resource reference to seamlessly refer to
a resource, or its revision.
It also had several disadvantages:
- List revisions is a custom method (:listRevisions) on the resource collection
- Delete revision is a custom method on the resource collection
- Not visible in API discovery doc
- Resource ID cannot use `@`
The guidance was modified ultimately to enable revisions to behave like a
resource, which reduces the users cognitive load and allows resource-oriented
clients to easily list, get, create, and update revisions.
### Using resource ID instead of tag
In the previous design, revisions had a separate identifer for a revision known
as a `tag`, that would live in a revision.
Tags were effectively a shadow resource ID, requiring methods to create, get and
filter revisions based on the value of the tag.
By consolidating the concept of a tag into the revision ID, the user no longer
needs to be familiar with a second set of retrieval and identifier methods.
## Changelog
- **2023-09-01**: AIP was updated to be a sub-collection.
- **2021-04-27**: Added guidance on returning the resource from Delete Revision.
[aip-142]: ./0142.md
[alias]: ./0122.md#resource-id-aliases
[cascading delete]: ./0135.md#cascading-delete
[UUID4]: https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random)
---
id: 163
state: approved
created: 2019-12-16
placement:
category: design-patterns
order: 90
---
# Change validation
Occasionally, a user wants to validate an intended change to see what the
result will be before actually making the change. For example, a request to
provision new servers in a fleet will have an impact on the overall fleet size
and cost, and could potentially have unexpected downstream effects.
## Guidance
APIs **may** provide an option to validate, but not actually execute, a
request, and provide the same response (status code, headers, and response
body) that it would have provided if the request was actually executed.
To provide this option, the method **should** include a `bool validate_only`
field in the request message:
```proto
message ReviewBookRequest {
string name = 1 [(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
int32 rating = 2;
string comment = 3;
// If set, validate the request and preview the review, but do not actually
// post it.
bool validate_only = 4;
}
```
The API **must** perform permission checks and any other validation that would
be performed on a "live" request; a request using `validate_only` **must** fail
if it determines that the actual request would fail.
**Note:** It may occasionally be infeasible to provide the full output. For
example, if creating a resource would create an auto-generated ID, it does not
make sense to do this on validation. APIs **should** omit such fields on
validation requests in this situation.
### Declarative-friendly resources
A resource that is declarative-friendly (AIP-128) **must** include a
`validate_only` field on methods that mutate the resource.
## Changelog
- **2020-10-06:** Added declarative-friendly resource requirement.
---
id: 164
state: approved
created: 2020-10-06
placement:
category: design-patterns
order: 95
---
# Soft delete
There are several reasons why a client could desire soft delete and undelete
functionality, but one over-arching reason stands out: recovery from mistakes.
A service that supports undelete makes it possible for users to recover
resources that were deleted by accident.
## Guidance
APIs **may** support the ability to "undelete", to allow for situations where
users mistakenly delete resources and need the ability to recover.
If a resource needs to support undelete, the `Delete` method **must** simply
mark the resource as having been deleted, but not completely remove it from the
system. If the method behaves this way, it **should** return the updated
resource instead of `google.protobuf.Empty`.
Resources that support soft delete **should** have both a `delete_time` and
`purge_time` field as described in AIP-148. Additionally, resources **should**
include a `DELETED` state value if the resource includes a `state` field
(AIP-216).
### Undelete
A resource that supports soft delete **should** provide an `Undelete` method:
```proto
rpc UndeleteBook(UndeleteBookRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{name=publishers/*/books/*}:undelete"
body: "*"
};
}
```
- The HTTP verb **must** be `POST`.
- The `body` clause **must** be `"*"`.
- The response message **must** be the resource itself. There is no
`UndeleteBookResponse`.
- The response **should** include the fully-populated resource unless it is
infeasible to do so.
- If the undelete RPC is [long-running](#long-running-undelete), the response
message **must** be a `google.longrunning.Operation` which resolves to the
resource itself.
### Undelete request message
Undelete methods implement a common request message pattern:
```proto
message UndeleteBookRequest {
// The name of the deleted book.
// Format: publishers/{publisher}/books/{book}
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference).type = "library.googleapis.com/Book"];
}
```
- A `name` field **must** be included. It **should** be called `name`.
- The field **should** be [annotated as required][aip-203].
- The field **should** identify the [resource type][aip-123] that it
references.
- The comment for the field **should** document the resource pattern.
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
### Long-running undelete
Some resources take longer to undelete a resource than is reasonable for a
regular API request. In this situation, the API **should** use a long-running
operation (AIP-151) instead:
```proto
rpc UndeleteBook(UndeleteBookRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{name=publishers/*/books/*}:undelete"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "Book"
metadata_type: "OperationMetadata"
};
}
```
- The response type **must** be set to the resource (what the return type would
be if the RPC was not long-running).
- Both the `response_type` and `metadata_type` fields **must** be specified.
### List and Get
Soft-deleted resources **should not** be returned in `List` (AIP-132) responses
by default (unless `bool show_deleted` is true). `Get` (AIP-131) requests for
soft-deleted resources **should** return the resource (rather than a
`NOT_FOUND` error).
APIs that soft delete resources **may** choose a reasonable strategy for
purging those resources, including automatic purging after a reasonable time
(such as 30 days), allowing users to set an expiry time (AIP-214), or retaining
the resources indefinitely. Regardless of what strategy is selected, the API
**should** document when soft deleted resources will be completely removed.
### Declarative-friendly resources
Soft-deletable resources have a poorer experience than hard-deleted resources in
declarative clients: since an ID on a soft-deleted resource is not re-usable
unless a custom method (undelete) is called, an imperative client must be
introduced or hand-written code is required to incorporate the usage of the
custom method.
### Errors
If the user does not have permission to access the resource, regardless of
whether or not it exists, the service **must** error with `PERMISSION_DENIED`
(HTTP 403). Permission **must** be checked prior to checking if the resource
exists.
If the user does have proper permission, but the requested resource does not
exist (either it was never created or already expunged), the service **must**
error with `NOT_FOUND` (HTTP 404).
If the user calling a soft `Delete` has proper permission, but the requested
resource is already deleted, the service **must** succeed if `allow_missing` is
`true`, and **should** error with `NOT_FOUND` (HTTP 404) if `allow_missing` is
`false`.
If the user calling `Undelete` has proper permission, but the requested
resource is not deleted, the service **must** respond with `ALREADY_EXISTS`
(HTTP 409).
## Further reading
- For the `Delete` standard method, see AIP-135.
- For long-running operations, see AIP-151.
- For resource freshness validation (`etag`), see AIP-154.
- For change validation (`validate_only`), see AIP-163.
## Changelog
- **2024-09-24**: Included missing requirement for `delete_time`.
- **2023-07-13**: Renamed overloaded `expire_time` to `purge_time`.
- **2021-07-12**: Added error behavior when soft deleting a deleted resource.
- **2021-07-12**: Clarified that `ALREADY_EXISTS` errors apply to `Undelete`.
- **2021-07-12**: Changed the `expire_time` field to "should" for consistency
with AIP-148.
- **2020-09-23**: Soft delete material in AIP-135 migrated to this AIP.
[aip-123]: ./0123.md
[aip-203]: ./0203.md
---
id: 165
state: approved
created: 2019-12-18
updated: 2022-06-02
placement:
category: design-patterns
order: 100
---
# Criteria-based delete
Occasionally, an API may need to provide a mechanism to delete a large number
of resources based on some set of filter parameters, rather than requiring the
individual resource name of the resources to be deleted.
This is a rare case, reserved for situations where users need to delete
thousands or more resources at once, in which case the normal Batch Delete
pattern ([AIP-235][]) becomes unwieldy and inconvenient.
## Guidance
**Important:** Most APIs **should** use only Delete ([AIP-135][]) or Batch
Delete ([AIP-235][]) for deleting resources, and **should not** implement
deleting based on criteria. This is because deleting is generally irreversible
and this type of operation makes it easy for a user to accidentally lose
significant amounts of data.
An API **may** implement a Purge method to permit deleting a large number of
resources based on a filter string; however, this **should** only be done if
the Batch Delete ([AIP-235][]) pattern is insufficient to accomplish the
desired goal:
```proto
rpc PurgeBooks(PurgeBooksRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:purge"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "PurgeBooksResponse"
metadata_type: "PurgeBooksMetadata"
};
}
```
- The RPC's name **must** begin with the word `Purge`. The remainder of the
RPC name **should** be the plural form of the resource being purged.
- The request message **must** match the RPC name, with a `Request` suffix.
- The response type **must** be a `google.longrunning.Operation` (see
AIP-151) that resolves to a message whose name matches the RPC name, with
a `Response` suffix.
- The HTTP verb **must** be `POST`, and the `body` **must** be `"*"`.
- The URI path **should** represent the collection for the resource.
- The `parent` field **should** be included in the URI. If the API wishes to
support deletion across multiple parents, it **should** accept the `-`
character consistent with [AIP-159][].
### Request message
Purge methods implement a common request message pattern:
```proto
message PurgeBooksRequest {
// The publisher to purge books from.
// To purge books across publishers, send "publishers/-".
string parent = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// A filter matching the books to be purged.
string filter = 2 [(google.api.field_behavior) = REQUIRED];
// Actually perform the purge.
// If `force` is set to false, the method will return a sample of
// resource names that would be deleted.
bool force = 3;
}
```
- A singular `string parent` field **should** be included, unless the resource
is top-level.
- The field **should** be [annotated as required][aip-203].
- The field **should** identify the [resource type][aip-123] that it
references.
- A singular `string filter` field **must** be included and **must** follow the
same semantics as in List methods (AIP-160).
- It **should** be [annotated as required][aip-203].
- A wildcard value of `"*"` **may** be supported for deleting everything.
- A singular `bool force` field **must** be included. If it is not set, the API
**must** return a count of the resources that would be deleted as well as a
sample of those resources, without actually performing the deletion.
### Response message
Purge methods implement a common response message pattern:
```proto
message PurgeBooksResponse {
// The number of books that this request deleted (or, if `force` is false,
// the number of books that will be deleted).
int32 purge_count = 1;
// A sample of the resource names of books that will be deleted.
// Only populated if `force` is set to false.
repeated string purge_sample = 2 [(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
}
```
- A singular `int32 purge_count` field **should** be included, and provide the
number of resources that were deleted (or would be deleted). This count
**may** be an estimate similar to `total_size` in AIP-158 (but the service
**should** document this if so).
- A `repeated string purge_sample` field **should** be included: If `force` is
`false`, it **should** provide a sample of resource names that will be
deleted. If `force` is true, this field **should not** be populated.
- The sample **should** be a sufficient size to catch clearly obvious
mistakes: A good rule of thumb is 100. The API **should** document the
size, and **should** document that it is a maximum (it is possible to send
fewer).
- The sample **may** be random or **may** be deterministic (such as the first
matched resource names). The API **should** document which approach is
used.
- The field **should** identify the [resource type][aip-123] that it
references.
**Note:** Even if `purge_count` and `purge_sample` are not included, the
`force` field **must** still be included in the request.
## Changelog
- **2025-07-23:** Explicitly state filter wildcard may be supported.
- **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-".
- **2020-10-29**: Expanded guidance on HTTP, field behavior, and resource
reference annotations.
[aip-123]: ./0123.md
[aip-135]: ./0135.md
[aip-159]: ./0159.md
[aip-203]: ./0203.md
[aip-235]: ./0235.md
---
id: 180
state: approved
created: 2019-07-23
placement:
category: compatibility
order: 10
---
# Backwards compatibility
APIs are fundamentally contracts with users, and users often write code against
APIs that is then launched into a production service with the expectation that
it continues to work (unless the API has a [stability level][aip-181] that
indicates otherwise). Therefore, it is important to understand what constitutes
a backwards compatible change and what constitutes a backwards incompatible
change.
## Guidance
Existing client code **must not** be broken by a service updating to a new
minor or patch release. Old clients **must** be able to work against newer
servers (with the same major version number).
**Important:** It is not always clear whether a change is compatible or not.
The guidance here **should** be treated as indicative, rather than as a
comprehensive list of every possible change.
There are three distinct types of compatibility to consider:
1. Source compatibility: Code written against a previous version **must**
compile against a newer version, and successfully run with a newer version
of the client library.
2. Wire compatibility: Code written against a previous version **must** be able
to communicate correctly with a newer server. In other words, not only are
inputs and outputs compatible, but the serialization and deserialization
expectations continue to match.
3. Semantic compatibility: Code written against a previous version **must**
continue to receive what most reasonable developers would expect. (This can
be tricky in practice, however, and sometimes determining what users will
expect can involve a judgment call.)
**Note:** In general, the specific guidance here assumes use of protocol
buffers and JSON as transport formats. Other transport formats may have
slightly different rules.
**Note:** This guidance assumes that APIs are intended to be called from a
range of consumers, written in multiple languages and with no control over
how and when consumers update. Any API which has a more limited scope (for
example, an API which is only called by client code written by the same team
as the API producer, or deployed in a way which can enforce updates) should
carefully consider its own compatibility requirements.
### Adding components
In general, new components (interfaces, methods, messages, fields, enums, or
enum values) **may** be added to existing APIs in the same major version.
However, keep the following guidelines in mind when doing this:
- Code written against the previous surface (and thus is unaware of the new
components) **must** continue to be treated the same way as before.
- New required fields **must not** be added to existing request messages or
resources.
- Any field being populated by clients **must** have a default behavior
matching the behavior before the field was introduced.
- This can be tricky to do in some cases. For example, adding pagination
after the fact where previously all items were returned (i.e. `page_size`
is infinite, which is not advised). If the default for the new `page_size`
field is _less_ than what was previously returned, older clients will
incorrectly assume all results were returned.
- Any field previously populated by the server **must** continue to be
populated, even if it introduces redundancy.
- For enum values specifically, be aware that it is possible that user code
does not handle new values gracefully.
- Enum values **may** be freely added to enums which are only used in request
messages.
- Enums that are used in response messages or resources and which are
expected to receive new values **should** document this. Enum values still
**may** be added in this situation; however, appropriate caution **should**
be used.
**Note:** It is possible when adding a component closely related to an existing
component (for example, `string foo_value` when `string foo` already exists) to
enter a situation where generated code will conflict. Service owners **should**
be aware of subtleties in the tooling they or their users are likely to use
(and tool authors **should** endeavor to avoid such subtleties if possible).
### Removing or renaming components
Existing components (interfaces, methods, messages, fields, enums, or enum
values) **must not** be removed from existing APIs in the same major version.
Removing a component is a backwards incompatible change.
**Important:** Renaming a component is semantically equivalent to "remove and
add". In cases where these sorts of changes are desirable, a service **may**
add the new component, but **must not** remove the existing one. In situations
where this can allow users to specify conflicting values for the same semantic
idea, the behavior **must** be clearly specified.
### Moving components between files
Existing components **must not** be moved between files.
Moving a component from one proto file to another within the same package is
wire compatible, however, the code generated for languages like C++ or Python
will result in breaking change since `import` and `#include` will no longer
point to the correct code location.
### Moving into oneofs
Existing fields **must not** be moved into or out of a oneof. This is a
backwards-incompatible change in the Go protobuf stubs.
### Changing the type of fields
Existing fields and messages **must not** have their type changed, even if the
new type is wire-compatible, because type changes alter generated code in a
breaking way.
### Changing string length
APIs **should** avoid increasing the upper bound for the size or limit (if
accepted as input) of `string` fields. APIs **should** treat expected size upper
bound increases as incompatible changes (see [Changing resource
names](#changing-resource-names) as an example). APIs **may** pad out values with
filler characters if reserving a consistent size is necessary, but this **must**
be documented if done.
### Changing resource names
A resource **must not** change its [name][aip-122].
Unlike most breaking changes, this affects major versions as well: in order for
a client to expect to use v2.0 access a resource that was created in v1.0 or
vice versa, the same resource name **must** be used in both versions.
More subtly, the set of valid resource names **should not** change either, for
the following reasons:
- If resource name formats become more restrictive, a request that would
previously have succeeded will now fail.
- If resource name formats become less restrictive than previously documented,
then code making assumptions based on the previous documentation could break.
Users are very likely to store resource names elsewhere, in ways that may be
sensitive to the set of permitted characters and the length of the name.
Alternatively, users might perform their own resource name validation to
follow the documentation.
- For example, Amazon gave customers [a lot of warning][ec2] and had a
migration period when they started allowing longer EC2 resource IDs.
### Semantic changes
Code will often depend on API behavior and semantics, _even when such behavior
is not explicitly supported or documented_. Therefore, APIs **must not** change
visible behavior or semantics in ways that are likely to break reasonable user
code, as such changes will be seen as breaking by those users.
**Note:** This does involve some level of judgment; it is not always clear
whether a proposed change is likely to break users, and an expansive reading of
this guidance could ostensibly prevent _any_ change (which is not the intent).
#### Changing value format or construction
APIs **must not** change the expected format or algorithm used to construct the
value of an existing field - even if the field is `OUTPUT_ONLY` and populated by
the API service - within an API version. Doing so requires a new API version.
For example, changing the format of a field `ip_address` conforming to IPv4
format to instead contain IPv6 values is a breaking change.
#### Default values must not change
Default values are the values set by servers for resources when they are not
specified by the client. This section only applies to static default values within
fields on resources and does not apply to dynamic defaults such as the default IP
address of a resource.
Changing the default value is considered breaking and **must not** be done. The
default behavior for a resource is determined by its default values, and this
**must not** change across minor versions.
For example:
```proto
message Book {
// google.api.resource and other annotations and fields
// The genre of the book
// If this is not set when the book is created, the field will be given a value of FICTION.
enum Genre {
UNSPECIFIED = 0;
FICTION = 1;
NONFICTION = 2;
}
}
```
Changing to:
```proto
message Book {
// google.api.resource and other annotations and fields
// The genre of the book
// If this is not set when the book is created, the field will be given a value of NONFICTION.
enum Genre {
UNSPECIFIED = 0;
FICTION = 1;
NONFICTION = 2;
}
}
```
would constitute a breaking change.
#### Serializing defaults
APIs **must not** change the way a field with a default value is serialized. For
example if a field does not appear in the response if the value is equal to the
default, the serialization **must not** change to include the field with the
default. Clients may depend on the presence or absence of a field in a resource
as semantically meaningful, so a change to how serialization is done for absent
values **must not** occur in a minor version.
Consider the following proto, where the default value of `wheels` is `2`:
```proto
// A representation of an automobile
message Automobile {
// google.api.resource and other annotations and fields
// The number of wheels on the automobile.
// The default value is 2, when no value is sent by the client.
int wheels = 2;
}
```
First the proto serializes to JSON when the value of `wheels` is `2` as follows:
```json
{
"name": "my-car"
}
```
Then, the API service changes the serialization to include `wheel` even if the
value is equal to the default value, `2` as follows:
```json
{
"name": "my-car",
"wheels": 2
}
```
This constitutes a change that is not backwards compatible within a major
version.
## Further reading
- For compatibility around field behavior, see [AIP-203][].
- For compatibility around pagination, see [AIP-158][].
- For compatibility around long-running operations, see [AIP-151][].
- For understanding stability levels and expectations, see [AIP-181][].
- For compatibility with client library resource name parsing, see [AIP-4231][]
- For compatibility with client library method signatures, see [AIP-4232][]
- For compatibility around field presence changes, see [AIP-149][].
- For compatibility around resource types, see [AIP-123][].
## Rationale
### Risk of string length changes
End users may store resource properties, like the `name`, in a dedicated
database column with a limited length. If the service starts returning values
for the `name` that are twice the originally documented/observed length, this
may unexpectedly break the customer's database. Furthermore, string properties
that appear in URLs (including query parameters) are especially likely to have
client-side limits, making them more sensitive to length changes.
### Risk of changing value format or construction
Customers often depend on the format or algorithmic construction of a field for
client-side parsing, hashing, or database table construction. Changing it in
an existing field could break that client-side consumption.
## Changelog
- **2025-10-21**: Added guidance for string length changes, changing formats,
and an example for carefully adding components.
- **2024-08-07**: Added reference to resource type compatibility.
- **2024-06-05**: Added reference to field presence compatibility.
- **2023-07-26**: Added reference to field behavior compatibility.
- **2023-07-26**: Added note on APIs which have limited clients.
- **2022-08-11**: Added "Moving components between files" section.
- **2022-06-01**: Added more links to other AIPs with compatibility concerns
- **2019-12-16**: Clarified that moving existing fields into oneofs is
breaking.
<!-- prettier-ignore-start -->
[aip-122]: ./0122.md
[aip-123]: ./0123.md
[aip-149]: ./0149.md
[aip-151]: ./0151.md
[aip-158]: ./0158.md
[aip-181]: ./0181.md
[aip-203]: ./0203.md
[aip-4231]: https://google.aip.dev/client-libraries/4231
[aip-4232]: https://google.aip.dev/client-libraries/4232
[ec2]: https://aws.amazon.com/blogs/aws/theyre-here-longer-ec2-resource-ids-now-available/
<!-- prettier-ignore-end -->
---
id: 181
state: approved
created: 2019-02-18
placement:
category: compatibility
order: 20
---
# Stability levels
While different organizations (both inside Google and outside) have different
product life cycles, AIPs refer to the _stability_ of an API component using
the following terms.
**Note:** These stability levels roughly correspond to the product launch
stages (alpha, beta, GA) in Google Cloud, but are not identical. GCP imposes
its own additional expectations and commitments on top of what is outlined
here.
## Alpha
An _alpha_ component undergoes rapid iteration with a known set of users who
**must** be tolerant of change. The number of users **should** be a
curated, manageable set, such that it is feasible to communicate with all
of them individually.
Breaking changes **must** be both allowed and expected in alpha components, and
users **must** have no expectation of stability.
## Beta
A _beta_ component **must** be considered complete and ready to be declared
stable, subject to public testing. Beta components **should** be exposed to an
unknown and potentially large set of users. In other words, beta components
**should not** be behind an allowlist; instead, they **should** be available to
the public.
Because users of beta components tend to have a lower tolerance of change, beta
components **should** be as stable as possible; however, the beta component
**must** be permitted to change over time. These changes **should** be minimal
but **may** include backwards-incompatible changes to beta components.
Backwards-incompatible changes **must** be made only after a reasonable
deprecation period to provide users with an opportunity to migrate their code.
This deprecation period **must** be defined at the time of being marked beta.
Beta components **should** be time-boxed and promoted to stable if no issues
are found in the specified timeframe, which **should** be specified at the time
of being marked beta. A reasonable time period **may** vary, but a good rule of
thumb is 90 days.
## Stable
A _stable_ component **must** be fully-supported over the lifetime of the major
API version. Because users expect such stability from components marked stable,
there **must** be no breaking changes to these components, subject to the
caveats described below.
### Major versions
When breaking changes become necessary, the API producer **should** create the
next major version of the API, and start a deprecation clock on the existing
version.
Turn-down of any version containing stable components **must** have a formal
process defined at the time of being marked stable. This process **must**
specify a deprecation period for users which provides them with reasonable
advance warning.
### Isolated changes
On very rare occasions, it could be preferable to make a small, isolated
breaking change, if this will only cause inconvenience to a small subset of
users. (Creating a new major version is an inconvenience to all users.) In this
case, the API producer **may** deprecate the component, but **must** continue
to support the component for the normal turndown period for a stable component.
**Important:** Making an in-place breaking change in a stable API is considered
an extreme course of action, and should be treated with equal or greater
gravity as creating a new major version. For example, at Google, this requires
the approval of the API Governance team.
### Emergency changes
In certain exceptional cases, such as security concerns or regulatory
requirements, any API component **may** be changed in a breaking manner
regardless of its stability level, and a deprecation is not promised in these
situations.
---
id: 182
state: reviewing
created: 2021-08-13
placement:
category: compatibility
order: 30
---
# External software dependencies
Some services have a particular type of dependency on external software: they
allow users to create resources that run on or expose the external software in
some way. For example:
- A database admin service can allow users to create databases running on a
particular version of a particular database engine (for example, PostgreSQL
13.4).
- A virtual machine service can allow users to create VMs running a particular
operating system (for example, Ubuntu 20.04).
- An application or function platform service can allow users to write code
that runs against a particular version of a programming language (for
example, Node.js 16.6).
Services that provide external software to users in this way will eventually
need to address the fact that all of these types of software have release
lifecycles, and the versions they currently expose will eventually reach
end-of-life.
## Guidance
Services that expose external software dependencies **should** allow users to
create resources using any currently-supported LTS (long-term support) version
of the supported software, and **may** allow users to create resources using
non-LTS versions.
Services **should not** indefinitely allow users to create new resources using
versions that have reached end-of-life, although they **may** have a transition
period between when the software version reaches end-of-life and when support
for creating new resources with that version is removed.
**Note:** Restricting or removing the ability to create resources using
end-of-life versions of software is **not** considered a breaking change for
the service for the purpose of AIP-181, even though it actually is one.
However, because the change can break existing users' workflows, services
**must** notify users who are using resources approaching end-of-life.
If possible, services **should** allow previously-created resources to remain,
and **may** warn users of the risks associated with continuing to use
end-of-life software. Services **should not** proactively remove resources
using end-of-life software, or impose other restrictions on existing resources,
unless critical security concerns require the service to do so.
### Continued support
If supporting a version that has reached end-of-life is necessary for business
reasons (usually because the end-of-life software still has significant
adoption), the service **may** choose to officially support the end-of-life
version, but **must** take on the responsibility of patching and maintaining
the software if it does so.
---
id: 185
state: approved
created: 2024-10-22
placement:
category: compatibility
order: 40
---
# API Versioning
This topic describes the versioning strategies used by Google APIs. In
general, these strategies apply to all Google-managed services.
## Guidance
All Google API interfaces **must** provide a _major version number_, which is
encoded at the end of the protobuf package, and included as the first part of
the URI path for REST APIs. In the event an API needs to make an incompatible
change, consult [AIP-180][] and [AIP-181][] for necessary steps based on the
stability level of the surface in question.
Note: The use of the term "major version number" above is taken from
[semantic versioning][]. However, unlike in traditional semantic versioning,
Google APIs **must not** expose minor or patch version numbers. For
example, Google APIs use `v1`, not `v1.0`, `v1.1`, or `v1.4.2`. From a user's
perspective, major versions are updated in place with minor/patch equivalent
changes, and users receive new functionality without migration.
A new major version of an API **must not** depend on a previous major version of
the same API. An API surface **must not** depend on other APIs, except for in
the cases outlined in [AIP-213][] and [AIP-215][].
Different versions of the same API **must** be able to work at the same time
within a single client application for a reasonable transition period. This time
period allows the client to transition smoothly to the newer version. An older
version **must** go through a reasonable, well-communicated deprecation period
before being shut down.
For releases which have [alpha or beta stability][AIP-181], APIs **must** append
the stability level after the major version number in the protobuf package and
URI path using one of these strategies:
- Channel-based versioning (recommended)
- Release-based versioning
- Visibility-based versioning
### Channel-based versioning
A *stability channel* is a long-lived release at a given stability level that
receives in-place updates. There is no more than one channel per stability level
for a major version. Under this strategy, there are up to three channels
available: alpha, beta, and stable.
The alpha and beta channel **must** have their stability level appended to the
version, but the stable channel **must not** have the stability level appended.
For example, `v1` is an acceptable version for the stable channel, but `v1beta`
or `v1alpha` are not. Similarly, `v1beta` or `v1alpha` are acceptable versions
for the respective beta and alpha channel, but `v1` is not acceptable for
either. Each of these channels receives new features and updates "in-place".
The beta channel's functionality **must** be a superset of the stable channel's
functionality, and the alpha channel's functionality **must** be a superset of
the beta channel's functionality.
#### Deprecating API functionality
API elements (fields, messages, RPCs) **may** be marked deprecated in any
channel to indicate that they should no longer be used:
```proto
// Represents a scroll. Books are preferred over scrolls.
message Scroll {
option deprecated = true;
// ...
}
```
Deprecated API functionality **must not** graduate from alpha to beta, nor beta
to stable. In other words, functionality **must not** arrive "pre-deprecated"
in any channel.
The beta channel's functionality **may** be removed after it has been deprecated
for a sufficient period; we recommend 180 days. For functionality that exists
only in the alpha channel, deprecation is optional, and functionality **may** be
removed without notice. If functionality is deprecated in an API's
alpha channel before removal, the API **should** apply the same annotation, and
**may** use any timeframe it wishes.
### Release-based versioning
**Important:** This pattern is not commonly used for new services. There are
existing services that follow it, but Channel-based Versioning is the preferred
mechanism.
An *individual release* is an alpha or beta release that is expected to be
available for a limited time period before its functionality is incorporated
into the stable channel, after which the individual release will be shut down.
When using release-based versioning strategy, an API may have any number of
individual releases at each stability level.
Note: Both the channel-based and release-based strategies update the _stable_
version in-place. There is a single stable channel, rather than individual
stable releases, even when using the release-based strategy.
Alpha and beta releases **must** have their stability level appended to the
version, followed by an incrementing release number. For example, `v1beta1` or
`v1alpha5`. APIs **should** document the chronological order of these versions
in their documentation (such as comments).
Each alpha or beta release **may** be updated in place with backwards-compatible
changes. For beta releases, backwards-incompatible updates **should** be made by
incrementing the release number and publishing a new release with the change.
For example, if the current version is `v1beta1`, then `v1beta2` is released
next.
Alpha and beta releases **should** be shut down after their functionality
reaches the stable channel. An alpha release **may** be shut down at any time,
while a beta release **should** allow users a reasonable transition period; we
recommend 180 days.
### Visibility-based versioning
[API visibility][] is an advanced feature provided by Google API infrastructure.
It allows API producers to expose multiple external API views from one internal
API surface, and each view is associated with an API _visibility label_, such
as:
```proto
import "google/api/visibility.proto";
message Resource {
string name = 1;
// Preview. Do not use this feature for production.
string display_name = 2
[(google.api.field_visibility).restriction = "PREVIEW"];
}
```
A visibility label is a case-sensitive string that can be used to tag any API
element. By convention, visibility labels should always use UPPER case.
An implicit `PUBLIC` label is applied to all API elements unless an explicit
visibility label is applied as in the example above.
Each visibility label is an allow-list. API producers need to grant visibility
labels to API consumers for them to use API features associated with the labels.
In other words, an API visibility label is like an ACL'ed API version.
Multiple visibility labels **may** be applied to an element by using a
comma-separated string (e.g. `"PREVIEW,TRUSTED_TESTER"`). When multiple
visibility labels are used, then the client needs only _one_ of the visibility
labels (logical `OR`).
By default, the visibility labels granted to the API consumer are used to verify
incoming requests. However, a client can send requests with an explicit
visibility label as follows:
```
GET /v1/projects/my-project/topics HTTP/1.1
Host: pubsub.googleapis.com
Authorization: Bearer y29....
X-Goog-Visibilities: PREVIEW
```
A single API request can specify at most one visibility label.
API producers can use API visibility for API versioning, such as
`INTERNAL` and `PREVIEW`. A new API feature starts with the `INTERNAL` label,
then moves to the `PREVIEW` label. When the feature is stable and becomes
generally available, all API visibility labels are removed from the API
definition.
In general, API visibility is easier to implement than API versioning for
incremental changes, but it depends on sophisticated API infrastructure support.
Google Cloud APIs often use API visibility for Preview features.
[AIP-180]: https://aip.dev/180
[AIP-181]: https://aip.dev/181
[AIP-213]: https://aip.dev/213
[AIP-215]: https://aip.dev/215
[api visibility]: https://github.com/googleapis/googleapis/blob/master/google/api/visibility.proto
[semantic versioning]: https://semver.org/
---
id: 190
state: approved
created: 2025-06-10
placement:
category: polish
order: 0
---
# Naming conventions
This topic describes the naming conventions used in Google APIs. In
general, these conventions apply to all Google-managed services.
## Guidance
In order to provide consistent developer experience across many APIs and
over a long period of time, all names used by an API **should** be:
- straightforward
- intuitive
- consistent
This includes names of interfaces, resources, collections, methods, and
messages.
Since English is a second language for many developers, one goal of these
naming conventions is to make every API name understandable to the majority of
developers. It does this by encouraging the use of a simple, consistent, and
small vocabulary when naming methods and resources.
- Names used in APIs **should** be in correct American English. For
example, license (instead of licence), color (instead of colour).
- Commonly accepted short forms or abbreviations of long words **may**
be used for brevity. For example, API is preferred over Application
Programming Interface.
- Unless otherwise specified, definitions **must** use UpperCamelCase names,
as defined by
[Google Java Style](https://google.github.io/styleguide/javaguide.html#s5.3-camel-case).
- Use intuitive, familiar terminology where possible. For example,
when describing removing (and destroying) a resource, delete is
preferred over erase.
- Use the same name or term for the same concept, including for
concepts shared across APIs.
- Avoid name overloading. Use different names for different concepts.
- Avoid overly general names that are ambiguous within the context of
the API and the larger ecosystem of Google APIs. They can lead to
misunderstanding of API concepts. Rather, choose specific names that
accurately describe the API concept. This is particularly important
for names that define first-order API elements, such as resources.
There is no definitive list of names to avoid, as every name must be
evaluated in the context of other names. Instance, info, and service
are examples of names that have been problematic in the past. Names
chosen should describe the API concept clearly (for example:
instance of what?) and distinguish it from other relevant concepts
(for example: does "alert" mean the rule, the signal, or the
notification?).
- Carefully consider use of names that may conflict with keywords in
common programming languages. Such names **may** be used but will
likely trigger additional scrutiny during API review. Use them
judiciously and sparingly.
### Interface names
To avoid confusion with [Service Names](./0009.md#api-service-name) such as
`pubsub.googleapis.com`, the term *interface name* refers to the name
used when defining a `service` in a .proto file:
```proto
// Library is the interface name.
service Library {
rpc ListBooks(...) returns (...);
rpc ...
}
```
You can think of the *service name* as a reference to the actual
implementation of a set of APIs, while the *interface name* refers to
the abstract definition of an API.
An interface name **should** use an intuitive noun such as Calendar or
BlobStore. The name **should not** conflict with any well-established
concepts in programming languages and their runtime libraries (for
example, File).
In the rare case where an *interface name* would conflict with another
name within the API, a suffix (for example `Api` or `Service`)
**should** be used to disambiguate.
### Method names
A service **may**, in its IDL specification, define one or more API
methods that correspond to methods on collections and resources. The
method names **should** follow the naming convention of `VerbNoun` in
UpperCamelCase, where the noun is typically the resource type.
Standard methods, and their Batch variants, define their naming guidance in
the following documents:
Method | Standard | Batch
------ | -------- | -----
`Get` | [AIP-131][] | [AIP-231][]
`List` | [AIP-132][] | N/A
`Create` | [AIP-133][] | [AIP-233][]
`Update` | [AIP-134][] | [AIP-234][]
`Delete` | [AIP-135][] | [AIP-235][]
All other methods are considered Custom Methods and adhere to AIP-136 naming
guidance.
### Message names
Message names **should** be short and concise. Avoid unnecessary or redundant
words. Adjectives can often be omitted if there is no corresponding message
without the adjective. For example, the `Shared` in `SharedProxySettings` is
unnecessary if there are no _unshared_ proxy settings.
Message names **should not** include prepositions (e.g. "With", "For").
Generally, message names with prepositions are better represented with
optional fields on the message.
#### Request and response messages
For request and response message names, see AIP-136 for custom methods and the
appropriate AIP for
[standard methods](https://google.aip.dev/general#operations).
## Further reading
- For proto and language package naming, see AIP-191.
- For collection ID naming conventions, see
[AIP-122](./0122.md#collection-identifiers).
- For Enum names, see AIP-126.
- For field names, see AIP-140.
- For repeated field names, see [AIP-140](./0140#repeated-fields).
- For fields representing times and durations, see AIP-142.
- For fields representing dates and times of day, see
[AIP-142](./0142#civil-dates-and-times).
- For fields representing a quantity, see AIP-141.
- For the canonical `List` method `filter` field, see
[AIP-132](./0132#filtering).
- For the canonical `List` response message, see
[AIP-132](./0132#response-message).
- For well known abbreviations, see [AIP-140](./0140#abbreviations).
<!-- Need these link values for the table entries which won't be hot-linked by
the site-generator like plain text would be -->
[AIP-131]: ./0131.md
[AIP-132]: ./0132.md
[AIP-133]: ./0133.md
[AIP-134]: ./0134.md
[AIP-135]: ./0135.md
[AIP-231]: ./0231.md
[AIP-233]: ./0233.md
[AIP-234]: ./0234.md
[AIP-235]: ./0235.md
---
id: 191
state: approved
created: 2019-07-25
placement:
category: polish
order: 10
---
# File and directory structure
A consistent file and directory structure, while making minimal difference
technically, makes API surface definitions easier for users and reviewers to
read.
## Guidance
**Note:** The following guidance applies to APIs defined in protocol buffers,
such as those used throughout Google. While the spirit of this guidance applies
to APIs defined using other specification languages or formats, some of the
particular recommendations might be irrelevant.
### Syntax
APIs defined in protocol buffers **must** use `proto3` syntax.
### Single package
APIs defined in protocol buffers **must** define each individual API in a
single package, which **must** end in a version component. For example:
```proto
syntax = "proto3";
package google.cloud.translation.v3;
```
Google APIs **must** reside in a directory that matches the protocol buffer
`package` directive. For example, the package above dictates that the directory
be `google/cloud/translation/v3`.
### File names
It is often useful to divide API definitions into multiple files. File names
**must** use `snake_case`.
APIs **should** have an obvious "entry" file, generally named after the API
itself. An API with a small number of discrete services (Google Cloud Pub/Sub's
`Publisher` and `Subscriber` is a good example) **may** have a separate entry
file per service.
APIs with only one file **should** use a filename corresponding to the name of
the API.
API `service` definitions and associated RPC request and response `message`
definitions **should** be defined in the same file.
Bear in mind that the file names often become module names in client libraries,
and customers use them in `import` or `use` statements. Therefore, choosing a
descriptive and language keyword-free filename does matter. For example, a file
called `import.proto` may be problematic in Python.
**Note:** The version **must not** be used as a filename, because this creates
bizarre imports in client libraries. Filenames such as `v3.proto` or
`v1beta1.proto` are prohibited.
### File layout
Individual files **should** place higher level and more important definitions
before lower level and less important definitions.
In a proto file, components **should** be in the following order, and each of
these **should** be separated by a blank line:
- Copyright and license notice (if applicable).
- The proto `syntax` statement.
- The proto `package` statement.
- Any `import` statements, in alphabetical order.
- Any file-level `option` statements.
- Any `service` definitions.
- Methods **should** be grouped by the resource they impact, and standard
methods **should** precede custom methods.
- Resource `message` definitions. A parent resource **must** be defined before
its child resources.
- The RPC request and response `message` definitions, in the same order of the
corresponding methods. Each request message **must** precede its
corresponding response message (if any).
- Any remaining `message` definitions.
- Any top-level `enum` definitions.
### Packaging annotations
Protocol buffers ships with annotations to declare the package or namespace
(depending on the vocabulary of the target language) of the generated files.
For example, setting `go_package` or `csharp_namespace` will override the
inferred package name.
When defining APIs, the following rules apply:
- Java
- The `java_package` annotation **must** be set. The correct value is usually
the proto package with the appropriate TLD prefixed. Example:
`com.google.example.v1`.
- The `java_multiple_files` annotation **must** be set to `true`.
- The `java_outer_classname` annotation **must** be set, and **should** be
set to the name of the proto filename, in `PascalCase`, with `Proto`
appended. Example: `LibraryProto`.
- Other languages
- Package or namespace directives for other languages **must** be set either
in every file in the proto package, or none of them. If they are set, the
values **must** be identical in every file.
- If any part of the protobuf package is a compound name (such as
accessapproval), C#, Ruby and PHP options **must** be specified in order
to take account of the word breaks using PascalCase (UpperCamelCase).
Example:
```proto
option csharp_namespace = "Google.Cloud.AccessApproval.V1";
option php_namespace = "Google\\Cloud\\AccessApproval\\V1";
option ruby_package = "Google::Cloud::AccessApproval::V1";
```
- The `go_package` value depends directly on how the Go code is managed i.e.
if the module name is based on the VCS provider or using a remote import
path, but often has a consistent structure.
- The module **may** differ based on product area e.g.
`google.cloud.accessapproval.v1` would be in module
`cloud.google.com/go/accessapproval`.
- The package import path **should** be derived from the proto package.
- An API version in the proto package **should** be prefixed with `api` e.g.
the proto package segment `v1` becomes `apiv1`.
- The terminal import path segment **should** be based on the product name
found within the proto package and **must** be suffixed with `pb` e.g.
`accessapproval` becomes `accessapprovalpb`.
- This value **should** be left to the team owning the generated code to
decide on.
All packaging annotations **should** be specified in alphabetical order of
name. Refer to the [Protobuf documentation][package docs] for more about
language package options.
**Important:** While languages other than Java have sensible defaults for APIs
which don't include compound names, be aware that _adding_ this annotation
(with a value not equivalent to the default) constitutes a breaking change in
that language. When releasing protos, be sure that omissions are intentional.
## Rationale
### Java packaging options
Set the option, `java_multiple_files`, to true to get a cleaner file structure.
Doing so instructs `protoc` to create one output file per Protobuf type, which
allows for more fine-grained imports. The option, `java_outer_classname`, is
required in combination with `java_multiple_files`. It instructs `protoc` to
wrap each compiled Protobuf type in a Java class whose name is the value of the
option. This prevents potential naming collisions between generated types.
### Go packaging option
The Go packaging option needs to be decided by the team that owns the generated
code, because it is directly tied to the source code management practices of the
team. Allowing every proto package to decide on their own Go package creates
inconsistencies and friction in management of the code. Within that owning team,
having a consistent structure in the Go package naming is critical to a
consistent end user experience.
## Changelog
- **2024-06-13**: Added guidance for Go packaging annotation.
- **2024-06-05**: Added rationale for Java packaging options.
- **2023-02-24**: Added guidance on protobuf syntax.
- **2022-10-18**: Added guidance on Ruby/PHP/C# options.
- **2019-11-18**: Added guidance on the packaging annotations.
[package docs]: https://protobuf.dev/programming-guides/proto3/#packages
---
id: 192
state: approved
created: 2019-07-25
placement:
category: polish
order: 20
---
# Documentation
Documentation is one of the most critical aspects of API design. Users of your
API are unable to dig into the implementation to understand the API better;
often, the API surface definition and its corresponding documentation will be
the only things a user has. Therefore, it is important that documentation be as
clear, complete, and unambiguous as possible.
## Guidance
In APIs defined in protocol buffers, public comments **must** be included over
every component (service, method, message, field, enum, and enum value) using
the protocol buffers comment format. This is important even in cases where the
comment is terse and uninteresting, as numerous tools read these comments and
use them.
Services, in particular, **should** have descriptive comments that explain what
the service is and what users are able to do with it.
**Note:** Many readers will not be native English speakers. Comments **should**
avoid jargon, slang, complex metaphors, pop culture references, or anything
else that will not easily translate. Additionally, many readers will have
different backgrounds and viewpoints; if writing examples involving people,
comments **should** use people who are non-controversial and no longer alive.
### Style
Comments **should** be in grammatically correct American English. However, the
first sentence of each comment **should** omit the subject and be in the
third-person present tense:
```proto
// Creates a book under the given publisher.
rpc CreateBook(CreateBookRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books"
body: "book"
};
}
```
### Descriptions
Descriptions of messages and fields **should** be brief but complete. Sometimes
comments are necessarily perfunctory because there is little to be said;
however, before jumping to that conclusion, consider whether some of the
following questions are relevant:
- What is it?
- How do you use it?
- What does it do if it succeeds? What does it do if it fails?
- Is it idempotent?
- What are the units? (Examples: meters, degrees, pixels)
- What are the side effects?
- What are common errors that may break it?
- What is the expected input format?
- What range of values does it accept? (Examples: `[0.0, 1.0)`, `[1, 10]`)
- Is the range inclusive or exclusive?
- For strings, what is the minimum and maximum length, and what characters
are allowed?
- If a value is above the maximum length, do you truncate or send an error?
- Is it always present? (Example: "Container for voting information. Present
only when voting information is recorded.")
- Does it have a default setting? (Example: "If `page_size` is omitted, the
default is 50.")
### Formatting
Any formatting in comments **must** be in [CommonMark][]. Headings and tables
**must not** be used, as these cause problems for several tools, and are
unsuitable for client library reference documentation.
Comments **should** use `code font` for field or method names and for literals
(such as `true`).
Raw HTML **must not** be used.
"ASCII art" attempts to present a diagram within the protos **must not** be
used. The Markdown within the protos is consumed by a large number of renderers,
and any ASCII art is very unlikely to be well-presented by all of them. If
a diagram is useful in order to understand the API, include a link to a
documentation page containing the diagram as an image.
### Cross-references
A comment can "link" to another component (service, method, message, field,
enum, or enum value) as a Markdown reference link. The reference **must** be one
of the following forms:
- The fully-qualified name of the element e.g. `[Book][google.example.v1.Book]`
- A scope-relative reference qualified e.g. `[Sci-Fi genre][Genre.GENRE_SCI_FI]`
- An implied reference e.g. `[Book][]` which equates to `[Book][Book]`
These references are resolved as per [name resolution][] rules.
Containing fields names **must not** be used in references. They will not
resolve. The original definition **must** be referenced instead. For example,
`[author][Book.author.family_name]` where `author` is a field of `Book`, will
not resolve, but `[author][Author.family_name]` will.
[name resolution]: https://protobuf.dev/programming-guides/proto3/#name-resolution
### External links
Comments **may** link to external pages to provide background information
beyond what is described in the public comments themselves. External links
**must** use absolute (rather than relative) URLs, including the protocol
(usually `https`), and **should not** assume the documentation is located on
any particular host. For example:
`[Spanner Documentation](https://cloud.google.com/spanner/docs)`
### Trademarked names
When referring to the proper, trademarked names of companies or products in
comments, acronyms **should not** be used, unless the acronym is such dominant
colloquial use that avoiding it would obscure the reference (example: [IBM][]).
Comments **should** spell and capitalize trademarked names consistent with the
trademark owner's current branding.
### Deprecations
To deprecate a component (service, method, message, field, enum, or enum value),
the `deprecated` [option](https://developers.google.com/protocol-buffers/docs/proto#options)
**must** be set to `true`, and the first line of the respective comment
**must** start with `"Deprecated: "` and provide alternative solutions for
developers. If there is no alternative solution, a deprecation reason **must**
be given.
### Internal comments
<!-- TODO: This does not work outside of Google.
We should probably try to get that fixed. -->
Comments **may** be explicitly marked as internal by wrapping internal content
in `(--` and `--)`.
Non-public links, internal implementation notes (such as `TODO` and `FIXME`
directives), and other such material **must** be marked as internal.
**Note:** Comments **should** use only leading comments (not trailing comments
or detached comments). In particular, comments **must not** use both a leading
and trailing comment to describe any component, because this is a common source
of inadvertent omissions of the internal content annotation.
[commonmark]: https://commonmark.org/
[ibm]: https://en.wikipedia.org/wiki/IBM
## Changelog
- **2024-10-29**: Include cross-reference resolution rules.
- **2023-08-11**: Expand deprecated comment requirement to all components.
- **2021-04-20**: Added guidance for deprecated services and RPCs.
- **2020-04-01**: Added guidance requiring absolute URLs for external links.
- **2020-02-14**: Added guidance around the use of trademarked names.
- **2019-09-23**: Added guidance about not using both leading and trailing
comments.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
---
id: 193
state: approved
created: 2019-07-26
placement:
category: polish
order: 30
---
# Errors
Effective error communication is an important part of designing simple
and intuitive APIs. Services returning standardized error responses
enable API clients to construct centralized common error handling logic.
This common logic simplifies API client applications and eliminates the
need for cumbersome custom error handling code.
## Guidance
Services **must** return a [`google.rpc.Status`][Status] message when an
API error occurs, and **must** use the canonical error codes defined in
[`google.rpc.Code`][Code]. More information about the particular codes
is available in the [gRPC status code documentation][].
Error messages **should** help a reasonably technical user *understand*
and *resolve* the issue, and **should not** assume that the user is an
expert in your particular API. Additionally, error messages **must not**
assume that the user will know anything about its underlying
implementation.
Error messages **should** be brief but actionable. Any extra information
**should** be provided in the `details` field. If even more information
is necessary, you **should** provide a link where a reader can get more
information or ask questions to help resolve the issue. It is also
important to [set the right tone][writing-tone] when writing messages.
The following sections describe the fields of `google.rpc.Status`.
### Status.message
The `message` field is a developer-facing, human-readable "debug message"
which **should** be in English. (Localized messages are expressed using
a `LocalizedMessage` within the `details` field. See
[`LocalizedMessage`](#localizedmessage) for more details.) Any dynamic aspects of
the message **must** be included as metadata within the `ErrorInfo` that appears
in [`details`](#status-details).
The message is considered a problem description. It is intended for
developers to understand the problem and is more detailed than
[`ErrorInfo.reason`][ErrorInfo-reason], discussed [later](#errorinfo).
Messages **should** use simple descriptive language that is easy to understand
(without technical jargon) to clearly state the problem that results in an
error, and offer an actionable resolution to it.
For pre-existing (brownfield) APIs which have previously returned errors
without machine-readable identifiers, the value of `message` **must**
remain the same for any given error. For more information, see
[Changing Error Messages](#changing-error-messages).
### Status.code
The `code` field is the status code, which **must** be the numeric value of
one of the elements of the [`google.rpc.Code`][Code] enum.
For example, the value `5` is the numeric value of the `NOT_FOUND`
enum element.
### Status.details
The `details` field allows messages with additional error information to
be included in the error response, each packed in a `google.protobuf.Any`
message.
Google defines a set of [standard detail payloads][details] for error
details, which cover most common needs for API errors.
Services **should** use these standard detail payloads when feasible.
Each type of detail payload **must** be included at most once. For
example, there **must not** be more than one [`BadRequest`][BadRequest]
message in the `details`, but there **may** be a `BadRequest` and a
[`PreconditionFailure`][PreconditionFailure].
All error responses **must** include an `ErrorInfo` within `details`. This
provides machine-readable identifiers so that users can write code against
specific aspects of the error.
The following sections describe the most common standard detail payloads.
#### ErrorInfo
The [`ErrorInfo`][ErrorInfo] message is the primary way to send a
machine-readable identifier. Contextual information **should** be
included in `metadata` in `ErrorInfo` and **must** be included if it
appears within an error message.
The `reason` field is a short snake_case description of the cause of the
error. Error reasons are unique within a particular domain of errors.
The reason **must** be at most 63 characters and match a regular expression of
`[A-Z][A-Z0-9_]+[A-Z0-9]`. (This is UPPER_SNAKE_CASE, without leading
or trailing underscores, and without leading digits.)
The reason **should** be terse, but meaningful enough for a human reader to
understand what the reason refers to.
Good examples:
- `CPU_AVAILABILITY`
- `NO_STOCK`
- `CHECKED_OUT`
- `AVAILABILITY_ERROR`
Bad examples:
- `THE_BOOK_YOU_WANT_IS_NOT_AVAILABLE` (overly verbose)
- `ERROR` (too general)
The `domain` field is the logical grouping to which the `reason` belongs.
The domain **must** be a globally unique value, and is typically the name of the service
that generated the error, e.g. `pubsub.googleapis.com`.
The (reason, domain) pair form a machine-readable way of identifying a particular error.
Services **must** use the same (reason, domain) pair for the same error, and
**must not** use the same (reason, domain) pair for logically different errors.
The decision about whether two errors are "the same" or not is not always clear, but
**should** generally be considered in terms of the expected action a client might take
to resolve them.
The `metadata` field is a map of key/value pairs providing additional
dynamic information as context. Each key within `metadata` **must** be at most
64 characters long, and conform to the regular expression `[a-z][a-zA-Z0-9-_]+`.
Any request-specific information which contributes to the `Status.message` or
`LocalizedMessage.message` messages **must** be represented within `metadata`.
This practice is critical so that machine actors do not need to parse error
messages to extract information.
For example consider the following message:
> An &lt;e2-medium&gt; VM instance with &lt;local-ssd=3,nvidia-t4=2&gt; is currently unavailable
> in the &lt;us-east1-a&gt; zone. Consider trying your request in the &lt;us-central1-f,us-central1-c&gt;
> zone(s), which currently has/have capacity to accommodate your request. Alternatively,
> you can try your request again with a different VM hardware configuration
> or at a later time. For more information, see the troubleshooting documentation.
The `ErrorInfo.metadata` map for the same error could be:
- `"zone": "us-east1-a"`
- `"vmType": "e2-medium"`
- `"attachment": "local-ssd=3,nvidia-t4=2"`
- `"zonesWithCapacity": "us-central1-f,us-central1-c"`
Additional contextual information that does not appear in an error message
**may** also be included in `metadata` to allow programmatic use by the client.
The metadata included for any given (reason,domain) pair can evolve over time:
- New keys **may** be included
- All keys that have been included **must** continue to be included (but may have empty values)
In other words, once a user has observed a given key for a (reason, domain) pair, the
service **must** allow them to rely on it continuing to be present in the future.
The set of keys provided in each (reason, domain) pair is independent from other pairs,
but services **should** aim for consistent key naming. For example, two error reasons
within the same domain should not use metadata keys of `vmType` and `virtualMachineType`.
#### LocalizedMessage
[`google.rpc.LocalizedMessage`][LocalizedMessage] is used to provide an error
message which **should** be localized to a user-specified locale where
possible.
If the [`Status.message`](#status-message) field has a sub-optimal value
which cannot be changed due to the constraints in the
[Changing Error Messages](#changing-error-messages) section, `LocalizedMessage`
**may** be used to provide a better error message even when no user-specified
locale is available.
Regardless of how the locale for the message was determined, both the `locale`
and `message` fields **must** be populated.
The `locale` field specifies the locale of the message,
following [IETF bcp47](https://www.rfc-editor.org/rfc/bcp/bcp47.txt) (Tags for
Identifying Languages). Example values: `"en-US"`, `"fr-CH"`, `"es-MX"`.
The `message` field contains the localized text itself. This
**should** include a brief description of the error and a call to action
to resolve the error. The message **should** include contextual information
to make the message as specific as possible. Any contextual information
in the message **must** be included in `ErrorInfo.metadata`. See
[`ErrorInfo`](#errorinfo) for more details of how contextual information
may be included in a message and the corresponding metadata.
The `LocalizedMessage` payload **should** contain the complete resolution
to the error. If more information is needed than can reasonably fit in this
payload, then additional resolution information **must** be provided in
a `Help` payload. See the [Help](#help) section for guidance.
#### Help
When other textual error messages (in `Status.message` or
`LocalizedMessage.message`) don't provide the user sufficient
context or actionable next steps, or if there are multiple points of
failure that need to be considered in troubleshooting, a link to
supplemental troubleshooting documentation **must** be provided in the
`Help` payload.
Provide this information in addition to a clear problem definition and
actionable resolution, not as an alternative to them. The linked
documentation **must** clearly relate to the error. If a single page
contains information about multiple errors, the
[`ErrorInfo.reason`](#errorinfo) value **must** be used to narrow down
the relevant information.
The `description` field is a textual description of the linked information.
This **must** be suitable to display to a user as text for a hyperlink.
This **must** be plain text (not HTML, Markdown etc).
Example `description` value: `"Troubleshooting documentation for STOCKOUT errors"`
The `url` field is the URL to link to. This **must** be an absolute URL,
including scheme.
Example `url` value:
`"https://cloud.google.com/compute/docs/resource-error"`
For publicly-documented services, even those with access controls on actual
usage, the linked content **must** be accessible without authentication.
For privately-documented services, the linked content **may** require
authentication.
### Error messages
Textual error messages can be present in both `Status.message` and
`LocalizedMessage.message` fields. Messages **should** be succinct but
actionable, with request-specific information (such as a resource name
or region) providing precise details where appropriate. Any request-specific
details **must** be present in [`ErrorInfo.metadata`](#errorinfo).
#### Changing error messages
Changing the content of `Status.message` over time must be done carefully,
to avoid breaking clients who have previously had to rely on the message
for all information. See [the rationale section](#updating-statusmessage)
for more details.
For a given RPC:
- If the RPC has *always* returned `ErrorInfo` with machine-readable
information, the content of `Status.message` **may** change over time.
(For example, the API producer may provide a clearer explanation,
or more request-specific information.)
- Otherwise, the content of `Status.message` **must** be stable,
providing the same text with the same request-specific information.
Instead of changing `Status.message`, the API **should** include a
[`LocalizedMessage`](#localizedmessage) within `Status.details`.
Even if an RPC has always returned `ErrorInfo`, the API **may** keep
the existing `Status.message` stable and add a
[`LocalizedMessage`](#localizedmessage) within `Status.details`.
The content of `LocalizedMessage.details` **may** change over time.
### Partial errors
APIs **should not** support partial errors. Partial errors add
significant complexity for users, because they usually sidestep the use
of error codes, or move those error codes into the response message,
where the user **must** write specialized error handling logic to
address the problem.
However, occasionally partial errors are necessary, particularly in bulk
operations where it would be hostile to users to fail an entire large
request because of a problem with a single entry.
Methods that require partial errors **should** use [long-running
operations][], and the method **should** put partial failure information
in the metadata message. The errors themselves **must** still be
represented with a [`google.rpc.Status`][Status] object.
### Permission Denied
If the user does not have permission to access the resource or parent,
regardless of whether or not it exists, the service **must** error with
`PERMISSION_DENIED` (HTTP 403). Permission **must** be checked prior to
checking if the resource or parent exists.
If the user does have proper permission, but the requested resource or
parent does not exist, the service **must** error with `NOT_FOUND` (HTTP
404).
## HTTP/1.1+JSON representation
When clients use HTTP/1.1 as per [AIP-127](./0127.md), the error information
is returned in the body of the response, as a JSON object. For backward
compatibility reasons, this does not map precisely to `google.rpc.Status`,
but contains the same core information. The schema is defined in the following proto:
```proto
message Error {
message Status {
// The HTTP status code that corresponds to `google.rpc.Status.code`.
int32 code = 1;
// This corresponds to `google.rpc.Status.message`.
string message = 2;
// This is the enum version for `google.rpc.Status.code`.
google.rpc.Code status = 4;
// This corresponds to `google.rpc.Status.details`.
repeated google.protobuf.Any details = 5;
}
Status error = 1;
}
```
The most important difference is that the `code` field in the JSON is an HTTP status code,
*not* the direct value of `google.rpc.Status.code`. For example, a `google.rpc.Status`
message with a `code` value of 5 would be mapped to an object including the following
code-related fields (as well as the message, details etc):
```json
{
"error": {
"code": 404, // The HTTP status code for "not found"
"status": "NOT_FOUND" // The name in google.rpc.Code for value 5
}
}
```
The following JSON shows a fully populated HTTP/1.1+JSON representation of an error response.
```json
{
"error": {
"code": 429,
"message": "The zone 'us-east1-a' does not have enough resources available to fulfill the request. Try a different zone, or try again later.",
"status": "RESOURCE_EXHAUSTED",
"details": [
{
"@type": "type.googleapis.com/google.rpc.ErrorInfo",
"reason": "RESOURCE_AVAILABILITY",
"domain": "compute.googleapis.com",
"metadata": {
"zone": "us-east1-a",
"vmType": "e2-medium",
"attachment": "local-ssd=3,nvidia-t4=2",
"zonesWithCapacity": "us-central1-f,us-central1-c"
}
},
{
"@type": "type.googleapis.com/google.rpc.LocalizedMessage",
"locale": "en-US",
"message": "An <e2-medium> VM instance with <local-ssd=3,nvidia-t4=2> is currently unavailable in the <us-east1-a> zone. Consider trying your request in the <us-central1-f,us-central1-c> zone(s), which currently has/have capacity to accommodate your request. Alternatively, you can try your request again with a different VM hardware configuration or at a later time. For more information, see the troubleshooting documentation."
},
{
"@type": "type.googleapis.com/google.rpc.Help",
"links": [
{
"description": "Additional information on this error",
"url": "https://cloud.google.com/compute/docs/resource-error"
}
]
}
]
}
}
```
## Rationale
### Requiring ErrorInfo
`ErrorInfo` is required because it further identifies an error. With
only approximately twenty [available values][Code] for `Status.status`,
it is difficult to disambiguate one error from another across an entire
[API Service][API Service].
Also, error messages often contain dynamic segments that express
variable information, so there needs to be machine-readable component of
*every* error response that enables clients to use such information
programmatically.
### Including LocalizedMessage
`LocalizedMessage` was selected as the location to present alternate
error messages. While `LocalizedMessage` **may** use a locale specified
in the request, a service **may** provide a `LocalizedMessage` even without
a user-specified locale, typically to provide a better error message in
[situations where `Status.message` cannot be changed](#updating-statusmessage).
Where the locale is not specified by the user, it **should** be `en-US`
(US English).
A service **may** include `LocalizedMessage` even when the same message is
provided in `Status.message` and when localization into a user-specified locale
is not supported. Reasons for this include:
- An intention to support user-specified localization in the near future, allowing
clients to consistently use `LocalizedMessage` and not change their error-reporting
code when the functionality is introduced.
- Consistency across all RPCs within a service: if some RPCs include
`LocalizedMessage` and some only use `Status.message` for error messages, clients
have to be aware of which RPCs will do what, or implement a fall-back mechanism.
Providing `LocalizedMessage` on all RPCs allows simple and consistent client code
to be written.
### Updating Status.message
If a client has ever observed an error with `Status.message` populated
(which it always will be) but without `ErrorInfo`, the developer of that client
may well have had to resort to parsing `Status.message` in order to find out
information beyond just what `Status.code` conveys. That information may be
found by matching specific text (e.g. "Connection closed with unknown cause")
or by parsing the message to find out metadata values (e.g. a region with
insufficient resources). At that point, `Status.message` is implicitly part
of the API contract, so **must not** be updated - that would be a breaking
change. This is one reason for introducing `LocalizedMessage` into the
`Status.details`.
RPCs which have **always** included `ErrorInfo` are in a better position:
the contract is then more about the stability of `ErrorInfo` for any given
error. The reason and domain need to be consistent over time, and the
metadata provided for any given (reason,domain) can only be expanded.
It's still possible that clients could be parsing `Status.message` instead of
using `ErrorInfo`, but they will always have had a more robust option
available to them.
## Further reading
- For which error codes to retry, see [AIP-194][aip-194].
- For how to retry errors in client libraries, see
[AIP-4221][aip-4221].
## Changelog
- **2024-10-18**: Rewrite/restructure for clarity.
- **2024-01-10**: Incorporate guidance for writing effective messages.
- **2023-05-17**: Change the recommended language for `Status.message` to be
the service's native language rather than English.
- **2023-05-17**: Specify requirements for changing error messages.
- **2023-05-10**: Require [`ErrorInfo`][ErrorInfo] for all error responses.
- **2023-05-04**: Require uniqueness by message type for error details.
- **2022-11-04**: Added guidance around PERMISSION_DENIED errors previously
found in other AIPs.
- **2022-08-12**: Reworded/Simplified intro to add clarity to the intent.
- **2020-01-22**: Added a reference to the [`ErrorInfo`][ErrorInfo] message.
- **2019-10-14**: Added guidance restricting error message mutability to if
there is a machine-readable identifier present.
- **2019-09-23**: Added guidance about error message strings being able to
change.
<!-- prettier-ignore-start -->
[aip-4221]: https://google.aip.dev/client-libraries/4221
[details]: https://github.com/googleapis/googleapis/blob/master/google/rpc/error_details.proto
[ErrorInfo]: https://github.com/googleapis/googleapis/blob/6f3fcc058ff29989f6d3a71557a44b5e81b897bd/google/rpc/error_details.proto#L27-L76
[ErrorInfo-reason]: https://github.com/googleapis/googleapis/blob/master/google/rpc/error_details.proto#L57
[PreconditionFailure]: https://github.com/googleapis/googleapis/blob/6f3fcc058ff29989f6d3a71557a44b5e81b897bd/google/rpc/error_details.proto#L139-L166
[BadRequest]: https://github.com/googleapis/googleapis/blob/477a59d764428136ba1d857a9633c0d231de6efa/google/rpc/error_details.proto#L168-L218
[LocalizedMessage]: https://github.com/googleapis/googleapis/blob/e9897ed945336e2dc967b439ac7b4be6d2c62640/google/rpc/error_details.proto#L275-L285
[grpc status code documentation]: https://github.com/grpc/grpc/blob/master/doc/statuscodes.md
[Code]: https://github.com/googleapis/googleapis/blob/master/google/rpc/code.proto
[Status]: https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto
[Status details]: https://github.com/googleapis/googleapis/blob/aeae5ea2b01ece6c0cee046ae84b881cdc62b95d/google/rpc/status.proto#L46-L48
[long-running operations]: ./0151.md
[API Service]: https://cloud.google.com/apis/design/glossary#api_service
[writing-tone]: https://developers.google.com/tech-writing/error-messages/set-tone
[Error model]: https://cloud.google.com/apis/design/errors#error_model
[Help]: https://github.com/googleapis/googleapis/blob/master/google/rpc/error_details.proto#L256
[resource-error]: https://cloud.google.com/compute/docs/resource-error
[InvalidArgument]: https://github.com/googleapis/googleapis/blob/d4acb64370d333024a167551e7da854506109ba2/google/rpc/code.proto#L52-L58
<!-- prettier-ignore-end -->
---
id: 194
state: approved
created: 2019-08-16
placement:
category: polish
order: 40
---
# Automatic retry configuration
RPCs sometimes fail. When one does, the client performing the RPC needs to know
whether it is safe to retry the operation. When status codes are used
consistently across multiple APIs, clients can respond to failures
appropriately.
## Guidance
Clients **should** automatically retry requests for which repeated runs would
not cause unintended state changes, which are non-transactional, and which are
[unary](https://grpc.io/docs/guides/concepts/).
Clients **should not** automatically retry transactional requests; instead
these requests **should** have application-level retry logic that retries the
entire transaction block from the start.
Clients **should not** automatically retry requests in which repeated runs
would cause unintended state changes.
**Note:** This AIP does not cover client streaming or bi-directional streaming.
**Note:** For client side retry behavior in the client libraries: see
[AIP-4221](https://aip.dev/4221).
### Retryable codes
For methods listed as retryable above, clients **should** retry the following
error codes:
- `UNAVAILABLE`: This code generally results from network hiccups, and is
generally transient. It is retryable under the expectation that the
connection will become available (soon).
### Non-retryable codes
The following codes **should not** be automatically retried for any request:
- `OK`: The request succeeded.
- `CANCELLED`: An application can cancel a request, which **must** be honored.
- `DEADLINE_EXCEEDED`: An application can set a deadline, which **must** be
honored.
- `INVALID_ARGUMENT`: Retrying a request with an invalid argument will never
succeed.
- `DATA_LOSS`: This is an unrecoverable error and **must** immediately be
surfaced to the application.
### Generally non-retryable codes:
The following codes generally **should not** be automatically retried for any
request:
- `RESOURCE_EXHAUSTED`: This code may be a signal that quota is exhausted.
Retries therefore may not be expected to work for several hours; meanwhile
the retries may have billing implications. If `RESOURCE_EXHAUSTED` is used
for other reasons than quota and the expected time for the resource to become
available is much shorter, it may be retryable.
- `INTERNAL`: This code generally means that some internal part of the system
has failed, and usually means a bug should be filed against the system. These
**should** immediately be surfaced to the application.
- `UNKNOWN`: Unlike `INTERNAL`, this code is reserved for truly
unknown-to-the-system errors, and therefore may not be safe to retry. These
**should** immediately be surfaced to the application.
- `ABORTED`: This code typically means that the request failed due to a
sequencer check failure or transaction abort. These **should not** be retried
for an individual request; they **should** be retried at a level higher (the
entire transaction, for example).
Some codes **may** be automatically retried if a system is designed without
synchronization or signaling between various components. For example, client
might retry `NOT_FOUND` on a read operation, which is designed to hang forever
until the resource is created. However, these types of systems are generally
discouraged.
Therefore, the following codes **should not** be automatically retried for any
request:
- `NOT_FOUND`: A client **should not** retry until a resource is created.
- `ALREADY_EXISTS`: A client **should not** retry until a resource is deleted.
- `PERMISSION_DENIED`: A client **should not** retry until it has permission.
- `UNAUTHORIZED`: A client **should not** retry until it is authorized.
- `UNAUTHENTICATED`: A client **should not** retry until it is authenticated.
- `FAILED_PRECONDITION`: A client **should not** retry until system state
changes.
- `OUT_OF_RANGE`: A client **should not** retry until the range is extended.
- `UNIMPLEMENTED`: A client **should not** retry until the RPC is implemented.
## Further reading
- For parallel or retried request disambiguation, see [AIP-154](./0154.md).
---
id: 200
state: approved
created: 2018-06-28
placement:
category: meta
order: 30
redirect_from: /not-precedent
---
# Precedent
Many times, APIs are written in ways that do not match new guidance that is
added to these standards after those APIs have already been released.
Additionally, sometimes it can make sense to intentionally violate standards
for particular reasons, such as maintaining consistency with established
systems, meeting stringent performance requirements, or other practical
concerns. Finally, as carefully as everyone reviews APIs before they are
released, sometimes mistakes can slip through.
Since it often is not feasible to fix past mistakes or make the standards serve
every use case, APIs may be stuck with these exceptions for quite some time.
Further, since new APIs often base their designs (names, types, structures,
etc) on existing APIs, it is possible that a standards violation in one API
could spill over into other APIs, even if original reason for the exception is
not applicable to the other APIs.
As a result of this problem, it is important to "stop the bleeding" of these
standards exceptions into new APIs, and additionally document the reasons for
each exception so that historical wisdom is not lost.
## Guidance
If an API violates the AIP standards for any reason, there **must** be an
internal comment linking to this document using its descriptive link
([aip.dev/not-precedent]()) to ensure others do not copy the violations or cite
the errors as precedent of a "previously approved API".
The comment should also include an explanation of what violates standards and
why it is necessary. For example:
```proto
message DailyMaintenanceWindow {
// Time within the maintenance window to start the maintenance operations.
// It must use the format "HH MM", where HH : [00-23] and MM : [00-59] GMT.
// (-- aip.dev/not-precedent: This was designed for consistency with crontab,
// and preceded the AIP standards.
// Ordinarily, this type should be `google.type.TimeOfDay`. --)
string start_time = 2;
// Output only. Duration of the time window, automatically chosen to be
// smallest possible in the given scenario.
// (-- aip.dev/not-precedent: This preceded the AIP standards.
// Ordinarily, this type should be `google.protobuf.Duration`. --)
string duration = 3;
}
```
**Important:** APIs should only be considered to be precedent-setting if they
are in beta or GA.
### Local consistency
If an API violates a standard throughout, it would be jarring and frustrating
to users to break the existing pattern only for the sake of adhering to the
global standard.
For example, if all of an API's resources use `creation_time` (instead of the
standard field `create_time` described in [AIP-142][]), a new resource in that
API should continue to follow the local pattern.
However, others who might otherwise copy that API should be made aware that
this is contra-standard and not something to cite as precedent when launching
new APIs.
```proto
// ...
message Book {
// (-- aip.dev/not-precedent: This field was present before there was a
// standard field.
// Ordinarily, it should be spelled `create_time`. --)
google.protobuf.Timestamp creation_time = 1;
}
// ...
message Author {
// (-- aip.dev/not-precedent: `Book` had `creation_time` before there was
// a standard field, so we match that here for consistency. Ordinarily,
// this would be spelled `create_time`. --)
google.protobuf.Timestamp creation_time = 1;
}
```
### Pre-existing functionality
Standards violations are sometimes overlooked before launching, resulting in
APIs that become stable and therefore can not easily be modified. Additionally,
a stable API may pre-date a standards requirement.
In these scenarios, it is difficult to make the API fit the standard. However,
the API should still cite that the functionality is contra-standard so that
other APIs do not copy the mistake and cite the existing API as a reason why
their design should be approved.
### Adherence to external spec
Occasionally, APIs must violate standards because specific requests are
implementations of an external specification (for example, OAuth), and their
specification may be at odds with AIP guidelines. In this case, it is likely to
be appropriate to follow the external specification.
### Adherence to existing systems
Similar to the example of an external specification above, it may be proper for
an API to violate AIP guidelines to fit in with an existing system in some way.
This is a fundamentally similar case where it is wise to meet the customer
where they are. A potential example of this might be integration with or
similarity to a partner API.
### Expediency
Sometimes there are users who need an API surface by a very hard deadline or
money walks away. Since most APIs serve a business purpose, there will be times
when an API could be better but cannot get it that way and into users' hands
before the deadline. In those cases, API review councils **may** grant
exceptions to ship APIs that violate guidelines due to time and business
constraints.
### Technical concerns
Internal systems sometimes have very specific implementation needs (e.g., they
rely on operation transforms that speak UTF-16, not UTF-8) and adhering to AIP
guidelines would require extra work that does not add significant value to API
consumers. Future systems which are likely to expose an API at some point
should bear this in mind to avoid building underlying infrastructure which
makes it difficult to follow AIP guidelines.
## Changelog
- **2020-03-27**: Reworded much of this AIP to follow [AIP-8][], and remove
first and second person. No semantic changes.
- **2019-05-04**: Changed to a public link ([aip.dev/not-precedent]()), and
changed references to "the style guide" to use the more generic term
"standards" (to account for a general shift to AIPs).
[aip-8]: ./0008.md
[aip-142]: ./0142.md
---
id: 202
state: approved
created: 2023-09-22
placement:
category: fields
order: 0
---
# Fields
The [`google.api.FieldInfo`][field info proto] type, through its accompanying
extension `google.api.field_info`, enriches a field's schema beyond the basic
name and type information.
## Guidance
Decorating a field with `google.api.field_info` is only necessary when
explicitly stated in this AIP or another that leverages `google.api.FieldInfo`
information. As such, the guidance herein applies to those scenarios as well.
### Format
Fields with a primitive type can still have a specific format. To convey that
type format, the `FieldInfo.Format` enumeration is used via the
`(google.api.field_info).format` extension field. The following guidance conveys
the meaning of and requirements for use of each `FieldInfo.Format` value.
#### UUID4
The `UUID4` format represents a UUID version 4 value as governed by
[RFC 4122][]. It **must** only be used on a field of type `string`.
Such a value **may** be normalized by the service to entirely lowercase letters.
For example, the value `F47AC10B-58CC-0372-8567-0E02B2C3D479` would be
normalized to `f47ac10b-58cc-0372-8567-0e02b2c3d479`.
As such, equivalence comparison **must not** be done via primitive text
comparison. Instead, an [RFC 4122][] compliant implementation **must** be used.
#### IPv4
The `IPV4` format represents an IP v4 address as governed by [RFC 791][]. It
**must** only be used on a field of type `string`.
Such a value **may** be condensed by the service, with leading zeros in each
octet stripped. For example, `001.022.233.040` would be condensed to
`1.22.233.40`.
As such, equivalence comparison **must not** be done via primitive text
comparison. Instead, an [RFC 791][] compliant implementation **must** be used.
#### IPv6
The `IPV6` format represents an IP v6 address as governed by [RFC 4291][]. It
**must** only be used on a field of type `string`.
Such a value **may** be normalized by the service to entirely lowercase letters
with zeros compressed, following [RFC 5952][]. For example, the value
`2001:0DB8:0::0` would be normalized to `2001:db8::`.
As such, equivalence comparison **must not** be done via primitive text
comparison. Instead, an [RFC 4291][] compliant implementation **must** be used.
#### IPv4 or IPv6
The `IPV4_OR_IPV6` value indicates that the field can be either an IP v4 or v6
address, as described in the [IPv4](#ipv4) and [IPv6](#ipv6) sections.
#### Format Compatibility
Adding a format specifier to an existing, unspecified field **is not** backwards
compatible, *unless* the field in question has always conformed to the format
being specified.
Changing an existing format specifier to a different one in all cases **is not**
backwards compatible.
#### Extending Format
Any new `FieldInfo.Format` value **must** be governed by an
[IETF-approved RFC][ietf rfc] or a [Google-approved AIP](./0001.md).
## Rationale
#### Why add a format specifier?
The format of a primitive-typed field can be critical to its usability. Some
programming languages may convey a specific type format as a standalone type,
as Java does with [UUID][java uuid]. Most have specific structural requirements
that are validated by the service, so conveying the format to the user ahead of
time is critical to their experience.
#### Why discourage primitive equality comparisons?
The text representations of the supported formats have many nuances and
transforming the value into a canonical representation is non-trivial. As such,
aligning implementations between each consumer and each service without any
issue is infeasible.
#### Why document value normalizations?
While primitive comparison is not recommended for any of the supported formats,
uniform normalization of values is important to set consumer expectations, and
create a user-friendly surface.
#### Why require an RFC or AIP for new formats?
Those formats which are sufficiently standardized to merit an RFC or AIP are
stable enough and widely enough known to be incorporated as a supported value
and see usage in Google APIs. Requiring such extra guidance means that governing
the format specification is not the responsibility of the `FieldInfo.Format`
enumeration itself.
[field info proto]: https://github.com/googleapis/googleapis/blob/master/google/api/field_info.proto
[rfc 4122]: https://datatracker.ietf.org/doc/html/rfc4122
[rfc 791]: https://datatracker.ietf.org/doc/html/rfc791
[rfc 4291]: https://datatracker.ietf.org/doc/html/rfc4291#section-2.2
[rfc 5952]: https://datatracker.ietf.org/doc/html/rfc5952
[ietf rfc]: https://www.ietf.org/standards/rfcs
[java uuid]: https://docs.oracle.com/javase/8/docs/api/java/util/UUID.html
---
id: 203
state: approved
created: 2018-07-17
placement:
category: fields
order: 15
---
# Field behavior documentation
When defining fields in protocol buffers, it is customary to explain to users
certain aspects of the field's behavior (such as whether it is required or
optional). Additionally, it can be useful for other tools to understand this
behavior (for example, to optimize client library signatures).
## Guidance
APIs use the `google.api.field_behavior` annotation to describe well-understood
field behavior, such as a field being required or immutable.
```proto
// The audio data to be recognized.
RecognitionAudio audio = 2 [(google.api.field_behavior) = REQUIRED];
```
- APIs **must** apply the `google.api.field_behavior` annotation on every field
on a message or sub-message used in a request.
- The annotation **must** include any [google.api.FieldBehavior][] values that
accurately describe the behavior of the field.
- `FIELD_BEHAVIOR_UNSPECIFIED` **must not** be used.
- APIs **must** at minimum use one of `REQUIRED`, `OPTIONAL`, or `OUTPUT_ONLY`.
**Warning:** Although `field_behavior` does not impact proto-level behavior,
many clients (e.g. CLIs and SDKs) rely on them to generate code. Thoroughly
review and consider which values are relevant when adding a new field.
Fields with no annotation are interpreted as `OPTIONAL` for
backwards-compatility. Nontheless, this annotation **must not** be omitted.
**Note:** The vocabulary given in this document is for _descriptive_ purposes
only, and does not itself add any validation. The purpose is to consistently
document this behavior for clients.
### field behavior of nested messages
`google.api.field_behavior` annotations on a nested message are independent of
the annotations of the parent.
For example, a nested message can have a field behavior of `REQUIRED` while the
parent field can be `OPTIONAL`:
```proto
message Title {
string text = 1 [(google.api.field_behavior) = REQUIRED];
}
message Slide {
Title title = 1 [(google.api.field_behavior) = OPTIONAL];
}
```
In the case above, if a `title` is specified, the `text` field is required.
## Vocabulary
### Identifier
The use of `IDENTIFIER` indicates that a field within a resource message is used
to identify the resource. It **must** be attached to the `name` field and **must
not** be attached to any other field (see [fields representing resource names]).
The `IDENTIFIER` value conveys that the field is not accepted as input (i.e.
`OUTPUT_ONLY`) in the context of a create method, while also being considered
`IMMUTABLE` and accepted as input for mutation methods that accept the
resource as the primary input e.g. [Standard Update][aip-134].
This annotation **must not** be applied to references to other resources within
a message.
### Immutable
The use of `IMMUTABLE` indicates that a field on a resource cannot be changed
after its creation. This can apply to either fields that are input or outputs,
required or optional.
When a service receives an immutable field in an update request (or similar),
even if included in the update mask, the service **should** ignore the field if
the value matches, but **should** error with `INVALID_ARGUMENT` if a change is
requested.
Potential use cases for immutable fields (this is not an exhaustive list) are:
- Attributes of resources that are not modifiable for the lifetime of the
application (e.g. a disk type).
**Note:** Fields which are "conditionally immutable" **must not** be given the
immutable annotation.
### Input only
The use of `INPUT_ONLY` indicates that the field is provided in requests and
that the corresponding field will not be included in output.
Additionally, a field **should** only be described as input only if it is a
field in a resource message or a field of a message included within a resource
message. Notably, fields in request messages (a message which only ever acts as
an argument to an RPC, with a name usually ending in `Request`) **should not**
be described as input only because this is already implied.
Potential use cases for input only fields (this is not an exhaustive list) are:
- The `ttl` field as described in AIP-214.
**Warning:** Input only fields are rare and should be considered carefully
before use.
### Optional
The use of `OPTIONAL` indicates that a field is not required.
A field **may** be described as optional if it is a field on a request message
(a message that is an argument to an RPC, usually ending in `Request`), or a
field on a submessage.
### Output only
The use of `OUTPUT_ONLY` indicates that the field is provided in responses, but
that including the field in a message in a request does nothing (the server
**must** clear out any value in this field and **must not** throw an error as a
result of the presence of a value in this field on input). Similarly, services
**must** ignore the presence of output only fields in update field masks (see:
AIP-161).
Additionally, a field **should** only be described as output only if it is a
field in a resource message, or a field of a message farther down the tree.
Notably, fields in response messages (a message which only ever acts as a
return value to an RPC, usually ending in `Response`) **should not** be
described as output only because this is already implied.
Output only fields **may** be set to empty values if appropriate to the API.
Potential use cases for output only fields (this is not an exhaustive list)
are:
- Create or update timestamps.
- Derived or structured information based on original user input.
- Properties of a resource assigned by the service which can not be altered.
### Required
The use of `REQUIRED` indicates that the field **must** be present (and set to
a non-empty value) on the request or resource.
A field **should** only be described as required if _either_:
- It is a field on a resource that a user provides somewhere as input. In this
case, the resource is only valid if a "truthy" value is _stored_.
- When [creating][aip-133] the resource, a value **must** be provided for the
field on the create request.
- When [updating][aip-134] the resource, the user **may** omit the field
provided that the field is also absent from the field mask, indicating no
change to the field (otherwise it **must** be provided).
- It is a field on a request message (a message that is an argument to an RPC,
with a name usually ending in `Request`). In this case, a value **must** be
provided as part of the request, and failure to do so **must** cause an error
(usually `INVALID_ARGUMENT`).
We define the term "truthy" above as follows:
- For primitives, values other than `0`, `0.0`, empty string/bytes, and `false`
- For repeated fields maps, values with at least one entry
- For messages, any message with at least one "truthy" field.
Fields **should not** be described as required in order to signify:
- A field which will always be present in a response.
- A field which is conditionally required in some situations.
- A field on any message (including messages that are resources) which is never
used as user input.
**Note:** In most cases, empty values (such as `false` for booleans, `0` for
integers, or the unspecified value for enums) are indistinguishable from unset
values, and therefore setting a required field to a falsy value yields an
error. A corollary to this is that a required boolean must be set to `true`.
### Unordered List
The use of `UNORDERED_LIST` on a repeated field of a resource indicates that
the service does not guarantee the order of the items in the list.
A field **should** be described as an unordered list if the service does not
guarantee that the order of the elements in the list will match the order that
the user sent, including a situation where the service will sort the list on
the user's behalf.
A resource with an unordered list **may** return the list in a stable order, or
**may** return the list in a randomized, unstable order.
## Backwards compatibility
Adding or changing `google.api.field_behavior` values can represent a semantic
change in the API that is perceived as incompatible for existing clients. The
following are examples of backwards incompatible changes with
`google.api.field_behavior`:
* Adding `REQUIRED` to an existing field previously considered `OPTIONAL`
(implicitly or otherwise)
* Adding a new field annotated as `REQUIRED` to an existing request message
* Adding `OUTPUT_ONLY` to an existing field previously accepted as input
* Adding `INPUT_ONLY` to an existing field previously emitted as output
* Adding `IMMUTABLE` to an existing field previously considered mutable
* Removing `OUTPUT_ONLY` from an existing field previously ignored as input
* Removing `IDENTIFIER` from an existing field.
There are some changes that *are* backwards compatible, which are as follows:
* Adding `OPTIONAL` to an existing field
* Adding `IDENTIFIER` to an existing `name` field
* Changing from `REQUIRED` to `OPTIONAL` on an existing field
* Changing from `OUTPUT_ONLY` and/or `IMMUTABLE` to `IDENTIFIER` on an existing
field
* Removing `REQUIRED` from an existing field
* Removing `INPUT_ONLY` from an existing field previously excluded in responses
* Removing `IMMUTABLE` from an existing field previously considered immutable
## Rationale
### Identifier field behavior
Resource names, the primary identifiers for any compliant resource, are never
fully constructed by the user on create. Such fields are typically assigned
`OUTPUT_ONLY` field behavior. They are, however, also often consumed as
the primary identifier in scenarios where the resource itself is the primary
request payload. Such fields could *not* be considered `OUTPUT_ONLY`.
Furthermore, in mutation requests, like Standard Update, the resource name as
the primary identifier cannot be changed in place. Such fields are typically
assigned `IMMUTABLE` field behavior. These conflicting and context-dependent
field behaviors meant that a new value was necessary to single out and convey
the behavior of the resource name field.
### Required set of annotations
A field used in a request message must be either an input or an output.
In the case of an output, the `OUTPUT_ONLY` annotation is sufficient.
In the case of an input, a field is either required or optional, and therefore
should have at least the `REQUIRED` or `OPTIONAL` annotation, respectively.
Only providing `INPUT_ONLY` does not convey the necessity of the field, so
specifying either `REQUIRED` or `OPTIONAL` is still necessary.
### Requiring field behavior
By including the field behavior annotation for each field, the overall behavior
that the resource exhibits is more clearly defined. Clearly defined field
behavior improves programmatic clients and user understanding.
Requiring the annotation also forces the API author to explicitly consider the
behavior when initially authoring of the API.
Modifying field behavior after initial authoring can result in
backwards-incompatible changes in clients. For example, making an optional field
required results in backwards-incompatible changes in the method signature of an
RPC or a resource in a [Declarative client][]. See the
[Backwards compatibility](#backwards-compatibility) section for more detailed
compatibility guidance.
## History
In 2023-05 field_behavior was made mandatory. Prior to this change, the
annotation was often omitted. Its values are relied upon to produce high quality
clients. Furthermore, adding or changing some of the field_behavior values after
the fact within a major version can be backwards-incompatible. See the
[Backwards compatibility](#backwards-compatibility) section for more detailed
compatibility guidance.
The benefits of requiring field_behavior at the time that the API is authored
surpass the costs to clients and API users of not doing so.
[aip-133]: ./0133.md
[aip-134]: ./0134.md
[aip-180]: ./0180.md
[google.api.FieldBehavior]: https://github.com/googleapis/googleapis/blob/master/google/api/field_behavior.proto#L49
[Declarative client]: ./0009.md#declarative-clients
[fields representing resource names]: ./0122.md#fields-representing-resource-names
## Changelog
- **2023-09-14**: Clarify that nested behavior and parent behavior are
independent.
- **2023-08-25**: Add guidance on `IDENTIFIER`.
- **2023-07-20**: Describe compatibility guidance with new section.
- **2023-05-24**: Clarify that `IMMUTABLE` does not imply input nor required.
- **2023-05-10**: Added guidance to require the annotation.
- **2020-12-15**: Added guidance for `UNORDERED_LIST`.
- **2020-05-27**: Clarify behavior when receiving an immutable field in an
update.
- **2019-12-05**: Added guidance on output only fields in field masks.
- **2019-06-18**: Use the machine-readable annotation, not comments.
---
id: 205
state: approved
created: 2019-02-18
placement:
category: process
order: 20
redirect_from: /beta-blocker
---
# Beta-blocking changes
APIs often release an Alpha version of their API in order to get early feedback
from customers. This API is provisional and can change many times before the
important feedback is incorporated and the API is made stable for Beta.
Since the purpose of Alpha is to gather feedback, the API does not need to be
perfect yet, and it's not strictly necessary for API authors to address every
usability concern or address every point in the API standards. Often, API
authors and API reviewers will not agree on the best design, and the best way
to find out is by having users try out the API.
However, once the feedback has been collected and the API is going to be
promoted to Beta, usability concerns and style issues do need to be addressed.
In order to ensure that these issues are not forgotten, they should be
explicitly documented in the API.
## Guidance
If an API has usability concerns or violates API standards, and the present
design should receive additional scrutiny before being carried through to the
Beta version, there **must** be an internal comment linking to this document
using its descriptive link ([aip.dev/beta-blocker]()) to ensure that the design
is corrected before the API is released to Beta.
The comment **must** also indicate what kind of change should be made for Beta.
For example:
```proto
message InputConfig {
// Parameters for input.
// (-- aip.dev/beta-blocker: Convert well-known parameters into explicit
// fields before the Beta launch. --)
map<string, string> parameters = 1;
}
```
If an exception to API standards _does_ need to be carried through to Beta and
GA, see [AIP-200][].
[aip-200]: ./0200.md
---
id: 210
state: approved
created: 2018-08-20
placement:
category: design-patterns
order: 110
---
# Unicode
APIs should be consistent on how they explain, limit, and bill for string
values and their encodings. This ranges from little ambiguities (like fields
"limited to 1024 characters") all the way to billing confusion (are names and
values of properties in Datastore billed based on characters or bytes?).
In general, if we talk about limits measured in bytes, we are discriminating
against non-ASCII text since it takes up more space. On the other hand, if we
talk about "characters", we are ambiguous about whether those are Unicode "code
points", "code units" for a particular encoding (e.g. UTF-8 or UTF-16),
"graphemes", or "grapheme clusters".
## Unicode primer
Character encoding tends to be an area we often gloss over, so a quick primer:
- Strings are just bytes that represent numbers according to some encoding
format.
- When we talk about **characters**, we sometimes mean Unicode **code points**,
which are numbers in the Unicode spec (up to 21 bits).
- Other times we might mean **graphemes** or **grapheme clusters**, which may
have multiple numeric representations and may be represented by more than one
code point. For example, `á` may be represented as a composition of
`U+0061 + U+0301` (the `a` + the accent combining mark) or as a single code
point, `U+00E1`.
- Protocol buffers uses **UTF-8** ("Unicode Transformation Format") which is a
variable-length encoding scheme using up to 4 **code units** (8-bit bytes)
per code point.
## Guidance
### Character definition
**TL;DR:** In our APIs, "characters" means "Unicode code points".
In API documentation (e.g., API reference documents, blog posts, marketing
documentation, billing explanations, etc), "character" **must** be defined as a
Unicode code point.
### Length units
**TL;DR:** Set size limits in "characters" (as defined above).
All string field length limits defined in API comments **must** be measured and
enforced in characters as defined above. This means that there is an underlying
maximum limit of (`4 * characters`) bytes, though this limit will only be hit
when using exclusively characters that consist of 4 UTF-8 code units (32 bits).
If you use a database system (e.g. Spanner) which allows you to define a limit
in characters, it is safe to assume that this byte-defined requirement is
handled by the underlying storage system.
### Billing units
APIs **may** use either code points or bytes (using the UTF-8 encoding) as the
unit for billing or quota measurement (e.g., Cloud Translation chooses to use
characters). If an API does not define this, the assumption is that the unit of
billing is characters (e.g., $0.01 _per character_, not $0.01 _per byte_).
### Unique identifiers
**TL;DR:** Unique identifiers **should** limit to ASCII, generally only
letters, numbers, hyphens, and underscores, and **should not** start with a
number.
Strings used as unique identifiers **should** limit inputs to ASCII characters,
typically letters, numbers, hyphens, and underscores
(`[a-zA-Z][a-zA-Z0-9_-]*`). This ensures that there are never accidental
collisions due to normalization. If an API decides to allow all valid Unicode
characters in unique identifiers, the API **must** reject any inputs that are
not in Normalization Form C. Generally, unique identifiers **should not** start
with a number as that prefix is reserved for Google-generated identifiers and
gives us an easy way to check whether we generated a unique numeric ID for or
whether the ID was chosen by a user.
Unique identifiers **should** use a maximum length of 64 characters, though
this limit may be expanded as necessary. 64 characters should be sufficient for
most purposes as even UUIDs only require 36 characters.
**Note:** See AIP-122 for recommendations about resource ID segments.
### Normalization
**TL;DR:** Unicode values **should** be stored in [Normalization Form C][].
Values **should** always be normalized into Normalization Form C. Unique
identifiers **must** always be stored in Normalization Form C (see the next
section).
Imagine we're dealing with Spanish input "estar<b>é</b>" (the accented part
will be bolded throughout). This text has what we might visualize as 6
"characters" (in this case, they are grapheme clusters). It has two possible
Unicode representations:
- Using 6 code points: `U+0065` `U+0073` `U+0074` `U+0061` `U+0072`
**`U+00E9`**
- Using 7 code points: `U+0065` `U+0073` `U+0074` `U+0061` `U+0072` **`U+0065`
`U+0301`**
Further, when encoding to UTF-8, these code points have two different
serialized representations:
- Using 7 code-units (7 bytes): `0x65` `0x73` `0x74` `0x61` `0x72` **`0xC3`
`0xA9`**
- Using 8 code-units (8 bytes): `0x65` `0x73` `0x74` `0x61` `0x72` **`0x65`
`0xCC` `0x81`**
To avoid this discrepancy in size (both code units and code points), use
[Normalization Form C][] which provides a canonical representation for strings.
[normalization form c]: https://unicode.org/reports/tr15/
### Uniqueness
**TL;DR:** Unicode values **must** be normalized to [Normalization Form C][]
before checking uniqueness.
For the purposes of unique identification (e.g., `name`, `id`, or `parent`),
the value **must** be normalized into [Normalization Form C][] (which happens
to be the most compact). Otherwise we may have what is essentially "the same
string" used to identify two entirely different resources.
In our example above, there are two ways of representing what is essentially
the same text. This raises the question about whether the two representations
should be treated as equivalent or not. In other words, if someone were to use
both of those byte sequences in a string field that acts as a unique
identifier, would it violate a uniqueness constraint?
The W3C recommends using Normalization Form C for all content moving across the
internet. It is the most compact normalized form on Unicode text, and avoids
most interoperability problems. If we were to treat two Unicode byte sequences
as different when they have the same representation in NFC, we'd be required to
reply to possible "Get" requests with content that is **not** in normalized
form. Since that is definitely unacceptable, we **must** treat the two as
identical by transforming any incoming string data into Normalized Form C or
rejecting identifiers not in the normalized form.
There is some debate about whether we should view strings as sequences of code
points represented as bytes (leading to uniqueness determined based on the
byte-representation of said string) or to interpret strings as a higher level
abstraction having many different possible byte-representations. The stance
taken here is that we already have a field type for handling that: `bytes`.
Fields of type `string` already express an opinion of the validity of an input
(it must be valid UTF-8). As a result, treating two inputs that have identical
normalized forms as different due to their underlying byte representation seems
to go against the original intent of the `string` type. This distinction
typically doesn't matter for strings that are opaque to our services (e.g.,
`description` or `display_name`), however when we rely on strings to uniquely
identify resources, we are forced to take a stance.
Put differently, our goal is to allow someone with text in any encoding (ASCII,
UTF-16, UTF-32, etc) to interact with our APIs without a lot of "gotchas".
## References
- [Unicode normalization forms](https://unicode.org/reports/tr15/)
- [Datastore pricing "name and value of each property"](https://cloud.google.com/datastore/pricing)
doesn't clarify this.
- [Natural Language pricing](https://cloud.google.com/natural-language/pricing)
uses charges based on UTF-8 code points rather than code units.
- [Text matching and normalization](https://sites.google.com/a/google.com/intl-eng/apis/matching?pli=1)
---
id: 211
state: approved
created: 2021-02-24
placement:
category: design-patterns
order: 115
---
# Authorization checks
The majority of operations, whether reads or writes, require authorization:
permission to do the thing the user is asking to do. Additionally, it is
important to be careful how much information is provided to _unauthorized_
users, since leaking information can be a security concern.
## Guidance
Services **must** check authorization before validating any request, to ensure
both a secure API surface and a consistent user experience. An operation
**may** require multiple permissions or preconditions in order to grant
authorization.
If a request can not pass the authorization check for any reason, the service
**must** error with `PERMISSION_DENIED`, and the corresponding error message
**should** look like: "Permission '`{p}`' denied on resource '`{r}`' (or it
might not exist)." This avoids leaking resource existence.
If it is not possible to determine authorization for a resource because the
resource does not exist, the service **should** check authorization to read
children on the parent resource, and return `NOT_FOUND` if the authorization
check passes.
### Multiple operations
A service could encounter a situation where it has two different operations
with two different permissions, either of which would reveal the existence of a
resource if called, but a user only has permission to call one of them.
In this situation, the service **should** still only check for authorization
applicable to the operation being called, and **should not** try to "help out"
by checking for related authorization that would provide permission to reveal
existence, because such algorithms are complicated to implement correctly and
prone to accidental leaks.
For example, posit a scenario where:
- A resource exists within a given collection that a user is unable to read.
- The user _does_ have the ability to create other resources, and the
collection uses user-specified IDs (meaning that a failure because of a
duplicate ID would reveal existence).
In this situation, the get or create methods **should** still only check
_their_ permissions when determining what error to return, and not one
another's.
## Rationale
[RFC 7231 §6.5.3][] states that services are permitted to use `404 Not Found`
in lieu of `403 Forbidden` in situations where the service does not want to
divulge existence, whereas this AIP argues for the use of `PERMISSION_DENIED`
(which corresponds to `403 Forbidden` in HTTP) instead. We take this position
for the following reasons:
- The practice of "getting `404 Not Found` until you have enough permission to
get `403 Forbidden`" is counter-intuitive and increases the difficulty of
troubleshooting.
- A service _could_ ameliorate this by sending information about missing
permissions while still using the `404 Not Found` status code, but this
constitutes a mixed message.
- While `403 Forbidden` is essentially always an error requiring manual action,
`404 Not Found` is often a valid response that the application can handle
(e.g. "get or create"); overloading it for permission errors deprives
applications of this benefit.
- RFC 7231 §6.5.4 states that `404 Not Found` results are cacheable, but
permission errors are not generally cacheable. Sending explicit cache
controls on a conditional basis could ameliorate this, but would defeat the
purpose.
- The guidance here is more consistent with most other real-world authorization
systems.
[rfc 7231 §6.5.3]: https://tools.ietf.org/html/rfc7231#section-6.5.3
## Changelog
- **2021-05-11:** Added a more detailed "Rationale" section.
---
id: 213
state: approved
created: 2018-08-17
placement:
category: protobuf
order: 20
---
# Common components
As specified in [AIP-215][], APIs must be self-contained except for the use
of "common component" packages which are intended for use by multiple APIs.
There are two kinds of common component packages:
- Organization-specific common components, covering organization-specific
concepts such as a "viewport" in Maps.
- Global common components which are generic (i.e. not domain-specific),
such as "timestamp" or "postal address".
Where it is safe to share a single representation across multiple APIs,
common components can make it easier for clients to interact with those APIs.
Concept-specific client code can be written once, and messages can be used from
the response of one API in the request of another without clunky copying, for
example.
This benefit comes with significant restrictions and limitations, however,
and should not be attempted lightly.
Note that even if the *domain* of a component is common, the requirements of
a component may be organization-specific. For example, some organizations may
have particular requirements of how financial values are represented, leading
to multiple finance-oriented organization-specific common components - because
any global common component would either not meet the organization-specific
requirements, or be too complex for general use.
## Guidance
- Organization-wide common component packages **must** end with `.type`, e.g.
`google.geo.type` or `google.shopping.type`.
- Organizations **must** consult the API design team before creating a new
organization-wide common component package.
- Organization-wide common component packages **must** be published in [the
`googleapis` repository](https://github.com/googleapis/googleapis).
- Organizations creating and publishing a new organization-wide common component
package **must** update this AIP to include it in the
[list below](#existing-organization-specific-common-component-packages).
- Organizations **must not** define generic components in organization-specific
common component packages, instead preferring global common components.
- Common components **must not** be "moved" (that is, deleted from one common
component package and added to a different one) from an organization-specific
common component package to a global common component package or vice versa.
- A common component **may** be copied from an organization-specific common
component package to a global common component package (without deleting the
original component) if it is found to be more widely-applicable than
originally expected.
- Fields **should not** be added to existing messages.
- Values **should not** be added to existing enums.
- Fields **must not** be removed from existing messages.
- Values **must not** be removed from existing enums.
- While documentation **may** be clarified, it **should not** change the
meanings of existing values, including the validity of any given message
or set of messages.
- New proto messages and enums **may** be added to common component packages.
- API teams **should** allow sufficient time for propagation to clients
before using the new messages and enums in their APIs. Fields may take some
time for any changes to propagate through publication to client libraries
and other surfaces.
- API teams **should** consult widely within their organization, and ideally
with the API design team, before adding a new message or enum, due to the
limitations listed above.
## Existing global common components
The global common components, which public-facing protos for an API **may** safely
import, are as follows:
- [`google.api.*`](https://github.com/googleapis/googleapis/blob/master/google/api) (but *not* subpackages of `google.api`)
- [`google.longrunning.Operation`](https://github.com/googleapis/googleapis/blob/master/google/longrunning/operations.proto)
- [`google.protobuf.*`](https://github.com/protocolbuffers/protobuf/tree/master/src/google/protobuf)
- [`google.rpc.*`](https://github.com/googleapis/googleapis/blob/master/google/rpc/)
- [`google.type.*`][type]
Note that some common components may have internal-only fields. APIs **should**
generally only rely on fields which have been
[released into open source](https://github.com/googleapis/googleapis).
Google APIs **may** also import [`google.iam.v1.*`][iam], which provides the
IAM messages used throughout Google.
<!-- prettier-ignore -->
[iam]: https://github.com/googleapis/googleapis/tree/master/google/iam/v1
**Note:** Many APIs also import components from other packages for internal-only
use (e.g. to apply visibility labels or provide instructions to internal
infrastructure). This is acceptable provided that the _public_ components do not
contain such references.
### Protobuf types
The `google.protobuf` package is somewhat special in that it is shipped with
protocol buffers itself, rather than with API tooling. (For most API designers,
this should be an implementation detail).
This package includes a small library of types useful for representing common
programming language constructs:
- [`google.protobuf.Duration`][duration]: Durations, with nanosecond-level
precision. The protobuf runtime provides helper functions to convert to and
from language-native duration objects where applicable (such as Python's
[`timedelta`][timedelta]).
- [`google.protobuf.Struct`][struct]: JSON-like structures (a dictionary of
primitives, lists, and other dictionaries). The protobuf runtime provides
helper functions in most languages to convert struct objects to and from
JSON.
- [`google.protobuf.Timestamp`][timestamp]: Timestamps, with nanosecond-level
precision. The protobuf runtime provides helper functions in most languages
to convert to and from language-native timestamp objects (such as Python's
[`datetime`][datetime]).
<!-- prettier-ignore-start -->
[datetime]: https://docs.python.org/3/library/datetime.html#datetime.datetime
[duration]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/duration.proto
[struct]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/struct.proto
[timedelta]: https://docs.python.org/3/library/datetime.html#datetime.timedelta
[timestamp]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/timestamp.proto
<!-- prettier-ignore-end -->
### API Types
The `google.type` package provides a "standard library" of types useful for
representing common concepts in APIs. While types are added from time to time
and the definitive list is always [the code][type], several types deserve note:
- [`google.type.Color`][color]: RGB or RGBA colors.
- [`google.type.Date`][date]: Calendar dates, with no time or time zone
component.
- [`google.type.DayOfWeek`][day_of_week]: The day of the week, with no other
date, time, or time zone component.
- [`google.type.LatLng`][lat_lng]: Geographic coordinates.
- [`google.type.Money`][money]: Currency.
- [`google.type.PostalAddress`][postal_address]: Postal addresses in most
countries.
- [`google.type.TimeOfDay`][time_of_day]: Wall-clock time, with no date or time
zone component.
<!-- prettier-ignore-start -->
[type]: https://github.com/googleapis/googleapis/tree/master/google/type
[color]: https://github.com/googleapis/googleapis/blob/master/google/type/color.proto
[date]: https://github.com/googleapis/googleapis/blob/master/google/type/date.proto
[day_of_week]: https://github.com/googleapis/googleapis/blob/master/google/type/dayofweek.proto
[lat_lng]: https://github.com/googleapis/googleapis/blob/master/google/type/latlng.proto
[money]: https://github.com/googleapis/googleapis/blob/master/google/type/money.proto
[postal_address]: https://github.com/googleapis/googleapis/blob/master/google/type/postal_address.proto
[time_of_day]: https://github.com/googleapis/googleapis/blob/master/google/type/timeofday.proto
<!-- prettier-ignore-end -->
### Adding to common protos
Occasionally, it may be useful to add protos to these packages or to add to the
list of commonly-available protos. In order to do this, [open an issue][] on
the AIP repository in GitHub, noting the guidelines above.
## Existing organization-specific common component packages
The following organization-specific common component packages exist and conform with the above guidance:
- [`google.apps.script.type`](https://github.com/googleapis/googleapis/tree/master/google/apps/script/type)
Common component package for Google Apps Script.
- [`google.geo.type`](https://github.com/googleapis/googleapis/tree/master/google/geo/type)
Common component package for Google Maps and the Geo organization.
- [`google.actions.type`](https://github.com/googleapis/googleapis/tree/master/google/actions/type)
Common component package for Actions on Google APIs.
## Non-conformant common component packages
The following common component packages exist, but do not conform with the above guidance,
and do not form a precedent for further such packages.
- [`google.cloud.common`](https://github.com/googleapis/googleapis/tree/master/google/cloud/common):
This does not conform to the requirement for the package name to end in `.type`. (This would otherwise
be acceptable, and this package should be considered as the Cloud common component package.)
- [`google.logging.type`](https://github.com/googleapis/googleapis/tree/master/google/logging/type):
This appears to be API-specific, although it's used from multiple APIs; some aspects should probably
be global or in a Cloud common component package.
- [`google.cloud.workflows.type`](https://github.com/googleapis/googleapis/tree/master/google/cloud/workflows/type):
API-specific types.
- [`google.cloud.oslogin.common`](https://github.com/googleapis/googleapis/tree/master/google/cloud/oslogin/common):
API-specific types, and a non-conformant name.
- [`google.identity.accesscontextmanager.type`](https://github.com/googleapis/googleapis/tree/master/google/identity/accesscontextmanager/type):
API-specific types.
- [`google.networking.trafficdirector.type`](https://github.com/googleapis/googleapis/tree/master/google/networking/trafficdirector/type):
API-specific types.
## Rationale
Common components are effectively unversioned: APIs evolve independently of
each other, both in terms of definition and implementation. A change such as
adding a field is backward-compatible and predictable in specific APIs, and the
API team can ensure that the server implementation is available before the API
definition is published. By contrast, a change in a common component would
effectively be universally available even if most API implementations did not
take it into account.
Adding a new message or enum is backward-compatible, as it does not affect
existing APIs that may import other messages or enums from the same common
component package.
Consultation with the API design team is required for global common components
and suggested for organization-specific common components as the border between
"generic" and "organization-specific" is a gray area; some generic *concepts*
have organization-specific use cases which surface through the components.
## Changelog
- **2023-06-27**: Restructured AIPs 215 and 213 for clarity, and introduced the
concept of organization-wide common protos more formally.
- **2018-08-17**: Initial AIP written.
[open an issue]: https://github.com/googleapis/aip/issues
[aip-215]: ./0215.md
---
id: 214
state: approved
created: 2018-06-19
placement:
category: design-patterns
order: 120
---
# Resource expiration
Customers often want to provide the time that a given resource or resource
attribute is no longer useful or valid (e.g. a rotating security key). Currently
we recommend that customers do this by specifying an exact "expiration time"
into a `google.protobuf.Timestamp expire_time` field; however, this adds
additional strain on the user when they want to specify a relative time offset
until expiration rather than a specific time until expiration.
Furthermore, the world understands the concept of a "time-to-live", often
abbreviated to TTL, but the typical format of this field (an integer, measured
in seconds) results in a sub-par experience when using an auto-generated client
library.
## Guidance
1. APIs wishing to convey an expiration **must** rely on a
[`google.protobuf.Timestamp`][timestamp] field called `expire_time`.
2. APIs wishing to allow a relative expiration time **must** define a `oneof`
called `expiration` (or `{something}_expiration`) containing both the
`expire_time` field and a separate [`google.protobuf.Duration`][duration]
field called `ttl`, the latter marked as input only.
3. APIs **must** always return the expiration time in the `expire_time` field
and leave the `ttl` field blank when retrieving the resource.
4. APIs that rely on the specific semantics of a "time to live" (e.g., DNS
which must represent the TTL as an integer) **may** use an `int64 ttl`
field (and **should** provide an [aip.dev/not-precedent](./0200.md) comment
in this case).
### Example
```proto
message ExpiringResource {
// google.api.resource and other annotations and fields
oneof expiration {
// Timestamp in UTC of when this resource is considered expired.
// This is *always* provided on output, and is calculated using
// the value of [ttl][] if set when created.
google.protobuf.Timestamp expire_time = 2;
// Input only. The TTL for this resource.
google.protobuf.Duration ttl = 3 [(google.api.field_behavior) = INPUT_ONLY];
}
}
```
## Rationale
### Alternatives considered
#### A new standard field called `ttl`
We considered allowing a standard field called `ttl` as an alternative way of
defining the expiration, however doing so would require that API services
continually update the field, like a clock counting down. This could
potentially cause problems with the read-modify-write lifecycle where a
resource is being processed for some time, and effectively has its life
extended as a result of that processing time.
#### Always use `expire_time`
This is the current state of the world with a few exceptions. In this scenario,
we could potentially push the computation of `now + ttl = expire_time` into
client libraries; however, this leads to a somewhat frustrating experience in
the command-line and using REST/JSON. Leaving things as they are is typically
the default, but it seems many customers want the ability to define relative
expiration times as it is quite a bit easier and removes questions of time
zones, stale clocks, and other silly mistakes.
<!-- prettier-ignore-start -->
[duration]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/duration.proto
[timestamp]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/timestamp.proto
<!-- prettier-ignore-end -->
---
id: 215
state: approved
created: 2018-10-01
placement:
category: protobuf
order: 30
---
# API-specific protos
APIs are mostly defined in terms of protos which are API-specific, with
occasional dependencies on common components. Keeping APIs isolated from each
other avoids versioning problems and client library packaging problems.
## Guidance
- All protos specific to an API **must** be within a package with a major version
(e.g., `google.library.v1`).
- References to resources in other APIs **must** be expressed in terms of
resource names ([AIP-122][]), rather than using the resource messages.
- When two versions of an API use effectively the same (API-specific) proto
that proto **must** be duplicated in each version. (In other words, APIs
**must not** create their own "API-specific common component" packages.)
- Organization-specific common components **may** be placed in a common package,
as described in [AIP-213][], but **must not** be used by any API outside
that organization.
- Global common components (also described in AIP-213) **may** be freely used by any API.
## Rationale
When one API depends on protos defined by another API, this introduces uncertainty
in terms of customer-expected behavior and client library dependency management.
Suppose `google.cloud.library.v1` depends on the protos (rather than abstract resources)
in `google.cloud.movies.v2`. Any change to `google.cloud.movies.v2` can cause problems.
For example:
- If a field is added to a message in `google.cloud.movies.v2`, should customers using
`google.cloud.library.v1` expect to see it? If so, how soon after the field has
been added? What about other API changes?
- If the whole major version `google.cloud.movies.v2` is deprecated
(typically after v3 has been released), does that mean `google.cloud.library.v1` has
to change to use `google.cloud.movies.v3`, and if so, does that require a new major version
for the library API as well?
- How should client library versioning reflect changes to dependent APIs?
Keeping APIs isolated from each other, with a limited set of common components which are
maintained in a highly disciplined way, reduces a lot of the issues with dependencies.
API-specific common components shared across versions add complexity for client
library generation and packaging, and are inflexible in terms of versioning.
When protos are duplicated because they *start* off the same in multiple versions,
they can still diverge over time as they are isolated from each other.
## Changelog
- **2023-06-27**: Restructured AIPs 215 and 213 for clarity.
- **2023-05-11**: Changed "PA" to "organization".
- **2018-10-01**: Initial AIP written.
[aip-213]: ./0213.md
[aip-122]: ./0122.md
---
id: 216
state: approved
created: 2018-10-01
updated: 2022-06-02
placement:
category: fields
order: 110
---
# States
Many API resources carry a concept of "state": ordinarily, the resource's place
in its life cycle. For example, a virtual machine may be being provisioned,
available for use, being spun down, or potentially be in one of several other
situations. A job or query may be preparing to run, be actively running, have
completed, and so on.
## Guidance
Resources needing to communicate their state **should** use an enum, which
**should** be called `State` (or, if more specificity is required, end in the
word `State`). This enum **should** be nested within the message it describes
when only used as a field within that message.
**Important:** We use the term `State`, and _not_ `Status` (which is reserved
for the HTTP and gRPC statuses).
### Enum values
Ideally, Google APIs use the same terminology throughout when expressing the
same semantic concepts. There are usually many words available to express a
given state, but our customers often use multiple APIs together, and it is
easier for them when our terms are consistent.
At a high level:
- Resources that are available for use are `ACTIVE` (preferred over terms such
as "ready" or "available").
- Resources that have completed a (usually terminal) requested action use past
participles (usually ending in `-ED`), such as `SUCCEEDED` (not
"successful"), `FAILED` (not "failure"), `DELETED`, `SUSPENDED`, and so on.
- Resources that are currently undergoing a state change use present
participles (usually ending in `-ING`), such as `RUNNING`, `CREATING`,
`DELETING`, and so on. In this case, it is expected that the state is
temporary and will resolve to another state on its own, with no further user
action.
**Note:** Remember to only add states that are useful to customers. Exposing a
large number of states simply because they exist in your internal system is
unnecessary and adds confusion for customers. Each state must come with a use
case for why it is necessary.
### Output only
The field referencing the `State` enum in a resource **should** behave and be
documented as "Output only", in accordance with [AIP-203](./0203.md).
APIs **should not** allow a `State` enum to be directly updated through an
"update" method (or directly set through the "create" method), and **should**
instead use custom state transition methods.
This is because update methods are generally not expected to have side effects,
and also because updating state directly implies that it is possible to set the
state to any available value, whereas states generally reflect a resource's
progression through a lifecycle.
### State transition methods
State transition methods are a special type of custom method that are
responsible for transitioning a state field from one enum value to another. As
part of the transition, other fields may also change, e.g. an `update_time`
field. The method definition should look like the following:
```proto
// Publishes a book.
// The `state` of the book after publishing is `PUBLISHED`.
// `PublishBook` can be called on Books in the state `DRAFT`; Books in a
// different state (including `PUBLISHED`) returns an error.
rpc PublishBook(PublishBookRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{name=publishers/*/books/*}:publish"
body: "*"
};
}
```
- The name of the method **should** be a verb followed by the singular form of
the resource's message name.
- The request message **must** match the RPC name, with a `Request` suffix.
- The response message **should** be the resource itself.
- If the RPC is [long-running][], the response message **should** be a
`google.longrunning.Operation` which resolves to the resource itself.
- The HTTP verb **must** be `POST`.
- The HTTP URI **must** use a `:` character followed by the custom verb
(`:publish` in the above example), and the verb in the URI **must** match the
verb in the name of the RPC.
- If word separation is required, `camelCase` **must** be used.
- The `body` clause in the `google.api.http` annotation **must** be `"*"`.
- The request message field receiving the resource name **should** map to the
URI path.
- This field **should** be called `name`.
- The `name` field **should** be the only variable in the URI path. All
remaining parameters **should** map to URI query parameters.
- If the state transition is not allowed, the service **must** error with
`FAILED_PRECONDITION` (HTTP 400).
The request message should look like this:
```proto
message PublishBookRequest {
// The name of the book to publish.
// Format: publishers/{publisher}/books/{book}
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
}
```
- A resource name field **must** be included. It **should** be called `name`.
- The comment for the field **should** document the resource pattern.
- Other fields **may** be included.
## Additional Guidance
### Default value
The zero value of each state enum **should** adhere to the following
convention:
```proto
enum State {
// The default value. This value is used if the state is omitted.
STATE_UNSPECIFIED = 0;
// Other values...
}
```
Resources **should not** provide an unspecified state to users, and this value
**should not** actually be used.
### Value uniqueness
Multiple top-level enums within the same package **must** not share the same
values. This is because the C++ protoc code generator flattens top-level enum
values into a single namespace.
State enums **should** live inside the resource definition.
### Prefixes
Using a `STATE_` prefix on every enum value is unnecessary. State enum values
**should not** be prefixed with the enum name, except for the default value
`STATE_UNSPECIFIED`.
### Breaking changes
**TL;DR:** Clearly communicate to users that state enums may receive new values
in the future, and be conscientious about adding states to an existing enum.
Even though adding states to an existing states enum _can_ break existing user
code, adding states is not considered a breaking change. Consider a state with
only two values: `ACTIVE` and `DELETED`. A user may add code that checks
`if state == ACTIVE`, and in the else cases simply assumes the resource is
deleted. If the API later adds a new state for another purpose, that code will
break.
We ultimately can not control this behavior, but API documentation **should**
actively encourage users to code against state enums with the expectation that
they may receive new values in the future.
APIs **may** add new states to an existing State enum when appropriate, and
adding a new state is _not_ considered a breaking change.
### When to avoid states
Sometimes, a `State` enum may not be what is best for your API, particularly in
situations where a state has a very small number of potential values, or when
states are not mutually exclusive.
Consider the example of a state with only `ACTIVE` and `DELETED`, as discussed
above. In this situation, the API may be better off exposing a
`google.protobuf.Timestamp delete_time`, and instructing users to rely on
whether it is set to determine deletion.
### Common states
The following is a list of states in common use. APIs **should** consider prior
art when determining state names, and **should** value local consistency above
global consistency in the case of conflicting precedent.
#### Resting states
"Resting states" are lifecycle states that, absent user action, are expected to
remain indefinitely. However, the user can initiate an action to move a
resource in a resting state into certain other states (resting or active).
- `ACCEPTED`
- `ACTIVE`
- `CANCELLED`
- `DELETED`
- `FAILED`
- `SUCCEEDED`
- `SUSPENDED`
- `VERIFIED`
#### Active states
"Active states" are lifecycle states that typically resolve on their own into a
single expected resting state.
**Note:** Remember only to expose states that are useful to customers. Active
states are valuable only if the resource will be in that state for a sufficient
period of time. If state changes are immediate, active states are not
necessary.
- `CREATING` (usually becomes `ACTIVE`)
- `DELETING` (usually becomes `DELETED`)
- `PENDING` (usually becomes `RUNNING`)
- `REPAIRING` (usually becomes `ACTIVE`)
- `RUNNING` (usually becomes `SUCCEEDED`)
- `SUSPENDING` (usually becomes `SUSPENDED`)
## Further reading
- For information on enums generally, see [AIP-126][].
## Changelog
- **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-".
- **2020-10-20**: Added guidance on prefixing enum values with enum name.
- **2020-09-02**: Clarified that states are not directly set on create either.
- **2019-12-05**: Changed guidance on state transition methods, downgrading
**must** to **should** on the response type.
- **2019-08-16**: Added guidance for state transition methods.
- **2019-07-18**: Added explicit guidance on the unspecified value.
[aip-126]: ./0126.md
[long-running]: ./0151.md
---
id: 217
state: approved
created: 2019-08-26
placement:
category: design-patterns
order: 130
---
# Unreachable resources
Occasionally, a user may ask for a list of resources, and some set of resources
in the list are temporarily unavailable. The most typical use case is while
supporting [Reading Across Collections][aip-159]. For example, a user may ask to
list resources across multiple parent locations, but one of those locations is
temporarily unreachable. In this situation, it is still desirable to provide
the user with all the available resources, while indicating that something is
missing.
## Guidance
If a method to retrieve data is capable of partially failing due to one or more
resources being temporarily unreachable, the response message **must** include
a field to indicate this:
```proto
message ListBooksResponse {
// The books matching the request.
repeated Book books = 1;
// The next page token, if there are more books matching the
// request.
string next_page_token = 2;
// Unreachable resources.
repeated string unreachable = 3 [
(google.api.field_behavior) = UNORDERED_LIST
];
}
```
- The field **must** be a repeated string, and **should** be named
`unreachable`.
- The field **must** contain the resource names of the resources that are
unreachable or those that impede reaching the requested collection, such as
the parent resource of the collection that could not be reached.
- For example, if an entire location is unreachable, preventing access to the
localized collection of resources requested, the location resource is
included.
- The field **must** contain _service-relative_ resource names, and **must not**
contain full resource names, resource URIs, or simple resource IDs. See
[AIP-122][aip-122] for definitions.
- For example, if a `Book` resource is unreachable, the _service-relative_
resource name `"shelves/scifi1/books/starwars4"` is included in
`unreachable`, as opposed to the _full_ resource name
`"//library.googleapis.com/shelves/scifi1/books/starwars4"`, the
_parent-relative_ resource `"books/starwars4"`, the resource ID
`"starwars4"`, or the resource URI.
- The response **must not** provide any other information about the issue(s)
that made the listed resources unreachable.
- For example, the response cannot contain an extra field with error reasons
for each `unreachable` entry.
- The service **must** provide a way for the user to make a more specific
request and receive an error with additional information e.g. via a Standard
Get or a Standard List targeted at the unreachable collection parent.
- The service **must** also allow the user to repeat the original call with
more restrictive parameters.
- The resource names that appear in `unreachable` **may** be heterogeneous.
- The `unreachable` field definition **should** document what potential
resources could be provided in this field, and note that it might expand
later.
- For example, if both an entire location and a specific resource in a
different location are unreachable, the unreachable location's name
e.g. `"projects/example123/locations/us-east1"` and the unreachable
resource's name e.g.
`"projects/example123/locations/europe-west2/instances/example456"` will
both appear in `unreachable`.
- The `unreachable` field **must not** have semantically meaningful ordering or
structure within the list. Put differently, `unreachable` **must** be an
unordered list.
- As such, the `unreachable` field **must** be annotated with `UNORDERED_LIST`
field behavior (see [AIP-203][unordered]).
[aip-122]: ./0122.md
[aip-160]: ./0160.md
[unordered]: ./0203.md#unordered-list
**Important:** If a single unreachable location or resource prevents returning
any data by definition (for example, a list request for a single publisher
where that publisher is unreachable), the service **must** fail the entire
request with an error.
### Pagination
While preparing a page of results to fulfill a page fetch RPC e.g. an
[AIP-132][aip-132] Standard List call, if the service encounters any unreachable
resources or collections they **must** do the following:
- Include the resource name for the unreachable resource in the `unreachable`
response field.
- The resource name **must** be the most appropriately scoped for the
unreachable resource or collection.
- For example, if a specific zone within a region is unreachable, the
unreachable resource name would be a zonal Location e.g.
`projects/example/locations/us-west1-a`, but if an entire region is
unreachable, the resource name would be a regional Location e.g.
`projects/example/locations/us-west1`.
- The resource name **must** be included, regardless of restrictive paging
parameters e.g. `order_by`, when it is identified as unreachable.
- Populate results that were previously considered unreachable on a following
page if their availability is restored and the paging parameters allow for
their inclusion.
- Determining inclusion eligibility based on paging parameters also includes
any documented default ordering behavior in the absence of user-specified
ordering in the request.
- For example, if region `projects/example/locations/us-west1` was unavailable
in the first page of an ordered paging call, and including its resources
would violate the ordering, those out-of-order resources are not included in
the following page.
- Similarly, if the same exact request is made, and resources previously
considered unreachable are available again, they **must** be populated,
within the constraints of the paging parameters.
- Limit the number of unreachable resource names returned in a given response
if, even after up-scoping the unreachable resource name, the number of
unreachable resource names exceeds a documented maximum.
- This maximum **must** be documented in the `unreachable` field comments
directly.
- This is independent of the `page_size` set by the caller.
#### Retaining previous behavior
Services **may** continue with previously implemented `unreachable` pagination
behavior where changing it would induce an incompatible change as per
[AIP-180][aip-180], but **must** document said behavior on the `unreachable`
field(s) directly.
[aip-180]: ./0180.md
### Adopting partial success
In order for an existing API that has a default behavior *differing* from the
aforementioned guidance i.e. the API call returns an error status instead of a
partial result, to adopt the `unreachable` pattern the API **must** do the
following:
- The default behavior **must** be retained to avoid incompatible behavioral
changes
- For example, if the default behavior is to return an error if any location
is unreachable, that default behavior **must** be retained.
- The request message **must** have a `bool return_partial_success` field
- The response message **must** have the standard
`repeated string unreachable` field
- The two aforementioned fields **must** be added simultaneously
When the `bool return_partial_success` field is set to `true` in a request, the
API **must** behave as described in the aforementioned guidance with regards to
populating the `repeated string unreachable` response field.
```proto
message ListBooksRequest {
// Standard List request fields...
// Setting this field to `true` will opt the request into returning the
// resources that are reachable, and into including the names of those that
// were unreachable in the [ListBooksResponse.unreachable] field. This can
// only be `true` when reading across collections e.g. when `parent` is set to
// `"projects/example/locations/-"`.
bool return_partial_success = 4;
}
message ListBooksResponse {
// Standard List Response fields...
// Unreachable resources. Populated when the request opts into
// `return_partial_success` and reading across collections e.g. when
// attempting to list all resources across all supported locations.
repeated string unreachable = 3 [
(google.api.field_behavior) = UNORDERED_LIST
];
}
```
#### Partial success granularity
If the `bool return_partial_success` field is set to `true` in a request that is
scoped beyond the supported granualirty of the API's ability to reasonably
report unreachable resources, the API **should** return an `INVALID_ARGUMENT`
error with details explaining the issue. For example, if the API only supports
`return_partial_success` when [Reading Across Collections][aip-159], it returns
an `INVALID_ARGUMENT` error when given a request scoped to a specific parent
resource collection. The supported granularity **must** be documented on the
`return_partial_success` field.
## Rationale
### Using service-relative resource names
In general, relative resource names, as defined in AIP-122, are the best
practice for referring to resources by name _within_ a service and in other
services when that other service is obvious. The full resource name format is
strictly less consumable (e.g., requires extra parsing client side), and
over-specified for the uses of `unreachable`. Resource URIs are not transport
agnostic, as they are unusable in standard methods for gRPC users, and simple
resource IDs do not provide enough information about exactly which resource
was unreachable in a heterogenous list of resources.
### Minimizing extra error details in response
The context in which an unreachable resource is discovered may be sensitive and
the state of the system fluid between calls. As such, it is preferred to defer
to the service by making a more specific RPC to get more details about a
specific resource or parent. This allows the parent to handle all necessary RPC
checks and system state resolution on at time of request, rather than by
shoehorning potentially privileged or stale information into the broader list
call it was unreachable for.
### Unordered `unreachable` contents
It is important for broad API consistency that the contents of `unreachable` not
have a specific or order semantic structure. If each API baked a specific
ordering into a standard field, no single implementation, client or server side,
would be correct.
### Per page `unreachable` resources
Populating `unreachable` resources on a per page basis allows end users to
identify immediately when a page is incomplete, rather than _after_ paging
through all results. Paging to completion is not guaranteed, so it is important
to communicate as soon as possible when there are unreachable resource missing
from a given page. Furthermore, it allows users to identify when there is a
potential issue that they need to account for in subsequent calls. Finally,
retaining unreachable resources until the end of paging results requires
services to retain the state for what should be indepedent and fully isolated
API calls.
### Using request field to opt-in
Introducing a new request field as means of opting into the partial success
behavior is the best way to communicate user intent while keeping the
default behavior backwards compatible. The alternative, changing the default
behavior with the introduction of the `unreachable` response field, presents
a backwards incompatible change. Users that previously expected failure when any
resource was unreachable, assume the successful response means all resources
are accounted for in the response.
### Introducing fields simultaneously
Introducing the request and response fields simultaneously is to prevent an
invalid intermediate state that is presented by only adding one or the other. If
only `unreachable` is added, then it could be assumed that it being empty means
all resources were returned when that may not be true. If only
`return_partial_success` is added, then the user wouldn't have a means of
knowing which resources were unreachable.
### Partial success granularity limitations
At a certain level of request scope granularity, an API is simply unable to
enumerate the resources that are unreachable. For example, global-only APIs may
be unable to provide granularity at a localized collection level. In such a
case, preemptively returning an error when `return_partial_success=true`
protects the user from the risks of the alternative - expecting unreachable
resources if there was an issue, but not getting any, thus falsely assuming
everything was retrieved. This aligns with guidance herein that suggests failing
requests that cannot be fulfilled preemptively.
## History
### Pagination guidance
The original guidance for how to populate the `unreachable` field revolved
around consuming the contents as if they were the paged results. This meant that
paged resources and unreachable resources couldn't be returned in the same
response i.e. page, and users needed to completely page through all results
in order to see if any were unreachable. See the Rationale section for the
reasoning around the changes.
## Further reading
- For listing across collections, see [AIP-159][].
## Changelog
- **2024-07-29**: Reformat guidance, add explicit resource name format
- **2024-07-26**: Change pagination guidance.
requirement.
- **2024-07-19**: Add guidance for brownfield adoption of partial success.
[aip-159]: ./0159.md
---
id: 231
state: approved
created: 2019-06-18
updated: 2022-06-02
placement:
category: operations
order: 200
---
# Batch methods: Get
Some APIs need to allow users to get a specific set of resources at a
consistent time point (e.g. using a read transaction). A batch get method
provides this functionality.
## Guidance
APIs **may** support Batch Get using the following pattern:
```proto
rpc BatchGetBooks(BatchGetBooksRequest) returns (BatchGetBooksResponse) {
option (google.api.http) = {
get: "/v1/{parent=publishers/*}/books:batchGet"
};
}
```
- The RPC's name **must** begin with `BatchGet`. The remainder of the RPC name
**should** be the plural form of the resource being retrieved.
- The request and response messages **must** match the RPC name, with
`Request` and `Response` suffixes.
- The HTTP verb **must** be `GET`.
- The HTTP URI **must** end with `:batchGet`.
- The URI path **should** represent the collection for the resource, matching
the collection used for simple CRUD operations. If the operation spans
parents, a dash (`-`) **may** be accepted as a wildcard.
- There **must not** be a body key in the `google.api.http` annotation.
- The operation **must** be atomic: it **must** fail for all resources or
succeed for all resources (no partial success). For situations requiring
partial failures, `List` ([AIP-132][]) methods **should** be used.
- If the operation covers multiple locations and at least one location is
down, the operation **must** fail.
### Request message
The request for a batch get method **should** be specified with the following
pattern:
```proto
message BatchGetBooksRequest {
// The parent resource shared by all books being retrieved.
// Format: publishers/{publisher}
// If this is set, the parent of all of the books specified in `names`
// must match this field.
string parent = 1 [
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The names of the books to retrieve.
// A maximum of 1000 books can be retrieved in a batch.
// Format: publishers/{publisher}/books/{book}
repeated string names = 2 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
}
```
- A `parent` field **should** be included, unless the resource being retrieved
is a top-level resource, to facilitate inclusion in the URI as
well to permit a single permissions check. If a caller sets this field, and
the parent collection in the name of any resource being retrieved does not
match, the request **must** fail.
- This field **should** be required if only 1 parent per request is allowed.
- The field **should** identify the [resource type][aip-122-parent] that it
references.
- The comment for the field **should** document the resource pattern.
- The request message **must** include a repeated field which accepts the
resource names specifying the resources to retrieve. The field **should** be
named `names`.
- If no resource names are provided, the API **should** error with
`INVALID_ARGUMENT`.
- The field **should** be required.
- The field **should** identify the [resource type][aip-122-names] that it
references.
- The comment for the field **should** document the resource pattern.
- Other fields besides `name` **may** be "hoisted" from the [standard Get
request][request-message]. There is no way to allow for these fields to
accept different values for different resources; if this is needed, use the
[alternative request message form](#nested-request-objects).
- Batch get **should not** support pagination because transactionality across
API calls would be extremely difficult to implement or enforce, and the
request defines the exact scope of the response anyway.
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
- The comment above the `names` field **should** document the maximum number of
requests allowed.
### Response message
The response for a batch get method **should** be specified with the following
pattern:
```proto
message BatchGetBooksResponse {
// Books requested.
repeated Book books = 1;
}
```
- The response message **must** include one repeated field corresponding to the
resources being retrieved.
- The order of books in the response **must** be the same as the names in the
request.
[aip-122-names]: ./0122.md#fields-representing-resource-names
[aip-122-parent]: ./0122.md#fields-representing-a-resources-parent
[request-message]: ./0131.md#request-message
### Nested request objects
If the [standard Get request message][request-message] contains a field besides
the resource name that needs to be different between different resources being
requested, the batch message **may** alternatively hold a `repeated` field of
the [standard Get request message][request-message]. This is generally
discouraged unless your use case really requires it.
The request for a batch get method using this approach **should** be specified
with the following pattern:
```proto
message BatchGetBooksRequest {
// The parent resource shared by all books being retrieved.
// Format: publishers/{publisher}
// If this is set, the parent field in the GetBookRequest messages
// must either be empty or match this field.
string parent = 1 [
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The requests specifying the books to retrieve.
// A maximum of 1000 books can be retrieved in a batch.
repeated GetBookRequest requests = 2
[(google.api.field_behavior) = REQUIRED];
}
```
- A `parent` field **should** be included. If a caller sets this field, and the
parent collection in the name of any resource being retrieved does not match,
the request **must** fail.
- This field **should** be required if only 1 parent per request is allowed.
- The field **should** identify the [resource type][aip-122-parent] that it
references.
- The comment for the field **should** document the resource pattern.
- The request message **must** include a repeated field which accepts the
request messages specifying the resources to retrieve, as specified for
[standard Get methods][request-message]. The field **should** be named
`requests`.
- The field **should** be required.
- Other fields **may** be "hoisted" from the [standard Get
request][request-message], which means that the field can be set at either
the batch level or child request level. Similar to `parent`, if both the
batch level and child request level are set for the same field, the values
**must** match.
- Batch get **should not** support pagination because transactionality across
API calls would be extremely difficult to implement or enforce, and the
request defines the exact scope of the response anyway.
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
- The comment above the `requests` field **should** document the maximum number
of requests allowed.
[aip-132]: https://aip.dev/132
## Changelog
- **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-".
- **2020-09-16**: Suggested annotating `parent`, `names`, and `requests` fields.
- **2020-08-27**: Removed parent recommendations for top-level resources.
- **2020-03-24**: Clarified behavior if no resource names are sent.
- **2019-09-11**: Changed the primary recommendation to specify a repeated
string instead of a repeated standard Get request message. Moved the original
recommendation into its own section.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
---
id: 233
state: approved
created: 2019-06-18
updated: 2022-06-02
placement:
category: operations
order: 210
---
# Batch methods: Create
Some APIs need to allow users to create multiple resources in a single
transaction. A batch create method provides this functionality.
## Guidance
APIs **may** support Batch Create using the following two patterns:
Returning the response synchronously
```proto
rpc BatchCreateBooks(BatchCreateBooksRequest) returns (BatchCreateBooksResponse) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:batchCreate"
body: "*"
};
}
```
Returning an Operation which resolves to the response asynchronously
```proto
rpc BatchCreateBooks(BatchCreateBooksRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:batchCreate"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "BatchCreateBooksResponse"
metadata_type: "BatchCreateBooksOperationMetadata"
};
}
```
- The RPC's name **must** begin with `BatchCreate`. The remainder of the RPC
name **should** be the plural form of the resource being created.
- The request and response messages **must** match the RPC name, with
`Request` and `Response` suffixes.
- If the batch method returns an `google.longrunning.Operation`, both the
`response_type` and `metadata_type` fields **must** be specified.
- The HTTP verb **must** be `POST`.
- The HTTP URI **must** end with `:batchCreate`.
- The URI path **should** represent the collection for the resource, matching
the collection used for simple CRUD operations. If the operation spans
parents, a dash (`-`) **may** be accepted as a wildcard.
- The body clause in the `google.api.http` annotation **should** be `"*"`.
### Atomic vs. Partial Success
- The batch create method **may** support atomic (all resources created or none
are) or partial success behavior. To make a choice, consider the following
factors:
- **Complexity of Ensuring Atomicity:** Operations that are simple
passthrough database transactions **should** use an atomic operation,
while operations that manage complex resources **should** use partial
success operations.
- **End-User Experience:** Consider the perspective of the API consumer.
Would atomic behavior be preferable for the given use case, even if it
means that a large batch could fail due to issues with a single or a few
entries?
- Synchronous batch create **must** be atomic.
- Asynchronous batch create **may** support atomic or partial success.
- If supporting partial success, see
[Operation metadata message](#operation-metadata-message) requirements.
### Request message
The request for a batch create method **should** be specified with the
following pattern:
```proto
message BatchCreateBooksRequest {
// The parent resource shared by all books being created.
// Format: publishers/{publisher}
// If this is set, the parent field in the CreateBookRequest messages
// must either be empty or match this field.
string parent = 1 [
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The request message specifying the resources to create.
// A maximum of 1000 books can be created in a batch.
repeated CreateBookRequest requests = 2
[(google.api.field_behavior) = REQUIRED];
}
```
- A `parent` field **should** be included, unless the resource being created is
a top-level resource. If a caller sets this field, and the
`parent` field of any child request message does not match, the request
**must** fail. The `parent` field of child request messages can be omitted if
the `parent` field in this request is set.
- This field **should** be required if only 1 parent per request is allowed.
- The field **should** identify the [resource type][aip-122-parent] that it
references.
- The comment for the field **should** document the resource pattern.
- The request message **must** include a repeated field which accepts the
request messages specifying the resources to create, as specified for
[standard Create methods][request-message]. The field **should** be named
`requests`.
- The field **should** be required.
- Other fields **may** be "hoisted" from the [standard Create
request][request-message], which means that the field can be set at either
the batch level or child request level. Similar to `parent`, if both the
batch level and child request level are set for the same field, the values
**must** match.
- Fields which must be unique cannot be hoisted (e.g. Customer-provided id
fields).
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
- The comment above the `requests` field **should** document the maximum number
of requests allowed.
### Response message
The response for a batch create method **should** be specified with the
following pattern:
```proto
message BatchCreateBooksResponse {
// Books created.
repeated Book books = 1;
}
```
- The response message **must** include one repeated field corresponding to the
resources that were created.
### Operation metadata message
- The `metadata_type` message **must** either match the RPC name with
`OperationMetadata` suffix, or be named with `Batch` prefix and
`OperationMetadata` suffix if the type is shared by multiple Batch methods.
- If batch create method supports partial success, the metadata message **must**
include a `map<int32, google.rpc.Status> failed_requests` field to communicate
the partial failures.
- The key in this map is the index of the request in the `requests` field in
the batch request.
- The value in each map entry **must** mirror the error(s) that would normally
be returned by the singular Standard Create method.
- If a failed request can eventually succeed due to server side retries, such
transient errors **must not** be communicated using `failed_requests`.
- When all requests in the batch fail, `Operation.error` **must** be set with
`code = google.rpc.Code.Aborted` and `message = "None of the requests
succeeded, refer to the BatchCreateBooksOperationMetadata.failed_requests
for individual error details"`
- The metadata message **may** include other fields to communicate the
operation progress.
### Adopting Partial Success
In order for an existing Batch API to adopt the partial success pattern, the API
must do the following:
- The default behavior must be retained to avoid incompatible behavioral
changes.
- If the API returns an Operation:
- The request message **must** have a `bool return_partial_success` field.
- The Operation `metadata_type` **must** include a
`map<int32, google.rpc.Status> failed_requests` field.
- When the `bool return_partial_success` field is set to true in a request,
the API should allow partial success behavior, otherwise it should continue
with atomic behavior as default.
- If the API returns a direct response synchronously:
- Since the existing clients will treat a success response as an atomic
operation, the existing version of the API **must not** adopt the partial
success pattern.
- A new version **must** be created instead that returns an Operation and
follows the partial success pattern described in this AIP.
## Rationale
### Restricting synchronous batch methods to be atomic
The restriction that synchronous batch methods must be atomic is a result of
the following considerations.
The previous iteration of this AIP recommended batch methods must be atomic.
There is no clear way to convey partial failure in a sync response status code
because an OK implies it all worked. Therefore, adding a new field to the
response to indicate partial failure would be a breaking change because the
existing clients would interpret an OK response as all resources created.
On the other hand, as described in [AIP-193](https://aip.dev/193), Operations
are more capable of presenting partial states. The response status code for an
Operation does not convey anything about the outcome of the underlying operation
and a client has to check the response body to determine if the operation was
successful.
### Communicating partial failures
The AIP recommends using a `map<int32, google.rpc.Status> failed_requests` field
to communicate partial failures, where the key is the index of the failed
request in the original batch request. The other options considered were:
- A `repeated google.rpc.Status` field. This was rejected because it is not
clear which entry corresponds to which request.
- A `map<string, google.rpc.Status>` field, where the key is the request id of
the failed request. This was rejected because:
- Client will need to maintain a map of request_id -> request in order to use
the partial success response.
- Populating a request id for the purpose of communicating errors could
conflict with [AIP-155](https://aip.dev/155) if the service can not
guarantee idempotency for an individual request across multiple batch
requests.
- A `repeated FailedRequest` field, where FailedRequest contains the individual
create request and the `google.rpc.Status`. This was rejected because echoing
the request payload back in response is discouraged due to additional
challenges around user data sensitivity.
[aip-122-parent]: ./0122.md#fields-representing-a-resources-parent
[request-message]: ./0133.md#request-message
## Changelog
- **2025-03-06**: Added detailed guidance for partial success behavior, and
decision framework for choosing between atomic and partial success
- **2023-04-18**: Changed the recommendation to allow returning partial
successes.
- **2022-06-02**: Changed suffix descriptions to eliminate superfluous "-".
- **2020-09-16**: Suggested annotating `parent` and `requests` fields.
- **2020-08-27**: Removed parent recommendations for top-level resources.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
---
id: 234
state: approved
created: 2019-06-18
updated: 2022-06-02
placement:
category: operations
order: 220
---
# Batch methods: Update
Some APIs need to allow users to modify a set of resources in a single
transaction. A batch update method provides this functionality.
## Guidance
APIs **may** support Batch Update using the following two patterns:
Returning the response synchronously
```proto
rpc BatchUpdateBooks(BatchUpdateBooksRequest) returns (BatchUpdateBooksResponse) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:batchUpdate"
body: "*"
};
}
```
Returning an Operation which resolves to the response asynchronously
```proto
rpc BatchUpdateBooks(BatchUpdateBooksRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:batchUpdate"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "BatchUpdateBooksResponse"
metadata_type: "BatchUpdateBooksOperationMetadata"
};
}
```
- The RPC's name **must** begin with `BatchUpdate`. The remainder of the RPC
name **should** be the plural form of the resource being updated.
- The request and response messages **must** match the RPC name, with
`Request` and `Response` suffixes.
- If the batch method returns an `google.longrunning.Operation`, both the
`response_type` and `metadata_type` fields **must** be specified.
- The HTTP verb **must** be `POST`.
- The HTTP URI **must** end with `:batchUpdate`.
- The URI path **should** represent the collection for the resource, matching
the collection used for simple CRUD operations. If the operation spans
parents, a dash (`-`) **may** be accepted as a wildcard.
- The body clause in the `google.api.http` annotation **should** be `"*"`.
### Atomic vs. Partial Success
- The batch update method **may** support atomic (all resources updated or none
are) or partial success behavior. To make a choice, consider the following
factors:
- **Complexity of Ensuring Atomicity:** Operations that are simple
passthrough database transactions **should** use an atomic operation,
while operations that manage complex resources **should** use partial
success operations.
- **End-User Experience:** Consider the perspective of the API consumer.
Would atomic behavior be preferable for the given use case, even if it
means that a large batch could fail due to issues with a single or a few
entries?
- Synchronous batch update **must** be atomic.
- Asynchronous batch update **may** support atomic or partial success.
- If supporting partial success, see
[Operation metadata message](#operation-metadata-message) requirements.
### Request message
The request for a batch update method **should** be specified with the
following pattern:
```proto
message BatchUpdateBooksRequest {
// The parent resource shared by all books being updated.
// Format: publishers/{publisher}
// If this is set, the parent field in the UpdateBookRequest messages
// must either be empty or match this field.
string parent = 1 [
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The request message specifying the resources to update.
// A maximum of 1000 books can be modified in a batch.
repeated UpdateBookRequest requests = 2
[(google.api.field_behavior) = REQUIRED];
}
```
- A `parent` field **should** be included, unless the resource being updated is
a top-level resource. If a caller sets this field, and the
parent collection in the name of any resource being updated does not match,
the request **must** fail.
- This field **should** be required if only 1 parent per request is allowed.
- The field **should** identify the [resource type][aip-122-parent] that it
references.
- The comment for the field **should** document the resource pattern.
- The request message **must** include a repeated field which accepts the
request messages specifying the resources to update, as specified for
[standard Update methods][request-message]. The field **should** be named
`requests`.
- The field **should** be required.
- Other fields **may** be "hoisted" from the [standard Update
request][request-message], which means that the field can be set at either
the batch level or child request level. Similar to `parent`, if both the
batch level and child request level are set for the same field, the values
**must** match.
- The `update_mask` field is a good candidate for hoisting.
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
- The comment above the `requests` field **should** document the maximum number of
requests allowed.
### Response message
The response for a batch update method **should** be specified with the
following pattern:
```proto
message BatchUpdateBooksResponse {
// Books updated.
repeated Book books = 1;
}
```
- The response message **must** include one repeated field corresponding to the
resources that were updated.
### Operation metadata message
- The `metadata_type` message **must** either match the RPC name with
`OperationMetadata` suffix, or be named with `Batch` prefix and
`OperationMetadata` suffix if the type is shared by multiple Batch methods.
- If batch update method supports partial success, the metadata message **must**
include a `map<int32, google.rpc.Status> failed_requests` field to communicate
the partial failures.
- The key in this map is the index of the request in the `requests` field
in the batch request.
- The value in each map entry **must** mirror the error(s) that would normally
be returned by the singular Standard Update method.
- If a failed request can eventually succeed due to server side retries, such
transient errors **must not** be communicated using `failed_requests`.
- When all requests in the batch fail, `Operation.error` **must** be set with
`code = google.rpc.Code.Aborted` and `message = "None of the requests
succeeded, refer to the BatchUpdateBooksOperationMetadata.failed_requests
for individual error details"`
- The metadata message **may** include other fields to communicate the
operation progress.
### Adopting Partial Success
In order for an existing Batch API to adopt the partial success pattern, the API
must do the following:
- The default behavior must be retained to avoid incompatible behavioral
changes.
- If the API returns an Operation:
- The request message **must** have a `bool return_partial_success` field.
- The Operation `metadata_type` **must** include a
`map<int32, google.rpc.Status> failed_requests` field.
- When the `bool return_partial_success` field is set to true in a request,
the API should allow partial success behavior, otherwise it should continue
with atomic behavior as default.
- If the API returns a direct response synchronously:
- Since the existing clients will treat a success response as an atomic
operation, the existing version of the API **must not** adopt the partial
success pattern.
- A new version **must** be created instead that returns an Operation and
follows the partial success pattern described in this AIP.
## Rationale
### Restricting synchronous batch methods to be atomic
The restriction that synchronous batch methods must be atomic is a result of
the following considerations.
The previous iteration of this AIP recommended batch methods must be atomic.
There is no clear way to convey partial failure in a sync response status code
because an OK implies it all worked. Therefore, adding a new field to the
response to indicate partial failure would be a breaking change because the
existing clients would interpret an OK response as all resources updated.
On the other hand, as described in [AIP-193](https://aip.dev/193), Operations
are more capable of presenting partial states. The response status code for an
Operation does not convey anything about the outcome of the underlying operation
and a client has to check the response body to determine if the operation was
successful.
### Communicating partial failures
The AIP recommends using a `map<int32, google.rpc.Status> failed_requests` field
to communicate partial failures, where the key is the index of the failed
request in the original batch request. The other options considered were:
- A `repeated google.rpc.Status` field. This was rejected because it is not
clear which entry corresponds to which request.
- A `map<string, google.rpc.Status>` field, where the key is the request id of
the failed request. This was rejected because:
- Client will need to maintain a map of request_id -> request in order to use
the partial success response.
- Populating a request id for the purpose of communicating errors could
conflict with [AIP-155](https://aip.dev/155) if the service can not
guarantee idempotency for an individual request across multiple batch
requests.
- A `repeated FailedRequest` field, where FailedRequest contains the individual
update request and the `google.rpc.Status`. This was rejected because echoing
the request payload back in response is discouraged due to additional
challenges around user data sensitivity.
[aip-122-parent]: ./0122.md#fields-representing-a-resources-parent
[request-message]: ./0134.md#request-message
## Changelog
- **2025-03-06**: Changed recommendation to allow partial success, along with
detailed guidance
- **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-".
- **2020-09-16**: Suggested annotating `parent` and `requests` fields.
- **2020-08-27**: Removed parent recommendations for top-level resources.
- **2019-09-11**: Fixed the wording about which child field the `parent` field
should match.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
---
id: 235
state: approved
created: 2019-06-18
updated: 2022-06-02
placement:
category: operations
order: 230
---
# Batch methods: Delete
Some APIs need to allow users to delete a set of resources in a single
transaction. A batch delete method provides this functionality.
## Guidance
APIs **may** support Batch Delete using the following two patterns:
Returning the response synchronously
```proto
rpc BatchDeleteBooks(BatchDeleteBooksRequest) returns (google.protobuf.Empty) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:batchDelete"
body: "*"
};
}
```
Returning an Operation which resolves to the response asynchronously
```proto
rpc BatchDeleteBooks(BatchDeleteBooksRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:batchDelete"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "google.protobuf.Empty"
metadata_type: "BatchDeleteBooksOperationMetadata"
};
}
```
- The RPC's name **must** begin with `BatchDelete`. The remainder of the RPC
name **should** be the plural form of the resource being deleted.
- The request message **must** match the RPC name, with a `Request` suffix.
- The response message **should** be `google.protobuf.Empty`.
- If the resource is [soft deleted][soft-delete], the response message
**should** be a response message containing the updated resources.
- If the batch method returns an `google.longrunning.Operation`, both the
`response_type` and `metadata_type` fields **must** be specified.
- If the resource is [soft deleted][soft-delete], the `response_type`
**should** be a response message containing the updated resources.
- The HTTP verb **must** be `POST` (not `DELETE`).
- The HTTP URI **must** end with `:batchDelete`.
- The URI path **should** represent the collection for the resource, matching
the collection used for simple CRUD operations. If the operation spans
parents, a dash (`-`) **may** be accepted as a wildcard.
- The body clause in the `google.api.http` annotation **should** be `"*"`.
### Atomic vs. Partial Success
- The batch delete method **may** support atomic (all resources deleted or none
are) or partial success behavior. To make a choice, consider the following
factors:
- **Complexity of Ensuring Atomicity:** Operations that are simple
passthrough database transactions **should** use an atomic operation,
while operations that manage complex resources **should** use partial
success operations.
- **End-User Experience:** Consider the perspective of the API consumer.
Would atomic behavior be preferable for the given use case, even if it
means that a large batch could fail due to issues with a single or a few
entries?
- Synchronous batch delete **must** be atomic.
- Asynchronous batch delete **may** support atomic or partial success.
- If supporting partial success, see
[Operation metadata message](#operation-metadata-message) requirements.
### Request message
The request for a batch delete method **should** be specified with the
following pattern:
```proto
message BatchDeleteBooksRequest {
// The parent resource shared by all books being deleted.
// Format: publishers/{publisher}
// If this is set, the parent of all of the books specified in `names`
// must match this field.
string parent = 1 [
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The names of the books to delete.
// A maximum of 1000 books can be deleted in a batch.
// format: publishers/{publisher}/books/{book}
repeated string names = 2 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
}
```
- A `parent` field **should** be included, unless the resource being deleted is
a top-level resource. If a caller sets this field, and the
parent collection in the name of any resource being deleted does not match,
the request **must** fail.
- This field **should** be required if only 1 parent per request is allowed.
- The field **should** identify the [resource type][aip-122-parent] that it
references.
- The comment for the field **should** document the resource pattern.
- The request message **must** include a repeated field which accepts the
resource names specifying the resources to delete. The field **should** be
named `names`.
- The field **should** be required.
- The field **should** identify the [resource type][aip-122-names] that it
references.
- The comment for the field **should** document the resource pattern.
- Other fields besides `name` **may** be "hoisted" from the [standard Delete
request][request-message]. There is no way to allow for these fields to
accept different values for different resources; if this is needed, use the
[alternative request message form](#request-message-containing-standard-delete-request-messages).
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
- The comment above the `names` field **should** document the maximum number of
requests allowed.
- Filter-based matching **must not** be supported.
### Request message containing standard delete request messages
If the [standard Delete request message][request-message] contains a field
besides the resource name that needs to be different between different
resources being requested, the batch message **may** alternatively hold a
`repeated` field of the [standard Delete request message][request-message].
This is generally discouraged unless your use case really requires it.
The request for a batch delete method **should** be specified with the
following pattern:
```proto
message BatchDeleteBooksRequest {
// The parent resource shared by all books being deleted.
// Format: publishers/{publisher}
// If this is set, the parent of all of the books specified in the
// DeleteBookRequest messages must match this field.
string parent = 1 [
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The requests specifying the books to delete.
// A maximum of 1000 books can be deleted in a batch.
repeated DeleteBookRequest requests = 2
[(google.api.field_behavior) = REQUIRED];
}
```
- A `parent` field **should** be included. If a caller sets this field, and the
parent collection in the name of any resource being deleted does not match,
the request **must** fail.
- This field **should** be required if only 1 parent per request is allowed.
- The field **should** identify the [resource type][aip-122-parent] that it
references.
- The comment for the field **should** document the resource pattern.
- The request message **must** include a repeated field which accepts the
request messages specifying the resources to delete, as specified for
[standard Delete methods][request-message]. The field **should** be named
`requests`.
- The field **should** be required.
- Other fields **may** be "hoisted" from the [standard Delete
request][request-message], which means that the field can be set at either
the batch level or child request level. Similar to `parent`, if both the
batch level and child request level are set for the same field, the values
**must** match.
- Fields which must be unique cannot be hoisted (e.g. `etag`).
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
- The comment above the `requests` field **should** document the maximum number
of requests allowed.
- Filter-based matching **must not** be supported unless it is infeasible to
support critical use cases without it, because it makes it too easy for users
to accidentally delete important data. If it is unavoidable, see [AIP-165][].
### Response message (soft-delete only)
In the case where a response message is necessary because the resource is
soft-deleted, the response **should** be specified with the following pattern:
```proto
message BatchDeleteBooksResponse {
// Books deleted.
repeated Book books = 1;
}
```
- The response message **must** include one repeated field corresponding to the
resources that were soft-deleted.
### Operation metadata message
- The `metadata_type` message **must** either match the RPC name with
`OperationMetadata` suffix, or be named with `Batch` prefix and
`OperationMetadata` suffix if the type is shared by multiple Batch methods.
- If batch delete method supports partial success, the metadata message **must**
include a `map<int32, google.rpc.Status> failed_requests` field to communicate
the partial failures.
- The key in this map is the index of the request in the `requests` field in
the batch request.
- The value in each map entry **must** mirror the error(s) that would normally
be returned by the singular Standard Delete method.
- If a failed request can eventually succeed due to server side retries, such
transient errors **must not** be communicated using `failed_requests`.
- When all requests in the batch fail, `Operation.error` **must** be set with
`code = google.rpc.Code.Aborted` and `message = "None of the requests
succeeded, refer to the BatchDeleteBooksOperationMetadata.failed_requests
for individual error details"`
- The metadata message **may** include other fields to communicate the
operation progress.
### Adopting Partial Success
In order for an existing Batch API to adopt the partial success pattern, the API
must do the following:
- The default behavior must be retained to avoid incompatible behavioral
changes.
- If the API returns an Operation:
- The request message **must** have a `bool return_partial_success` field.
- The Operation `metadata_type` **must** include a
`map<int32, google.rpc.Status> failed_requests` field.
- When the `bool return_partial_success` field is set to true in a request,
the API should allow partial success behavior, otherwise it should continue
with atomic behavior as default.
- If the API returns a direct response synchronously:
- Since the existing clients will treat a success response as an atomic
operation, the existing version of the API **must not** adopt the partial
success pattern.
- A new version **must** be created instead that returns an Operation and
follows the partial success pattern described in this AIP.
## Rationale
### Restricting synchronous batch methods to be atomic
The restriction that synchronous batch methods must be atomic is a result of
the following considerations.
The previous iteration of this AIP recommended batch methods must be atomic.
There is no clear way to convey partial failure in a sync response status code
because an OK implies it all worked. Therefore, adding a new field to the
response to indicate partial failure would be a breaking change because the
existing clients would interpret an OK response as all resources created.
On the other hand, as described in [AIP-193](https://aip.dev/193), Operations
are more capable of presenting partial states. The response status code for an
Operation does not convey anything about the outcome of the underlying operation
and a client has to check the response body to determine if the operation was
successful.
### Communicating partial failures
The AIP recommends using a `map<int32, google.rpc.Status> failed_requests` field
to communicate partial failures, where the key is the index of the failed
request in the original batch request. The other options considered were:
- A `repeated google.rpc.Status` field. This was rejected because it is not
clear which entry corresponds to which request.
- A `map<string, google.rpc.Status>` field, where the key is the request id of
the failed request. This was rejected because:
- Client will need to maintain a map of request_id -> request in order to use
the partial success response.
- Populating a request id for the purpose of communicating errors could
conflict with [AIP-155](https://aip.dev/155) if the service can not
guarantee idempotency for an individual request across multiple batch
requests.
- A `repeated FailedRequest` field, where FailedRequest contains the individual
create request and the `google.rpc.Status`. This was rejected because echoing
the request payload back in response is discouraged due to additional
challenges around user data sensitivity.
[aip-122-names]: ./0122.md#fields-representing-resource-names
[aip-122-parent]: ./0122.md#fields-representing-a-resources-parent
[aip-165]: ./0165.md
[request-message]: ./0135.md#request-message
[soft-delete]: ./0135.md#soft-delete
## Changelog
- **2025-03-06**: Changed recommendation to allow partial success, along with
detailed guidance
- **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-".
- **2020-09-16**: Suggested annotating `parent`, `names`, and `requests` fields.
- **2020-08-27**: Removed parent recommendations for top-level resources.
- **2020-03-27**: Added reference to AIP-165 for criteria-based deletion.
- **2019-10-11**: Changed the primary recommendation to specify a repeated
string instead of a repeated standard Delete message. Moved the original
recommendation into its own section.
- **2019-09-11**: Fixed the wording about which child field the `parent` field
should match.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
---
id: 236
state: approved
created: 2023-03-30
updated: 2023-03-30
placement:
category: resource-design
order: 240
---
# Policy preview
A policy is a resource that provides rules that admit or deny access to other
resources. Generally, the outcome of a policy can be evaluated to a specific set
of outcomes.
Changes to policies without proper validation may have unintended consequences
that can severely impact a customer’s overall infrastructure setup. To safely
update resources, it is beneficial to test these changes via policy rollout
APIs.
Preview is a rollout safety mechanism for policy resources, which gives the
customer the ability to validate the effect of their proposed changes against
production traffic prior to the changes going live. The result of the policy
evaluation against traffic is logged in order to give the customer the data
required to test the correctness of the change.
Firewall policies exemplify a case that is suitable for previewing. A new
configuration can be evaluated against traffic to observe which IPs would be
allowed or denied. This gives the customer the data to guide a decision on
whether to promote the proposed changes to live.
The expected flow for previewing a policy is as follows:
1. The user creates an experiment containing a new policy configuration
intended to replace the live policy.
2. The user uses the "startPreview" method to start generating logs which compare
the live and experiment policy evaluations against live traffic.
3. The user inspects the logs to determine whether the experiment has the
intended result.
4. The user uses the "commit" method to promote the experiment to live.
## Guidance
### Non-goals
This proposal is for a safety mechanism for policy rollouts only. Safe rollouts
for non-policy resources are not in scope.
### Experiments
A new configuration of a policy to be previewed is stored as a nested collection
under the policy. These nested collections are known as experiments.
A hypothetical policy resource called, `Policy`, is used throughout. It has the
following resource name pattern:
`projects/{project}/locations/{location}/policies/{policy}`
The experimental versions of the resource used for previewing or other safe
rollout practices are represented as a nested collection under `Policy` using a
new resource type. The resource type **must** follow the naming convention
*RegularResourceType*`Experiment`.
The following pattern is used for the experiment collection:
`projects/{project}/locations/{location}/policies/{policy}/experiments/{experiment}`
A proto used to represent an experiment **must** contain the following:
1. The required top-level fields for a resource, like `name` and `etag`
2. The policy message that is being tested itself
3. The field, `preview_metadata`, which contains metadata specific to
previewing the experiment of a specific resource type.
```proto
message PolicyExperiment {
// google.api.resource, name, and other annotations and fields
// The policy experiment. This Policy will be used to preview the effects of
// the change but will not affect live traffic.
Policy policy = 2;
// The metadata associated with this policy experiment.
PolicyPreviewMetadata preview_metadata = 3
[(google.api.field_behavior) = OUTPUT_ONLY];
// Allows clients to store small amounts of arbitrary data.
map<string, string> annotations = 4;
}
```
- The experiment proto **must** have a top-level field with the same type as the
live policy.
- It **must** be named as the live resource type. For example, if the
experiment is for FirewallPolicy, then this field **must** be named
`firewall_policy`.
- The name inside the embedded `policy` message **must** be the name of the
live policy.
- When the user is ready to promote an experiment, they **must** copy the
`policy` message into the live policy and delete the experiment. This can be
done manually or via a "commit" custom method.
- A product **may** support multiple experiments concurrently being previewed
for a single live policy.
- Each experiment must generate logs having each entry preceded by log_prefix
so that the user can compare the results of the experiment with the behavior
of the live policy.
- The number of experimental configurations for a given live policy **may** be
capped at a certain number and the cap **must** be documented.
- Cascading deletes **must** occur: if the live policy is deleted, all
experiments **must** also be deleted.
- `map<string,string>` [annotations][aip-128-annotations] **must** allow clients
to store small amounts of arbitrary data.
### Metadata
`preview_metadata` tracks all metadata of previewing the experiment. The
messages **must** follow the convention: *RegularResourceType*`PreviewMetadata`.
This is so the proto can be defined uniquely for each resource type in the
same service with experiments.
```proto
message PolicyPreviewMetadata {
// Possible values of the state of previewing the experiment.
enum State {
// Default value. This value is unused.
STATE_UNDEFINED = 0;
// The experiment is actively previewing.
ACTIVE = 1;
// The previewing of the experiment has been stopped.
SUSPENDED = 2;
}
// The state of previewing the experiment.
State state = 1;
// An identifying string common to all logs generated when previewing the
// experiment. Searching all logs for this string will isolate the results.
string log_prefix = 2;
// The most recent time at which this experiment started previewing.
google.protobuf.Timestamp start_time = 3;
// The most recent time at which this experiment stopped previewing.
google.protobuf.Timestamp stop_time = 4;
}
```
- `PolicyPreviewMetadata` **must** have the fields defined in the proto above.
- It **may** have additional fields if the service or resource requires it.
- When an experiment is first previewed, `preview_metadata` **must** be
absent.
- It is present on the experiment once the "startPreview" method is used.
- All `preview_metadata` fields **must** be output only.
- `state` changes between `ACTIVE` and `SUSPENDED` when previewing is started
or stopped. This happens when the "startPreview" or "stopPreview custom methods
are invoked, respectively.
- The first time the "startPreview" custom method is used, the system **must**
create `preview_metadata` and do the following:
- It **must** set the `state` to `ACTIVE`
- It **must** populate `start_time` with the current time.
- `start_time` **must** be updated every time `state` is changed to
`ACTIVE`.
- It **must** set a system generated `log_prefix` string, which is a
predefined constant hard coded by the system developers.
- The same value is used for previewing experiments for the given resource
type. For example, "FirewallPolicyPreviewLog" for FirewallPolicy.
- When the "stopPreview" custom method is used, the system **must** do the
following:
- It **must** set the `state` to `SUSPENDED`
- It **must** populate the `stop_time` with the current time.
### Methods
#### create
- The resource **must** be created using long-running
[Create][aip-133-long-running] and
`google.longrunning.operation_info.response_type` **must** be
`PolicyExperiment`.
- Creating a new experiment to preview **must** support the following use
cases:
- Preview a new policy.
- Preview an update to an already live policy.
- Preview a deletion of a current policy.
- For the update and delete use cases, the `policy` field in the experiment
**must** have the full payload of the live policy copied into it, including
the name.
- The user **must** set the rules to the new intended state to preview an
update.
- The user **must** set set the rules to represent a no-op to preview a
delete.
- To preview a new policy, the system must do the following:
- If the system does not support a nested collection without a live policy,
the user **must** create a live policy and set the rules to represent a
no-op. For example, the rules of a no-op policy **may** be empty.
- An experiment is created as a child of the no-op policy.
- If the system supports previewing multiple experiments for a live policy,
calling "create" more than once **must** create multiple experiments.
#### update
- The resource **must** be updated using long-running
[Update][aip-134-long-running] and
`google.longrunning.operation_info.response_type` **must** be
`PolicyExperiment`.
- The name inside `policy` **must not** change but the other fields can in
order to change the experiment being previewed because this `policy` is
intended to replace the live policy, and the name of the live policy
**must not** change.
- The system **must** set the `state` to `SUSPENDED` if the `state` was `ACTIVE`
at the time of an update.
- This is so the user can easily distinguish between different versions of
the experiment being previewed.
#### get
- The standard method, [Get][aip-131], **must** be included for
`PolicyExperiment` resource types.
#### list
- The standard method, [List][aip-132], **must** be included for
`PolicyExperiment` resource types.
- Filtering on `PolicyPreviewMetadata` indicates which experiments are actively
previewed.
- For example, the following filter string returns a List response with
experiments being previewed: preview_metadata.state = ACTIVE.
#### delete
- The resource **must** be deleted using long-running
[Delete][aip-135-long-running] and
`google.longrunning.operation_info.response_type` **must** be
`PolicyExperiment`.
#### startPreview
```proto
// Starts previewing a PolicyExperiment. This triggers the system to start
// generating logs to evaluate the PolicyExperiment.
rpc StartPreviewPolicyExperiment(StartPreviewPolicyExperimentRequest)
returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{name=policies/*/experiments/*}:startPreview"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "PolicyExperiment"
metadata_type: "StartPreviewPolicyExperimentMetadata"
};
}
// The request message for the startPreview custom method.
message StartPreviewPolicyExperimentRequest {
// The name of the PolicyExperiment.
string name = 1;
}
```
- This custom method is required.
- `google.longrunning.Operation.metadata_type` **must** follow guidance on
[Long-running operations][aip-151]
- This method **must** trigger the system to start generating logs to preview
the experiment.
- Whenever the method is called successfully, the system **must** set the
following values in the `PolicyPreviewMetadata`:
- `log_prefix` to the predefined constant.
- `start_time` to the current time
- `state` to `ACTIVE`.
- If the method is called on an experiment with the rules representing a no-op,
then the system **must** preview the deletion of the live policy.
#### stopPreview
```proto
// Stops previewing a PolicyExperiment. This triggers the system to stop
// generating logs to evaluate the PolicyExperiment.
rpc StopPreviewPolicyExperiment(StopPreviewPolicyExperimentRequest)
returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{name=policies/*/experiments/*}:stopPreview"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "PolicyExperiment"
metadata_type: "StopPreviewPolicyExperimentMetadata"
};
}
// The request message for the stopPreview custom method.
message StopPreviewPolicyExperimentRequest {
// The name of the PolicyExperiment.
string name = 1;
}
```
- This custom method is required.
- `google.longrunning.Operation.metadata_type` **must** follow guidance on
[Long-running operations][aip-151]
- This method **must** trigger the system to stop generating logs to preview the
experiment.
- Whenever the method is called successfully, the system **must** set the
following values in the `PolicyPreviewMetadata`:
- `stop_time` to the current time
- `state` to `SUSPENDED`
#### commit
The resource **may** expose a new custom method called "commit" to promote an
experiment. The system copies `policy` from the experiment into the live policy
and then deletes the experiment.
Declarative clients **may** manually copy fields from an experiment into the
live policy and then delete the experiment rather than calling "commit" if
preferable.
```proto
// Commits a PolicyExperiment. This copies the PolicyExperiment's policy message
// to the live policy then deletes the PolicyExperiment.
rpc CommitPolicyExperiment(CommitPolicyExperimentRequest)
returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{name=policies/*/experiments/*}:commit"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "google.protobuf.Empty"
metadata_type: "CommitPolicyExperimentMetadata"
};
}
// The request message for the commit custom method.
message CommitPolicyExperimentRequest {
string name = 1;
string etag = 2;
string parent_etag = 3;
}
```
- `google.longrunning.Operation.metadata_type` **must** follow guidance on
[Long-running operations][aip-151]
- The method **must** atomically copy `policy` from the experiment into the live
policy, and then delete the experiment.
- If any experiment fails "commit", previewing it **must not** stop, and the
live policy **must not** be updated.
- The method can be called on an experiment in any state.
- The `etag` **must** match that of the experiment in order for commit to be
successful. This is so the user does not commit an unintended version of the
experiment.
- If no `etag` is provided, the API **must not** succeed to prevent the user
from unintentionally committing a different version of the experiment as
intended.
- A `parent_etag` **may** be provided to guarantee that the experiment
overwrites a specific version of the live policy.
- The method is not idempotent and calling it twice on the same experiment
**must** return a 404 NOT_FOUND as the experiment is deleted as part of the
first call.
### Changes to live policy API methods
#### delete
- A delete of the live policy **must** delete all experiments.
- To maintain the experiments while negating the effect of the live policy, the
live policy **must** be changed to a no-op policy instead of using this
method.
### Logging
Logging is crucial for the user to evaluate whether an experiment should be
promoted to live.
Logs **must** contain the results of the evaluated experiment, the `etag`
associated with that experiment alongside that of the live policy, and be
preceded by the value of `log_prefix`.
- The `etag` fields help the user identify which
configurations of the live and experiment are evaluated in the log.
- `log_prefix` helps the user separate logs specifically generated for
previewing the experiment from other use cases.
Overall, these logs help the user make a decision about whether to promote the
experiment to live.
## Changelog
- **2023-04-27:** Methods for start and stop renamed. State to enum. Annotations
added.
- **2023-03-30:** Initial AIP written.
[aip-128-annotations]: https://aip.dev/128#annotations
[aip-131]: https://aip.dev/131
[aip-132]: https://aip.dev/132
[aip-133-long-running]: https://aip.dev/133#long-running-create
[aip-134-long-running]: https://aip.dev/134#long-running-update
[aip-135-long-running]: https://aip.dev/135#long-running-delete
[aip-151]: https://google.aip.dev/151
---
title: General
order: 0
categories:
- code: meta
- code: process
- code: api-concepts
title: API Concepts
- code: resource-design
title: Resource Design
- code: operations
- code: fields
- code: design-patterns
title: Design Patterns
- code: compatibility
title: Compatibility and Versioning
- code: polish
- code: protobuf
title: Protocol buffers
- code: misc
title: Miscellaneous
default: true
Reference in New Issue View Git Blame Copy Permalink