Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions doc/content/design/pool-certificates.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ revision: 2
status: released (22.6.0)
---

This design is modified by [trusted-certificates.md](trusted-certificates.md).

## Overview

Xenserver has used TLS-encrypted communications between xapi daemons in a pool since its first release.
Expand Down Expand Up @@ -353,22 +355,30 @@ This feature needs clients to behave differently when initiating pool joins, to
Several alerts are introduced:
* POOL_CA_CERTIFICATE_EXPIRING_30, POOL_CA_CERTIFICATE_EXPIRING_14, POOL_CA_CERTIFICATE_EXPIRING_07, POOL_CA_CERTIFICATE_EXPIRED: Similar to host certificates, now the user-installable pool's CA certificates are monitored for expiry dates and alerts are generated about them. The body for this type of message is:

```
<body><message>The trusted TLS server certificate {is expiring soon|has expired}.</message><date>20210302T02:00:01Z</date></body>
```

* HOST_INTERNAL_CERTIFICATE_EXPIRING_30, HOST_INTERNAL_CERTIFICATE_EXPIRING_14, HOST_INTERNAL_CERTIFICATE_EXPIRING_07, HOST_INTERNAL_CERTIFICATE_EXPIRED: Similar to host certificates, the newly-introduced hosts' internal server certificates are monitored for expiry dates and alerts are generated about them. The body for this type of message is:

```
<body><message>The TLS server certificate for internal communications {is expiring soon|has expired}.</message><date>20210302T02:00:01Z</date></body>
```

* TLS_VERIFICATION_EMERGENCY_DISABLED: The host is in emergency mode and is not enforcing tls verification anymore, the situation that forced the disabling must be fixed and the verification enabled ASAP.

```
<body><host>HOST-UUID</host></body>
```

* FAILED_LOGIN_ATTEMPTS: An hourly alert that contains the number of failed attempts and the 3 most common origins for these failed alerts. The body for this type of message is:

```
<body>
<total>35</total>
<known><username>usr5</username><originator>origin5</originator><ip>5.4.3.2</ip><number>10</number><date>20200922T15:03:13Z</date></known>
<known><username>usr4</username><useragent>UA</useragent><number>6</number><date>20200922T15:03:13Z</date></known>
<known><useragent>UA</useragent><ip>4.3.2.1</ip><number>4</number><date>20200922T14:57:11Z</date></known>
<unknown>10</unknown>
</body>
```
127 changes: 111 additions & 16 deletions doc/content/design/trusted-certificates.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@
title: Trusted certificates for identity validation in TLS connections
layout: default
design_doc: true
revision: 1
revision: 2
status: draft
---

# Overview

In various use cases, TLS connections are established on the host on which XAPI runs.
When establishing a TLS connection, the peer identity needs to be validated.
This is done using either a root CA certificate to perform certificate chain validation, or a known peer certificate for validation with certificate pinning.
The root CA certificates and peer certificates involved in this process are referred to as trusted certificates.
This is done using either a root CA certificate to perform certificate chain validation, or a pinned certificate for validation with certificate pinning.
The root CA certificates and pinned certificates involved in this process are referred to as trusted certificates.
When a trusted certificate is installed, the local endpoint can validate the peer identity during TLS connection establishment.
Certificate chain validation is a general-purpose, standards-based approach but requires additional steps, such as getting the peer's certificate signed by a CA.
In contrast, certificate pinning offers a quicker way to set up trust in some cases without the overhead of CA signing.
Expand All @@ -21,13 +21,13 @@ This allows the use case to start in quicker and easier way without prior CA sig

As the unified API for the whole system, XAPI also exposes interfaces for users to install and manage trusted certificates that are used by system components for different purposes.

The base design described in [pool-certificates.md](https://github.com/minglumlu/xen-api/blob/5d1ea1520825d502c57a90a02db476cd7d6a9132/doc/content/design/pool-certificates.md) defines the database, API, and trust store in the filesystem for managing trusted certificates.
The base design described in [pool-certificates.md](pool-certificates.md) defines the database, API, and trust store in the filesystem for managing trusted certificates.
This document introduces the following enhancements to that design:

* Explicit separation of root CA certificates and peer certificates:
* Explicit separation of root CA certificates and pinned certificates:
In the base design, both certificate types share the same database schema, APIs, and are stored together in a single bundle file.
This makes it difficult to determine the appropriate validation approach based on the certificate type.
The improvement introduces a type value to separate root CA certificates and peer certificates explicitly.
The improvement introduces a type value to separate root CA certificates and pinned certificates explicitly.

* Add a "purpose" attribute for trusted certificates:
According to the base design, only certificates used for internal TLS connections among XAPI processes within a pool are stored separately.
Expand All @@ -49,13 +49,13 @@ This case benefits from the improvements introduced in this design as well.
## Database schema
The *Certificate* class in database is defined to represent general certificates, including trusted certificates.
One existing class field "type" supports the following enumeration values:
* "ca": trusted certificates including both root CA and peer.
* "ca": trusted certificates including both root CA and pinned.
* "host": identity certificate of a host for communication with entities outside the pool.
* "host_internal": identity certificate of a host for communication with other pool members.

Two improvements in this design:
* A new value "peer" is introduced in this design so that the existing "ca" now represents trusted root CA only.
The new "peer" will represent trusted peer certificates.
* A new value "pinned" is introduced in this design so that the existing "ca" now represents trusted root CA only.
The new "pinned" will represent trusted pinned certificates.

* A new enumeration type "purpose" is introduced to indicate the intended usage of a trusted certificate.
A new *Certificate* class field "purpose" (a set of values of enumeration type "purpose") will be added to represent all applicable purposes of a trusted certificate.
Expand All @@ -79,27 +79,122 @@ For the same reason, "pool.uninstall_ca_certificate" will also be deprecated.
This is a new API introduced in this design with its arguments being defined as:
* session (ref session_id): reference to a valid session;
* self (ref Pool): reference to the pool;
* ca (boolean): the trusted certificate is a root CA certificate used to verify a chain (true), or a peer certificate used for certificate pinning (false);
* ca (boolean): the trusted certificate is a root CA certificate used to verify a chain (true), or a pinned certificate used for certificate pinning (false);
* cert (string): the trusted certificate in PEM format;
* purpose (string list): the purposes of the trusted certificate.

This new API is used to install trusted certificate.
When *purpose* is an empty set, it stands for a root CA certificate for general purpose.
The *purpose* can not be an empty set when the *ca* is false, because each peer certificate is specific to a single server and therefore unsuitable for a shared trusted certificate for general purpose.
The *purpose* can not be an empty set when the *ca* is false, because each pinned certificate is specific to a single server and therefore unsuitable for a shared trusted certificate for general purpose.

It returns *void* when succeed. Otherwise, return corresponding API error.

### pool.uninstall_trusted_certificate
This is a new API introduced in this design to uninstall a trusted certificate with its arguments being defined as:
* session (ref session_id): reference to a valid session;
* certificate (ref Certificate): reference to the trusted certificate;
* force (bool): remove the database entry even if the file doesn't exist.

It returns *void* when succeed. Otherwise, return corresponding API error.

### pool.join
Prior to this design, trusted certificates are exchanged between the pool and the joining host during the pre‑join phase.
This design preserves that behavior to ensure the joiner works correctly both before and after joining the pool.
According to the base design, trusted certificates are exchanged between the pool and the joining host during the pre‑join phase.
This design basically preserves that behavior to ensure the joiner works correctly both before and after joining the pool.
However, a number of modifications have been introduced compared with the base design.

~~~mermaid

sequenceDiagram
participant clnt as Client
participant join as Joiner
participant coor as Coordinator
participant memb as Member
clnt->>join: Pool.join coordinator_ip coordinator_username coordinator_password
join->>coor:login_with_password rpc_no_verify coordinator_ip coordinator_username coordinator_password
coor-->>join:

Note over join: pre_join_checks
rect rgba(0,0,0,0.05)
join->>join: assert_tls_verification_matches
alt fails
Note over join: interrupt join, raise error
end
end

Note over join: exchnage trusted intra-pool host identity certificates
rect rgba(0,0,0,0.05)
join->>coor: Pool.exchange_certificates_on_join <Joiner's trusted host identity cert>
coor->>coor: Cert_distrib.exchange_certificates_with_joiner start
coor->>memb: Host.cert_distrib_atom Write
memb-->>coor:
coor->>coor: Cert_distrib.get_local_pool_certs
coor-->>coor: Cert_distrib.exchange_certificates_with_joiner done
coor-->>join: <trusted host identity certs in pool>
join->>join: Cert_distrib.import_joining_pool_certs <trusted host identity certs in pool>
end

join->>coor:login_with_password rpc_verify coordinator_ip coordinator_username coordinator_password
coor-->>join:

Note over join: exchange legacy ca certificates
rect rgba(0,0,0,0.05)
join->>coor: Pool.exchange_ca_certificates_on_join <Joiner's legacy ca certs>
coor->>coor: Cert_distrib.exchange_ca_certificates_with_joiner
coor-->>join: <legacy ca certs in pool>
join->>join: Cert_distrib.import_joining_pool_ca_certificates <legacy ca certs in pool>
end

Note over join: exchange trusted certificates
rect rgba(0,0,0,0.05)
join->>coor: Pool.exchange_trusted_certificates_on_join <Joiner's trusted certs>
loop for every <trusted> in Joiner
coor->>coor: Pool.install_trusted_certificate start
coor->>memb: Host.cert_distrib_atom Write
memb-->>coor:
coor->>memb: Host.certificate_sync
memb-->>coor:
coor-->>coor: Pool.install_trusted_certificate done
end
coor->>coor: Cert_distrib.collect_trusted_certs
coor-->>join: <trusted certs in pool>
loop for every <trusted> in pool
join->>join: Pool.install_trusted_certificate
end
end

Note over join: exchange CRLs
rect rgba(0,0,0,0.05)
join->>coor: Pool.exchange_crls_on_join <Joiner's CRLs>
loop for every <CRL> in Joiner
coor->>coor: Pool.crl_install start
coor->>memb: Host.cert_distrib_atom Write
memb-->>coor:
coor->>memb: Host.certificate_sync
memb-->>coor:
coor-->>coor: Pool.crl_install done
end
coor->>coor: Cert_distrib.collect_crls
coor-->>join: <CLRs in pool>
loop for every <CRL> in pool
join->>join: Pool.crl_install
end
end

join->>coor: Host.add joiner
coor-->>join:

join->>join: restart_as_slave

Note over join: Copy all certificates from coordinator
rect rgba(0,0,0,0.05)
join->>coor: Host.copy_primary_host_certs
coor->>join: Host.cert_distrib_atom Write
join-->>coor:
coor->>join: Host.cert_distrib_atom GenBundle
join-->>coor:
coor-->>join: return from Host.copy_primary_host_certs
end

~~~

### pool.eject
The trusted certificates will be removed from any host which is being eject from the pool.
Expand Down Expand Up @@ -130,10 +225,10 @@ The stores for the certificates installed via "pool.install_trusted_certificate"
| Name | Filesystem location | Used for |
| ---- | ------------------- | -------- |
| Trusted General CA | /etc/trusted-certs/ca-general/ | Trusted root CA certificates that users can install to validate a peer’s identity when establishing a TLS connection for general purpose.
| Trusted Peer | /etc/trusted-certs/peer-\<PURPOSE\>/ | Trusted peer certificates that users can install to validate a peer’s identity when establishing a TLS connection for \<PURPOSE\>.
| Trusted Peer | /etc/trusted-certs/pinned-\<PURPOSE\>/ | Trusted pinned certificates that users can install to validate a peer’s identity when establishing a TLS connection for \<PURPOSE\>.
| Trusted CA | /etc/trusted-certs/ca-\<PURPOSE\>/ | Trusted root CA certificates that users can install to validate a peer’s identity when establishing a TLS connection for \<PURPOSE\>.
| General Bundle | /etc/trusted-certs/ca-bundle-general.pem | Bundle of trusted root CA certificates under /etc/trusted-certs/ca-general/ to verify a peer's identity when establishing a TLS connection for general purpose.
| Peer Bundle | /etc/trusted-certs/peer-bundle-\<PURPOSE\>.pem | Bundle of trusted peer certificates under /etc/trusted-certs/peer-\<PURPOSE\>/ to verify a peer's identity when establishing a TLS connection for \<PURPOSE\>.
| Peer Bundle | /etc/trusted-certs/pinned-bundle-\<PURPOSE\>.pem | Bundle of trusted pinned certificates under /etc/trusted-certs/pinned-\<PURPOSE\>/ to verify a peer's identity when establishing a TLS connection for \<PURPOSE\>.
| CA Bundle | /etc/trusted-certs/ca-bundle-\<PURPOSE\>.pem | Bundle of trusted root CA certificates under /etc/trusted-certs/ca-\<PURPOSE\>/ to verify a peer's identity when establishing a TLS connection for \<PURPOSE\>.

The filesystem location is derived from the \<PURPOSE\>. Each \<PURPOSE\> string corresponds to a predefined value of the "purpose" type in the database, implemented as predefined constants.
Expand Down
18 changes: 16 additions & 2 deletions ocaml/alerts/certificate/certificate_check.ml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ type cert =
| CA of API.ref_Certificate * API.datetime
| Host of API.ref_host * API.datetime
| Internal of API.ref_host * API.datetime
| Pinned of API.ref_Certificate * API.datetime

let get_certificates rpc session_id =
XenAPI.Certificate.get_all_records ~rpc ~session_id
Expand All @@ -22,14 +23,18 @@ let get_certificates rpc session_id =
)
| `ca ->
CA (cert_ref, certificate.API.certificate_not_after)
| `pinned ->
Pinned (cert_ref, certificate.API.certificate_not_after)

let certificate_description = function
| Host _ ->
"The TLS server certificate"
| Internal _ ->
"The internal TLS server certificate"
| CA _ ->
"The CA pool certificate"
"The pool-wide trusted root CA certificate"
| Pinned _ ->
"The pool-wide trusted pinned leaf certificate"

let alert_conditions = function
| Host _ ->
Expand All @@ -53,16 +58,25 @@ let alert_conditions = function
; (14, Api_messages.pool_ca_certificate_expiring_14)
; (30, Api_messages.pool_ca_certificate_expiring_30)
]
| Pinned _ ->
[
(0, Api_messages.pool_pinned_certificate_expired)
; (7, Api_messages.pool_pinned_certificate_expiring_07)
; (14, Api_messages.pool_pinned_certificate_expiring_14)
; (30, Api_messages.pool_pinned_certificate_expiring_30)
]

let alert_message_cls_and_obj_uuid rpc session_id cert =
match cert with
| Host (host, _) | Internal (host, _) ->
(`Host, XenAPI.Host.get_uuid ~rpc ~session_id ~self:host)
| CA (cert, _) ->
(`Certificate, XenAPI.Certificate.get_uuid ~rpc ~session_id ~self:cert)
| Pinned (cert, _) ->
(`Certificate, XenAPI.Certificate.get_uuid ~rpc ~session_id ~self:cert)

let get_expiry = function
| Host (_, exp) | Internal (_, exp) | CA (_, exp) ->
| Host (_, exp) | Internal (_, exp) | CA (_, exp) | Pinned (_, exp) ->
exp

let alert rpc session_id =
Expand Down
1 change: 1 addition & 0 deletions ocaml/alerts/certificate/certificate_check.mli
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ type cert =
| CA of API.ref_Certificate * API.datetime
| Host of API.ref_host * API.datetime
| Internal of API.ref_host * API.datetime
| Pinned of API.ref_Certificate * API.datetime

val certificate_description : cert -> string

Expand Down
10 changes: 10 additions & 0 deletions ocaml/idl/datamodel_certificate.ml
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,16 @@ let certificate_type =
; ( "host_internal"
, "Certificate that identifies a single host to other pool members"
)
; ("pinned", "Pinned leaf certificate that is trusted by the whole pool")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer if we used a name that signifies the purpose of the certificate rather than the method used to trust it. This means using peer. One way to think about it is that a peer certificate is going to be pinned to a service with a particular purpose.

Copy link
Member Author

@minglumlu minglumlu Jan 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The confusion of using peer comes from the self-signed certificate. It can be either a root CA, or a leaf certificate for certificate pinning. We hope when a user or a client manipulates the certificates, it's clear for them to know the difference.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The confusion of using peer comes from the self-signed certificate. It can be either a root CA, or a leaf certificate for certificate pinning.

I don't follow the reasoning, I understand that a self-signed certificate can be used in many roles, but the enum makes clear in which role it's being used, as a ca certificate, or a peer certificate. Maybe I don't see the confusion because a peer certificate for me is the certificate that identifies the other peer in a TLS handshake, and is completely separate from the certificate chain, the latter includes the root or CA certificates.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It did lead to confusion with several people involved. If "peer" generally means just the certificate of the remote entity, then it does not clearly imply (in this type field) how to use it in TLS verification, that is verify the chain and hostname or just the "pinned" certificate. If we call it pinned, then it is unambiguous.

]
)

let certificate_purpose =
Enum
( "certificate_purpose"
, [("licensing", "Trusted certificates that are for licensing purpose.")]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"that are for" could be simplified to "Trusted certificates for licensing"

)

let t =
create_obj ~name:_certificate
~descr:"An X509 certificate used for TLS connections" ~doccomments:[]
Expand Down Expand Up @@ -75,5 +82,8 @@ let t =
; field ~qualifier:StaticRO ~lifecycle:[] ~ty:String "fingerprint_sha1"
~default_value:(Some (VString ""))
"The certificate's SHA1 fingerprint / hash"
; field ~qualifier:StaticRO ~lifecycle:[] ~ty:(Set certificate_purpose)
"purpose" ~default_value:(Some (VSet []))
"The purposes of the certificate"
]
~messages:[] ()
2 changes: 1 addition & 1 deletion ocaml/idl/datamodel_common.ml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ open Datamodel_roles
to leave a gap for potential hotfixes needing to increment the schema version.*)
let schema_major_vsn = 5

let schema_minor_vsn = 793
let schema_minor_vsn = 794

(* Historical schema versions just in case this is useful later *)
let rio_schema_major_vsn = 5
Expand Down
30 changes: 28 additions & 2 deletions ocaml/idl/datamodel_errors.ml
Original file line number Diff line number Diff line change
Expand Up @@ -878,6 +878,13 @@ let _ =
"The host joining the pool has different CA certificates from the pool \
coordinator while using the same name, uninstall them and try again."
() ;
error Api_errors.pool_joining_host_trusted_certificates_conflict
["ref_in_pool"; "ref_on_host"]
~doc:
"The joining host has a trusted certificate identical to one on the pool \
coordinator but with different purpose. Uninstall it then install it on \
the host again with the pool-compatible purpose, and try again."
() ;
error Api_errors.pool_joining_sm_features_incompatible
["pool_sm_ref"; "candidate_sm_ref"]
~doc:
Expand Down Expand Up @@ -1659,6 +1666,8 @@ let _ =
~doc:"The specified certificate does not exist." () ;
error Api_errors.certificate_already_exists ["name"]
~doc:"A certificate already exists with the specified name." () ;
error Api_errors.trusted_certificate_already_exists ["fingerprint"]
~doc:"A trusted certificate already exists with the same purpose." () ;
error Api_errors.certificate_name_invalid ["name"]
~doc:"The specified certificate name is invalid." () ;
error Api_errors.certificate_corrupt ["name"]
Expand All @@ -1685,7 +1694,7 @@ let _ =
() ;

error Api_errors.server_certificate_invalid []
~doc:"The provided certificate is not in a PEM-encoded X509." () ;
~doc:"The provided certificate is not in a PEM-encoded X509 format." () ;
error Api_errors.server_certificate_key_mismatch []
~doc:
"The provided key does not match the provided certificate's public key."
Expand All @@ -1701,9 +1710,26 @@ let _ =
() ;

error Api_errors.server_certificate_chain_invalid []
~doc:"The provided intermediate certificates are not in a PEM-encoded X509."
~doc:
"The provided intermediate certificates are not in a PEM-encoded X509 \
format."
() ;

error Api_errors.not_trusted_certificate ["ref"]
~doc:"The provided certificate is not a trusted certificate." () ;

error Api_errors.certificate_lacks_purpose []
~doc:"No purpose is specified for the provided certificate." () ;

error Api_errors.trusted_certificate_expired ["now"; "not_after"]
~doc:"The provided certificate has expired." () ;

error Api_errors.trusted_certificate_not_valid_yet ["now"; "not_before"]
~doc:"The provided certificate is not valid yet." () ;

error Api_errors.trusted_certificate_invalid []
~doc:"The provided certificate is not in a PEM-encoded X509 format." () ;

error Api_errors.vmpp_has_vm []
~doc:"There is at least one VM assigned to this protection policy." () ;
error Api_errors.vmpp_archive_more_frequent_than_backup []
Expand Down
Loading
Loading