Improve handling of trusted certificates#6826
Improve handling of trusted certificates#6826minglumlu wants to merge 20 commits intoxapi-project:feature/trusted-certsfrom
Conversation
f490581 to
ac35717
Compare
| let certificate_purpose = | ||
| Enum | ||
| ( "certificate_purpose" | ||
| , [("licensing", "Trusted certificates that are for licensing purpose.")] |
There was a problem hiding this comment.
"that are for" could be simplified to "Trusted certificates for licensing"
ocaml/idl/datamodel_host.ml
Outdated
| let install_trusted_certificate = | ||
| call ~flags:[`Session] ~pool_internal:true ~hide_from_docs:true | ||
| ~name:"install_trusted_certificate" | ||
| ~doc:"Install a TLS trusted certificate on this host." |
There was a problem hiding this comment.
It's not clear how TLS and trusted are related: is it a TLS-trusted certificate or a trusted TLS certificate?
There was a problem hiding this comment.
It's the trusted certificates used to verify peer identify when establishing TLS connection.
There was a problem hiding this comment.
Would use trusted TLS certificate: it is a TLS certificate that we trust for this purpose.
ocaml/idl/datamodel_pool.ml
Outdated
|
|
||
| let install_trusted_certificate = | ||
| call ~name:"install_trusted_certificate" | ||
| ~doc:"Install a TLS trusted certificate, pool-wide." |
There was a problem hiding this comment.
See above for the ambiguity about how trust and TLS are related.
ocaml/xapi/certificates.ml
Outdated
| Xapi_stdext_threads.Threadext.Mutex.execute m (fun () -> | ||
| let (_ : string), (_ : string) = | ||
| Forkhelpers.execute_command_get_output | ||
| "/opt/xensource/bin/update-ca-bundle.sh" [cert_dir; bundle_path] |
There was a problem hiding this comment.
This script so far does not take arguments and I don't see it being changed but maybe I missed the change. How is this supposed to work and does this mean other invocations of this script need to be updated?
There was a problem hiding this comment.
The script seems can be replaced with https://github.com/xapi-project/xen-api/blob/master/ocaml/xapi/cert_distrib.ml#L45
I didn't find revoking from other components so far. But if there are, they should change as well.
There was a problem hiding this comment.
The code is in
- https://github.com/xapi-project/xen-api/blob/master/scripts/update-ca-bundle.sh
and the way I read it, it does not take arguments. Is this PR changing it?
There was a problem hiding this comment.
Yes. This PR is changing it.
There was a problem hiding this comment.
cert_distrib does not replace update-ca-bundle.sh, they are for different purposes:
-cert_distrib distributes certificates across the pool, and update-ca-bundle creates the bundles from the distributed certificates.
Note that cert_distrib calls Helpers.update_ca_bundle, which runs update-ca-bundle.sh.
| ] | ||
| | Pinned _ -> | ||
| [ | ||
| (0, Api_messages.pool_ca_certificate_expired) |
There was a problem hiding this comment.
Should the alerts about the pinned certificates reuse the same message as the CA ones?
There was a problem hiding this comment.
Add new messages for pool_pinned_certificate_...
ocaml/xapi/db_gc_util.ml
Outdated
| all_certificates | ||
| |> List.filter (fun (cert, record) -> | ||
| record.API.certificate_type <> `ca | ||
| && record.API.certificate_type <> `pinned |
There was a problem hiding this comment.
It's now tempting to change the condition to match host or host_internal certificates instead of the ones that are not that..
| ) | ||
| ; ( "pool-install-trusted-certificate" | ||
| , { | ||
| reqd= ["filename"] |
There was a problem hiding this comment.
I would prefer if ca was a required parameter, there's nothing indicating what is the default mode to the user.
| Option.bind (List.assoc_opt "purpose" params) (fun ss -> | ||
| String.split_on_char ',' ss | ||
| |> List.map Record_util.certificate_purpose_of_string | ||
| |> Option.some | ||
| ) | ||
| |> Option.to_list | ||
| |> List.flatten |
There was a problem hiding this comment.
If the parameter allows more than one values, separated by comas, it should be documented in cli_frontend so users can see it.
Code here also looks more complex than necessary:
| Option.bind (List.assoc_opt "purpose" params) (fun ss -> | |
| String.split_on_char ',' ss | |
| |> List.map Record_util.certificate_purpose_of_string | |
| |> Option.some | |
| ) | |
| |> Option.to_list | |
| |> List.flatten | |
| List.assoc_opt "purpose" params | |
| |> Option.map (fun ss -> | |
| String.split_on_char ',' ss | |
| |> List.map Record_util.certificate_purpose_of_string | |
| ) | |
| |> function Some purposes -> purposes | None -> [] |
ocaml/xapi-consts/api_errors.ml
Outdated
| let not_allowed_when_ntp_is_enabled = | ||
| add_error "NOT_ALLOWED_WHEN_NTP_IS_ENABLED" | ||
|
|
||
| let not_trusted_certificate = add_error "NOT_TRUSTED_CERTIFICTE" |
There was a problem hiding this comment.
| let not_trusted_certificate = add_error "NOT_TRUSTED_CERTIFICTE" | |
| let not_trusted_certificate = add_error "NOT_TRUSTED_CERTIFICATE" |
ocaml/xapi/certificates.ml
Outdated
| * Note that the bundles (e) and (f) are generated automatically using the contents of (c) and (d) respectively *) | ||
|
|
||
| type t_trusted = CA_Certificate | CRL | ||
| type t_trusted = CA_Certificate | CRL | Root_CA | Leaf_Pinned |
There was a problem hiding this comment.
It's not clear the difference between CA_Certificate and Root_CA just by looking at the names. Since be don't need to maintain the names here (it's not exposed in the API) we can change the names to:
type t_trusted = Root | Peer | Root_legacy | CRLThe comment before the type should also be updated with the new directories
Also the types in the functions further down seem to imply that the legacy certs are never linked with a purpose, but the new ones might be, I recommend using types to enforce this:
module Trusted = sig
type t = Root of string list | Peer of string list | Root_legacy | CRL
endThis way the functions lose the purpose parameter while allowing existing ones to disregard the purpose if they don't need it:
let library_path = function
| Root_legacy ->
!Xapi_globs.trusted_certs_dir
| CRL ->
Stunnel.crl_path
| Root _ | Peer _ ->
!Xapi_globs.trusted_certs_by_purpose_dir
let all_trusted_kinds =
let all_purposes =
[] :: List.map (fun x -> [x]) API.certificate_purpose__all
in
List.concat [
List.map (fun p -> Root p) all_purposes
; List.map (fun p -> Peer p) all_purposes
; [Root_legacy; CRL]
]
let trusted_cert_dirs kind =
let parent = library_path kind in
match kind with
| CA_Certificate | CRL ->
[parent]
| Root_CA p ->
[parent // ps "ca-%s" (string_of_purpose p)]
| Leaf_Pinned p ->
[parent // ps "pinned-%s" (string_of_purpose p)]
let trusted_bundle_location kind = ...
let with_cert_store kind f = ...
let rehash () =
let ( let* ) l f = List.iter f l in
let* kind = all_trusted_kinds in
with_cert_store kind
@@ fun ~cert_dir ~bundle_dir:_ ~bundle_name:_ ->
mkdir_cert_path kind ;
rehash' cert_dir| ; ( "host_internal" | ||
| , "Certificate that identifies a single host to other pool members" | ||
| ) | ||
| ; ("pinned", "Pinned leaf certificate that is trusted by the whole pool") |
There was a problem hiding this comment.
I would prefer if we used a name that signifies the purpose of the certificate rather than the method used to trust it. This means using peer. One way to think about it is that a peer certificate is going to be pinned to a service with a particular purpose.
There was a problem hiding this comment.
The confusion of using peer comes from the self-signed certificate. It can be either a root CA, or a leaf certificate for certificate pinning. We hope when a user or a client manipulates the certificates, it's clear for them to know the difference.
There was a problem hiding this comment.
The confusion of using peer comes from the self-signed certificate. It can be either a root CA, or a leaf certificate for certificate pinning.
I don't follow the reasoning, I understand that a self-signed certificate can be used in many roles, but the enum makes clear in which role it's being used, as a ca certificate, or a peer certificate. Maybe I don't see the confusion because a peer certificate for me is the certificate that identifies the other peer in a TLS handshake, and is completely separate from the certificate chain, the latter includes the root or CA certificates.
There was a problem hiding this comment.
It did lead to confusion with several people involved. If "peer" generally means just the certificate of the remote entity, then it does not clearly imply (in this type field) how to use it in TLS verification, that is verify the chain and hostname or just the "pinned" certificate. If we call it pinned, then it is unambiguous.
psafont
left a comment
There was a problem hiding this comment.
For some reason, github is asking me add a global comment instead of allowing me to simply comment on pieces of code
ac35717 to
dde36b8
Compare
Signed-off-by: Ming Lu <ming.lu@cloud.com>
Signed-off-by: Ming Lu <ming.lu@cloud.com>
Signed-off-by: Ming Lu <ming.lu@cloud.com>
dde36b8 to
b2124dc
Compare
ocaml/xapi/certificates.ml
Outdated
| |> List.map (fun kind -> | ||
| mkdir_cert_path kind ; | ||
| trusted_store_locations kind | ||
| |> List.map (fun store -> | ||
| list_dir store.cert_dir |> List.map (fun n -> (n, store.purpose)) | ||
| ) | ||
| |> List.flatten | ||
| ) | ||
| |> List.flatten |
There was a problem hiding this comment.
| |> List.map (fun kind -> | |
| mkdir_cert_path kind ; | |
| trusted_store_locations kind | |
| |> List.map (fun store -> | |
| list_dir store.cert_dir |> List.map (fun n -> (n, store.purpose)) | |
| ) | |
| |> List.flatten | |
| ) | |
| |> List.flatten | |
| |> List.flat_map (fun kind -> | |
| mkdir_cert_path kind ; | |
| trusted_store_locations kind | |
| |> List.flat_map (fun store -> | |
| list_dir store.cert_dir |> List.map (fun n -> (n, store.purpose)) | |
| ) | |
| ) |
|
cert_distrib hasn't been changed in this PR, it's the module that handles copying (CA) certificates on pool join. Has it been tested? In general, I ddidn't expect this amount of code being added to cope with the new certificates. Probably cert_distrib should be used instead, by adding there a couple of new functions to install CA certificates across the pool. |
Well, just found there are lots of duplications between |
|
I simply don't understand why is so much code added to list certificates from the filesystem, with even API calls. It doesn't help at all that the commit message contains no reasoning as to why it's needed. Normally to list certificates, only the database would be needed because the certificate lifecycle is controlled using the API. SO why is all this filesystem access needed in this case?
I wanted to do this when we introduced cert_distrib, but never had the time to do it. Seeing that it needs to be modified anyway, I'd apreciate if certificates is only used for local operatons, and cert_distrib to distribute certificates. |
The data retrieved from database is the primary source of certificates. When doing
This is done duplicatedly in both certificates.ml and cert_distrib.ml but with different approaches:
Now I understand they need to be consolidated as cert_distrib only.
I didn't look into cert_distrib thoroughly before. Now I can eliminate the duplications. |
b2124dc to
ab810b7
Compare
|
Pushed the changes based on @psafont's comments. Not tested yet but I pushed them as I would like to see if it's on the right track. Will test tomorrrow if it is. |
psafont
left a comment
There was a problem hiding this comment.
Thanks for making the changes, I think there's all the trusted certificates should be exchanged on pool join, currently they arent. Other than that it looks good
ocaml/xapi/cert_distrib.ml
Outdated
| (diff |> StringSet.elements |> String.concat "; ") ; | ||
| not in_sync_with_remote | ||
| in | ||
| let ( let*? ) l f = List.map f l |> List.exists Fun.id in |
There was a problem hiding this comment.
Isn't this simply let ( let*? ) l f = List.exists f l in ?
I don't see a need to evaluate all the list in the users: they don't have visible side-effects and it seems safe to short-circuit.
| Worker.local_write_cert_fs ~__context HostPoolCert Merge pool_certs ; | ||
| Worker.local_regen_bundle ~__context | ||
|
|
||
| let collect_ca_certs ~__context ~names = |
There was a problem hiding this comment.
I think this function is missing the exchange of CRL, RootCert and PinnedCert. This will make sure all the certificates are copied on pool join. Maybe it's worth changing the function to collect_trusted_certificates, and similar with the following function.
Sure. Will append some commits for these missing pieces with the current approach. Thanks. |
cbeab03 to
fff055f
Compare
|
No. The API can work. But the SDK is complaining it. Probably due to the Not too bad. |
fff055f to
8cb0d7c
Compare
|
@minglumlu Could you squash the fixups please? |
| |> List.map WireProtocol.pair_of_certificate_file | ||
|
|
||
| (* This function is called on the pool that is incorporating a new host *) | ||
| let exchange_ca_certificates_with_joiner ~__context ~import ~export = |
There was a problem hiding this comment.
I'm surprised the exchange of trusted certificates and CRLs between the joiner and the pool is not done here, and instead is done in xapi_pool. Are you sure it's working correctly? I would have renamed this function (exchange_ca_certificates_with_joiner) and import_joining_pool_ca_certificates to add the exchange of trusted certificates and CRLs
There was a problem hiding this comment.
I didn't merge these 2 (actually 3) functions as the type of import argument of their corresponding API functions are quite different. I would have to merge them as well if using a single Cert_distrib.exchange_ca_certificates_with_joiner. And I found the logic of Pool.exchange_trusted_certificates_on_join and Pool.exchange_crls_on_join seems very simple and different with Pool.exchange_ca_certificates_on_join also. For example, for merging logic,
- legacy ca: filename is the primary info used for merging; but database is still needed to update;
- trusted: purpose and fingerprint are the primary info used for merging;
- crl: filename is the primary info used for merging, but no database at all.
So I didn't change the existing Cert_distrib.exchange_ca_certificates_with_joiner.
There was a problem hiding this comment.
But let me have a try to see how it will look like ...
There was a problem hiding this comment.
Sorry, Pau, I attempted to merge these functions, but the differences among them are too extensive to get them merged effectively.
The current work flow is like below, so it works.
[host] [pool]
exchange_trusted_certificates
-> Client.Pool.exchange_trusted_certificates_on_join --> exchange_trusted_certificates_on_join
-> install_trusted_certificate
-> Certificates.host_install
-> Cert_distrib.copy_certs_to_all
-> Cert_distrib.collect_trusted_certs
<--
-> install_trusted_certificate
There was a problem hiding this comment.
Ok, can you explain this modification in the design doc? This changes the previous design regarding certificate exchanges (https://xapi-project.github.io/new-docs/design/pool-certificates/#u2-pool-join)
I need to check what's wrong with the mermaid thing, it used to work :/
Signed-off-by: Ming Lu <ming.lu@cloud.com>
Signed-off-by: Ming Lu <ming.lu@cloud.com>
Signed-off-by: Ming Lu <ming.lu@cloud.com>
Signed-off-by: Ming Lu <ming.lu@cloud.com>
Signed-off-by: Ming Lu <ming.lu@cloud.com>
Signed-off-by: Ming Lu <ming.lu@cloud.com>
Signed-off-by: Ming Lu <ming.lu@cloud.com>
Signed-off-by: Ming Lu <ming.lu@cloud.com>
Signed-off-by: Ming Lu <ming.lu@cloud.com>
Signed-off-by: Ming Lu <ming.lu@cloud.com>
Signed-off-by: Ming Lu <ming.lu@cloud.com>
Signed-off-by: Ming Lu <ming.lu@cloud.com>
Signed-off-by: Ming Lu <ming.lu@cloud.com>
Signed-off-by: Ming Lu <ming.lu@cloud.com>
Signed-off-by: Ming Lu <ming.lu@cloud.com>
Signed-off-by: Ming Lu <ming.lu@cloud.com>
450c76f to
a6e390e
Compare
psafont
left a comment
There was a problem hiding this comment.
This is great thank, only one comment that should be easy to resolve
| design_doc: true | ||
| revision: 2 | ||
| status: released (22.6.0) | ||
| status: draft (update based on release 22.6.0 revision 2) |
There was a problem hiding this comment.
I'd rather indicate when this was changed:
status: released (22.6.0), modified (26.2.0)
Or if we want to follow Rob's suggestion, and the one which I prefer, keep this as it is because it's a historic document; add the new diagram and other changes to trusted-certificates.md, and state here that another suggestion updates it:
status: released (22.6.0), modified by [trusted-certificates](trusted-certificates.md)
There was a problem hiding this comment.
Thanks. I updated in the new design trusted-certificates.md and placed a reference in the base design pool-certificates.md.
Signed-off-by: Ming Lu <ming.lu@cloud.com>
a11e52f to
36890b6
Compare
|
the failing check can be ignored, it happens because ocaml 5.3.0 was removed from xs-opam for 5.4.0 |
The design is https://github.com/xapi-project/xen-api/blob/master/doc/content/design/trusted-certificates.md
(It's a little bit out-of-date. Will update it later. And remaining changes will be raised in following PRs)
Manual tests are done with a new purpose
testing(change is not included in this PR) like:And
pool.joinandpool.ejectfor exchanging trusted certificates and CRLs.The certificate stores under
/etc/trusted-certs/on coordinator and supporter hosts' dom0 filesystem.