Revision f0476905
b/doc/design-2.2.rst | ||
---|---|---|
234 | 234 |
only be valid for the time necessary to move the instance. |
235 | 235 |
|
236 | 236 |
For additional protection of the instance data, the two clusters can |
237 |
verify the certificates exchanged via the third party by signing them |
|
238 |
using HMAC with a key shared among the involved clusters. If the third |
|
239 |
party does not know this secret, it can't forge the certificates and |
|
240 |
redirect the data. Unless disabled by a new cluster parameter, verifying |
|
241 |
the HMAC must be mandatory. The HMAC will be prepended to the |
|
242 |
certificate and only covers the certificate (from ``-----BEGIN |
|
243 |
CERTIFICATE-----`` to ``-----END CERTIFICATE-----``). |
|
237 |
verify the certificates and destination information exchanged via the |
|
238 |
third party by checking an HMAC signature using a key shared among the |
|
239 |
involved clusters. By default this secret key will be a random string |
|
240 |
unique to the cluster, generated by running SHA1 over 20 bytes read from |
|
241 |
``/dev/urandom`` and the administrator must synchronize the secrets |
|
242 |
between clusters before instances can be moved. If the third party does |
|
243 |
not know the secret, it can't forge the certificates or redirect the |
|
244 |
data. Unless disabled by a new cluster parameter, verifying the HMAC |
|
245 |
signatures must be mandatory. The HMAC signature for X509 certificates |
|
246 |
will be prepended to the certificate similar to an RFC822 header and |
|
247 |
only covers the certificate (from ``-----BEGIN CERTIFICATE-----`` to |
|
248 |
``-----END CERTIFICATE-----``). The header name will be |
|
249 |
``X-Ganeti-Signature``. |
|
244 | 250 |
|
245 | 251 |
On the web, the destination cluster would be equivalent to an HTTPS |
246 | 252 |
server requiring verifiable client certificates. The browser would be |
... | ... | |
270 | 276 |
|
271 | 277 |
#. Third party tells source cluster to shut down instance, asks for the |
272 | 278 |
instance specification and for the public part of an encryption key |
279 |
|
|
280 |
- Instance information can already be retrieved using an existing API |
|
281 |
(``OpQueryInstanceData``). |
|
282 |
- An RSA encryption key and a corresponding self-signed X509 |
|
283 |
certificate is generated using the "openssl" command. This key will |
|
284 |
be used to encrypt the data sent to the destination cluster. |
|
285 |
|
|
286 |
- Private keys never leave the cluster. |
|
287 |
- The public part (the X509 certificate) is signed using HMAC with |
|
288 |
salting and a secret shared between Ganeti clusters. |
|
289 |
|
|
273 | 290 |
#. Third party tells destination cluster to create an instance with the |
274 | 291 |
same specifications as on source cluster and to prepare for an |
275 | 292 |
instance move with the key received from the source cluster and |
276 | 293 |
receives the public part of the destination's encryption key |
294 |
|
|
295 |
- The current API to create instances (``OpCreateInstance``) will be |
|
296 |
extended to support an import from a remote cluster. |
|
297 |
- A valid, unexpired X509 certificate signed with the destination |
|
298 |
cluster's secret will be required. By verifying the signature, we |
|
299 |
know the third party didn't modify the certificate. |
|
300 |
|
|
301 |
- The private keys never leave their cluster, hence the third party |
|
302 |
can not decrypt or intercept the instance's data by modifying the |
|
303 |
IP address or port sent by the destination cluster. |
|
304 |
|
|
305 |
- The destination cluster generates another key and certificate, |
|
306 |
signs and sends it to the third party, who will have to pass it to |
|
307 |
the API for exporting an instance (``OpExportInstance``). This |
|
308 |
certificate is used to ensure we're sending the disk data to the |
|
309 |
correct destination cluster. |
|
310 |
- Once a disk can be imported, the API sends the destination |
|
311 |
information (IP address and TCP port) together with an HMAC |
|
312 |
signature to the third party. |
|
313 |
|
|
277 | 314 |
#. Third party hands public part of the destination's encryption key |
278 | 315 |
together with all necessary information to source cluster and tells |
279 | 316 |
it to start the move |
317 |
|
|
318 |
- The existing API for exporting instances (``OpExportInstance``) |
|
319 |
will be extended to export instances to remote clusters. |
|
320 |
|
|
280 | 321 |
#. Source cluster connects to destination cluster for each disk and |
281 | 322 |
transfers its data using the instance OS definition's export and |
282 | 323 |
import scripts |
324 |
|
|
325 |
- Before starting, the source cluster must verify the HMAC signature |
|
326 |
of the certificate and destination information (IP address and TCP |
|
327 |
port). |
|
328 |
- When connecting to the remote machine, strong certificate checks |
|
329 |
must be employed. |
|
330 |
|
|
283 | 331 |
#. Due to the asynchronous nature of the whole process, the destination |
284 | 332 |
cluster checks whether all disks have been transferred every time |
285 |
after transfering a single disk; if so, it destroys the encryption |
|
333 |
after transferring a single disk; if so, it destroys the encryption
|
|
286 | 334 |
key |
287 | 335 |
#. After sending all disks, the source cluster destroys its key |
288 | 336 |
#. Destination cluster runs OS definition's rename script to adjust |
... | ... | |
291 | 339 |
by the third party |
292 | 340 |
#. Source cluster removes the instance if requested |
293 | 341 |
|
342 |
Instance move in pseudo code |
|
343 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
|
344 |
|
|
345 |
.. highlight:: python |
|
346 |
|
|
347 |
The following pseudo code describes a script moving instances between |
|
348 |
clusters and what happens on both clusters. |
|
349 |
|
|
350 |
#. Script is started, gets the instance name and destination cluster:: |
|
351 |
|
|
352 |
(instance_name, dest_cluster_name) = sys.argv[1:] |
|
353 |
|
|
354 |
# Get destination cluster object |
|
355 |
dest_cluster = db.FindCluster(dest_cluster_name) |
|
356 |
|
|
357 |
# Use database to find source cluster |
|
358 |
src_cluster = db.FindClusterByInstance(instance_name) |
|
359 |
|
|
360 |
#. Script tells source cluster to stop instance:: |
|
361 |
|
|
362 |
# Stop instance |
|
363 |
src_cluster.StopInstance(instance_name) |
|
364 |
|
|
365 |
# Get instance specification (memory, disk, etc.) |
|
366 |
inst_spec = src_cluster.GetInstanceInfo(instance_name) |
|
367 |
|
|
368 |
(src_key_name, src_cert) = src_cluster.CreateX509Certificate() |
|
369 |
|
|
370 |
#. ``CreateX509Certificate`` on source cluster:: |
|
371 |
|
|
372 |
key_file = mkstemp() |
|
373 |
cert_file = "%s.cert" % key_file |
|
374 |
RunCmd(["/usr/bin/openssl", "req", "-new", |
|
375 |
"-newkey", "rsa:1024", "-days", "1", |
|
376 |
"-nodes", "-x509", "-batch", |
|
377 |
"-keyout", key_file, "-out", cert_file]) |
|
378 |
|
|
379 |
plain_cert = utils.ReadFile(cert_file) |
|
380 |
|
|
381 |
# HMAC sign using secret key, this adds a "X-Ganeti-Signature" |
|
382 |
# header to the beginning of the certificate |
|
383 |
signed_cert = utils.SignX509Certificate(plain_cert, |
|
384 |
utils.ReadFile(constants.X509_SIGNKEY_FILE)) |
|
385 |
|
|
386 |
# The certificate now looks like the following: |
|
387 |
# |
|
388 |
# X-Ganeti-Signature: $1234$28676f0516c6ab68062b[…] |
|
389 |
# -----BEGIN CERTIFICATE----- |
|
390 |
# MIICsDCCAhmgAwIBAgI[…] |
|
391 |
# -----END CERTIFICATE----- |
|
392 |
|
|
393 |
# Return name of key file and signed certificate in PEM format |
|
394 |
return (os.path.basename(key_file), signed_cert) |
|
395 |
|
|
396 |
#. Script creates instance on destination cluster and waits for move to |
|
397 |
finish:: |
|
398 |
|
|
399 |
dest_cluster.CreateInstance(mode=constants.REMOTE_IMPORT, |
|
400 |
spec=inst_spec, |
|
401 |
source_cert=src_cert) |
|
402 |
|
|
403 |
# Wait until destination cluster gives us its certificate |
|
404 |
dest_cert = None |
|
405 |
disk_info = [] |
|
406 |
while not (dest_cert and len(disk_info) < len(inst_spec.disks)): |
|
407 |
tmp = dest_cluster.WaitOutput() |
|
408 |
if tmp is Certificate: |
|
409 |
dest_cert = tmp |
|
410 |
elif tmp is DiskInfo: |
|
411 |
# DiskInfo contains destination address and port |
|
412 |
disk_info[tmp.index] = tmp |
|
413 |
|
|
414 |
# Tell source cluster to export disks |
|
415 |
for disk in disk_info: |
|
416 |
src_cluster.ExportDisk(instance_name, disk=disk, |
|
417 |
key_name=src_key_name, |
|
418 |
dest_cert=dest_cert) |
|
419 |
|
|
420 |
print ("Instance %s sucessfully moved to %s" % |
|
421 |
(instance_name, dest_cluster.name)) |
|
422 |
|
|
423 |
#. ``CreateInstance`` on destination cluster:: |
|
424 |
|
|
425 |
# … |
|
426 |
|
|
427 |
if mode == constants.REMOTE_IMPORT: |
|
428 |
# Make sure certificate was not modified since it was generated by |
|
429 |
# source cluster (which must use the same secret) |
|
430 |
if (not utils.VerifySignedX509Cert(source_cert, |
|
431 |
utils.ReadFile(constants.X509_SIGNKEY_FILE))): |
|
432 |
raise Error("Certificate not signed with this cluster's secret") |
|
433 |
|
|
434 |
if utils.CheckExpiredX509Cert(source_cert): |
|
435 |
raise Error("X509 certificate is expired") |
|
436 |
|
|
437 |
source_cert_file = utils.WriteTempFile(source_cert) |
|
438 |
|
|
439 |
# See above for X509 certificate generation and signing |
|
440 |
(key_name, signed_cert) = CreateSignedX509Certificate() |
|
441 |
|
|
442 |
SendToClient("x509-cert", signed_cert) |
|
443 |
|
|
444 |
for disk in instance.disks: |
|
445 |
# Start socat |
|
446 |
RunCmd(("socat" |
|
447 |
" OPENSSL-LISTEN:%s,…,key=%s,cert=%s,cafile=%s,verify=1" |
|
448 |
" stdout > /dev/disk…") % |
|
449 |
port, GetRsaKeyPath(key_name, private=True), |
|
450 |
GetRsaKeyPath(key_name, private=False), src_cert_file) |
|
451 |
SendToClient("send-disk-to", disk, ip_address, port) |
|
452 |
|
|
453 |
DestroyX509Cert(key_name) |
|
454 |
|
|
455 |
RunRenameScript(instance_name) |
|
456 |
|
|
457 |
#. ``ExportDisk`` on source cluster:: |
|
458 |
|
|
459 |
# Make sure certificate was not modified since it was generated by |
|
460 |
# destination cluster (which must use the same secret) |
|
461 |
if (not utils.VerifySignedX509Cert(cert_pem, |
|
462 |
utils.ReadFile(constants.X509_SIGNKEY_FILE))): |
|
463 |
raise Error("Certificate not signed with this cluster's secret") |
|
464 |
|
|
465 |
if utils.CheckExpiredX509Cert(cert_pem): |
|
466 |
raise Error("X509 certificate is expired") |
|
467 |
|
|
468 |
dest_cert_file = utils.WriteTempFile(cert_pem) |
|
469 |
|
|
470 |
# Start socat |
|
471 |
RunCmd(("socat stdin" |
|
472 |
" OPENSSL:%s:%s,…,key=%s,cert=%s,cafile=%s,verify=1" |
|
473 |
" < /dev/disk…") % |
|
474 |
disk.host, disk.port, |
|
475 |
GetRsaKeyPath(key_name, private=True), |
|
476 |
GetRsaKeyPath(key_name, private=False), dest_cert_file) |
|
477 |
|
|
478 |
if instance.all_disks_done: |
|
479 |
DestroyX509Cert(key_name) |
|
480 |
|
|
481 |
.. highlight:: text |
|
482 |
|
|
294 | 483 |
Miscellaneous notes |
295 | 484 |
^^^^^^^^^^^^^^^^^^^ |
296 | 485 |
|
Also available in: Unified diff