History | View | Annotate | Download (105.7 kB)
Merge branch 'devel-2.4'
RPC/Backend: Make UploadFile uid and gid agnostic
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Implement grow dry-run at RPC level
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Implement dryrun mode for BlockDev.Grow()
This is always called with False from backend for now.
Add 2 new variables to the OS scripts environment
Add INSTANCE_PRIMARY_NODE and INSTANCE_SECONDARY_NODES. These newvalues are useful for OS scripts that needs to know the nodes wherethe instance lives.. or has lived.
Signed-off-by: Iustin Pop <iustin@google.com>...
Cluster verify: check for missing bridges
Currently cluster verify doesn't check for bridge information; theonly checks are done at instance create and failover/migratetime. This means a cluster that seems healthy will fail creation jobs.
This patch implements a simple verification that all nodes (in the...
Shared storage instance migration
Modify LUMigrateInstance and TLMigrateInstance to allow instance migrations forinstances with DTS_EXT_MIRROR disk templates.
Migrations of shared storage instances require either a target node, or aniallocator to determine the target node. If none is given, the cluster default...
Add bdev_sizes RPC call
The bdev_sizes multi-node RPC call returns the sizes of the requestedblock devices on the desired nodes. Its intended use is to verify theexistence of a block device on a given node for shared block storagesupport.
Block device paths are expected to lie under constants.BLOCKDEV_DIR...
Core shared file storage support
This patch introduces core file storage support, consisting of the following:
A configure-time switch for enabling/disabling shared file storagesupport and controlling the shared file storage location:--with-shared-file-storage-dir=. Shared file storage configuration is then...
Merge branch 'devel-2.3' into devel-2.4
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
backend: Disable compression in export info file
The new import/export infrastructure in Ganeti 2.2 and up handlescompression differently. It no longer writes compressed files to thedestination. Unfortunately changing this behaviour would be non-trivial,...
Re-create instance disk symlinks on activate
This patch implements recreation of instance disk symlinks when theactivate-disks operation is run. Until now, it was not possible tore-create these symlinks without stopping and starting or migrating aninstance as the RPC call where this is done was in instance startup...
cluster verify: add hvparams verification
Currently, the validity of the hypervisor parameters is only checkedat init/modification time, and not in the cluster verify. This is bad,as it can lead to inconsistent state that is only detected when thenext modification (which can be unrelated) is made, leading to...
Verify disks: increase parallelism and other fixes
The recent work on multi-VG support has converted LUClusterVerifyDisksinto doing serialised calls to each node, as each node can havedifferent VGs. This is suboptimal, especially for big clusters, where...
Improve import/export timeout settings
With this patch, the exporting node will retry to connect a few times.The receiving node will make use of the master's increased timeout (seeprevious patch).
utils: Move I/O-related code into separate file
Fix a typo in backend.py
Sorry I thought I did run commit-check but must not have paid attentionto its output. There was a typo in the docstring. This patch fixes this.
Add backend method for pause/resume sync of devices
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Adding additional VerifyNode checks to backend
This adds checks for out of band support. The helpers have to exist andthey have to be executable.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Allow customisation of the disk index separator
As per issue 124, some Xen versions (or packaging) don't deal nicelywith the colon being part of a disk name. Therefore we add aconfigure-time option for customising this.
Note: setting the separator to interesting values like / is not...
Make snapshots multi-vg aware
Currently, the Snapshot() function of LogicalVolume returns only thelogical volume path, with the assumption that we only have one VG. Butwith the recent changes, it makes more sense to return the full data (vgand lv) from it, so as to not require computing it in the master....
Make rpc.call_lv_list() VG-aware
Changes to backend.GetVolumeList():- now accepts a list of VGs instead of one VG- returns LV names in the form "vg_name/lv_name"
Corresponding changes are done in: VerifyDisks, VerifyNode,LUCreateInstance (for both disk creation and adoption cases)...
Adding backend functionality to call oob helper
backend: Add support for IPv6 in import/export
Move compilation of some regexes to init time
I have found a few regexes which are static and thus can be moved toload time, rather than run time, creation.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
RPC call_node_info: change protocol
Currently, the call_node_info RPC does always check both the VG freespace and the hypervisor information. However, in ⅔ of the uses, we onlycare about one or the other. Therefore, we change it so that if any ofthe passed parameters is None, we don't perform the respective check. We...
Fix disk checks in “gnt-cluster verify”
Tests have shown that the changes in commit b8d26c6e5 don't work aswanted. If any disk wasn't found on the node, all disks located on thesame node would show as faulty. The cause was incorrect exceptionhandling on the node....
Add support for vm_capable in cluster verify
The method to make vm_capable integrate easily into cluster verify is as follows:
- we add a new NV_VMNODES that represents nonvm-capable nodes the LU populates this list (it's expected that non-vm_capable nodes...
Second iteration over backend.BlockdevWipe
This patch now uses dd entirely to wipe the disk, make itmuch easier to wipe in blocks so we can give interactive feedbackabout the status.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>...
Simplify and extend the instance OS env
Some parameters were missing (uuid, c/mtime). We simplify the exportmethod; unfortunately we cannot simply iterate over slots since themapping is not 1:1.
backend.Upload: switch to utils.SafeWriteFile
This allows serialization of updates to a given file, with respect toother cooperating writers.
Adding backend method to wipe a block device
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'devel-2.2'
Abstract OS name/variant functions
Currently, the computation of the 'pure' name or the variant ishardcoded and spread around the functions that need it. This is notnice, and in the future we'd spread it even more with more usage ofvariants/pure os names....
Migrate call from backend._GetVGInfo to bdev.LogicalVolume.GetVGInfo
This patch removes duplicate code found in backend which also needs toget VG infos. To make it simpler we moved to bdev.LogicalVolume.GetVGInfo.
Signed-off-by: René Nussbaumer <rn@google.com>...
Fix OS_VARIANT variable setting
This was introduced in efaa9b06d1e1e6d1678d0edd75b1ba37cf0de3d9.
in OSCoreEnv: inst_os.name is pure operating system name (without variant) as variant is stripped in OSFromDisk(). So we always get variant = inst_os.supported_variants[0] (first...
Move job queue to new ganeti.runtime
Log warning instead of raising OpExecError for ndisc6
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add RPC calls to update /etc/hosts
Use family in backend.StartMaster
This patches changes the StartMaster method to consult the clusterprimary ip version when deciding whether to use arping or ndisc6 afteractivating the master ip.
Signed-off-by: Manuel Franceschini <livewire@google.com>...
Removing all ssh setup code from the core
Support IPv6 cluster init
Support for resolving hostnames to IPv6 addresses
This patch enables IPv6 name resolution by using socket.getaddrinfoinstead of socket.gethostbyname_ex.
It renames the HostInfo class to Hostname and unifies its use throughoutthe code. This is achieved by using static calls where no object is...
Introduce new IPAddress classes
This patch unifies the netutils functions dealing with IP addresses tothree classes:- IPAddress: Common IP address functionality- IPv4Address: IPv4 specific functionality- IPv6address: IPv6-specific functionality
Furthermore it adds methods to check whether an address is a loopback...
Instance migration: remove error on missing link
Since we don't support upgrades from 1.2.4 without restarting theinstance, the 'not restarted since 1.2.5' check/error iswrong/misleading.
Since the live migration works anyway without the links (it recreates...
Change the meaning of call_node_start_master
Currently, backend.StartMaster (the function behind this RPC call) willactivate the master IP and then, if the start_daemons parameter is true,it will also activate the master role.
While this works, it has two issues:...
Introduce lib/netutils.py
This patch moves network utility functions to a dedicated module.
Add drbd_helper rpc call
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
VerifyNode: add usermode helper reply
Rename some constants to facilitate IPv6 support
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Two more fixes for OS params and opcode defaults
If the OS is not using API v20, the parameter verification should beentirely skipped.
The second change is a simple typo.
Fix breakage due to OS parameters
I was using wrong python installation path (thanks Guido!), so I was notactually testing the new backend.py module. Two immediate things bugsare fixed, and after these burnin passes again…
Add OS verification support to cluster verify
For this, we needed to extend the NodeImage class with a few extravariables, and we do a trick in the node verification where we pick thefirst node that returned valid OS data as the reference node, and then...
Add OS parameters to cluster and instance objects
The patch also modifies the instance RPC calls to fill the osparameterscorrectly with the cluster defaults, and exports the OS parameters inthe instance/OS environment.
Add support for OS parameters during import/export
Nothing special here, just copy/adjust the beparams code.
LUDiagnoseOS: add more fields, cleanup
This patch exports all the way from backend a new field ‘api_version’which holds the list of support API versions, and exposes the (alreadycomputed) ‘parameters’ field.
The patch also reworks (again) the field calculation in its Exec()...
Add reading of OS parameters from disk
The patch also modifies the internal methods in LUDiagnoseOS and gnt-osto deal with the format change of call_os_diagnose.
Introduce an RPC call for OS parameters validation
While we only support the 'parameters' check today, the RPC call isgeneric enough that will be able to support other checks in the future.The backend function will both validate the parameters list (so as to...
ListVisibleFiles: do not sort output
Among all users, turns out just one may need the output to be sorted.All the others can cope without.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Split the core-OS and instance-specific env
Since we'll need to be able to generate the OS-specific environmentseparately from the instance one, we move it to a separate function. Wealso add a new OS_NAME env. var which is identical to the INSTANCE_OSone (which won't exist for OS-only environments)....
Merge branch 'devel-2.1' into master
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
Fix unsafe variant initializer in _TryOSFromDisk
In case an OS has inconsistent declarations, we might get into a casewhere one node reports a valid variants list (with OS API >=15), andanother node has OS API < 15, in which case its supported_variants gets...
backend: Add support for import/export magic
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Handle ESRCH when sending signals
Upon sending signals, ESRCH can be reported when the target nolonger exists.
Remove the job queue drain rpc call
This call was introduced but never used. In two years.Since it's just creating/removing a file it can also be in simpler ways,without a special rpc call, if/when we need it again. In the meantime,let's give it to history....
backend: Enable export size prediction
Distribute cluster domain secret
The cluster domain secret file was not distributed to other nodes.
Allow control of import/export compression method
For exports to/imports from the same machine, compression willnot be used anymore.
Put common import/export daemon options into object
The X509 key name and CA are passed from cmdlib all the way tothe backend import/export daemon. With the addition of an optionto choose the compression method, another parameter would haveto be passed all the way. By moving these options to a separate...
Merge branch 'devel-2.1'
Add checks for master IP in cluster verify
This also updates a comment in the unittest for utils.py. We unittestthe new function for two things: correct reporting on real case (forlocalhost), and correct reporting with a mocked-out TcpPing that returns...
Conflicts: daemons/ganeti-noded lib/daemon.py lib/rapi/baserlib.py lib/rapi/rlib2.py lib/utils.py
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix some pylint warnings
Disable warnings for:- except Exception,- use of __errno_location,- redeclaration of handleError()
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Lock PowercycleNode child in memory
Fix import/export
63bcea2a5 added file checks for import/export, but unfortunately theywere broken.
backend: remove a couple of useless mkdir calls
Those directories must exist for the node daemon to run (it's in thenode daemon's list of ensured directories) and those functions are onlycalled by the node daemon, so there's no point in those checks+mkdir...
backend: Check paths and always write CA file for import/export daemon
Once the import/export daemon uses separate users, the node daemon file (whichis used for intra-cluster transfers) might not be readable anymore. Alwayswriting it to a daemon-specific file will make this easier....
Remove two unused RPC functions
Both of these functions, “snapshot_export” and “instance_os_import”,have been replaced by the instance import/export daemon.
Add RPC call to send SIGTERM to import/export daemon
This will be used to stop the daemon without doing complete cleanup (yet).
utils: Add function to read locked PID file
This is useful in combination with utils.StartDaemon and will be used forreading the import/export daemon's PID file.
Conflicts: doc/security.rst trivial lib/cli.py trivial
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add CleanupInstance hypervisor call
Currently some hypervisors (namely kvm) need to do some cleanup aftermaking sure an instance is stopped. With the moving of the retry cyclein backend those cleanups were never done. In order to solve this we adda new optional hypervisor function, CleanupInstance, which gets called...
Add RPC calls to import and export instance data
These RPC calls can be used to start, monitor and stop the instance dataimport/export daemon.
backend: Consolidate code opening real block device
Merge remote branch 'devel-2.1'
Export more instance parameters in instance export
Currently the backend parameters are not exported automatically, butonly a few directly in the '[instance]' section. Hypervisor type andhypervisor parameters are not exported at all.
This patch creates two separate sections for the be and hv parameters,...
Export the nicparams too during instance export
The patch tries to export all params (based on the dict defined inconstants), using None for missing keys.
Fix backend.VerifyNode behaviour for VG problems
In case LVM is broken, backend.GetVolumeList will raise an RPC exception(as expected since it's a function exposed over RPC). Therefore we mustbe prepared to catch any such exceptions, so that we don't fail the...
Add RPC calls to create and remove X509 certificates
Certificates and keys generated using these functions will be used forinter-cluster instance moves. As per design, the private key should neverleave the node.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
backend: Two small style fixes
- Pass keyword parameter as such- Replace “not x == y” with “x != y”
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Rename SSL_CERT_FILE to NODED_CERT_FILE
To be consistent with RAPI_CERT_FILE, the rather generic named“SSL_CERT_FILE” constant is renamed to “NODED_CERT_FILE”. The actual filename is not changed.
Rightname confd's HMAC key
Currently, the ganeti-confd's HMAC key is called “cluster HMAC key” orsimply “HMAC key” everywhere. With the implementation of inter-clusterinstance moves, another HMAC key will be introduced for signing criticaldata. They can not be the same, so this patch clarifies the purpose of the...
utils.CreateBackup: Use human-readable instead of seconds since Epoch
Seconds since the Epoch are not easily readable by a human. Using aformatted timestamp makes it easier (e.g.“….backup-2010-03-12_14_02_43.…”). This patch also makes OS logfiles usethis formatted timestamp....
Improve cluster verify with hypervisor errors
In case the hypervisor has issues on one node, currentlybackend.VerifyNode will exit via an exception (two exit paths possible,one via HypervisorError from hypervisor.Verify(), and one via RPCFailfrom GetInstanceList). This is bad as it invalidates all other checks of...
Fix node volumes list for stripped volumes
Currently backend.NodeVolumes() drops everything except the first PV,thus we get a truncated result. The patch is not the nicest, as Pythondoesn't have a simple `concat' function, so I had to change the listcomprehension to an explicit loop....