History | View | Annotate | Download (138.3 kB)
Merge branch 'stable-2.8' into stable-2.9
Conflicts: lib/cmdlib/instance.py: manually apply 0973f9ed on...
Export and import Disk/NIC name
Name of Disk/NIC were not exported during backup until now.Use the exported info during gnt-backup import.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Conflicts: NEWS: trivial configure.ac: ignore version bump on stable-2.8 lib/bdev.py: manually apply the part of commit...
DRBD: ensure peers are UpToDate for dual-primary
DrbdAttachNet supports both, normal primary/secondary node operation, and(during live migration) dual-primary operation. When resources are newlyattached, we poll until we find all of them in connected or syncing operation....
Use secondary IP when moving instances
All data traffic usually goes over the secondary network, butgnt-instance move didn't. This patch corrects this problem by using thetarget nodes secondary IP as move target.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Replace LD_* constants with DT_* constants
LD_* constants are basically like DT_* constants, exceptfor that both file and shared file were mapped to file.In order to not having to maintain three slightly differentsets of disk-related constants (DT, LD and ST), we merge...
Lint improvements to regexps
Fix some regular expressions so that they pass lint checks with newer versionsof pylint.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
gnt-cluster verify: consider shared file storage
This patch enhances 'gnt-cluster verify' in a way that itnow validates the acceptance and existance of the sharedstorage directory.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Verify file storage path
This patch adds two verification steps to 'gnt-clusterverify':- The configured file storage directory is checked against the allowed file storage directories file.- We check whether the configured file storage directory is existing and writable on each node....
Prepare verification code for new file path verification
This patch prepares the verification code for addinga new verification step for the file storage paths:- It moves a couple of file storage helper functions from bdev to filestorage (since they make more sense there...
backend: remove ENABLE_FILE_STORAGE
This patch removes the usage of the ENABLE_FILE_STORAGEconstant in the backend code. To avoid having to passit through various RPC calls, we instead move the checkto cmdlib.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Factor out check for storage params
This patch factors out the sanity checks for storageparameters in its own function. Unit tests included.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Tighten sanity checks for '_GetLvmVgSpaceInfo'
This patch tightens the sanity checks for the input of'_GetLvmVgSpaceInfo' and provides unit tests for it.
More sanity checks for spindle space reporting
This patch adds a wrapper around the space reportingfor spindles which includes more sanity checks forthe input. Unit tests provided.
Unit tests for _GetVgSpindlesInfo
This patch provides unit tests for _GetVgSpindlesInfo.In order to mock the used function for the 'bdev' moduleI made the low-level storage function an optional parameterof _GetVgSpindlesInfo. I also found out that mocked...
Unit tests for _GetVgInfo
This patch provides unit tests for the backend's_GetVgInfo function. In order to mock the underlyingbackend function, it was necessary to make it anoptional parameter of the function.
Fix propagation of storage parameters to/from backend
This patch fixes two problems with the storage reportingthat showed up in the QA for exclusive storage:
- The processing of storage space information for instance operations wrongly assumed that the volume group's...
Check DRBD status on verify-disks
Enhance `gnt-cluster verify-disks` such that it checks the DRBD statusof all disks. If the status of at least one disk of an instance iseither StandAlone or Diskless, the instance disks are activatedautomatically.
Add general storage parameters to node info call
As described in the design doc about storage types,we plan to generalize the RPC call "node info" wrt tostorage types. This patch extends the call by acceptinga list of storage units, that is not only identified...
Limit the PVs used for disk growth with exclusive storage
The exclusive storage flag is passed along in the backend functions fordisk growth. LVM volumes handles the flag by reusing the same PVs, as disksare constrained by the number of their spindles....
Merge branch 'stable-2.8' into 'master'
Refactor NodeInfo RPC regarding storage reporting
The NodeInfo RPC call is refactored to handle now more thanjust storage reporting for volume groups.
Since NodeInfo now returns storage space information notnecessarily for volume groups, but also for other storage...
Integrate free space reporting for file storage
This patch integrates the free space reporting for filestorage. So far, it is only called, when the defaultdisk template is a file-based disk template ("file" or"sharedfile").
Add storage type to NodeInfo result
So far, the storage information returned from RPC callNodeInfo contained only information about volume groups.In order to extend the storage reporting to other storagetype, we include another field "type" in the result of...
Index nodes by their UUID
No longer index nodes by their name but by their UUID in the clusterconfig. This change changes large parts of the code, as the followingadjustments were necessary: * Change the index key to UUID in the configuration and the ConfigWriter, including all methods....
Add cluster name to instance migration RPC
This fixes a 'FIXME' in the xen hypervisor. The instancemigration requires the cluster's name. This should comefrom the RPC call, but so far was hackishly taken fromssconf. Ssconf is intended to be used by scripts only and...
Remove XEN_CMD from constants, adjust to PowercycleNode
This patch extend the RPC to powercycle a node by ahvparams parameter. This parameter is used by thexen hypervisor to determine which xen command to use.The respective LU is adjusted as well.
Since this was the last operation depending on the...
Use hvparams in GetAllInstancesInfo
This patch extends the GetAllInstancesInfo method of thehypervisors by a hvparams parameter. This change propagatesup to the RPC call all_instances_info, which also neededto be extended. All callers of this call were adjusted...
Use hvparams in GetInstanceInfo
This patch extends the GetInstanceInfo function of thehypervisors by an hvparams parameters. This leads toan extension of the instance_info RPC call and theadjustmend of all classes using this call.
backend.py: remove fallback implementation of GetNodeInfo
This removes the fallback implementation of the backend'sGetNodeInfo function to the old implementation. By now,all calling code is adjusted to the new hvparams parameterand thus, the old implementation should not be necessary...
backend.py: use hvparams in GetNodeInfo
This patch extends the GetNodeInfo function of the backendmodule by a hvparams parameter. For now, it contains afallback solution for when the parameter is None. Thisfallback will be removed in a later patch of this series....
Verify: node info and instance list
This patch enhances node verification by:- Factoring out the verification of the node info and the instance list to increase testability.- Make both of them use the hvparams parameter.
Verify: xen toolstack, hypervisor and hvparams
This patch extends the node verification by:- Adding a check for the xen toolstack when the hypervisor is verified.- Factoring out the hypervisor verification in a subfunction to increase testability.- Factoring out the hvparams verification in a...
backend.py: use instance's hvparams whenever possible
The backend module's function 'ListInstances' is calledwithin the module several times. In many cases, it ispossible to reuse the instance's hvparams for the'ListInstances' call and thus avoiding having to extend...
backend.py: GetInstanceListForHypervisor
In same cases, the call 'GetInstanceList' is only done forone hypervisor. In those cases, it is more convenient tonot provide the full collection of hypervisor parametersfor all hypervisors, but only the ones for this hypervisor....
backend.py: hvparams in 'GetInstanceList'
Propagating the use of hvparams further up the callinghierarchy, this patch makes the backend module use thehvparams in 'GetInstanceList'. Unit tests provided.
node_info RPC reports info about free spindles
node_info RPC can be used to get information about free/total spindles.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Export device UUIDs to hooks and OS scripts
Export UUIDs and names of instance NICs and disks to the environmentof OS scripts and instance related hooks.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>Signed-off-by: Klaus Aehlig <aehlig@google.com>...
New RPC to get size and spindles of disks
This RPC replaces the existing one that only returned disk size.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
RPC 'node_info': <storage_type,key> instead of vg_names
This replaces the field 'vg_names' in the RPC call of 'node info' by'storage_units'. A storage unit is a tuple <storage_type,key>and a generalization of a vg_name. The list of vg names is replaced by...
Rename dir 'block' to 'storage'
Renaming the 'block' directory to 'storage', because I plan toplace code there that is related to file storage and leavingit named 'block' would be misleading.
Split DRBD8Dev into DRBD8 and DRBD8Dev
All functionality specific to a single DRBD8 devide is now in DRBD8Dev,whereas functionality which is valid for the whole DRBD "installation" on a device is collected in DRBD8.
This makes it possible to remove a couple of FIXME's and clarifies the...
Rename DRBD8 to DRBD8Dev
Right now the DRBD8 class has multiple responsibilities: a) itreprensents a device which can be set up, grown, etc. and b) itrepresents the whole DRBD system on a node which has a usermode helper,which knows how to shut down all devices, etc. Therefore, the DRBD8Dev...
Make DRBD version queryable from noded
gnt-cluster verify should issue a warning if there are multiple DRBDversions present in a node group. In order to do so, the DRBD versionhas to be queryable from noded.
Merge branch 'stable-2.8' into master
Extract /proc/drbd parsing code into DRBD8Info
As the DRBD8 class got bigger due to the previous merge of BaseDRBD, nowparts of it are ripped out into DRBD8Info. This new class parses/proc/drbd and exposes the information in an easily accessible way. This...
Remove BaseDRBD
BaseDRBD was probably useful when DRBD 0.7 and DRBD 8 were supported.However, there is only one subclass of BaseDRBD remaining (DRBD8), andthe separation of responsibilities between those two classes was ratherrandomly chosen.
The unification into one class also eases the introduction of...
Merge branch 'stable-2.7' into stable-2.8
Conflicts: NEWS doc/rapi.rst
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Reason trail implementation for "start"
Opcode-specific implementation of the reason trail for the instancestartup operation.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Reason trail implementation for "shutdown"
Opcode-specific implementation of the reason trail for the instance shutdownoperation.
Extract DRBD-related classes into block/drbd.py
This is in preparation to introducing support for DRBD8.4.base.py had to be extracted as well in order to avoid cylic importsbetween bdev.py and drbd.py. It now contains the BlockDev class andutility functions needed by bdev.py and drbd.py....
Move lib/bdev.py to lib/block/bdev.py
That's in preparation of extracting DRBD related code from bdev.py. Asbdev.py is already rather long, new features will require to split itinto more manageable pieces. That's why it's moved in an own directory.
Reason trail implementation for instance reboot
This commits allows ganeti to correctly forward the reason trail informationregarding instance reboot.
Add function for storing the reason trail of an instance
We are interested in storing the reason trail for operations that changedthe state of instances. This commit adds a function to do this.
Also, the GetInstReasonFilename() function name is prepended with an...
Remove old "reason" implementation
Remove the useless parts of the old, partial, implementation of the support fortracking the reason of instances state change, before implementing the newreason trail support, as per the design document.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Move HooksMaster out of the mcpu module
We need to do this, so that backend.py doesn't need to import mcpu, andthus indirectly cmdlib. This reduces the size of the node daemon byabout half, which is very important as it is pinned in memory.
This solves Issue 419....
Merge branch 'devel-2.7'
Fix job queue directory permission problems
If split users are used, the queue directory could only be accessedby masterd, but also confd needs to be able to read it, e.g. when itis queried as part of "gnt-job list"
This commit fixes the permissions in such a way to allow proper access rights....
Status change reason support for Reboot
Add support to the Reboot command for specifying the reason for the laststatus change.
Some features are implemented as functions, even if used only once, becausethey will be used by the future patches introducing reason support for all...
Infrastructure for specifying instance status change reason
This patch introduces some infrastructural modifications that will be used bythe following commits to implement the support for specifying the reason forthe last status change of an instance....
Remove useless code in backend for network hooks
In backend NICs arrive with netinfo filled. If nic.network is not Nonenic.netinfo is not too. Thus all the info is derived from HooksDict()and nic.network must not be checked.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>...
Finish the remote→restricted commands rename
The documentation still points to /etc/ganeti/remote-commands,although the code is already using restricted-command. Update thedocumentation and a few docstrings accordingly.
Signed-off-by: Iustin Pop <iustin@google.com>...
OS environment: add network information
1) Move the hooks environment dict generator inside the object. Thisalso adds missing values such as network family and uuid.2) Use the same generator both for the os environment and for theinstance hooks.3) Update manpage and hooks documentation....
Add test for backend._GetBlockDevSymlinkPath
Add a unit test for the trivial “_GetBlockDevSymlinkPath” function inbackend (small changes in the function were required).
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
cluster-verify checks that PVs are not shared
When exclusive_storage is set, cluster-verify complains if PVs are sharedamong unrelated LVs.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Return master ip script output in case of failure
Currently only the exit code is returned, which is somewhat pointless:for failures, we already know it's ≠ 0, so we don't have enoughinformation (reported on the mailing list). By logging the output aswell (if any), the user will have more information available for...
Add the gnt-storage client
Add a new client called 'gnt-storage'.The client interacts with the ExtStorage interface, similarly tothe way gnt-os interacts with the OS interface.
For now, only two commands are supported: 'info' and 'diagnose'.
'diagnose' calculates the node status of each provider on each node,...
Add exclusive_storage parameter to node_info RPC
The flag is passed to the low-level functions that need it, but it's notyet used. Also, RPCs get just a default value for now.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Added class to contain information about a PV
This makes the code more readable and easier to upgrade.bdev.LogicalVolume.GetPVInfo and the code that depends on it have beenrefactored to use the new class.
utils.CheckVolumeGroupSize() has been moved to lib/utils/lvm.py, where more...
Add exclusive_storage parameter to blockdev_create RPC
The flag is passed to the low-level functions that need it, but it's notyet used.
Replace frozenset with compat.UniqueFrozenset
This is not a trivial s/frozenset/compat.UniqueFrozenset/, but ratheronly replaces “frozenset” where appropriate. Most of the places are“static” information that doesn't change after the module has beenloaded....
Add RPC for setting watcher pause
The watcher pause file should be set/unset on all nodes at once, notonly the master node. For that a new RPC is needed.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Export network in FinalizeExport()
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Rename leftovers from remote to restricted commands
As per Iustin Pop's suggestion in <20121115131730.GX824@google.com> on<ganeti-devel@googlegroups.com>.
backend: Rename RunRemoteCommand to RunRestrictedCmd
Rename constants and directory for restricted commands
Rename configure option for restricted commands
It is no longer “--enable-remote-commands”, but rather“--enable-restricted-commands”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Peripheral changes for networks
Specifically: * export a new environment variable for the gnt-os-interface * export a new environment variable for KVM specific scripts * ovf support for networks
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>...
backend: Implement remote commands
As per design document (doc/design-remote-commands.rst), a number ofrather strict tests is applied to any incoming request, a delay isinserted upon errors and returned error messages are very generic(unless it's the actual command that failed). There are unit tests for...
Add utility to check if file is executable
This replaces direct calls to “os.access” and“os.path.exists”/“os.path.isfile”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
backend: Switch to new file storage directory verification
The configuration is no longer used for verifying file storage paths.
Check allowed file storage paths during cluster-verify
Some paths, such as /bin or /usr/lib, should not be used for filestorage. This patch implements a check during cluster verification tofail in case such a path has been used.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
bdev: Add verification for file storage paths
An earlier version of this patch series verified all paths in cmdlib inthe master daemon. With this change all that verification code is movedto bdev to run inside the node daemon. The checks are much stricter...
Update blockdev's "info" at instance rename
Currently, we set "info" metadata on block devices at device creationtime, but we never update it, leading to stale data in case ofinstance renames. This would not be a big problem in case of regularrenames (assuming this is a rare operation), but importing instances...
Ignore empty/comment lines in OS variants file
Per a conversation on ganeti@googlegroups.com:
- gnt-os diagnose ; gnt-os list take in consideration blank lines in /etc/ganeti/instance-image/variants.list that could be confusing.
Let's fix this and also let's ignore comment lines....
LUClusterVerifyGroup: Localize virtual file paths
The check for file consistency didn't properly handle virtual pathsin case of a virtual cluster. This didn't cause any breakage as ina standard virtual cluster setup with only one node all files arevisible for every node....
Move constant for /etc/hosts to pathutils
Needed for coming patches.
backend: Use utils.IsBelowDir instead of local code
utils.IsBelowDir is actually tested and doesn't allow writes to“…/queue*”, like the old code here did.
Check fingerprint of file with allowed file storage paths
This makes differences show up in “gnt-cluster verify”.
backend: Check for shared storage also
If normal file storage was disabled but shared storage enabled,“_TransformFileStorageDir” would still throw an exception.
in “opcodes._CheckStorageType” there's also a check, but I wasn't quitesure what the correct way of handling it was, so I added a TODO comment....
Move a function from backend to ssconf
The “WriteSsconfFiles” function is used to write ssconf files. By movingit we can avoid importing backend into bootstrap. The latter is importedby CLI programs and backend doesn't have much to do with them.
Remove constant for disk wipe block size
It is dangerous to have this block size as a global constant as thatcould give the impression of it being easily changed. Doing so withoutfurther adjustments to how “dd” is called will lead to disks not beingwiped properly....
Stop hardcoding root user
Some parts of the code still use a hardcoded user name: root. This patchreplaces all with a constant specified at build time. The end goal is tomake it possible to run a Ganeti cluster without any special privileges(of course this will prevent some functionality from working)....
instance_info now returns vcpus
instance_info and all_instances_info calls had different per-instancereturn types, this commit fixes the mismatch.
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Implement virtual cluster support in Python code
- pathutils: Prepend node-specific prefix path- RPC: Use virtual paths (see vcluster.py)- SSH: Pass environment variables, use destination's node directory when copying files using scp, use GANETI_HOSTNAME to determine hostname...
Migrate lib/backend.py from constants to pathutils
File system paths moved from constants to pathutils.
constants: Move most paths to separate module
This is inpreparation for the implementation of virtual clusters. Manypaths will change based on an environment variable and are no longerconstant and should no longer be in “constants.py”. Since “constants.py”...
Bump pep8 version to 1.2
Debian Wheezy will ship with this version, and it has many improved checks compared to 0.6, so let's:
- bump version in the docs- silence some new checks that are wrong due to our indent=2 instead of 4- fix lots of errors in the code where the indentation was wrong by 1...
Fix DRBD resize code
There are two bugs in the current resize code, affecting mostly DRBD.
First, due to bugs in old DRBD versions (pre 8.0.14), the code currentlycalls `drbdsetup resize' on both the primary or secondary. However,this is actually wrong per current DRBD (from drbdsetup(8)):...