update hroller man page: node group filtering is implemented
Filtering of nodes by node group was already implemented in2fce67b; fix the man page to reflect this change.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Use correct name for design-storagespace.rst
This was renamed in b284f50 but the old name was still in use in themakefile and draft designs list.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Rename design doc from space reporting to storage types
Since the design doc for storage space reporting ended up being mostlyabout disk templates and storage types, and the space reporting beingjust a small part of it, I am renaming the design doc.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Storage type design doc: ipolicy vs. enabled disk templates
Adding the design decision regarding the consistency of the enabled disktemplates in the ipolicy and in the cluster config.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
QA: Fix error regarding missing names of disk configs
This fixes an error in case there are no disk names in the QA config(which was the case till recently). Right now, QA tries to use 'None'as disk name and that makes instance creation fail.
Revert "Make invocation of rapi test configurable ..."
This reverts commit 405692eff3d6f8400834f2df93d597bc7e4dd89f, because itbreaks QA.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
devel/review: take default reviewer from environment
When adding a 'Reviewed-by'-line, also add the reviewerpassed through the REVIEWER environment variable. In thatway, adding the reviewer for patch series all reviewedby the same person gets easier....
Make invocation of rapi test configurable by QA config
This does not solve the underlying problem of the rapi test notbeing runnable locally, but it makes it at least configurableto avoid having to patch in/out all the time.
Fix warnings hlint 1.8.43 complained about
These lines are ok according to previous versions of hlint but triggeran error with version 1.8.43.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add nic/disk name and UUID fields in Haskell code
Add name and uuid fields to Disk and nic Haskell objects. Also,since they now have an UUID, make them instances of UuidObject.
Signed-off-by: Christor Stavrakakis <cstavr@grnet.gr>Reviewed-by: Helga Velroyen <helgav@google.com>
Add name to INicParams and IDiskParams
This commits adds name to NIC and Disk modification definition.
Signed-off-by: Chris Stavrakakis <cstavr@grnet.gr>Reviewed-by: Helga Velroyen <helgav@google.com>
Update QA tests to check disk and NIC names
Modify existing tests to use the name option for instance disks. Theconfiguration options 'disk' and 'disk-growth' are merged into a single'disks' option, which is a list of dictionaries with 'size', 'growth' and...
Update manpages wrt new device refering
This patch updates gnt-instance man page to include refering to devicesby their name and UUID.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>Reviewed-by: Helga Velroyen <helgav@google.com>
Preserve device names during instance move
Fix 'move-instance' tool to preserve the instance disk and NIC nameswhen moving instances from one cluster to another. Also, fix value errorsince an instance NIC is an 8-tuple, containing also name and UUID.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>...
Display UUIDs and names in gnt-instance info
Modify gnt-instance client to display the UUID and namefields of instances disks and NICs.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Reviewed-by: Helga Velroyen <helgav@google.com>
Add Disks and NICs to _AllUUIDObjects
Since disks and NICs have UUIDs, they must be consideredto _AllUUIDObjects.
Support quering for devices names and UUIDs
Extend instance queries to list 'uuid' and 'name' fields of instance'sDisks and NICs.
Check that device names are unique and valid
Extend the CheckArguments phase of LUInstanceCreate and CheckPrereqphase of LUInstanceSetParams to also check if the name parameters ofdisks and NICs are unique and valid.
Implement renaming Disks and NICs
Implement renaming Disks and NICs in LUInstanceSetParams. Remove codethat checked that changing disk mode was the only disk modificationsupported. Also, handle case of 'None' value for 'name' parameter.
Support UUIDs and names when refering to a device
Modify _ApplyContainerMods function to lookup NICs/Disks not only by theirindex inside the container, but also by their UUID or name. Abstract thelookup code in new GetItemFromContainer function.
Make type of identifier in "opcode._TestInstSetParamsModList" to be...
Support UUIDs and names when handling NICs/Disks
Handle UUID and name fields when creating/managing NICs and Disks. Also,export NICs and Disks names to instance hooks environment.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>...
Add IDISK_NAME and INIC_NAME constants
Declare IDISK_NAME and INIC_NAME constants and add them to theIDISK_PARAMS_TYPE and INIC_PARAMS_TYPE.
Add uuid and name slots to Disk/NIC ConfigObjects
This patch adds "uuid" and "name" slots to Disk and NIC ConfigObjects.
Add design doc for device UUIDs and names
This commit adds the design document for adding UUID and name slots to NIC andDisk objects and the ability to refer to them by these slots.
QA: Test enabled disk templates with known configuration
The test failed because it was executed also in a configuration that wasn'texpected, i.e., with exclusive storage enabled.
Also the docstring is shortened, as it was being truncated in the QA output....
QA: Decision about disk templates to use is more modular
qa_config.IsTemplateSupported() is already used to decide which disktemplates are supported by QA. Enabled templates now are part of thatdecision.
Unit tests are updated.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
Fix typo in documentation string
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Adapt test data to changed ipolicy format
The test data for hail allocating a multi-disk instance wasoriginally added in the 2.7 branch. On master the formatof the ipolicy had changed; "min" and "max" are no longertop-level of the ipolicy, but instead below the "minmax"...
Remove mistaken merge conflict line from NEWS
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Merge branch 'devel-2.7'
Update man page 'gnt-cluster' regarding disk templates
This documents the new 'gnt-cluster init/modify' option'--enabled-disk-templates' in the man page.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Michele Tartara <mtarara@google.com>
Removes the 'sharedfile' storage type
Since storage type are supposed to represent the underlying technologyof disk templates, the storage type 'sharedfile' is superfluous, becausetechnically both disk templates 'file' and 'sharedfile' use the file system....
Update NEWS file regarding disk types
This patch extends the NEWS file describing the essential user-facingchanges in this patch series.
Consistency checks for config wrt disk templates
This patch adds consistency checks for the configuration regardingdisk templates. In particular, it checks that the list of enableddisk templates is not empty, does not contain any bogus templates,and that all instances use actually enabled disk templates....
QA: creating instances of dis/enabled disk templates
This patch makes sure that instance creation is only possiblefor instances that use enabled disk templates.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
QA: dis/enabling of used disk templates
This extends the QA for 'gnt-cluster modify'. It ensuresthat disabling of a disk template is only possible if thecluster does not have any instances using the template.
gnt-instance: no instance creation with disabled templates
This patch makes sure that no instances can be created which usedisk templates which are not enabled on the cluster.
qa_instance: refactoring wrt disk templates
This patch is a preparation for later patches in QA in this seriesit refactors the instance QA to make it more flexible regardingcreation of instances with different disk templates. Right now weonly support creation of instances with disk templates 'drbd', 'plain',...
Removes obsolete 'enabled storage types'
Since managing of different storage units is now done usingdisk templates and not storage types, we remove the obsoleteenabled storage types.
QA: Test cluster init/modify wrt enabled disk templates
Fixes the QA of 'gnt-cluster init' and 'gnt-cluster modify' with respectto enabling disk templates.
gnt-cluster init: enabling of disk templates
This patch enhances 'gnt-cluster init' to accept a list of disk templatesto be enabled. It removes the list of enabled storage types.
gnt-cluster modify: verify instance's disk template usage
Adds a verification step to 'gnt-cluster modify' to make sure that nodisk template is disabled that is currently in use by at least one instance.
gnt-cluster modify: modify enabled disk templates
Adds setting of the list of enabled disk templates to 'gnt-cluster modify'.Note that this does not yet include any verification regarding disk templatescurrently used by instances.
gnt-cluster info: report enabled disk templates
Add reporting of the currently enabled disk templates to'gnt-cluster info'.
Add enabled disk templates to cluster cfg
This patch adds constants for enabling disk templates to the constants,and the cluster configuration in haskell and python. It removes theobsolete preference list for enabled storage types.
Fix burnin install path
Due to a bug in 8e55e20f burnin's installation path got changed.Moving tools/burnin to the right makefile target reverts this (whilepreserving the value of the original commit).
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Fix format of the NEWS file
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Add "less" to the chroot
Less is not part of the base debian system, so it has to be addedexplicitly.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
NEWS: Add news entry for the hail disk policy fix
hail now correctly applies the instance disk policies, whichleads to differnt allocations.
Add shelltests verifying hail applies disk ipolicy per disk
We test on two minimal examples, a positive and a negative one, wherethe possibility to allocate a node depends on whether the disk policyis checked on a per-disk level or on the total amount of disk space...
Make the disks parameter available to the constructor
In that way, tools building on Instance will benefit from the correctedverification semantics of the instance policy on disk space.
Verify individual disks in Instance
Instance policy on disks is specified on a per-disk basis. So extendthe instance description by the sizes of the individual disks and modifythe instance policy verification to correctly check individual disks.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Compatibility fix for GHC 7
Importing an instance, like JSON, automatically allows the names of allits functions to be used.
If one of this functions (namely showJSON) is imported separately, GHC 6is fine with that, but GHC 7 emits a warning (therefore, the compilation of...
Properly update iv_name of disks while changing templates
Trasforming the disk of an instance from DRBD to plain did not properlyupdate the iv_name of disks, leaving it to None.
This commit modifies _ConvertDrbdToPlain to properly set the iv_namevariables....
qa: fix TestInstanceModifyPrimaryAndBack flakiness
The test tries to scp the instance disks from the primary node to thenew destination, but connects to the master node instead. Thus it worksonly when the primary node is the master node. Fixing by correcting to...
Check minimum size of networks on creation
When creating a network, so far no size constraints were checked.We now limit the size of a network to a /30 or bigger, althoughtecnically, the ipaddr library supports even /32 networks.
Limit the size of networks to /16
This patch introduces an upper limit to the size of the networks that canbe created.
Fix job queue directory permission problems
If split users are used, the queue directory could only be accessedby masterd, but also confd needs to be able to read it, e.g. when itis queried as part of "gnt-job list"
This commit fixes the permissions in such a way to allow proper access rights....
Set the default editor inside the chroot
Allows the chroot builder to set the default editor to be used in thechroot.
Vim is used as the default choice.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
The disk size of a diskless instance is 0, not None
For diskless instances it is still reasonable to sum up the disk usageof all the (zero) disks, resulting in the empty sum. This uniformityalso has the advantage that iallocators (like hail) do not have to do...
Postpone non-urgent TODO from 2.7 to 2.9
Add backports to the chroot builder sources
This allows to have newer software than what is available in Debian Squeeze(the default version of the distribution used by the script), particularlyvim and git.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Add tool for creating users and groups
When executed in split user mode, Ganeti requires a specific set of usersand groups to be created in the appropriate way.
This commit introduces a tool that takes care of that automatically.
qa: Fix mistake in _DestroyInstanceDisks
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
qa: fix instance recreate disks for ST_FILE disks
More lvm commands "sprinkled" in qa :/ We really have to factor them outin the storage layer, once we have a good one.
Backwards compatibility fix for Lucid
The code introduced by the previous commit triggered a possible libraryconflict in Ubuntu Lucid.
This patch introduces an equivalent but more widely acceptable version ofthe same code.
Properly export errors while reading job list
In case of problems while reading the job list from disk (such as permissionerrors) confd would silently fail, writing a warning on the log file butsending an empty list and no error message to the client....
Fix typo and improve comment
qa: skip gnt-backup for file instances
Note that this fixes the "current issue" but doesn't fix the underlyingproblem. :/
qa: fix RemoveInstanceOfflineNode for file-based
- The cleanup should only happen for non externally mirrored disk templates- It should behave differently for file and lvm based instances- It should definitely not be explicit in the qa file :/ (but it's...
qa: disable inter-cluster-instance-move for file
This actually hides a real issue that was discovered by enablingfile-based qa, so bug was filed for it. On the other hand it's betterthan keeping the qa broken, since this functionality was broken anyway....
qa: fix rename instance for non-lvm backed storage
The rename instance test performs an lvm-specific check for tags on theinstance volume. This should be in the future abstracted inside thetemplate class, and not explicitly called from qa.
This would allow other disks to perform the checks that make sense for...
qa: fix modify primary test
This test uses a non-existing config value "file-storage-dir" and fails.Fixing it by using the default value, which is currently what qa does.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
qa: pass some extra options to make scp work
scp by itself won't work on a cluster: a few extra options are needed totell it where to find the "right" ssh global known hosts file managed byGaneti.
Check for python-yaml at configure time
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Fix a bug in the Runtime tests
A group name was erroneusly inserted in the list of user names.
Add script for building chroot
Setting up a proper environment for building a Ganeti can be quite difficult.
This patch adds a script for building a compressed chroot with all therequired libraries and tools.
Extension of storage reporting design doc
This patch rewrites and extends the design doc about storage reportingwith respect to disk templates and storage types. In constrast to theprevious version, we now consider disk templates as the user-facingentity, that the user can dis/enable for the cluster. Storage types...
gnt-group info uses a revised format
The code is more modular, the output is more similar to "gnt-cluster info" and is YAML-compliant.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
gnt-node info uses a revised format
The code is more modular, and the output is YAML-compliant. QA has beenupdated to reflect the new format.
Remove unused code
After changing the way gnt-xxx info print their output,cli.FormatParameterDict() is not used anymore.
Refactor out function to format instance policy
The new function can be used for group-level instance policies. Support forroman numeral formatting has been dropped to make the code simpler. Forother info items it's already unsupported, and it's not exactly documented...
Fix help string for gnt-group info
Document procedure to downgrade a cluster
This is based on the --donwgrade option of cfgupgrade, first introduced in1709435e.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix upgrade procedure in UPGRADE
The watcher should be disabled while upgrading.
Restrict instance move to templates based on local files
Moving an instance is done by copying over the disks. Restrictthis to disk templates that are copyable. This avoids accidentaluse on, e.g., the sharedfile template.
Introduce a constant for the copyable disk templates
This list contains the disk templates suitable for movingan instance by copying the files. A requirement is thatthey're not accessed externally or shared between nodes;in particular, sharedfile is not suitable....
Do not _RemoveDisks after failed _CreateDisks
Now that _CreateDisks cleans up after itself in case of failure,do not clean up at call sites, as there we have to overapproximatethus potentially causing data loss.
Make _CreateDisk clean up partially created disks on failure
_CreateDisk used to just throw an exception if _CreateBlockDev failedleaving the caller in the state that some disks were created, withoutprecise knowledge which. Usually, the clean up then overapproximated...
Fix typo in an error message
Fix error in the developer notes
The version numbers of two programs had been swapped.
Bump up version for 2.7.0~beta2
Also update the release date for beta2
check-news: fix behavior on invalid date line
Before:$ ./autotools/check-news < ./NEWSTraceback (most recent call last): File "./autotools/check-news", line 150, in <module> main() File "./autotools/check-news", line 120, in main parsed_ts = time.mktime(time.strptime(m.group("date"), "%d %b %Y"))...
Add dependency on PyYAML in devnotes
This is required because of 0e79564a; otherwise, the pylint-qa target willfail.
News updates for 2.7.0 beta2
Mention most relevant updates and user visible changes and fixes.
These entries will only be maintained during the beta/rc series, butwill then be replaced with a single "2.7.0" entry when we will release2.7.
Add QA for gnt-instance modify --new-primary
Add a simple walk through the intended use case ofgnt-instance modify --new-primary for the disk templatefile. The tested scenario is shutting down an instance,manually moving the disk to the new node, setting the...
Additionally support disk-type file in QA
Also perform QA tests for the file disk template. The main reasonis that testing gnt-instance modify --new-primary requires amanual move of disks, which is most easily simulated for files.
Add option --new-primary to gnt-instance modify
Extend gnt-instance modify by a new parameter --new-primarythat sets the pnode paramter of the opcode.
Make LUInstanceSetParams handle pnode parameter
If pnode is set LUInstanceSetParams now sets the primary nodeaccordingly. Unless force is set, it is verified that the instanceis no longer running on the old node.
Add parameter pnode to OpInstanceSetParams
This parameter will be used to set the new primary nodeof an instance, assuming that the disks have been movedby other means (outside ganeti's control).