QA: Test enabled disk templates with known configuration
The test failed because it was executed also in a configuration that wasn'texpected, i.e., with exclusive storage enabled.
Also the docstring is shortened, as it was being truncated in the QA output....
QA: Decision about disk templates to use is more modular
qa_config.IsTemplateSupported() is already used to decide which disktemplates are supported by QA. Enabled templates now are part of thatdecision.
Unit tests are updated.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
Fix typo in documentation string
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Adapt test data to changed ipolicy format
The test data for hail allocating a multi-disk instance wasoriginally added in the 2.7 branch. On master the formatof the ipolicy had changed; "min" and "max" are no longertop-level of the ipolicy, but instead below the "minmax"...
Remove mistaken merge conflict line from NEWS
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Merge branch 'devel-2.7'
Update man page 'gnt-cluster' regarding disk templates
This documents the new 'gnt-cluster init/modify' option'--enabled-disk-templates' in the man page.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Michele Tartara <mtarara@google.com>
Removes the 'sharedfile' storage type
Since storage type are supposed to represent the underlying technologyof disk templates, the storage type 'sharedfile' is superfluous, becausetechnically both disk templates 'file' and 'sharedfile' use the file system....
Update NEWS file regarding disk types
This patch extends the NEWS file describing the essential user-facingchanges in this patch series.
Consistency checks for config wrt disk templates
This patch adds consistency checks for the configuration regardingdisk templates. In particular, it checks that the list of enableddisk templates is not empty, does not contain any bogus templates,and that all instances use actually enabled disk templates....
QA: creating instances of dis/enabled disk templates
This patch makes sure that instance creation is only possiblefor instances that use enabled disk templates.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
QA: dis/enabling of used disk templates
This extends the QA for 'gnt-cluster modify'. It ensuresthat disabling of a disk template is only possible if thecluster does not have any instances using the template.
Signed-off-by: Helga Velroyen <helgav@google.com>...
gnt-instance: no instance creation with disabled templates
This patch makes sure that no instances can be created which usedisk templates which are not enabled on the cluster.
qa_instance: refactoring wrt disk templates
This patch is a preparation for later patches in QA in this seriesit refactors the instance QA to make it more flexible regardingcreation of instances with different disk templates. Right now weonly support creation of instances with disk templates 'drbd', 'plain',...
Removes obsolete 'enabled storage types'
Since managing of different storage units is now done usingdisk templates and not storage types, we remove the obsoleteenabled storage types.
QA: Test cluster init/modify wrt enabled disk templates
Fixes the QA of 'gnt-cluster init' and 'gnt-cluster modify' with respectto enabling disk templates.
gnt-cluster init: enabling of disk templates
This patch enhances 'gnt-cluster init' to accept a list of disk templatesto be enabled. It removes the list of enabled storage types.
gnt-cluster modify: verify instance's disk template usage
Adds a verification step to 'gnt-cluster modify' to make sure that nodisk template is disabled that is currently in use by at least one instance.
gnt-cluster modify: modify enabled disk templates
Adds setting of the list of enabled disk templates to 'gnt-cluster modify'.Note that this does not yet include any verification regarding disk templatescurrently used by instances.
gnt-cluster info: report enabled disk templates
Add reporting of the currently enabled disk templates to'gnt-cluster info'.
Add enabled disk templates to cluster cfg
This patch adds constants for enabling disk templates to the constants,and the cluster configuration in haskell and python. It removes theobsolete preference list for enabled storage types.
Fix burnin install path
Due to a bug in 8e55e20f burnin's installation path got changed.Moving tools/burnin to the right makefile target reverts this (whilepreserving the value of the original commit).
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Fix format of the NEWS file
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Add "less" to the chroot
Less is not part of the base debian system, so it has to be addedexplicitly.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
NEWS: Add news entry for the hail disk policy fix
hail now correctly applies the instance disk policies, whichleads to differnt allocations.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add shelltests verifying hail applies disk ipolicy per disk
We test on two minimal examples, a positive and a negative one, wherethe possibility to allocate a node depends on whether the disk policyis checked on a per-disk level or on the total amount of disk space...
Make the disks parameter available to the constructor
In that way, tools building on Instance will benefit from the correctedverification semantics of the instance policy on disk space.
Verify individual disks in Instance
Instance policy on disks is specified on a per-disk basis. So extendthe instance description by the sizes of the individual disks and modifythe instance policy verification to correctly check individual disks.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Compatibility fix for GHC 7
Importing an instance, like JSON, automatically allows the names of allits functions to be used.
If one of this functions (namely showJSON) is imported separately, GHC 6is fine with that, but GHC 7 emits a warning (therefore, the compilation of...
Properly update iv_name of disks while changing templates
Trasforming the disk of an instance from DRBD to plain did not properlyupdate the iv_name of disks, leaving it to None.
This commit modifies _ConvertDrbdToPlain to properly set the iv_namevariables....
qa: fix TestInstanceModifyPrimaryAndBack flakiness
The test tries to scp the instance disks from the primary node to thenew destination, but connects to the master node instead. Thus it worksonly when the primary node is the master node. Fixing by correcting to...
Check minimum size of networks on creation
When creating a network, so far no size constraints were checked.We now limit the size of a network to a /30 or bigger, althoughtecnically, the ipaddr library supports even /32 networks.
Limit the size of networks to /16
This patch introduces an upper limit to the size of the networks that canbe created.
Fix job queue directory permission problems
If split users are used, the queue directory could only be accessedby masterd, but also confd needs to be able to read it, e.g. when itis queried as part of "gnt-job list"
This commit fixes the permissions in such a way to allow proper access rights....
Set the default editor inside the chroot
Allows the chroot builder to set the default editor to be used in thechroot.
Vim is used as the default choice.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
The disk size of a diskless instance is 0, not None
For diskless instances it is still reasonable to sum up the disk usageof all the (zero) disks, resulting in the empty sum. This uniformityalso has the advantage that iallocators (like hail) do not have to do...
Postpone non-urgent TODO from 2.7 to 2.9
Add backports to the chroot builder sources
This allows to have newer software than what is available in Debian Squeeze(the default version of the distribution used by the script), particularlyvim and git.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Add tool for creating users and groups
When executed in split user mode, Ganeti requires a specific set of usersand groups to be created in the appropriate way.
This commit introduces a tool that takes care of that automatically.
qa: Fix mistake in _DestroyInstanceDisks
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
qa: fix instance recreate disks for ST_FILE disks
More lvm commands "sprinkled" in qa :/ We really have to factor them outin the storage layer, once we have a good one.
Backwards compatibility fix for Lucid
The code introduced by the previous commit triggered a possible libraryconflict in Ubuntu Lucid.
This patch introduces an equivalent but more widely acceptable version ofthe same code.
Properly export errors while reading job list
In case of problems while reading the job list from disk (such as permissionerrors) confd would silently fail, writing a warning on the log file butsending an empty list and no error message to the client....
Fix typo and improve comment
qa: skip gnt-backup for file instances
Note that this fixes the "current issue" but doesn't fix the underlyingproblem. :/
qa: fix RemoveInstanceOfflineNode for file-based
- The cleanup should only happen for non externally mirrored disk templates- It should behave differently for file and lvm based instances- It should definitely not be explicit in the qa file :/ (but it's...
qa: disable inter-cluster-instance-move for file
This actually hides a real issue that was discovered by enablingfile-based qa, so bug was filed for it. On the other hand it's betterthan keeping the qa broken, since this functionality was broken anyway....
qa: fix rename instance for non-lvm backed storage
The rename instance test performs an lvm-specific check for tags on theinstance volume. This should be in the future abstracted inside thetemplate class, and not explicitly called from qa.
This would allow other disks to perform the checks that make sense for...
qa: fix modify primary test
This test uses a non-existing config value "file-storage-dir" and fails.Fixing it by using the default value, which is currently what qa does.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
qa: pass some extra options to make scp work
scp by itself won't work on a cluster: a few extra options are needed totell it where to find the "right" ssh global known hosts file managed byGaneti.
Check for python-yaml at configure time
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Fix a bug in the Runtime tests
A group name was erroneusly inserted in the list of user names.
Add script for building chroot
Setting up a proper environment for building a Ganeti can be quite difficult.
This patch adds a script for building a compressed chroot with all therequired libraries and tools.
Extension of storage reporting design doc
This patch rewrites and extends the design doc about storage reportingwith respect to disk templates and storage types. In constrast to theprevious version, we now consider disk templates as the user-facingentity, that the user can dis/enable for the cluster. Storage types...
gnt-group info uses a revised format
The code is more modular, the output is more similar to "gnt-cluster info" and is YAML-compliant.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
gnt-node info uses a revised format
The code is more modular, and the output is YAML-compliant. QA has beenupdated to reflect the new format.
Remove unused code
After changing the way gnt-xxx info print their output,cli.FormatParameterDict() is not used anymore.
Refactor out function to format instance policy
The new function can be used for group-level instance policies. Support forroman numeral formatting has been dropped to make the code simpler. Forother info items it's already unsupported, and it's not exactly documented...
Fix help string for gnt-group info
Document procedure to downgrade a cluster
This is based on the --donwgrade option of cfgupgrade, first introduced in1709435e.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix upgrade procedure in UPGRADE
The watcher should be disabled while upgrading.
Restrict instance move to templates based on local files
Moving an instance is done by copying over the disks. Restrictthis to disk templates that are copyable. This avoids accidentaluse on, e.g., the sharedfile template.
Introduce a constant for the copyable disk templates
This list contains the disk templates suitable for movingan instance by copying the files. A requirement is thatthey're not accessed externally or shared between nodes;in particular, sharedfile is not suitable....
Do not _RemoveDisks after failed _CreateDisks
Now that _CreateDisks cleans up after itself in case of failure,do not clean up at call sites, as there we have to overapproximatethus potentially causing data loss.
Make _CreateDisk clean up partially created disks on failure
_CreateDisk used to just throw an exception if _CreateBlockDev failedleaving the caller in the state that some disks were created, withoutprecise knowledge which. Usually, the clean up then overapproximated...
Fix typo in an error message
Fix error in the developer notes
The version numbers of two programs had been swapped.
Bump up version for 2.7.0~beta2
Also update the release date for beta2
check-news: fix behavior on invalid date line
Before:$ ./autotools/check-news < ./NEWSTraceback (most recent call last): File "./autotools/check-news", line 150, in <module> main() File "./autotools/check-news", line 120, in main parsed_ts = time.mktime(time.strptime(m.group("date"), "%d %b %Y"))...
Add dependency on PyYAML in devnotes
This is required because of 0e79564a; otherwise, the pylint-qa target willfail.
News updates for 2.7.0 beta2
Mention most relevant updates and user visible changes and fixes.
These entries will only be maintained during the beta/rc series, butwill then be replaced with a single "2.7.0" entry when we will release2.7.
Add QA for gnt-instance modify --new-primary
Add a simple walk through the intended use case ofgnt-instance modify --new-primary for the disk templatefile. The tested scenario is shutting down an instance,manually moving the disk to the new node, setting the...
Additionally support disk-type file in QA
Also perform QA tests for the file disk template. The main reasonis that testing gnt-instance modify --new-primary requires amanual move of disks, which is most easily simulated for files.
Add option --new-primary to gnt-instance modify
Extend gnt-instance modify by a new parameter --new-primarythat sets the pnode paramter of the opcode.
Make LUInstanceSetParams handle pnode parameter
If pnode is set LUInstanceSetParams now sets the primary nodeaccordingly. Unless force is set, it is verified that the instanceis no longer running on the old node.
Add parameter pnode to OpInstanceSetParams
This parameter will be used to set the new primary nodeof an instance, assuming that the disks have been movedby other means (outside ganeti's control).
Update installation documentation
Some required dependencies were missing from the installation instructions.
Correct typos in comments
Correct four typos in the comments of the Drbd types file.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Add status information to the DRBD data collector report
Add status information as required by the design document.
Export the data collected by DRBD in JSON format
Write an intermediate function computing the JSON data before adding themto the DCReport.
Directly export the data computed by that function (with default parameters)allowing them to be used (in the future) by the monitoring agent....
Add DCStatus data type for the data collectors
Also adds the DCStatusCode, part of DCStatus, and the addStatusutility function for adding the "status" field to an already existingJSValue.
The design document is updated to have the status codes sorted by increasing...
Add Kind data type for data collectors
Also, add it to the DRBD data collector, and export it from there.
Add data type for data collector category
Also, update the DRBD data collector to use and export it.
Export the dcVersionInformation for the Drbd collector
Also, update the JSON output (and the design document) so that it is notin camelcase anymore. This is part of a bigger effort to remove camelcasefrom the exposed JSON.
Add data collector version data type
Define the new data type and update the DRBD data collector to use it.
Fix comment line position in DRBD data collector
Export dcName information of Drbd data collector
Change format of the empty status message
Having 'null' instead of an empty string for an empty status messageincreases the complexity of the code with no real improvement.
This patch modifies the monitoring agent design document solving this issue.
Improve INSTALL documentation
Additional clarification note added.
Document more installation dependencies
A couple of them were missing and are added by this patch.
Unit tests for config.ConfigWriter.VerifyConfig()
The test is aimed mostly at instance policies, which changed recently.
cfgupgrade: Add support for newly refactored ipolicies
Instance policies in the configuration are upgraded and downgraded. Bothoperations are idempotent.
Refactor ispecs in ipolicy structures
Minimum and maximum instance specs are put together into a single elementof the instance policy. This is in preparation for introducing multiplemin/max specs.
cfgupgrade: Add --downgrade option
It's now possible to go back to the previous stable version. Unit testsprovided.
This is mostly useful during development, when going from master tostable/devel.
cfgupgrade: Refactor code
All the upgrading code is now in one function. No functionality has beenchanged.
Configuration is verified on load
If there are errors in the configuration when master daemon is started,they are now logged and reported. Previously they were logged sometimes atstartup, and when performing any change to the configuration.
Refactor cli.CreateIPolicyFromOpts()
The code relative to instance specs has been moved into a differentfunction. This in preparation for further patches to have a more clear codestructure.
Remove duplicate filling of instance policy
Instance policy fields were manually filled in CreateIPolicyFromOpts() andthen filled again by the caller. Now they are filled once inside thefunction.
Unit test for cli.CreateIPolicyFromOpts()
Useful for subsequent patches.
Amend partitioned design doc for multiple ispecs
There will be only one standard specification in instance policies.