Introduce new module for IP pool management
Add new library module lib/network.py.Introduce new class: AddressPool.
AddressPool implements all operations needed for managing IPsinside the IP pool.
Given a Network config object (nobj), the class:
Basic IP pool management logic
Implement LUs for corresponding opcodes: * LUNetworkAdd: - Check for IP validity - Reserves all necessary IPs - Create new Network config object * LUNetworkRemove: - Checks if connected to any nodegroup - Remove a Network config object...
IP pool related objects, opcodes and constants
Config objects: * Introduce L{Network} with - IPv4 network field (mandatory) - IPv4 gateway, IPv6 (network/gateway), mac prefix, type (optional) * Modify existing config objects to support networks: - Add new slot 'network' to L{NIC} config object...
Fix computation of disk sizes in _ComputeDiskSize
Currently, hail fails with FailDisk when trying to add an instanceof type: 'file', 'sharedfile' and 'rbd'.
This is due to a "0" or None value in the corresponding dict inside_ComputeDiskSize, which results in a "O" or non Int value of the...
Add verification of RPC results in _WipeDisks
Due to an oversight, the pause/resume sync RPC calls in _WipeDiskslack the verification of the overall RPC status, and directly iterateover the payload. The code actually doing the wipe does verifycorrectly the results. This can result in jobs failing with a hard to...
Add test for checking that all gnt-* subcommands run OK
This is a bit of a shell munging trickery, but works for now. Makingit more generic can be done later.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Fix double use of PRIORITY_OPT in gnt-node migrate
This breaks the command, as optparse considers that an error.
Add new Makefile target to rebuild the whole dist
Due to the fact how the automake system works it doesn't rebuild alreadyprebuild files in distcheck. This lead to a bug, where a rebuild of thedocumentation was failing because we missed the fact that the files were...
Add missing rst files to Makefile.am
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Release version 2.6.0 (final)
Phew, it wasn't easy, but…
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Fix 'explicitely' common typo
It seems that 'explicitely' is wrong, and that the right form is'explicitly'. This is just fixing the typo plus adjusting affectedparagraphs.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix issue in LUClusterVerifyGroup with multi-group clusters
In case LUClusterVerifyGroup is run on a group which doesn't containthe master node, the following could happen:
- master node is selected due to the explicit check- if the order of nodes in the 'absent_nodes' list is such that the...
Add QA test for node group modification of ndparams
Additionally, we run hbal if the htools tests are enabled (newkey). Note that hbal will directly exit, since there are no instances,but at least it will load and parse the group details.
Signed-off-by: Iustin Pop <iustin@google.com>...
Fix node group modification of node parameters
Commit 904b3bfe tried to fix the deletion of custom ndparams fromgroup, but instead broke both modification and deletion: because werun ForceDictType on self.op.ndparams instead of the updatednew_ndparams, we can neither delete nor set properly spindle_count...
Merge commit 'v2.5.2' into stable-2.6
Conflicts: NEWS (trivial, merged the entries)...
Fix RST formatting in NEWS file
*FileID is interpreted as a start-of-bold, without corresponding endmarker. Sorry for not catching it in the first place.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
Update NEWS and bump version for release 2.5.2
This is a trivial release, with just bug fixes, so a small change log.
Fix boot=on flag for CDROMs
This generalises commit 4304964a to cdroms too, since they havesomewhat the same logic. We just abstract the needs_boot_flag into aseparate variable, and then reuse it in the cdrom section.
Note that the logic of what 'if=' type to pass to KVM was very...
KVM: only pass boot flag once
This addresses issue 230: passing two methods of booting to KVM can,depending on the KVM version, confuse it.
Note that commit 4304964a introduced a partial fix for this (but onlyfor disks, and keyed on KVM versions). However, it didn't fix cdrom...
Ensure a stable content of the bash completion file
Currently, the order of commands in the bash completion file israndom, because the sub-commands are not sorted. This makes it harderto investigate the differences in packaged Ganeti or in installedGaneti, since chunks in this file will have a random order....
Release Ganeti 2.6.0 rc4
Prepare NEWS for Ganeti 2.6.0rc4
Another release candidate is coming.
Add some rudimentary node group ipolicy checks
Fix setting ipolicy on node groups
On node groups we don't have the std field. However, the InstancePolicyobject always verifies that the std value is within a given range. As wefill it up with defaults if not set (as it happens to be on node groups)and the min value is higher than the default std value (taken from...
Fix --no-headers for the new list-drbd command
Sorry, I forgot that with GenerateTable this needs to be handledmanually. Fixed now and tested in both ways.
(But to be honest, this should be abstracted in GenerateTable, insteadof the 'if' test in all its callers.)...
Add a simple QA test for gnt-node list-drbd
This will be run only if 'haskell-confd' QA configuration has beenenabled.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Add a new gnt-node command list-drbd
This uses confd to query the DRBD minors, which is very special; noother command currently does so.
Since the backend is only implemented in the Haskell version of confd,we have checks that 1) confd is enable, and 2) hs confd is also...
Implement a node to drbd minors query function
This can be queried remotely since it's a pure configuration query.
Add a new unused confd query
This is not implemented currently.
Add support for computing instance all/secondary nodes
This fixes an old FIXME. Since we now how actual DRBD configurationdata, we can finally compute the instance's secondary nodes.
Add disk logical ID support in Objects.hs
This is a complex field, so we have to do a lot of manual work for now.
The complexity arises from the fact that the contents of the field,and the way to parse it, depends on the disk type field, so we don'thave a single, static way of parsing it. Hence we needed the...
Fix a docstring in bdev's DRBD8 class
It seems this was not updated since the move to static minors…
Extend the Template Haskell loadFn model
Currently, we only allow field-by-field de-serialisation. Since wehave cases where information about how to un-serialise a field issplit across two JSON fields (e.g. disk type and disk logical_id,hypervisor and hvparams, etc.), we need to pass the entire object to...
Change how customFields are built
Instead of passing an expression (which cannot come from the currentmodule), we pass a name, which is allowed to reference functions fromthe module we're in. Since we currently don't have custom fields, wedon't need to modify any callers....
Implement lookup of partial names in Config.hs
This uses the recently-moved functions to implement partial lookup ofnames on getNode and getInstance, similar to the Python codebase.
Reorganise the lookup functions
Currently, the LookupResult, MatchPriority and related functions arelocate in Loader.hs, since (so far) only hbal needs them in theselection of instances. However, with the new functionality on confdside, we need these functions there too, but we don't want to import...
Remove an unused function
This is not used, as we need a more complex serialisation, which isdone in the saveObjectField function.
Ensure that disk.params is always defined (and a dict)
Commit cce46164 fixed upgrading from other 2.6 versions, butaccidentally broke upgrading from 2.5 (disk.params was left as None,which breaks FillDict). The simplest way to handle params is to always...
Another small consistency fix with if branches
While looking at the testability of this piece of code, I found anotherconsistency problem. We have two if branches instead of one, withelif's.
Fix inconsistency in the LUXI protocol w.r.t. args
This inconsistency was found during rebalancing. Hbal failed because,Ganeti couldn't load the opcode. After digging through the cause, aninconsistency with the "args" field in the LUXI protocol was triggered...
Fix UpgradeConfig of Disk object regards disk params
This bug was found during disk parameter debugging. While looking at theconfig some values seem present on the disk parameters, but that's notexpected behaviour. This patch fixes this, and also fixes the "broken"...
Make Ganeti 2.6.0 rc3 release
Allow reinstall even when secondaries are offline
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Prepare NEWS file for Ganeti 2.6.0 rc3
Update NEWS regarding rc3 fixes and document changes in behaviour.
QA updated to test instance removing when sec. is offline
Added test checking that it is possible to remove an instance with drbddisk when non-primary node is offline.
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
Ignore offline node errors when removing disks
When we delete DRBD disks from some instance, we do not want to geterrors due to nodes other than that instance's primary being offline.
Allow instance disc activation with offline secondaries
Currently, this is not allowed, so one can't run a replace-disks; thisbreaks any non-invasive method of recovering the redundancy of theinstance if its disks are already stopped (but it still works if the...
RAPI regression beparams/memory fix
For compatibility with the old Ganeti version, we want to keep thebeparams/memory field around for another release. This patch fixes thisregression.
Fix redistribution of files w.r.t. offline nodes
Currently, _RedistributeAncillaryFiles computes two lists: the list ofonline nodes (for all files redistribution), and the list ofvm_capable nodes, for hypervisor-specific files. However, thevm_capable list includes offline nodes too, leading to warning...
Fix cluster verify error on master-ip-setup script
This error does not show up until we exceed the pool of mastercandidates and have nodes which are not master candidates.
The background is that we check for master-ip-setup script on mastercandidates and expect them not to be on the other nodes. However, we...
Add a Makefile rule to test unclean git status
This is trivial, but can be used easily from automated builds to checkthat the git tree is clean: files are not modified and untracked filesare not present.
Add test helpers to .gitignore
Commit 3096524e enabled building of test helpers, but forgot to addthem to .gitignore, resulting in "unclean" git status output.
Fix a typo in NEWS
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Release Ganeti 2.6.0rc2
Prepare NEWS file for 2.6.0 rc2
We are about to release another version of Ganeti 2.6, so we prepare theNEWS file with the changes from rc1 to rc2.
Update time on Haskell binaries after build
This patch works around an issue in our build system. Since we don'tuse cabal or 'ghc -M', we don't track actual dependencies in ourMakefile; this in turn means that editing a file that only is used inthe main 'htools' binary will keep triggering 'ghc --make' for the...
hcheck: add two simple type aliases for readability
The same types are reused a couple of times, so let's add a couple oftype aliases for easier change later and readability.
hcheck: rework output mode
Looking at the output of hcheck, in human readable mode, it looks likeit always starts with a blank line. This is not nice, so I wanted toredo this to start cleanly.
However, looking at the code, I realised that we need some internal...
hcheck: reword and fix typo in instance/offline msg
Add hlint warning for wrong use of 'length'
While removing split instances check from hcheck, I saw a wrong use of'length': this should only be used when one needs the actual length ofthe list, and not when one wants to check whether the list is empty or...
Remove can_rebalance functionality from hcheck
Since hbal can now run rebalances even in the presence of splitinstances, we can remove this check and always run the rebalance.
Fix building of hcheck man page
The code used a space, not a tab, in Makefile.am, which means itactually broke the variable declaring the man pages.
Furthermore, we add some blank lines in the man page, otherwise theparagraphs will be run-in together....
Fix tiny typo in htools.rst
Backtracking through git blame, it turns out than commit 49148d15changed "\fB20,102400,16384,4\fR" into "**B20,102400,16384,4**". Butof course that's a mistake, since the markup was \fB, not just \f.
Allow param `modify' during gnt-instance modify
With the new gnt-instance modify syntax for addition and removal ofdisks/NICs on arbitrary indexes, we hit an assertion if the userpasses `modify' as one of the disk's parameters. E.g::
gnt-instance modify --disk 2:modify,size=3G instance1...
Correct a wrong --help option for --simulate
This was not update when the alloc policy was added to the parameter.
Add a shell test for hbal and split instances
This is not perfect, as we only test that hbal completes successfullyand that it show a score improvement, but it's better than nothing.
hbal: relax restrictions on split instances
The recent patch series on hcheck made me realise how much pain wehave due to hbal not being able to work (at all) with split instances.
This is sub-optimal; ideally, hbal would degrade its behaviour, butnot completely refuse to work. This patch does this, by marking any...
Add newline at the end of shelltest files
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Add forgotten unittest changes for instance_os_add
The previous patch which fixed disk parameters didn't adapt theunittests so it lead to failing QA.
Annotate disk params on instance_os_add
We call _OpenRealBD during the process and this needs disk parameters towork. This was reported by Constantinos.
The fix is very ugly though.
Fix a few style issue in hcheck
This fixes a couple of issue I've seen while fixing the List import:
- removes over-use of printf with putStr/show- wraps some lines- removes superfluous parentheses in constructs like 'IO (a)'- makes sure that in type signatures, the '->' are placed at the...
Fix old-style import
Commit 1213f9d6 (re)added an old-style import, which fails with recentcompilers.
Style fix in hcheck
Style fixes in shelltests
Simplify stats printing
PrintGroupStats and printClusterStats had very similar codestructure. This patch abstracts those two into printStats, making ita little more generic. Also, printStats has its arguments documentedas requested in previous patch series....
Cleaner simulation of rebalance
This replaces the 'if .. then .. else ..' with a large block of code in'else' branch by separating this code into a separate function.
Small reordering and renaming in hcheck
Reordered functions and shortened one function name.
Move 'need rebalance' check from cluster check
As suggested by Rene, we should keep the check separate from printing.
Always print all variables in machine readable version
As discussed in cover letter of patch series introducing hcheck, inmachine readable version we should always print values of all variables.When hbal simulation was not run, FINAL values are equal to INIT values....
Print idx <-> uuid mappings in machine readable mode
As per Iustin Pop's suggestion, we are printing group mappingsfrom group idx (known only to htools) to group uuid, which identifiesthe group uniquely. This is required because group uuid and namemay contain characters such as '-' which are not valid parts of...
Human readable group name improved
Instead of printing group id in human readable version of hcheck,we now print group name.
Print info about ability to rebalance
As suggested during offline discussion, this patch adds a new bashvariable and description field in hcheck, which informs if therebalance was possible. This is to determine if there are splitinstances on the cluster, in which case rebalance is impossible....
Improve readability for hcheck-related changes
As suggested by Rene, I added comments to types for printKeysand printFinal. Also, realigned some lines in Hspace.
Annotate disks upon blockdev_shutdown
Annotate disks on blockdev_remove
This annotates the disks for the blockdev_remove where it isappropriate. It leaves out 2 cases were we can't reliably annotate diskparameters due to lack of knowledge what we should annotate. Those casesaffects only lvs used for drbd, so it doesn't affect the bug reported by...
Annotate disk params on blockdev_getmirrorstatus_multi
This is also related to the bug reported by Constantinos,as we've only one getmirrorstatus_multi call in whole cmdlib, we justannotate them while we are building the disk list.
Signed-off-by: René Nussbaumer <rn@google.com>...
Annotate disk parameters on blockdev_getmirrorstatus
Not annotating them works for DRBD but not for RBD as reported byConstantinos.
Fix for man entry for htools - added hinfo
Fixes in man page for hinfo
Update to the install file - hinotify dependency
Hinotify was not marked as a dependency for hconfd.
Release Ganeti 2.6.0rc1
Shelltestrunner tests for hcheck
Simple tests for hcheck using shelltestrunner. Among other, we checkthat we can run hcheck on multi-group cluster.
Machine readable hcheck functionality
Machine readable output for hcheck.
Human readable hcheck functionality
Full hcheck functionality is implemented in this patch. Hcheck runsper-group checks, provides summary for the entire cluster and simulatesrebalance if required, the printing the modified metrics.In this patch, only the human-readable output is available....
Allowing rebalance to run silently
Part of hcheck's simulation required us to silently run hbal. For thatpurpose we expose iterateDepth and allow it to run silently.
Generalize functions used by both hspace and hcheck
As hspace and hcheck both use machine readable options, they requiresimilar functions - which are now generalized and moved to CLI andUtils.
Initial commit for introducting hcheck tool
Introduce infrastructure required to add Hcheck and build itsuccessfuly.
NEWS: Improve formatting, wording
Clarify some things, use ```` where appropriate.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Prepare the update of the NEWS file
This updates the NEWS file for the upcoming rc1 release.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Extend burnin to do NIC changes
This would have caught our previous issue, since QA otherwise doesn'tdo add, remove or changes of NICs.
Signed-off-by: Iustin Pop <iustin@google.com>Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>