htools: add group restrictions to findBestAllocGroup
While a normal multi-group allocation doesn't need to restrict thelist of target groups beyond the unallocable status of some groups,when we relocate instances from one group to the other we need tolimit the subset of searched groups....
htools: generalise and rename availableLocalNodes
Currently this function works in the context of an instance, computingits local group nodes. We can use generalise it to take a group indexargument, so that it's not bound to an instance, and we rename it...
htools: add ChangeGroup to IAllocator types/loader
This patch updates the types and functions in IAllocator.hs andLoader.hs to the new design changes (elimination of multi-relocatemode and replacement with change-group).
It also removes an extra re-export of EvacMode from Loader.hs....
htools: split tryMGAlloc in two
Currently, this function does both identify the best group for aninstance, and format the results (partially). The former functionalityis useful in other contexts too, so let's split the function in two.
Signed-off-by: Iustin Pop <iustin@google.com>...
htools: add node-evacuation of DRBD all nodes
This is much more complicated than either change primary (which istrivial) and change secondary (which is also simple).
The main strategy is:
- choose a new secondary (that will become the final primary) and...
htools: small optimisation in Cluster.collapseFailures
This makes fewer temporary data structures, and thus brings a bit ofimprovement in total runtime.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
htools: add optional limit to allocation count
Currently, the tieredAlloc/iterateAlloc functions will not returnuntil the allocation fails; this means unit-testing theirfunctionality (e.g. that an instance can be allocated) is slow, sincethey will allocate all possible instances....
htools: add node-evacuation of DRBD secondaries
htools: rework compCV so that we can use [Node] too
In some cases, we have a [Node.Node] list, instead of a Node.Liststructure. Instead of converting [Node.Node] to the latter, let'srework compCV and add a compCVNodes such that we skip this(potentially expensive) step....
Adding a wrapper around connecting to kvm console
The wrapper will connect to the console, and check in the background ifthe instance is paused, unpausing it as necessary.
Signed-off-by: Stephen Shirley <diamond@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Adding a wrapper around "xm console"
Add opcode attribute for chained jobs
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Design for chained jobs
htools: fix a doc bug in the hbal man page
Similar to the previous hspace fix.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Mark htools/README as obsolete
And point to the main INSTALL file for up-to-date information.
Fix htool unittest failure due to invalid prereqs
Currently, the test prop_Node_rMem does some reserved memory checks onnodes, and for the test to work correctly (can add more instances) werequire that the node free memory is greater than zero via aprerequisite....
htools: add node-evacuate of DRBD8 primaries
htools: add more IAlloc data types and functions
These move the 'not supported' message deeper into the workflow;assuming that the workflow as added by this patch is correct, all thatremains is:
- for disk template DRBD8, plug in the currently existing mechanisms...
htools: add two monad-conversion utility functions
For monadic operations, these function will make it easier to convertother monads (or monad-like structure) to our 'Result' type.
htools: add a function for computing evacuated nodes
The new IAllocator interface is based on instances, not nodes, so weneed to backtrack and compute on which nodes we can't allocateinstances during the current operation. This patch adds a function for...
htools: rename an internal constant
'emptySolution' is fine while we have only one Solution type(AllocSolution), but in the future we'll introduce another one andthus we should rename it.
htools: abstract and simplify a multigroup function
In the future, this 'split/associated instance idx list' functionalitywill be used by other code, so let's move it to its own code.
The new method of computing the split also requires exporting one more...
hspace: also display the starting tiered specs
These are useful for checking what instance specifications we startwith (as they are not necessarily the first entry in the TSPEC map).
hspace: add support for units in the options
This patch converts hspace to allow units in all disk and memoryspecifications, and updates the man page accordingly. It also correcta mistake in the man page for the --max-cpu option.
htools: add parseUnit function
This is similar, but not identical, to Utils.ParseUnit. The biggestdifference is that we don't round up/down; as we only use integraltypes, the result will always be rounded down.
Moreover, since (real-world) disk sizes come in SI units, the function...
htools: simplify select tests
Rather than running all possible tests in one go, split them out inthree cases: "normal", undefined default value, undefined list values(both of which require a valid response before them).
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Set startup_paused to False when restarting
This fixes the lint error:
E1120:1220:InstanceReboot: No value passed for parameter'startup_paused' in function call
htools: lookup instance names in select/exclude
Currently we just match the passed name with fqdns. With this change wedo a proper search for either the fqdn or the initial name component, asin Ganeti itself.
htools: infrastructure for looking up names
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
htools: Add if' and select as functions
if' is just a functional version of the if-then-else construct. Using itwe can manipulate ifs as functions, and also build other functionality.
select allows to choose a result depending on truth value of statements,...
htools: remove obsolete option INodes
hspace no longer uses it, hence it can be removed.
hspace: also report the disk template
hspace: ask for the disk_template not number of nodes
This makes more sense, as the disk template is more meaningful thanthe number of nodes.
htools: add an utility function for disk templates
In the future we'll use this instead of requiring the number of nodesfrom the user.
htools: define the --disk-template option
Also sorts the export list correctly.
htools: read/save the disk template in Text backend
This requires that we change the signature of loadInst in order toproperly annotate the error messages, which in turn requires moreunittest changes.
Also, this invalidates yet again saved data files…...
htools: read the disk_template in the Luxi backend
We need to request the disk template in the Luxi query and then use itwhen parsing the response.
htools: read the disk template in Luxi and Rapi
Both these backends already have the data, we just need to use it.
htools: add DiskTemplate to instance definition
Currently always fixed to DRBD8, as that is what we assume already.
htools: add some unittests for Types.hs
htools: add a DiskTemplate type
This will be used later in the Instance type.
htools: small simplification to parseUtilisation
We eliminate the local variable as it's used only once.
htools: rename Instance.auto_balance
… to autoBalance, per the recommended style.
htools: simplify some JSON-related code
Instead of manually encoding the JSValue(s), we can use showJSON aslong as we use data types that can be auto-converted.
htools: allow different result types
This patch extends the previous patch changes to allow that eachCluster function returns a different result type, as long as we havean auxiliary function that processes that into the standardIAllocResult type.
htools: start reorganising the IAllocator workflow
Currently, all IAllocator requests return the same result type: a listof strings (usually nodes, but can also be instance:nodes). This won'tbe true in the future, with the new request types that will return...
htools: split more code from hail/main
The logic of request processing is not separated into its ownfunction, and (beside CLI interaction, e.g. verbosity handling) alllogic is now in IAllocator.hs.
htools: move code from hail.hs to IAllocator.hs
This will make it easier to unittest the code, and keeps all login in asingle module.
htools: add parsing of input node-evac messages
This is a lot of duplication with modeMreloc, which will get sortedout once we convert mreloc to change-group.
htools: remove pre-2.4 compatibility from RAPI
This was potentially hiding issues in the group query, so let's removeit since we're now bound to the current Ganeti version.
htools: introduce new data type for node-evac
The evac-mode goes into the Types.hs module since it needs to beavailable to Cluster.hs too.
htools: docstring fixes and improvements
No code changes (except one definition being moved around in QC.hs)are contained in this patch.
Fix some issues in htools apidoc generation
First, auto-generated files were not processed if HsColour thus the'source' link in Haddock documentation was dangling.
Second, we document the original Python constant name in theGaneti.Constants source, so that it's easily retrievable (as opposed...
Replace an mkdir -p with mkdir_p in Makefile.am
mkdir_p
Oops, this shouldn't have gone in in the first place…
Exclude auto-generated Haskell code from coverage
Especially for Ganeti.Constants, which has many declarations, itdoesn't make sense to check for coverage.
Update multi-reloc design doc with enhanced results
In order to increase the amount of information returned from theIAllocator plugin, we enhance the return values from the node-evac andchange-group operations.
gnt-cluster {command|copyfile}: Support per-group operations
This patch allows commands to be run on and files to be copied to allnodes within a specific group.
cli.GetOnlineNodes: Support node group filter, use query2
This patc changes cli.GetOnlineNodes to use query2, which does thefiltering in the master daemon, and adds a new parameter to filter bynode group.
Unittests were added for the old implementation and then adopted to...
Remove references to multi-relocate from iallocator doc
For some reason these were still in tere.
ht.WithDesc: Work around pylint warning
Explicitely defining “__call__” silences a pylint warning when wrappedtype check functions are used directly. I had no idea pylint is thisintelligent.
Merge branch 'devel-2.4'
ht: Add new check for numbers
Places which receive floats can usually also deal with integers, e.g.OpTestDelay. Tests are added and the new check function is used for theaforementioned opcode and verifying query results.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Fix off-by-one bug in job serial generation
Commit 009e73d0 (September 2009) changed the job queue to generatemultiple job serials at once. Ever since it would return one more thanrequested.
The “serial” file in the job queue directory is defined to contain the...
Reverts the patch series about console wrappers
This reverts commits 030a9cb8022b83bf43ec14dfbafd943299bc01c4 andae082df0000a785b693b2f4aa434650a81a94bdf.
There are two problems:
- Makefile.am breakage, which is trivial to revert- unittest breakage, which honestly I'm not sure how to fix and how...
Add gnt-instance start --pause
Creates the instance, but pauses execution before booting. This combinedwith 'gnt-instance console' unpausing instances means that the entireboot process can be viewed and monitored.
Signed-off-by: Stephen Shirley <diamond@google.com>...
Fix lint error
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
RAPI: Document all feature strings
- Use constants and an assertion- Update documentation for node migration
Remove old node evacuation opcode
LUNodeEvacStrategy has been replaced with LUNodeEvacuate.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Change RAPI for new node evacuation opcode
The change is not backwards compatible, see the updated NEWS file.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Change “gnt-node evacuate” to use new opcode
By default it'll now evacuate all instances from the node, notjust secondaries.
Add new opcode to evacuate node
This new opcode will replace LUNodeEvacStrategy, which used to return alist of instances and new secondary nodes. With the new opcode theiallocator (if available) is tasked to generate the necessary operationsin the form of opcodes. This moves some logic from the client to the...
Alias gnt-job show to gnt-job info
Am I the only one to make that mistake 10 times a week?
One Haskell and integer sizes fix
Haskell has two main integer types:
- Int, which is a native-type, and is guaranteed to have at least [-2²⁹, 2²⁹-1] range; on 64-bit platforms, it has much higher range- Integer, which is a software type (implemented using libgmp), and...
Fix cluster verify for empty node groups
There were some implicit assertions in the code that all node groupshave nodes, which is not necessarily true.
Additionally, the patch does a wrapping change.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Fix a typo and punctuation in iallocator.rst
Beside the 'dscription' typo, also make the punctuation moreconsistent.
Shorten some unbreakable lines in man pages
In order to make the display right on 80-columns terminals.
Correct some spelling mistakes
New lintian is even smarter:
- overriden → overridden- allows to → allows one to
Fix htools, QuickCheck library detection and tests
Just saw this while testing the migration to QuickCheck v2: whileconfigure.ac detects that QuickCheck-2.x is not available, the test inMakefile.am was against WANT_HTOOLS (overall htools compilation), not...
htools: fix prop_Node_rMem corner case
This patch fixes a bug in the test specification where we allowed nodeswith zero free memory (hence no instance can be added, at all) and addsa simple labeling of the way this test can fail.
htools: fix corner case in prop_Text_Load_Instance
This unittest had a corner case where it could fail if the sameprimary/secondary node names were generated.
htools: allow unittest to be replayed
This just adds glue to allow replaying of tests using a given RNG stateand test size (both are needed for exact replayability).
htools: improve more unittests
Using new functionality in QuickCheck 2 (the suchThat function), wegenerate now better test cases, such that (heh) we have no longerincomplete tests.
htools: more fixes to unsatisfiable tests
Currently the way we generate nodes in some cases is by creating atotally random node, then restricting the test based on whether the node'size' (as defined by multiples of base unit) satifies some high/lowrules. This results in hard-to-satisfy conditions, so we change this...
htools: Rework some unittests
The new scaffolding which replaced the batch driver of QuickCheck 1 nowshows how many passes we have for incomplete tests. Some tests show verylow pass counts, so we rework them to have more actually valid testcases.
htools: Switch to QuickCheck 2.x
Since current distros don't package anymore QuickCheck 1.x, let's moveto 2.x.
This requires also a few changes to the code:
- Test.QuickCheck.Batch doesn't exist anymore, so we need to write some scaffolding code to replace it...
htools: some lint fixes
Removal of duplicate parantheses, removal of extra 'do', conversion fromnested if to guards, use hierarchical imports. All per hlint.
htools: rewrite Cluster.filterMGResults
filterMGResults was built using a sequence of map and filter calls;while this was logically correct, it used some incomplete patternmatching which with the new GHC 7 triggers a warning.
The patch rewrites it using a single foldl that does both the filtering...
htools: use the new Group.isAllocable
… instead of the hardcoded test against AllocUnallocable.
htools: add a helper function
… that checks if a group is allocable.
Fix bug in recreate-disks for DRBD instances
The new functionality in 2.4.2 for recreate-disks to change nodes isbroken for DRBD instances: it simply changes the nodes without caringfor the DRBD minors mapping, which will lead to conflicts in non-empty...
Fix a lint warning
Patch db8e5f1c removed the use of feedback_fn, hence pylint warnnow.
KVM: configure bridged NICs at migration start
Commit 5d9bfd870 moved tap interface handling from KVM to Ganeti, partlyto also solve the problem of routed interfaces getting configured tooearly during live migrations, causing network anomalies. In that...
Fix RAPI documentation regarding master role
Fix bug in drbd8 replace disks on current nodes
Currently the drbd8 replace-disks on the same node (i.e. -p or -s) hasa bug in that it does modify the instance disk temporarily beforechanging it back to the same value. However, we don't need to, andshouldn't do that: what this operation do is simply change the LVM...
Enable using ghc parallel version 3
Currently htools cannot be compiled under sid because the parallelhaskell library is version 3. Using it issues a few warning, butcompiles and passes unit tests. Ship it?
Conflicts: lib/cmdlib.py - use RequireSharedFileStorage there
remove bootstrap._InitSharedFileStorage
This function is a copy of bootstrap._InitFileStorage with the followingdifferences: - check constants.ENABLE_SHARED_FILE_STORAGE and not constants.ENABLE_FILE_STORAGE - use different local variable names - one different error string...
LUInstanceCreate: use opcodes.RequireFileStorage
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>