History | View | Annotate | Download (64.2 kB)
Add a helper function for mirrorType computations
We most always use `templateMirrorType . diskTemplate`, so let's add ahelper function for this.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Enable evacuation/relocation for shared storage
This patch toggles the final bit and enables nodeEvacInstance to workon shared storage instances. Diskless instances are handled the same(as in, None is the same everywhere :).
Signed-off-by: Iustin Pop <iustin@google.com>...
Replace explicit case with 'either'
Instead of manually case-ing on the Either contents, let's just useeither with const functions (not really readable, but…).
Rename evacDrbdSecondaryInner to evacOneNodeInner
The function is now generic, so let's rename it and cleanup/improvethe docstrings.
Abstract the secondary evac into a separate function
This is generic enough (no longer depending on the replace type) thatit will be used for other replace modes too. So all we need to do isto abstract it to a separate function (that can be called from...
Extend the possible move types with FailoverToAny
This brings together all previous pieces and allows balancing to work(with the known caveats related to disk space calculation) for sharedstorage disk templates.
Add a simple type alias for nicer type signatures
Rework evacDrbdSecondaryInner to take the imove
Instead of hard-coding the instance move operation, let's accept is asa parameter from the caller. This is the single thing that makes thisDRBD-specific, so removing it will help for futurepatches. Furthermore, this abstract the imove op type into a single...
Use the mirror type in Cluster.possibleMoves
Currently the function possibleMoves always presumes DtDrbd, which iswrong for shared storage. To make it work with all disk templates, weadditionally pass the mirror type to it (we assume that the movebehaviour only depends on the mirror type, and not the individual...
Fix printSolutionLine to handle big scores
Currently we don't set a boundary on the length of the score field,which breaks all our nicely-aligned output efforts.
Fix printSolutionLine to handle non-DRBD steps
Currently Cluster.printSolutionLine function always wants to print theold and new secondary, and as such it fails (ugly) for moves ofnon-DRBD instances. Change it so that it looks explicitly fornoSecondary nodes, and format the output different in that case....
Small simplification in tryBalance
Instead of computing the bad/good node lists separately (in effectiterating twice over the all_nodes list and also creating that listtwice from the container), let's use 'partition' and do a smallimprovement.
htools: Adding printTable helper
This helper abstract away all the unlines and unwords call you've to doafter you got a formatTable back.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Use the spindles metric in cluster scores
This makes balancing use the new metric.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Shorten some function names
Since we use the functions always module-qualified, let's drop the`instance` prefix and use a shorter one.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
hinfo: Adding basic skeleton based on hbal
It prints the information which hbal prints in a nice formatted manner and ofcourse without doing any actions. An example output looks like this:
$ hinfo -m xen.example.comLoaded 4 nodes, 2 instancesCluster has 1 node group(s)...
htools: allow rbd disk template
Notes: - it'd be nice if there was a way to automatically generate the DiskTemplate list instead of manually specifying it. After all we have C.diskTemplates - Of course this actually does nothing, as for other...
Re-indent code for readability
Otherwise the docstrings for members take too many lines.
Track 'normalised' used CPUs in Cluster.CStats
This will be used in hspace for proper tracking of used/pool/unavailnpus.
Switch over some hspace stats from RSpec to AllocInfo
This patch changes the allocation delta stats from RSpec toAllocInfo. There's nothing interesting about it, but it should allowmore correct computations of Npus in hspace.
I also moved the AllocStats type alias from Cluster.hs to Types.hs...
Extend node evacuation unittests
Currently they only check for change secondary, via iallocator'srelocate method. We add two separate unittests that checks all thenode evacuate modes (ChangeAll, ChangePrimary) and also the groupchange request. All of these are done only for DRBD instances....
Remove some dead code
While wondering why Cluster.tryReloc is not unit-tested, I realisedthat this is no longer uses. Apparently, last summer's new IAllocatormodes and related changes have silently deprecated this function.
Add a complex allocation/serialisation/load test
This is unfortunately a complex test that checks if:
- starting from a empty cluster- allocating a few instances on it- serialising it via the Text backend- loading it back into internal data structures...
Obey instance policy in allocation routines
This currently checks the minimum/maximum sizes and the disktemplate. Unittests will come later.
Trivial performance optimisation
This is not in the hot-path, but still, no sense in doing O(n) insteadof O(1).
Cleanup hlint errors
First, we update the recommended hlint version to what I used to get aclean output (1.8.15). Most of the changes are:
- remove unneeded parentheses- some simplifications (intercalate " " → unwords, maybe … id → fromMaybe, etc.)- removal of some duplicate code (in previous patches)...
Abstract some common Cluster.hs code into a function
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
htools: finish re-indenting Cluster.hs
Start converting the indentation to 2-chars
… also do some other small style cleanups.
htools: remove 'running' from instance options
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Generalize the generation of ADT from raw types
Further cleanup in hspace
This moves the checking of results from the allocation functions to aseparate function, so that we have less code duplication. It also doesa bit of simplification in the printing functions.
Add a type synonym for the allocation function sig
Both iterateAlloc and tieredAlloc share the same signature, but it'snot documented nor exported (needed for refactoring).
Merge branch 'devel-2.5'
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Merge branch 'stable-2.5' into devel-2.5
hail: Fix result for node evacuation
According to the iallocator documentation the “node-evacuate” call needsto return a list of jobs, not a list of lists of jobs.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Rename some helper functions for consistency
This changes the names for some helper functions so that futurepatches are touching less unrelated code. The change replacesshortened prefixes with the full type name.
Small simplification in tryAlloc
Change how node pairs are generated/used
Currently, the node pairs used for allocation are a simple [(primary,secondary)] list of tuples, as this is how they were used before theprevious patch. However, for that patch, we use them separately perprimary node, and we have to unpack this list right after generation....
Parallelise instance allocation/capacity computation
This patch finally enables parallelisation in instance placement.
My original try for enabling this didn't work well, but it took awhile (and liberal use of threadscope) to understand why. The attempt...
Abstract comparison of AllocElements
This is moved outside of the concatAllocs as it will be needed inanother place in the future.
Change type of Cluster.AllocSolution
Originally, this data type was used both by instance allocation (1result), and by instance relocation (many results, one perinstance). As such, the field 'asSolutions' was a list, and thevarious code paths checked whether the length of the list matches the...
htools: remove dead code
The tryEvac/evacuateInstance functions are no longer used in the newmulti-group world order, so we remove them and change the unit-test totest the actual IAllocator function.
hail: don't select the primary as new secondary
This just adds the primary node of the instance as 'non-allocable'during the choosing of the new secondary.
htools: rework the algorithm for ChangeAll mode
I think I've identified the problem with the current ChangeAllmode. The current algorithm works as follows:
- identify a new primary by choosing the node which gives best score as new secondary- failover to it...
htools: replace two hardcoded uses of pri+sec nodes
These two cases use explicit uses of primary and secondary nodes withInstance.allNodes, which means the code is more flexible if theinternal layout of the instance changes.
I've verified that the output of involvedNodes is not required to be...
htools: add target_node member to migrate opcode
… and failover too. Not many changes otherwise except forserialisation and unittests.
htools: a few more hlint fixes
Tested only on GHC 7.x, will test on 6.1x too before commit.
htools: further docstring fixes
This adds parameter documentation for Cluster.iMoveToJob (I think itwas not clear if the new or old node list is needed) and fixes otherdocstring style issues.
After this patch, all modules except for CLI.hs (which has many...
htools: move tiered spec map helper to Hspace.hs
This is used just in hspace, so let's help in making Cluster.hssmaller. We also split the function in two, as computing the spec mapand formatting it are two different tasks.
htools: add a compatibility module
When compiling with the parallel-3.x library, we get a deprecationwarning, which makes understanding any other error messagesharder. This patch adds a compatibility module that will hold suchcode for transitioning libraries....
htools: remove no-longer-needed tryMG* functionality
… which was deprecated by the previous patch.
htools: fix potential bug in ialloc/change-group
Currently, the ChangeAll mode of nodeEvac computes the primary groupof the instance and then uses the resulting group index for computingthe group score. However, during the change-group operation (which...
htools: return new state from new IAllocator modes
The old modes already return the node list (as part of AllocSolution),this patch makes the new modes provide this new information.
htools: update the node-evac/change-group result
… per the new design changes. This also requires changes toupdateEvacSolution, as passing in the original instance is nothelpful; we just pass the instance index now and look up the actualinstance in the modified instance map....
htools: add change-group IAllocator mode
This builds a lot upon the node-evacuation mode, and itsimplementation is therefore quite easy (once node-evac was added).
htools: add group restrictions to findBestAllocGroup
While a normal multi-group allocation doesn't need to restrict thelist of target groups beyond the unallocable status of some groups,when we relocate instances from one group to the other we need tolimit the subset of searched groups....
htools: generalise and rename availableLocalNodes
Currently this function works in the context of an instance, computingits local group nodes. We can use generalise it to take a group indexargument, so that it's not bound to an instance, and we rename it...
htools: split tryMGAlloc in two
Currently, this function does both identify the best group for aninstance, and format the results (partially). The former functionalityis useful in other contexts too, so let's split the function in two.
htools: add node-evacuation of DRBD all nodes
This is much more complicated than either change primary (which istrivial) and change secondary (which is also simple).
The main strategy is:
- choose a new secondary (that will become the final primary) and...
htools: small optimisation in Cluster.collapseFailures
This makes fewer temporary data structures, and thus brings a bit ofimprovement in total runtime.
htools: add optional limit to allocation count
Currently, the tieredAlloc/iterateAlloc functions will not returnuntil the allocation fails; this means unit-testing theirfunctionality (e.g. that an instance can be allocated) is slow, sincethey will allocate all possible instances....
htools: add node-evacuation of DRBD secondaries
htools: rework compCV so that we can use [Node] too
In some cases, we have a [Node.Node] list, instead of a Node.Liststructure. Instead of converting [Node.Node] to the latter, let'srework compCV and add a compCVNodes such that we skip this(potentially expensive) step....
htools: abstract and simplify a multigroup function
In the future, this 'split/associated instance idx list' functionalitywill be used by other code, so let's move it to its own code.
The new method of computing the split also requires exporting one more...
htools: rename an internal constant
'emptySolution' is fine while we have only one Solution type(AllocSolution), but in the future we'll introduce another one andthus we should rename it.
htools: add a function for computing evacuated nodes
The new IAllocator interface is based on instances, not nodes, so weneed to backtrack and compute on which nodes we can't allocateinstances during the current operation. This patch adds a function for...
htools: add more IAlloc data types and functions
These move the 'not supported' message deeper into the workflow;assuming that the workflow as added by this patch is correct, all thatremains is:
- for disk template DRBD8, plug in the currently existing mechanisms...
htools: add node-evacuate of DRBD8 primaries
htools: rename Instance.auto_balance
… to autoBalance, per the recommended style.
htools: docstring fixes and improvements
No code changes (except one definition being moved around in QC.hs)are contained in this patch.
One Haskell and integer sizes fix
Haskell has two main integer types:
- Int, which is a native-type, and is guaranteed to have at least [-2²⁹, 2²⁹-1] range; on 64-bit platforms, it has much higher range- Integer, which is a software type (implemented using libgmp), and...
htools: rewrite Cluster.filterMGResults
filterMGResults was built using a sequence of map and filter calls;while this was logically correct, it used some incomplete patternmatching which with the new GHC 7 triggers a warning.
The patch rewrites it using a single foldl that does both the filtering...
htools: use the new Group.isAllocable
… instead of the hardcoded test against AllocUnallocable.
Implement balancing with no instance moves
Note that --no-disk-moves and --no-instance-moves are not incompatible,but if both are used no solution can possibly exist.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Show the auto_balance flag in the instance listing
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Adeodato Simo <dato@google.com>
htools: Make opcode naming consistent with Ganeti codebase
This patch just cleans up the htools codebase to make it more consistentwith the naming of the Ganeti codebase.
htools: Use OpMigrateInstance with allow_failover option
Before hbal decided on the fly if an instance is migratable or not. Aswe implemented failover fallback in commit d5cafd31456 we can start touse that.
Signed-off-by: René Nussbaumer <rn@google.com>...
Fix some lint warnings in htools code
hlint gives more suggestions, but some make the code (IMHO) harder toread.
Fix lint errors in the htools code
These are just changes from hlint suggestions. Still compiles andpasses unittests.
Merge remote branch 'htools/master'