htools: start reorganising the IAllocator workflow
Currently, all IAllocator requests return the same result type: a listof strings (usually nodes, but can also be instance:nodes). This won'tbe true in the future, with the new request types that will return...
htools: split more code from hail/main
The logic of request processing is not separated into its ownfunction, and (beside CLI interaction, e.g. verbosity handling) alllogic is now in IAllocator.hs.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
htools: move code from hail.hs to IAllocator.hs
This will make it easier to unittest the code, and keeps all login in asingle module.
htools: add parsing of input node-evac messages
This is a lot of duplication with modeMreloc, which will get sortedout once we convert mreloc to change-group.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
htools: remove pre-2.4 compatibility from RAPI
This was potentially hiding issues in the group query, so let's removeit since we're now bound to the current Ganeti version.
htools: introduce new data type for node-evac
The evac-mode goes into the Types.hs module since it needs to beavailable to Cluster.hs too.
htools: docstring fixes and improvements
No code changes (except one definition being moved around in QC.hs)are contained in this patch.
Fix some issues in htools apidoc generation
First, auto-generated files were not processed if HsColour thus the'source' link in Haddock documentation was dangling.
Second, we document the original Python constant name in theGaneti.Constants source, so that it's easily retrievable (as opposed...
Replace an mkdir -p with mkdir_p in Makefile.am
mkdir_p
Oops, this shouldn't have gone in in the first place…
Exclude auto-generated Haskell code from coverage
Especially for Ganeti.Constants, which has many declarations, itdoesn't make sense to check for coverage.
Update multi-reloc design doc with enhanced results
In order to increase the amount of information returned from theIAllocator plugin, we enhance the return values from the node-evac andchange-group operations.
Signed-off-by: Iustin Pop <iustin@google.com>...
gnt-cluster {command|copyfile}: Support per-group operations
This patch allows commands to be run on and files to be copied to allnodes within a specific group.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
cli.GetOnlineNodes: Support node group filter, use query2
This patc changes cli.GetOnlineNodes to use query2, which does thefiltering in the master daemon, and adds a new parameter to filter bynode group.
Unittests were added for the old implementation and then adopted to...
Remove references to multi-relocate from iallocator doc
For some reason these were still in tere.
ht.WithDesc: Work around pylint warning
Explicitely defining “__call__” silences a pylint warning when wrappedtype check functions are used directly. I had no idea pylint is thisintelligent.
Merge branch 'devel-2.4'
ht: Add new check for numbers
Places which receive floats can usually also deal with integers, e.g.OpTestDelay. Tests are added and the new check function is used for theaforementioned opcode and verifying query results.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Fix off-by-one bug in job serial generation
Commit 009e73d0 (September 2009) changed the job queue to generatemultiple job serials at once. Ever since it would return one more thanrequested.
The “serial” file in the job queue directory is defined to contain the...
Reverts the patch series about console wrappers
This reverts commits 030a9cb8022b83bf43ec14dfbafd943299bc01c4 andae082df0000a785b693b2f4aa434650a81a94bdf.
There are two problems:
- Makefile.am breakage, which is trivial to revert- unittest breakage, which honestly I'm not sure how to fix and how...
Add gnt-instance start --pause
Creates the instance, but pauses execution before booting. This combinedwith 'gnt-instance console' unpausing instances means that the entireboot process can be viewed and monitored.
Signed-off-by: Stephen Shirley <diamond@google.com>...
Adding a wrapper around connecting to kvm console
The wrapper will connect to the console, and check in the background ifthe instance is paused, unpausing it as necessary.
Signed-off-by: Stephen Shirley <diamond@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Adding a wrapper around "xm console"
Fix lint error
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
RAPI: Document all feature strings
- Use constants and an assertion- Update documentation for node migration
Remove old node evacuation opcode
LUNodeEvacStrategy has been replaced with LUNodeEvacuate.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Change RAPI for new node evacuation opcode
The change is not backwards compatible, see the updated NEWS file.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Change “gnt-node evacuate” to use new opcode
By default it'll now evacuate all instances from the node, notjust secondaries.
Add new opcode to evacuate node
This new opcode will replace LUNodeEvacStrategy, which used to return alist of instances and new secondary nodes. With the new opcode theiallocator (if available) is tasked to generate the necessary operationsin the form of opcodes. This moves some logic from the client to the...
Alias gnt-job show to gnt-job info
Am I the only one to make that mistake 10 times a week?
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
One Haskell and integer sizes fix
Haskell has two main integer types:
- Int, which is a native-type, and is guaranteed to have at least [-2²⁹, 2²⁹-1] range; on 64-bit platforms, it has much higher range- Integer, which is a software type (implemented using libgmp), and...
Fix cluster verify for empty node groups
There were some implicit assertions in the code that all node groupshave nodes, which is not necessarily true.
Additionally, the patch does a wrapping change.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Fix a typo and punctuation in iallocator.rst
Beside the 'dscription' typo, also make the punctuation moreconsistent.
Shorten some unbreakable lines in man pages
In order to make the display right on 80-columns terminals.
Correct some spelling mistakes
New lintian is even smarter:
- overriden → overridden- allows to → allows one to
Fix htools, QuickCheck library detection and tests
Just saw this while testing the migration to QuickCheck v2: whileconfigure.ac detects that QuickCheck-2.x is not available, the test inMakefile.am was against WANT_HTOOLS (overall htools compilation), not...
htools: Rework some unittests
The new scaffolding which replaced the batch driver of QuickCheck 1 nowshows how many passes we have for incomplete tests. Some tests show verylow pass counts, so we rework them to have more actually valid testcases.
htools: more fixes to unsatisfiable tests
Currently the way we generate nodes in some cases is by creating atotally random node, then restricting the test based on whether the node'size' (as defined by multiples of base unit) satifies some high/lowrules. This results in hard-to-satisfy conditions, so we change this...
htools: improve more unittests
Using new functionality in QuickCheck 2 (the suchThat function), wegenerate now better test cases, such that (heh) we have no longerincomplete tests.
htools: allow unittest to be replayed
This just adds glue to allow replaying of tests using a given RNG stateand test size (both are needed for exact replayability).
htools: fix corner case in prop_Text_Load_Instance
This unittest had a corner case where it could fail if the sameprimary/secondary node names were generated.
htools: fix prop_Node_rMem corner case
This patch fixes a bug in the test specification where we allowed nodeswith zero free memory (hence no instance can be added, at all) and addsa simple labeling of the way this test can fail.
htools: Switch to QuickCheck 2.x
Since current distros don't package anymore QuickCheck 1.x, let's moveto 2.x.
This requires also a few changes to the code:
- Test.QuickCheck.Batch doesn't exist anymore, so we need to write some scaffolding code to replace it...
htools: some lint fixes
Removal of duplicate parantheses, removal of extra 'do', conversion fromnested if to guards, use hierarchical imports. All per hlint.
htools: rewrite Cluster.filterMGResults
filterMGResults was built using a sequence of map and filter calls;while this was logically correct, it used some incomplete patternmatching which with the new GHC 7 triggers a warning.
The patch rewrites it using a single foldl that does both the filtering...
htools: use the new Group.isAllocable
… instead of the hardcoded test against AllocUnallocable.
htools: add a helper function
… that checks if a group is allocable.
Fix a lint warning
Patch db8e5f1c removed the use of feedback_fn, hence pylint warnnow.
Fix bug in recreate-disks for DRBD instances
The new functionality in 2.4.2 for recreate-disks to change nodes isbroken for DRBD instances: it simply changes the nodes without caringfor the DRBD minors mapping, which will lead to conflicts in non-empty...
KVM: configure bridged NICs at migration start
Commit 5d9bfd870 moved tap interface handling from KVM to Ganeti, partlyto also solve the problem of routed interfaces getting configured tooearly during live migrations, causing network anomalies. In that...
Fix RAPI documentation regarding master role
Fix bug in drbd8 replace disks on current nodes
Currently the drbd8 replace-disks on the same node (i.e. -p or -s) hasa bug in that it does modify the instance disk temporarily beforechanging it back to the same value. However, we don't need to, andshouldn't do that: what this operation do is simply change the LVM...
Enable using ghc parallel version 3
Currently htools cannot be compiled under sid because the parallelhaskell library is version 3. Using it issues a few warning, butcompiles and passes unit tests. Ship it?
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Conflicts: lib/cmdlib.py - use RequireSharedFileStorage there
remove bootstrap._InitSharedFileStorage
This function is a copy of bootstrap._InitFileStorage with the followingdifferences: - check constants.ENABLE_SHARED_FILE_STORAGE and not constants.ENABLE_FILE_STORAGE - use different local variable names - one different error string...
LUInstanceCreate: use opcodes.RequireFileStorage
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Don't add ",boot=on" to disks on kvm >= 0.14
Under newer kvm this prevents the vm from starting.Ah, change!
KVM: fix per-instance stored UID value
When using the pool security model, ExecuteKVMRuntime was storing theinstance's UID using str(uid), which would result in storing theLockedUid._repr__() result:
$ cat /var/run/ganeti/kvm-hypervisor/uid/xxxxxxxxxxxxx...
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Add one forgotten element to the file disk path
This was left out during the fix/refactoring
Conflicts: lib/cmdlib.py - use constants.DTS_FILEBASED...
Add DTS_FILEBASED constant
LUInstanceCreate: fix file storage dir calculation
- Move the calculation at the beginning of CheckPrereq, since it doesn't modify any state, but still keeps locks- Only perform the calculation if the actual disk template is filebased- Error out if there is no defined file storage dir...
Check that filestorage is enabled when requested
Remove self.op.file_storage_dir isabs check
As the manpage says, and the code does, self.op.file_storage_dir is anadditional relative path under the cluster file storage dir. As such itshould not be absolute.
--select-instances hbal manpage update
htools live test: test instance selection as well
Check that the selected instances are known
Loader.updateMovable: evaluate selected instances
This also adds docstrings for the function arguments and renames exinstto exinsts, which is how it is called in other functions, since it's alist.
Add instance selection list to Loader.mergeData
Add --select-instances hbal flag
Remove double whitespace in help string
Add gnt-network design doc
This design covers high level network block definition and poolmanagement.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>...
Replace iallocator's mreloc w/ change-group and node-evac
This patch removes all occurrences of the “multi-relocate” iallocatormode. Commit 25ee7fd845 updated the design document and introducedseparate modes, “change-group” and “node-evacuate”. The constants aren't...
Fix a couple of typos
Signed-off-by: Stephen Shirley <diamond@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Makefile: Add version check for iallocator.rst
iallocator.rst contains the Ganeti version at the top.
Update iallocator design for node group-aware operations
A while ago a new ``multi-relocate`` mode was proposed and documented.As it turned out, the interface had some deficiencies. With this patchThe relocation modes are reduced to two and split into separate...
jqueue: Allow loading of archived jobs
Chained jobs need to look at previous jobs, including archived ones. Anice side-effect of this change is the ability to look at archived jobsusing “gnt-job info <id>” as long as the ID is known.
Adding basic abstraction layer for caching
This includes an own simple cache implementation and aninterface to a memcache instance.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix _checkRsaPrivateKey for newer key generation
Keys generated under debian sid just read "BEGIN PRIVATE KEY" ratherthan "BEGIN RSA PRIVATE KEY".
Fix locking issues in LUClusterVerifyGroup
- Use functions in ConfigWriter instead of custom loops- Calculate nodes only once instances locks are acquired, removes one potential race condition- Don't retrieve lists of all node/instance information without locks...
cmdlib: Acquire BGL for LUClusterVerifyConfig
LUClusterVerifyConfig verifies a number of configuration settings. Fordoing so, it needs a consistent list of nodes, groups and instances. Sofar no locks were acquired at all (except for the BGL in shared mode)....
Export/import instance tags
Fix issue with tags on instance creation
Commit 720f56c85a added the ability to specify tags when creating aninstance. The “tags” attribute of an instance object needs to be a set,but the patch's code saved it as a list, causing breakage in other parts...
Fix incomplete merge
Commit 66bd7445 changed the semantics of _JobProcessor on finishedjobs, and updated the related unittests in the 2.4 branch. It was thenmerged to master, however on master there was an additional test forthis case, which was not updated....
Add tag handling to {Op,LU}InstanceCreate
Add a tag slot to opcodes.OpInstanceCreate. We do not reuse _PTags, as this isintended for OpTagsSet and thus:
a) is not documented b) does not carry a default value, making it mandatory
Also pass the tags to the iallocator during instance creation....
Add tagging option to gnt-instance create
Add TAG_ADD_OPT option to cli.py and use it in gnt-instance. Modifycli.GenericInstanceCreate() accordingly.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>Signed-off-by: Iustin Pop <iustin@google.com>...
Export instance tags to instance hooks
Instance hooks now get an INSTANCE_TAGS environment variable, which contains aspace-delimited list of the affected instance's tags.
Also update the documentation to reflect the change.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>...
http.client: Make debug log less noisy
The HTTP client code generates quite a lot of debug log messages. Withthis patch they're hidden unless explicitely enabled in the code.
htools: introduce a type alias for JSON objects
This makes the type definitions a bit more readable/simpler.
hail: stop using old-style 'nodes' key
hail: add parsing of multi-relocate request
This is not handled yet, this patch just adds parsing of the incomingrequest.
hail: add option for displaying the parsed request
This can be used for debugging.
hail: add new data types for the multi-reloc mode
Add --no-instance-moves to the htools live tests
Update hbal manpage for --no-instance-moves
Implement balancing with no instance moves
Note that --no-disk-moves and --no-instance-moves are not incompatible,but if both are used no solution can possibly exist.
Pass the instance moves option in hbal
While still being ignored, now it gets passed down to the iterationfunction.
Add --no-instance-moves cli htools option
This option doesn't currently do anything.
jqueue: Fix potential race condition when cancelling queued jobs
When a job was cancelled, its status would be changed and the filewritten again. Since this was a final status, the job file could bemoved anytime for archival. If the job was still in the queue, however,...