Implementation of InstanceList call and result
instance_list call takes a list of hypervisors and returns a listof running instances.
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Implementation of AllInstancesInfo call and result
all_instances_info call takes a list of hypervisors and returns a listof (name, memory, state, vcpus, time) - one element for eachinstance.
Infrastructure to execute RPC calls
Prepare and execute RPC call, prepare result for the call. Theseprocedures are generic in the sense that they only require theCall and Result types to be connected.We use curl library for sending http requests for RPC; as the library's...
Datatypes for haskell RPC calls
We introduce typeclasses for RPC call and result and create a typeclassthat binds the two together. For that we need to useMultiParamTypeClasses and FunctionalDependencies language pragmas, whichallow us to ensure that RPC result type can be deduced based on the...
Add Hypervisor and start exporting AdminState
Very simple Hypervisor object, that we want to pass in some of RPCcalls is added. We also export AdminState data type, as it is usedin one of the calls that will be implemented in this patch series.
Signed-off-by: Agata Murawska <agatamurawska@google.com>...
Move necessary constants from rpc & rpc_defs to constants
As timeout constants will now be used also on Haskell side, they havebeen moved from rpc_defs and rpc to constants. The same is true forconnection timeout for connecting to nodes.
Merge branch 'devel-2.6'
Merge branch 'stable-2.6' into devel-2.6
Make stable-2.6 compatible with newer pep8
This is done so that all current branches can run with newer pep8;note that instead of fixing the problems (like I did on master), I'vejust silenced more. These should not be merged onto master!
Signed-off-by: Iustin Pop <iustin@google.com>...
More descriptive Constants.hs header
Bump pep8 version to 1.2
Debian Wheezy will ship with this version, and it has many improved checks compared to 0.6, so let's:
- bump version in the docs- silence some new checks that are wrong due to our indent=2 instead of 4- fix lots of errors in the code where the indentation was wrong by 1...
Change node parameter oob_program to VTYPE_STRING
Since this is an inheritable parameter, having it as a MABYE_STRINGcreates only problems (per our derivation rules). We change it toSTRING, with the default "", meaning no program. Note that most of the...
Fix computation of disk sizes in _ComputeDiskSize
Currently, hail fails with FailDisk when trying to add an instanceof type: 'file', 'sharedfile' and 'rbd'.
This is due to a "0" or None value in the corresponding dict inside_ComputeDiskSize, which results in a "O" or non Int value of the...
Add verification of RPC results in _WipeDisks
Due to an oversight, the pause/resume sync RPC calls in _WipeDiskslack the verification of the overall RPC status, and directly iterateover the payload. The code actually doing the wipe does verifycorrectly the results. This can result in jobs failing with a hard to...
Adding design doc for bulk instance create
This is the design doc for the bulk instance creation. You can moredetails in the doc itself.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Add a server-side Luxi implementation
This is a trivial code change, but it allows us to finally test thesend-receive code on both client and server sides via a simplein-process server.
The unittest works, but it won't handle timeouts very nicely; it will...
Switch the Luxi interface from Strings to ByteStrings
I'm doing this change for future performance optimisations. Currentlywe use the Luxi interface just as a client, so not in the hot path,but when we'll use this as a server interface, we're interested to...
Add test for checking that all gnt-* subcommands run OK
This is a bit of a shell munging trickery, but works for now. Makingit more generic can be done later.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Fix double use of PRIORITY_OPT in gnt-node migrate
This breaks the command, as optparse considers that an error.
Add new Makefile target to rebuild the whole dist
Due to the fact how the automake system works it doesn't rebuild alreadyprebuild files in distcheck. This lead to a bug, where a rebuild of thedocumentation was failing because we missed the fact that the files were...
rapi client: accept arbitrary shutdown arguments
The "ignore_offline_nodes" parameter is unsupported. Rather thanexplicitely adding it, just pass all keyword arguments in the body ofthe query, and rapi on the other side will do the right thing.
Supports for old arguments that were passed via the query is unchanged....
Handle offline nodes for "instance down" checks
When offlining an instance because its primary node is down, we must beable to cope with the situation.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add missing rst files to Makefile.am
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Switch job IDs to numeric
This has been a long-standing cleanup item, which we've alwaysrefrained from doing due to the high estimated effort needed.
In reality, it turned out that after some infrastructure improvements(the previous patches), the actual job queue-related changes are quite...
Add infrastructure for building numeric namefield filters
Currently, all the CLI helpers in qlang.py and cli.py assume that allnamefields are string, which leads to various breakage in case the'reactually not.
To improve the flexibility of the helpers, we add a bit of...
Make jqueue unittests more generic
This patch removes/abstracts some hardcoded values in the jqueueunittests. Currently we have a per-resource type name field, but thename field values are hardcoded at the call sites, instead of beingabstracted into separate variables....
Expand debug messages in gnt-debug test-jobqueue
It wasn't clear for which tests we were ignoring the errors.
Fix uses of OpPrereqError without code info
A while back, we did cleanup the code and ensured (manually) that useof OpPrereqError includes an errors.ECODE_* field as secondargument. Since we cannot automate the check for this, it turns outthat more and more such usage has crept over the years, including in...
htools: abstract function for parsing job ids
Both the job id and submit job result parsing are abstracted intoseparate functions, so that later changes are more localised.
Also, this makes submitManyJobs itself easier to read.
Introduce a type for the ganeti job type
This will be used for easier change later.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Release version 2.6.0 (final)
Phew, it wasn't easy, but…
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Fix 'explicitely' common typo
It seems that 'explicitely' is wrong, and that the right form is'explicitly'. This is just fixing the typo plus adjusting affectedparagraphs.
Fix issue in LUClusterVerifyGroup with multi-group clusters
In case LUClusterVerifyGroup is run on a group which doesn't containthe master node, the following could happen:
- master node is selected due to the explicit check- if the order of nodes in the 'absent_nodes' list is such that the...
Add QA test for node group modification of ndparams
Additionally, we run hbal if the htools tests are enabled (newkey). Note that hbal will directly exit, since there are no instances,but at least it will load and parse the group details.
Fix node group modification of node parameters
Commit 904b3bfe tried to fix the deletion of custom ndparams fromgroup, but instead broke both modification and deletion: because werun ForceDictType on self.op.ndparams instead of the updatednew_ndparams, we can neither delete nor set properly spindle_count...
QA for allocator in recreate-disks
QA for recreate-disks called it twice with a list of node; the first callnow uses an allocator.
Signed-off-by: Adam Ingrassia <api@google.com>Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Front-end and doc to use allocator in recreate-disks
The recreate-disks command no longer requires an explicit list of nodes.
Allocator support in recreate-disks LU
This affects only the backend.
Added method to call an allocator for recreate-disks
Ideally this should have used relocation mode, but to due lacking ofsupport in current allocators, allocation mode has been used instead.
Signed-off-by: Adam Ingrassia <api@google.com>Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
Support for checking the group of only the primary node
_CheckInstanceNodeGroups behaves in the old way with the default valueof the new parameter.
Signed-off-by: Adam Ingrassia <api@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge commit 'v2.5.2' into stable-2.6
Conflicts: NEWS (trivial, merged the entries)...
Fix RST formatting in NEWS file
*FileID is interpreted as a start-of-bold, without corresponding endmarker. Sorry for not catching it in the first place.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
Update NEWS and bump version for release 2.5.2
This is a trivial release, with just bug fixes, so a small change log.
Fix boot=on flag for CDROMs
This generalises commit 4304964a to cdroms too, since they havesomewhat the same logic. We just abstract the needs_boot_flag into aseparate variable, and then reuse it in the cdrom section.
Note that the logic of what 'if=' type to pass to KVM was very...
KVM: only pass boot flag once
This addresses issue 230: passing two methods of booting to KVM can,depending on the KVM version, confuse it.
Note that commit 4304964a introduced a partial fix for this (but onlyfor disks, and keyed on KVM versions). However, it didn't fix cdrom...
Build epydoc.conf using standard replace_vars_sed
This is just begging to be converted to a standard replace_vars_sedrule, instead of custom sed calls.
Simplify some make rules
A rule of type "a/%: a/%.in" will also match "a/b/%: a/b/%.in", so noneed for the explicit examples/hooks rule. As for the man rules, theyare identical and thus can be collapsed.
We still have the problem that globally, not all our %.in to %...
Explicitly terminate some make rules
Generic rules like:
%: %.in
have the downside that the source (%.in) itself matches the target(via %.in: %.in.in). This leads to things like:
Looking for a rule with intermediate file `doc/examples/hooks/ipsec.in.in'....
Remove ancient implicit make rules
GNU Make contains some (ancient) implicit rules, that try toautomatically extract source files from RCS/SCCS version controlsystems. This is unneeded, and it pollutes the make -d outputsignificantly: after removing these rules (by defining empty targets...
Stop using BUILT_SOURCES
Commit dc7d2c49 introduced the use of BUILT_SOURCES to work aroundmissing dependencies. However, on closer reading of the gmake manual,BUILT_SOURCES is mainly used in cases where the dependencies are notknow before the build starts (e.g. with auto-generated C header...
Change how we create the 'ganeti' symlink
Currently, if one runs 'make' in an already fully-built tree, this isthe result:
cd . && test -h "ganeti" || { rm -f ganeti && ln -s lib ganeti; } make all-am make[1]: Entering directory `/tmp/test' cd . && test -h "ganeti" || { rm -f ganeti && ln -s lib ganeti; }...
Partial undo of "Makefile: Streamline directory creation"
Commit c964d962 changed the way we create directories, by two things:
- unifying all dependencies and ad-hoc directory creation into a single target (all_dirfiles)- changing how directories are created from a stamp file to .dir files...
A few style fixes in Makefile.am
Seen while debugging make rules.
Ensure a stable content of the bash completion file
Currently, the order of commands in the bash completion file israndom, because the sub-commands are not sorted. This makes it harderto investigate the differences in packaged Ganeti or in installedGaneti, since chunks in this file will have a random order....
Correctly do sub-$(MAKE) invocations
Reading the automake documentation, it seems that explicit calls to$(MAKE) need $(AM_MAKEFLAGS) passed, so let's do that.
Make error classes more consistent
This removes "pass" from a few definitions, making all errorsconsistent.
Add support for -cpu option in KVM
This adds supports for the (non-validated) cpu parameter to KVM.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add note in admin guide about drbd stacked devices
Fixes: http://code.google.com/p/ganeti/issues/detail?id=251Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'stable-2.6'
Re-indent kvm-ifup.in
By a strange accident, this file seems to be tabified, instead of ourown 2-space rule. Re-indent it and also break one long line.
Release Ganeti 2.6.0 rc4
Prepare NEWS for Ganeti 2.6.0rc4
Another release candidate is coming.
Add some rudimentary node group ipolicy checks
Fix setting ipolicy on node groups
On node groups we don't have the std field. However, the InstancePolicyobject always verifies that the std value is within a given range. As wefill it up with defaults if not set (as it happens to be on node groups)and the min value is higher than the default std value (taken from...
Force tap's MAC prefix to "fe"
Setting a high prefix discourages the bridge from adopting the tap'sMAC. Xen is not affected by this since the MAC is forced to"fe:ff:ff:ff:ff:ff".
This addresses issue #217.
Fix --no-headers for the new list-drbd command
Sorry, I forgot that with GenerateTable this needs to be handledmanually. Fixed now and tested in both ways.
(But to be honest, this should be abstracted in GenerateTable, insteadof the 'if' test in all its callers.)...
Add a simple QA test for gnt-node list-drbd
This will be run only if 'haskell-confd' QA configuration has beenenabled.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Add a new gnt-node command list-drbd
This uses confd to query the DRBD minors, which is very special; noother command currently does so.
Since the backend is only implemented in the Haskell version of confd,we have checks that 1) confd is enable, and 2) hs confd is also...
Implement a node to drbd minors query function
This can be queried remotely since it's a pure configuration query.
Add a new unused confd query
This is not implemented currently.
Add support for computing instance all/secondary nodes
This fixes an old FIXME. Since we now how actual DRBD configurationdata, we can finally compute the instance's secondary nodes.
Add disk logical ID support in Objects.hs
This is a complex field, so we have to do a lot of manual work for now.
The complexity arises from the fact that the contents of the field,and the way to parse it, depends on the disk type field, so we don'thave a single, static way of parsing it. Hence we needed the...
Fix a docstring in bdev's DRBD8 class
It seems this was not updated since the move to static minors…
Extend the Template Haskell loadFn model
Currently, we only allow field-by-field de-serialisation. Since wehave cases where information about how to un-serialise a field issplit across two JSON fields (e.g. disk type and disk logical_id,hypervisor and hvparams, etc.), we need to pass the entire object to...
Change how customFields are built
Instead of passing an expression (which cannot come from the currentmodule), we pass a name, which is allowed to reference functions fromthe module we're in. Since we currently don't have custom fields, wedon't need to modify any callers....
Implement lookup of partial names in Config.hs
This uses the recently-moved functions to implement partial lookup ofnames on getNode and getInstance, similar to the Python codebase.
Reorganise the lookup functions
Currently, the LookupResult, MatchPriority and related functions arelocate in Loader.hs, since (so far) only hbal needs them in theselection of instances. However, with the new functionality on confdside, we need these functions there too, but we don't want to import...
Remove an unused function
This is not used, as we need a more complex serialisation, which isdone in the saveObjectField function.
Ensure that disk.params is always defined (and a dict)
Commit cce46164 fixed upgrading from other 2.6 versions, butaccidentally broke upgrading from 2.5 (disk.params was left as None,which breaks FillDict). The simplest way to handle params is to always...
Another small consistency fix with if branches
While looking at the testability of this piece of code, I found anotherconsistency problem. We have two if branches instead of one, withelif's.
Fix inconsistency in the LUXI protocol w.r.t. args
This inconsistency was found during rebalancing. Hbal failed because,Ganeti couldn't load the opcode. After digging through the cause, aninconsistency with the "args" field in the LUXI protocol was triggered...
Fix UpgradeConfig of Disk object regards disk params
This bug was found during disk parameter debugging. While looking at theconfig some values seem present on the disk parameters, but that's notexpected behaviour. This patch fixes this, and also fixes the "broken"...
Make Ganeti 2.6.0 rc3 release
Allow reinstall even when secondaries are offline
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Prepare NEWS file for Ganeti 2.6.0 rc3
Update NEWS regarding rc3 fixes and document changes in behaviour.
QA updated to test instance removing when sec. is offline
Added test checking that it is possible to remove an instance with drbddisk when non-primary node is offline.
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
Ignore offline node errors when removing disks
When we delete DRBD disks from some instance, we do not want to geterrors due to nodes other than that instance's primary being offline.
Allow instance disc activation with offline secondaries
Currently, this is not allowed, so one can't run a replace-disks; thisbreaks any non-invasive method of recovering the redundancy of theinstance if its disks are already stopped (but it still works if the...
Replace a few explicit case expressions
Since we're just talking about converting Maybe into another monad, wecan do that via the maybe function, instead of explicit casing.
Fix Haskell coverage results
There are two current issues with the coverage values:
- we don't import all modules, thus leading to incomplete coverage results (too optimistic);- we use hpc in its default mode (intersection), which means that even modules which do have coverage results but are not used in all...
Also compute Haskell tags in "make tags"
Note however that this won't work correctly with older compilers, dueto ghc issue #4256; however, it's a handy way to build complete a TAGSfile for Emacs.
Cleanup the QC.hs file w.r.t. compiler options
Currently, we build the Haskell unittests with custom GHC flags,because we (I) were quite lazy when initially writing theunittests. This is not a problem for the tests themselves, but itcreates problem when (for example) one would want to pass all...
RAPI regression beparams/memory fix
For compatibility with the old Ganeti version, we want to keep thebeparams/memory field around for another release. This patch fixes thisregression.
Installation instructions update
This patch aims at improving documentation for installing ganeti - both for users and developers.
Fix Query2.hs to compile without warnings
Since this is not yet used by any targets, we didn't detect yet thecompilation warnings. Just trivial exports/imports cleanup.
Ensure that --wait-for-sync is used in QA
We don't have a specific test for activate disks, so let's add it inthe cases where we run (incidentally) activate-disks.
Add --wait-for-sync in gnt-instance
Note that this needs (like for the opcode) a new option, with thedefault reverted (False instead of True).
9Add wait_for_sync flag to OpInstanceActivateDisks
This can be used to ensure that after activate-disks has returned, theinstance's storage is consistent; currently there's no programmaticway to do this.
hbal: return exit status 0 in case of early exit
This derives from an internal bug, but the story is consistent acrossboth internal and external usage of hbal.
Basically right now, hbal returns exit code 1 if requested to exitearly, even if all jobs are successful. This is counter-intuitive due...