cmdlib: Use dict.fromkeys instead of custom loop
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Simplify InitConfig and remove SimpleConfigWriter
InitConfig currently creates the cluster config_data, then puts it intoa dict, passes it to SimpleConfigWriter to load it from a dict (whichjust reuses the dict value) and then saves it. The SimpleConfigWriter is...
InitCluster, don't use SimpleConfigWriter
InitConfig returns a SimpleConfigWriter to InitCluster, which thenpasses it on to ssh.WriteKnownHostsFile, which extracts a couple ofvalues from it. One line later the full ConfigWriter is initialized.
By initializing it one line before we can pass the full writer to...
Fix python 2.4 compatibility
I got overexcited and forgot we have to remain compatible with python2.4. With this patch we move from sha256 to sha1 for hmac authenticatedserialized messages, and we handle both newer and older python, byimporting the right module for each....
Merge branch 'next' into branch-2.1
Conflicts:
daemons/ganeti-masterd...
Remove SimpleConfigWriter.SetMasterNode
This function is not used.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
_GenerateDiskTemplate: use base_index in the name
Currently if a disk is added later the base_index is not considered, andall the disks are called disk0. This patch fixes it.
HMAC authenticated json messages
This patch includes HMAC authenticated json messages to the serializer.The new interface works on any json-encodable data type, and can sign itwith a private key and an optional salt. The same private key must beused upon message loading to verify the message....
rapi: Implement /2/nodes/[node_name]/role resource
This resource can be used to retrieve and set the role of a node.
rapi: Add generic “force” parameter
cmdlib: Fix typo in LUQueryClusterInfo
This was broken by my pylint fixes patch.
Extend call_node_start_master rpc with no_voting
When the parameter is set to True and start_daemons is also True,ganeti-masterd will be started with the new --no-voting --yes-do-itoptions.
This new option is set to True only on masterfailover, when no_voting is...
Merge branch 'master' into next
Create a new --no-voting option for masterfailover
This allows failing over in certain corner cases, such as a 2 nodecluster with one node down. The man page is also updated to documentthis dangerous option and how to recover from this situation.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Fix pylint warnings
bootstrap: Don't leak file descriptor when generating SSL certificate
Fix problem with EAGAIN on socket connection in clients
If a user used ^Z to stop the program, poll() in socket.recv would returnEAGAIN due to SIGSTOP. This patch changes luxi.Transport.Recv to ignore EAGAIN.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Fix some typos
Increase maximum accepted size for a DRBD meta dev
With the change to stripped LVs, the actual size of a meta device (whichis small) can be more than we expected (for non-stripped LVs). Thispatch increases from 160MB to 1GB the accepted size, and updates the...
Cleanup config data when draining nodes
Currently, when draining nodes we reset their master candidate flag, butwe don't instruct them to demote themselves. This leads to “ERROR: file'/var/lib/ganeti/config.data' should not exist on non master candidates...
Fix node readd issues
This patch fixes a few node readd issues.
Currently, the node readd consists of two opcodes: - OpSetNodeParms, which resets the offline/drained flags - OpAddNode (with readd=True), which reconfigures the node
The problem is that between these two, the configuration is inconsistent...
backend.DemoteFromMC: don't fail for missing files
If the config file is missing when the DemoteFromMC() function iscalled, it will raise a ProgrammerError. Instead of changing theutils.CreateBackup() file which is called from multiple places, for nowwe only change the DemoteFromMC() function to not call it if the file is...
Allow GetMasterCandidateStats to ignore some nodes
This patch modifies ConfigWriter.GetMasterCandidateStats to allow it toignore some nodes in the calculation, so that we can use it to predictcluster state without some nodes (which we know we will modify, and thus...
Fix error message for extra files on non MC nodes
Currently the message for extraneous files on non master candidates isconfusing, to say the least. This makes it hopefully more clear.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Olivier Tharan <olive@google.com>
Merge branch 'master' into branch-2.1
Rename the volume_list RPC call to lv_list
There are volume-related rpc calls. This patch renames the ‘volume_list’call to ‘lv_list’ to make more clear its purpose.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
GenericMain, handle ParameterError from _ParseArgs
Before this case was not covered, and printed a stack trace.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
check_ident_key_val, handle no_ and - prefixes
If an ident member of an IdentKeyVal relationship starts with no_ or -,handle it the same way we do for a key. Some unittests are added tocheck that check_ident_key_val behaves as expected.
This patch also changes ForceDictType to, for now, fail on such an...
_SplitKeyVal with no data return an empty dict
If an empty string is passed to _SplitKeyVal, we should return {},rather than {'': True}. Also test for the correct behavior.
Introduce OS api version 15
Also, since Ganeti 2.1 will be compatible with both 10 and 15, changethe OS_API_VERSION constant to be an OS_API_VERSIONS set, and update theplaces in the code that used that constat to use something else.
In particular: - in the qa for now we just create a fake version 10 OS...
_OSOndiskAPIVersion: save a loop
The api_versions list is first stripped and then converted to integer.Combining the two operations.
Fix adjustement of candidates in cluster modify
The code for adjusting the candidate pool size was done after the configupdate, and this means we triggered the save of the config file withoutfixing the candidate pool, which aborts with an error.
The patch just moves it above. The old comment was valid, but we anyway...
Add a new node list field
This patch adds a ‘role’ node list field, which shows a one-characternode status. This is a simpler way to see the node status than selectingall the flags individually.
Use ReadFile.splitlines() rather than readlines
A few places in the code open a file "manually" rather than using ourwrapper function, because they need an array with the lines. Combiningthe result of utils.ReadFile with splitlines() we get rid of theexceptions....
Rename _OSOndiskVersion to _OSOndiskAPIVersion
This makes what versions we're talking about clearer.
Convert ssconf._ReadFile to utils.ReadFile
Making ssconf._ReadFile a wrapper over utils.ReadFile
Signed-off-by: Guido Trotter <ultrotter@google.com>
backend.StartMaster: fix variable name
As per comments for patch “Convert node_start_master to new styleresult”, the ‘payload’ variable is renamed to ‘err_msgs’.
Fix HTTP server library handling of credentials
Currently the http library only checks credentials when authenticationis required. This means that any credentials are accepted on the rootresource, for example, which makes problems hard to diagnose - the...
Fix a typo in backend.InstanceReboot docstring
The documentation for the reboot was wrong. This patch fixes it andupdates the docstring with more details.
rapi: implement dry-run mode
This patch implements dry-run mode for the operations which modify thestate of the cluster. Dry-run mode is enabled by passing a 'dry-run'query argument with positive integer value.
Signed-off-by: Iustin Pop <iustin@google.com>...
LUCreateInstance: the node list as return value
Currently LUCreateInstance has no result; this patch changes it so thatboth the normal result and the dry-run result is the node list of theselected instance.
Implement dry-run mode at cli level (partially)
This patch adds support for the dry-run mode for all command lineoperations, and also makes use of this for commands using theSubmitOrSend function. For the ones not using it, the flag has noeffect (future patches)....
LU execution: implement dry-run framework
This patch adds a new (global) opcode flag 'dry_run' which, when True,causes early exit from the LU workflow, returning a special value fromthe LU object (initialized in the parent LogicalUnit class, and which if...
Introduce slots deriving in opcodes.py
This simple patch adds to all opcodes extension of the base opcodeslots. This way we can add slots across all opcodes, for example'dry-run'.
Fix some small epydoc warnings
Fix various pylint warnings
There were multiple issues: - copy-paste resulted in wrong indentation - wrong function name - missing spaces around assignment - overriding built-in names (type, dir) or already defines ones (errors, hypervisor)
Fix handling of 'vcpus' in instance list
Currently running “gnt-instance list -o+vcpus” fails with a cryptic message: Unhandled Ganeti error: vcpus
This is due to multiple issues: - in some corner cases cmdlib.py raises an errors.ParameterError but this is not handled by cli.py...
RAPI: move to nic parameters
In query we ask for nic.links, rather than nic.bridgesIn create we accept both "link" and "bridge" and let the opcode dealwith it. Note that we still can create only one nic per instance.
Update a forgot docstring for nic parameters
Properly document the expected nic format.
Fix QueryInstanceData for nic parameters
This CL updates QueryInstanceData to return NICs in the new format (mac,ip, mode, link) and fixes gnt-instance info to properly display them.
Update instance query for NIC parameters
Compatibility with the old parameters is maintained, by allowing toquery for "bridge" "nic.bridges" and "nic.bridge/N", but None isreturned in that case for routed nics.
Rename _PreBuildNICHooksList to _NICListToTuple
We're going to use this helper function for more than just hooks, sowe'll give it a more generic name.
Fix checking for valid OS in instance create
The current check in LUCreateInstance.CheckPrereq() is wrong - it only checksif we got an OS, but not if we got a valid OS. This patch fixes it.
Show disk size in instance info
The size of the instance's disk was not shown in “gnt-instance info”.This patch adds it and formats it nicely if possible.
LUQueryInstances: fix querying for nic data
Currently we support querying for "mac" "ip" or "bridge", meaning "theone of the first nic. We are not checking that there is a first nic,though, and thus could incur in errors. This patch fixes it by returning...
Specify the object type in two docstring
KVMHypervisor, fix a bug in MigrateInstance
This was introduced as a typo in commit7e66c35b5685525538dd65dd3dff75b5346d30d7 while removing duplicate codefrom the KVM hypervisor.
Fix backend.{Start,Stop}Master
Commit c26a6bd21c17641f718369caed88ae16947fa774 changed GetMasterInfonot to return a tuple anymore, but didn't update its two callers inbackend.py, which were trying to extract the values from the secondtuple element. This causes a stack trace in node-daemon.log....
Small whitespace change
Some indentation was wrong, and pylint rightfully complained.
Simplify RPC call result check in cmdlib.py
Now that all results are the same, we can even more simplify thehandling in cmdlib more. Almost all if result.RemoteFailMsg()…constructs are similar, and we resurect the RpcResult.Raise() functionto take a message argument, which it will process and raise an...
Simplify the RPC result framework in backend.py
Since now all functions fail via _Fail, the return True, … is redundantas all normal return paths have it, and thus the True value can be addedin the ganeti-noded handler.
This means that all functions can now forget about the special result...
Convert all backend function to exception
Instead of returning (False, msg) from rpc endpoints, we raise alwaysexceptions (the non-endpoint, internal functions can remain as is). Thismeans that the error paths are agnostic to how the failure is signalled...
Simplify usage of backend._FindDisks
Since all users of _FindDisks now return new-style results, we cansimply make it raise an exception and not deal with the status field.
Pre-compute error status in RpcResult
Since now all RPCs have the new style result type, we can pre-computethe error message at RpcResult() init time and not wait until it'srequested.
Remove old invalid-os related functionality
We no longer need OS objects to be able to represent invalid OSes. Thiscleans up the code handling those cases.
Big rewrite of the OS-related functions
Currently the OSes have a special, customized error handling: the OSobject can represent either a valid OS, or an invalid OS. The associatedfunction, instead of raising other exception or failing, create customOS objects representing failed OSes....
Convert the jobqueue rpc to new style result
This patch converts the job queue rpc calls to the new style result.It's done in a single patch as there are helper function (in both jqueueand backend) that are used by multiple rpcs and need synchronizedchange....
Convert the file storage rpcs to new style result
This patch converts all three file-storage rpc (create, delete, rename)to new style result. This is done in a single patch as they all use ahelper function which itself needs to/can be converted.
Convert test_delay rpc to new style result
Convert iallocator_runner rpc to new result style
This patch converts this rpc into the new style. Since the functionalready had some error handling, we remove this custom error reportingand replace it with our (new-style) result type. This allows significant...
Convert hooks_runner rpc to new style result
This also converts (and fixes) unittests and mock objects to deal withthis change, and the custom hook verifier in cmdlib.LUClusterVerify.
Convert os_diagnose rpc to new style result
This also removes custom post-processing from rpc.py; since this callhas only one user, it was simple to move it back to the caller.
Convert call_version rpc to new style result
This also cleans up its single use in cmdlib.py.
Convert write_ssconf_files to new style
The patch also adds logging of errors from the ConfigWriter in case theRPC fails (although today we don't have failure modes).
Convert master_info rpc to new style result
This was more tricky as the backend function is used by other functionin backend.py. As such, it must be handled specially - it must raisealways an exception and not simply return False, err.
Convert node_volumes rpc to new style result
Conver node_leave_cluster rpc to new style result
This patch converts this rpc call to the new style result, and alsochanges in the process the meaning of the QuitGanetiException'sarguments and the node daemon rpc call exception handler.
The problem with the exception handler is that we used a two-stage one,...
Convert node_stop_master rpc to new style result
Convert node_start_master to new style result
This is used in multiple places outside cmdlib.py, so it's a moreinteresting patch.
Convert node_verify rpc to new result style
Remove utils.CheckDict since it's no longer used
Only the custom handling in rpc.py for call_node_info used this, so nowit can be removed.
Convert node_info rpc to new style result
This patch also does some cleanup and enforces valid results (withproper type, i.e. int for memory/disk values) from remote node,otherwise we handle the result as failure.
We do this so that we can remove custom processing in rpc.py which is...
Convert node_has_ip_address rpc to new style
This should actually have a function in backend, but it's fine for now.
Convert instance_list rpc to new style result
Since backend.GetInstanceList() is used both as RPC endpoint and asinternal function, it can't return (status, value). Instead it returnsonly valid instance info, and failures are denoted by exceptions; and...
Convert all_instances_info rpc to new result style
Convert instance_info rpc to new style result
Convert instance_os_import rpc to new style result
This changes from a list of booleans to «status, error messages». Thismeans that instead knowing which disk has failed (position based), weget a list of all failures (with details how they failed).
Convert bridges_exist to new style result
This was a very simple (boolean) RPC, so converting it to actually havemore value with the new style results was more difficult.
Convert vg_list rpc to new style result
This doesn't have known failure modes but converting will help later.
We also now call directly utils.ListVolumeGroups() instead of thebackend.ListVolumeGroups() so that we don't have to undo the (status,value) result type....
Convert volume_list rpc to new style result
This is a big change, because we need to cleanup its users too.
The call and thus LUVerifyDisks LU used to differentiate between failureat node level and failure at LV level, by returning different types inthe RPC result. This is way too complicated for our needs....
Convert export_remove rpc to new style
This converts the export_remove rpc to new style result and also fixesand old TODO by adding exception handling (and conversion to failure).
Convert export_list rpc to new style result
This is used in multiple places, so it has a little more changes thanthe previous ones.
Convert export_info rpc to new style result
This also removes some code from ganeti-noded and rpc.py, which shouldnot do such processing of data (and be simply glue code). (Oralternatively they could, if we had better infrastructure).
Convert finalize_export rpc to new style results
This rpc doesn't have explicit failure modes, but once all RPCs areconverted and we can catch and convert expceptions to failure in noded,we will have meaningfull error messages.
Convert snapshot_export rpc to new style
Convert blockdev_snapshot rpc to new result style
This also slightly changes the code in cmdlib that uses it for moreclarity.