Split Luxi Client into a generic and a specific part
The generic part will be reused in WConfd.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Move Transport from luxi.py to a separate module
Also create a new module for RPC errors.This allows it to be reused for other clients as well.
Add a Python directory for RPC code to keep it at one place
Move rpc.py to rpc/node.py and modify imports in existing code.
Gluster: add the Shared File storage type
The shared file and gluster disk templates should not report their diskspace information like file does, because they do not behave the same.
If a cluster pulls from the same, shared source of storage then it is...
Gluster: add userspace access support
Add support for the QEMU gluster: protocol. Also change the accessmode routines so they check the access parameter for all templates.
Signed-off-by: Santi Raffa <rsanti@google.com>Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Gluster: mount automatically
Add parameters to the Gluster disk template so Gluster can manage themount point point autonomously.
Signed-off-by: Santi Raffa <rsanti@google.com>Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Gluster: use ssconf value for mountpoint directory
Gluster still does not mount anything autonomously, but this commitchanges where Gluster expects its mountpoint to be.
ssconf: Add Gluster mount directory
This commit adds the gluster storage directory to ssconf (withoutactually using its value just yet).
Gluster: add GlusterVolume class
This commit teaches Gluster what a volume is and how to use it.
Gluster: minimal implementation
Add Gluster to Ganeti by essentially cloning the shared file behavioureverywhere in the code base.
netutils: Add ValidatePortNumber method
This method accepts a port number and checks that it is in fact valid.
FileStorage: extract file logic to a FileDeviceHelper object
This will allow code reuse for Gluster through composition, ratherthan inheritance.
FileStorage: move to filesstorage.py
Move the FileStorage class in its own file, together with its helperfunctions.
PathJoin: improve error message when given one argument
PathJoin fails with an unclear message if only one argument is passedto it. Calling PathJoin("/foo") causes this exception:
Error: path joining resulted in different prefix (/foo != /foo)
However, /foo and /foo obviously share prefixes: what this function...
ComputeLDParams: do not spell out disk templates
A large part of the complexity in this function is due to the needto translate from "template-specific" parameter names to"template-agnostic" parameter names. This logic is complex and havingcomplex code for complex logic is okay....
bdev: Fix position of DEV_MAP
This rather important dictionary from constants to classes was hidingbetween function definitions. The dict cannot go to the top of the fileas the classes haven't been defined there yet, so it's been pushedto the bottom of the file....
gnt-cluster verify: demote orphan volume error to warning
Ganeti checks for orphan volume by making sure that it knows about allvolumes on disk; any additional orphan volume, even if created by theadministrator, causes a failure in gnt-cluster verify. Given that...
For the commandline, switch to query socket by default
As luxid now understands all the requests used by the command-line tools,switch the default luxi socket for those to be the socket of luxid.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Add nic.vlans to the query fields
In commit 3293332 this was only done for the Haskell side; doso for python as well, to have both views consistent.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Take a fresh Luxi client for each failing test
Luxid is more strict with closing the connection after receivinga syntactically incorrect request, gnt-debug cannot use the sameclient for several successive tests verifying that a syntacticallyincorrect request is recognized as such....
Remove queue inspection from masterd
When masterd was responsible for queue handling, it would, onrestart, pick up all non-started jobs from the queue. Now thatluxid is responsible for handling the queue, make masterd notdo this any more.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Parse spindles as integers
When parsing disk parameters, also convert spindles into integers.This is needed to obtain a syntactically correct request.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add default_iallocator_params cluster parameter
Add a cluster parameter to hold the iallocator parameters usedby the default instance allocator. Implement the option tomodify config.data, query config.data and upgrade man pages,tests and cfgupgrade tool. The new default_iallocator_params is...
Remove parseUnit for spindles
In patch e33c9e6881e6, it was a bit overkill to useparseUnit for converting the 'spindles' value since it isonly an int without a unit. This patch fixes that.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Use query client for instance removal
This patch fixes another occurrence of triggering querieswith a non-query client.
Destroy Xen domain only if necessary
Check if a given Xen domain still exists after a 'xm shutdown' beforedestroying it. This check is necessary in order to cope with Xendomains created with other version of Ganeti, given that an instancecreated in or prior to Ganeti 2.10 will be cleaned up automatically...
Fix missing '@raise' in docstring of 'StopInstance'
Fix missing '@raise' in docstring of 'StopInstance'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Fix integer parsing of spindles in gnt-instance add
'gnt-instance' did not parse the spindles value to integerbefore submitting the opcode.
Fix burnin error of instance name retrieval
Switching to Haskell queries revealed a bug in burninregarding the fetching of instance names from queries.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Instance queries: remove opcodes and LU
Removes the remains of the instance queries.
Export and network queries: remove opcodes and LUs
Removes the remains of the export (aka backup) and networkqueries.
Group queries: remove opcodes and LUs
Removes the remains of the group query code.
Node queries: remove opcodes and LUs
Removes the remains of the node query code.
Make burnin use luxi queries
Burnin was still using some op queries. With this patch,they are now replaced by luxi queries.
Remove op queries from masterd
Masterd still had some 'if' branches for the queries thatused to be op queries, but are now luxi queries. Thispatch removes them.
Remove instance query python code
This patch removes the python code for the instancequeries. So far, it replaces it by 'NotImplemented'exceptions. In a later patch of this series, theremaining part is remove completely.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Make watcher use queries properly
The watcher did not yet use the query client to makequeries. This patch fixes that.
Adjust console behavior wrt to operation state
Unlike its cli counterpart, the rapi console operation didnot properly take into account whether the instance isactually running. This patch fixes this problem.
User query client necessary for instance queries
There were some places left when code querying for instanceinformation did not use the query client yet.
Switch to Haskell for group queries
This patch removes the group query implementationin python in order to use the new Haskell implementation.
Use query client when neccessary for group queries
This patch makes code use the query client for groupqueries where necessary in order to remove the pythonqueries soon.
Switch to haskell for export (aka backup) queries
This patch removes the python implementation of export(aka backup) queries. So far, it is replaced by'NotImplemented' exceptions, but later in this seriesit will be replaced completely.
Switch to Haskell for network queries
This patch removes the python implementation of networkqueries and replaces it with 'NotImplemented' exceptions.It will be removed completely once all queries areswitched to Haskell.
Disable node query code
This patch removes the python query implementation fornodes. So far, the code is replaced by 'NotImplemented'exceptions, because the overall structure of query classescan be removed more easily at once when all queryimplementations are ready to be removed....
masterd: implement query via luxi
The master daemon so far still did queries via the pythonimplementation. This patch implements that it uses thehaskell implementation and removes the node queries fromthe list of OP-queriable entities.
Make watcher use query clients
So far, the watcher was still issuing node queriesvia the python implementation. This patche switches tothe haskell implementation.
Move GetClient() to runtime module
Soon both, the cli and the masterd, will use a luxiclient to run queries. The method to obtain and configurethe luxi client with the right address can be reused fromcli to masterd here. Therefore, this patch moves the...
Use a query client for node queries where necessary
To remove the node query's python implementation soon, thispatch makes all users of the queries call the haskellimplementation instead of the python one.
Remove --enable-split-query option
Switching from python to haskell queries, this patchremoves the option to dis/enable the haskell queriesat configure time.
Add Xen paused state as a Ganeti running state
The QA test 'TestClusterBurnin' pauses instances and the Xen statedomain 'paused' was not being handled properly by Ganeti because itwas not being classified as either a running or shutdown state. Thispatch adds the 'paused' state to the set of running states by...
Parse spindles as integer
The disk parameter "spindles", if present, has to be an integer.Enforce this already when parsing the command line, instead ofpassing around the string representation of that integer.
Merge branch 'stable-2.10' into master
Merge branch 'stable-2.9' into stable-2.10
Use custom SSH ports for other operations
In particular copyfile and renew-crypto.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Use configured SSH ports when connecting to a console
This is accomplished by passing the corresponding node group tohv_*.py. Tests for hv_*.py that call GetInstanceConsole updated.
Use custom SSH ports in node groups when working with nodes
Calling `gnt-instance console` with a custom SSH port doesn't work yet.
Add a method retrieving `NodeGroup` objects by group name
Also add a method for the retrieval of "ndparams" (node parameters)of groups, not just nodes.
Don't allow optional node parameters
Ganeti does not support optional fields in parameters(hypervisor-params, disk-params, etc.). OpenVSwitch related nodeparameters were the exception to this rule, which caused numerousproblems related to import/export and (de-)serialization....
Move OVS node parameters to the right place
OpenVSwitch node parameters were passed as separate command line flags.This was not only inconsistent with all other parameters(hypervisor-params, disk-params, ...), but also with the documentedbehavior in the man pages....
Make NIC VLAN queryable
Add an additional query field which makes it possible to query forVLANs of instance NICs.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Use constant instead of raw string
Use the appropriate constant instead of a raw string.
Fix instance info
Fix class methods in KVM hypervisor
Several methods in KVM hypervisor behave like 'classmethods', however,they did not have the '@classmethod' annotation. This patch adds'@classmethod' annotations to several of these methods with the goalof making '_StopInstance' available to the QA. This step is necessary...
Watcher cleans user shutdown domains
Update watcher to destroy shutdown domains when it detects that adomain is in the state 'USER_down'. This requires adding logicsimilar to that of restart instance, that is tracking the number ofcleanup attempts in the watcher's state and give up if a given number...
Python instance queries report 'USER_down'
Update instance queries on the Python codebase to report 'USER_down'in the situation where the user has shutdown a Xen domain but Ganetistill marks this domain as 'ADMIN_up' in its configuration file.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
Prevent instance start when user down
Prevent starting a given instance when that instance has been shutdownby the user, given that the instance must be first properly shutdown.
Xen handle domain shutdown
Update Xen backend to properly recognize when a domain has beenshutdown by the user and to properly cleanup a shutdown domain whenGaneti requests Xen to stop this domain.
Preserve Xen domain after shutdown
Change Xen domain configuration to preserve the domain state aftershutdown, as opposed to automatic reboot, which is the defaultbehaviour.
Add Python hypervisor instance state
Add 'HvInstanceState' which represents the state of an instance(either 'running' or 'shutdown') and fix the remaining hypervisorbackends to return a value of this type. Before this patch, eachbackend was returning their own value, some were returning strings,...
Fix typo in documentation
Fix retrieval of xen command in class method
This patch fixes issue 608. When introducing theconfigurability of the xen toolstack in commit8ef418bb92, the hypervisor api was accidentally changedin a way that let to this error in KVM.
Adapt TestJobSubmission to new semantics of SubmitManyJobs
Historically, Ganeti would partially handle syntactically incorrectluxi requests, if the outer structure was that of SubmitManyJobsrequest. Moreover, this behavior was verified by a test. We changed to...
Add a constructor to BaseOpCode
In order to guarantee all mandatory slots to be present, adda custom constructor to BaseOpCode adding those fields withdefault value, instead of inheriting the constructor fromoutils.ValidatedSlots.
FillDict: do not use exceptions for flow control
While it's generally better to beg for forgiveness than ask forpermission in Python, it's not a good idea to rely on exceptions tomodel control flow, especially if the resulting code is simpler.
Signed-off-by: Santi Raffa <rsanti@google.com>...
Remove hardcoded references to File, SharedFile templates
DTS_FILEBASED is a constant that exists and this commit makes surethat it is used whenever sensible, rather than resorting to hardcodingthe pair of templates in very many files.
Make masterd call luxid to write jobs to queue
As the responsibility for writing the job queue changedto luxid, make masterd call to luxid when it is necessaryto write a job.
Set reason trail for derived opcodes
As the base parameters also includes the reason trail,set it, if not set already, together with the otherbase parameters in the case of an opcode expanding toa list of opcodes.
Add Luxi Request to pick up a job in the queue
During the transition to the new daemon layout, from step 2onwards, luxid will write to the queue but masterd will triggerthe execution. Therefore, add a new luxi request to tell masterdto pick up a job that has already been written to the queue....
Factor out functionality to pickup a job from the queue
The current restart procedure for masterd includes functionalityto pick up a job from the queue and restart it, if it hasn't beenstarted before. Move this functionality into a separate functionto be able to have the enqueuing be done by luxid....
Hs2Py constants: additional module jstore
Add constants from additional modules ('ganeti.jstore') to the Haskellto Python constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Hs2Py constants: additional module errors
Add constants from module 'ganeti.errors' to the Haskell to Pythonconstant generation.
Hs2Py constants: additional module qlang
Add constants from additional modules ('ganeti.qlang') to the Haskellto Python constant generation.
Hs2Py constants: additional module luxi
Add constants from additional modules ('ganeti.luxi') to the Haskellto Python constant generation.
Eliminate unused constants
Hs2Py constants: add 'UUID_REGEX'
Add constant 'UUID_REGEX' to the Haskell to Python constantgeneration.
Readd nodes as online
Patch d0d7d7cf accidentally removed the offline-flag resetwhen readding a node. Readd it.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
When verifying BRBD version, ignore missing values
When comparing for consistency of the DRBD versions, someversions might not be available via RPC, typically, if thenode is offline. In this case, leave these nodes out of thetest, instead of failing with an internal python error....
Merge branch 'stable-2.8' into stable-2.9
Conflicts: NEWS: trivial configure.ac: ignore version bump on stable-2.8 lib/bdev.py: manually apply the part of commit...
DRBD: ensure peers are UpToDate for dual-primary
DrbdAttachNet supports both, normal primary/secondary node operation, and(during live migration) dual-primary operation. When resources are newlyattached, we poll until we find all of them in connected or syncing operation....
Conflicts: NEWS: trivial configure.ac: drop suffix bump of stable-2.9
Conflicts: lib/cmdlib/instance_storage.pyResolved by manually applying the node name to uuidtransition on the version of stable-2.9.
Improve error message for replace-disks
In some conditions, replace-disks will fail if the disks are not properlyactivated. Improve the error message suggesting to run activate-disks beforeexecuting replace-disks.
Fixes Issue 606.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Replace all constant definitions with re-exports
Remove all constant definitions in 'lib/constants.py' and re-exportthe module 'lib/_constants.py', which contains the generatedconstants.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Prepare constants for automatic reexport
Hs2Py constants: 'hvsParameterTypes' and 'hvsParameters'
Add constants 'hvsParameters' and 'hvsParameterTypes' to the Haskellto Python constant generation.
Fix indentation that triggers PEP8 error
Signed-off-by: Petr Pudlak <pudlak@google.com>Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Check if hotplug is supported in CheckPrereq
Introduce new RPC hotplug_supported that invokes the correspondinghypervisor's method which checks if hotplug is generallysupported. Call this RPC early in CheckPrereq() and abort ifhotplug is not supported. Currently only KVM hypervisor with...