Fix docstring for 'AsyncStreamServer'
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Add utility to compare versions
This will be needed, e.g., for post-upgrade task, as theyhave to decide whether a feature was not yet present atthe version started from.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Merge branch 'stable-2.10' into master
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Run postupgrade hook after upgrade
To allow for necessary last-moment adaptions, of the new cluster,we run the post-upgrade hook of the target version, providingthe version we originally started from.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Provide path to post-upgrade
Also add the current version to the intent-to-upgrade file
Our design states, that the intent-to-upgrade file contains "the currentversion of ganeti, the version to change to, and the process ID". Make theimplementation fit with that design.
admin.rst: update and reword disk template section
The disk template section was not updated for Gluster. This commitalso refactors the section slightly by unifying the different remarksabout /etc/ganeti/file-storage-paths.
sphinx_ext is also changed in order to not hardcode too much...
Remove certification on 2.11 to 2.10 downgrade
While version 2.10 ignores any leftover client certificates, theirpresence will prevent a the cluster working after an upgrade backto version 2.11 again. So we have to remove them right at thedowngrade.
Add support for version-specific downgrade tasks
Upgrading can have no specific knowledge about additionaltasks besides upgrading the configuration, as upgrades needto be able to go to any future version (within the same majorversion). Downgrading, however, is version specific and always...
Correct exception when ssconf file does not exist
After an upgrade to 2.11, the ssconf file for the mastercertificates might not exist. Based on the non-existance,noded falls back to a compatibility mode regarding dealingwith SSL certificates. The check for the ssconf file...
Create client certificate for normal nodes
The vcluster QA revealed a bug in the SSL certificatehandling code, where certificates were only createdwhen the node is a master-candidate. However, every nodeshould have a certificate, but only the digests of the...
Verify client certificates
This patch adds a step to 'gnt-cluster verify' to verifythe existence and validity of the nodes' clientcertificates. Since this is a crucial point of thesecurity concept, the verification is very detailed withexpressive error messages and well tested by unit tests....
Verify incoming RPCs against candidate map
From this patch on, incoming RPC calls are checked againstthe map of valid master candidate certificates. If no mapis present, the cluster is assumed to be inbootstrap/upgrade mode and compares the incoming call...
Handle promoting/demoting nodes wrt to client certificates
This patch makes Ganeti correctly handle the clientcertificates when nodes get promoted to master candidatesor demoted to normal nodes.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Extend RPC call to create SSL certificates
So far the RPC call 'node_crypto_tokens' did only retrievethe certificate digest of an existing certificate. Thiscall is now enhanced to also create a new certificate andreturn the respective digest. This will be used in various...
Create client SSL certificates on cluster init
This patch makes Ganeti create a client SSL certificate forthe master node on cluster initialization. Note that some ofthe code in this patch is later moved into an LU to serverequirements for crypto renewal and updates, but for this...
Store candidate certificates in ssconf
This patch enables Ganeti to store the candidatecertificate map in ssconf. A utility function toread it is provided as well.
Handle client certificates on node add/remove
This patch adds the certificate of a newly added orreadded master candidate node to the map of master candidatecertificates. It removes a master candidate node's certificatedigest from the candidate certificate map if the node is...
Add certificate for master node
On cluster initialization, the master node'sSSL certificate digest is added to the list of mastercandidate certificates.
Add candiate certificate map to configuration
At the end of this patch series, incoming RPC calls arelegitimized against a map of master candidate nodes'SSL certificate digests. This patch adds the map itselfto the cluster's configuration.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Retrieve a node's certificate digest
In various cluster operations, the master node needs toretrieve the digest of a node's SSL certificate. For thispurpose, we add an RPC call to retrieve the digest. Thefunction is designed in a general way to make it possible...
Utility functions to manipulate the candidate map
This patch adds a couple of utility functions to manipulatethe map of master candidate SSL certificate digests.
Replace errors re-export in luxi.py with proper imports
Instead of re-exporting errors in luxi.py, import rpc/errors.py in themodules that use them.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
luxi.py: Fix pylint warning about unused imports
Reexport exception classes more explicitly for pylint's convenience.
Signed-off-by: Santi Raffa <rsanti@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
rpc: Fix one more py-apidoc warnings
rpc: Fix py-apidoc warnings
The previous commits shuffled code around using import renames asglue. apidoc ignores import renames, however, and chokes on somenow invalid link targets.
This commit fixes the issue.
Signed-off-by: Santi Raffa <rsanti@google.com>...
Separate the LUXI protocol version from the generic client
This allows other daemons and their clients (such as WconfD) to use adifferent versioning sequence of their protocols.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Rename CallLuxiMethod to CallRPCMethod
Also update error messages and testing code to refer to RPC instead ofLUXI.
Split Luxi Client into a generic and a specific part
The generic part will be reused in WConfd.
Move Transport from luxi.py to a separate module
Also create a new module for RPC errors.This allows it to be reused for other clients as well.
Add a Python directory for RPC code to keep it at one place
Move rpc.py to rpc/node.py and modify imports in existing code.
Gluster: add the Shared File storage type
The shared file and gluster disk templates should not report their diskspace information like file does, because they do not behave the same.
If a cluster pulls from the same, shared source of storage then it is...
Gluster: add userspace access support
Add support for the QEMU gluster: protocol. Also change the accessmode routines so they check the access parameter for all templates.
Signed-off-by: Santi Raffa <rsanti@google.com>Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Gluster: mount automatically
Add parameters to the Gluster disk template so Gluster can manage themount point point autonomously.
Signed-off-by: Santi Raffa <rsanti@google.com>Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Gluster: use ssconf value for mountpoint directory
Gluster still does not mount anything autonomously, but this commitchanges where Gluster expects its mountpoint to be.
ssconf: Add Gluster mount directory
This commit adds the gluster storage directory to ssconf (withoutactually using its value just yet).
Gluster: add GlusterVolume class
This commit teaches Gluster what a volume is and how to use it.
Gluster: minimal implementation
Add Gluster to Ganeti by essentially cloning the shared file behavioureverywhere in the code base.
netutils: Add ValidatePortNumber method
This method accepts a port number and checks that it is in fact valid.
FileStorage: extract file logic to a FileDeviceHelper object
This will allow code reuse for Gluster through composition, ratherthan inheritance.
FileStorage: move to filesstorage.py
Move the FileStorage class in its own file, together with its helperfunctions.
PathJoin: improve error message when given one argument
PathJoin fails with an unclear message if only one argument is passedto it. Calling PathJoin("/foo") causes this exception:
Error: path joining resulted in different prefix (/foo != /foo)
However, /foo and /foo obviously share prefixes: what this function...
ComputeLDParams: do not spell out disk templates
A large part of the complexity in this function is due to the needto translate from "template-specific" parameter names to"template-agnostic" parameter names. This logic is complex and havingcomplex code for complex logic is okay....
bdev: Fix position of DEV_MAP
This rather important dictionary from constants to classes was hidingbetween function definitions. The dict cannot go to the top of the fileas the classes haven't been defined there yet, so it's been pushedto the bottom of the file....
gnt-cluster verify: demote orphan volume error to warning
Ganeti checks for orphan volume by making sure that it knows about allvolumes on disk; any additional orphan volume, even if created by theadministrator, causes a failure in gnt-cluster verify. Given that...
For the commandline, switch to query socket by default
As luxid now understands all the requests used by the command-line tools,switch the default luxi socket for those to be the socket of luxid.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Add nic.vlans to the query fields
In commit 3293332 this was only done for the Haskell side; doso for python as well, to have both views consistent.
Fix pylint 0.26.0/Python 2.7 warning
pylint 0.26.0 on Python 2.7 generates a warning on the string '\ ',recommending to use the r prefix. This patch adds the missing prefix.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Take a fresh Luxi client for each failing test
Luxid is more strict with closing the connection after receivinga syntactically incorrect request, gnt-debug cannot use the sameclient for several successive tests verifying that a syntacticallyincorrect request is recognized as such....
Merge branch 'stable-2.9' into stable-2.10
Pass hvparams to GetInstanceInfo
...so that the xen command to be called can be determined. Thisfixes another semantical conflict of the last merge.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Jose Lopes <jabolopes@google.com>
Adapt parameters that moved to instance variables
Due to a change in the code organization in stable-2.9, somemethod variables became instance variables, causing a semanticmerge conflict. Fix this.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Merge branch 'stable-2.8' into stable-2.9
Support reseting arbitrary params of ext disks
If param=default and the param already exists then we removeit from params dict. This is stolen by GetUpdatedParams() whichis used for hvparams modification/inheritance.
This means that 'default' value is not accepted for an arbitrary...
Allow modification of arbitrary params for ext
Disks of ext template are allowed to have arbitrary parametersstored in the Disk object's params slot. Those parameters can bepassed during creation of a new disk, either in LUInstanceCreate()or in LUInsanceSetParams(). Still those parameters can not be...
Do not clear disk.params in UpgradeConfig()
Commits 5dbee5e and cce4616 fix disk upgrades concerning paramsslot. Since 2.7 params slot should be empty and gets filledany time needed.
Still ext template allows passing arbitrary params per disk.These params should be saved in config file for future use....
SetDiskID() before accepting an instance
SetDiskID() fills physical_id slot of a Disk object.
LUInstanceSetParams() does not invoke SetDiskID() upon creation of anew disk. As a result the physical_id slot of the Disk object inconfig data is missing.
In case of ext disk template, in AcceptInstance() we invoke...
Lock group(s) when creating instances
This is required to prevent race conditions such as removing a networkfrom a group and adding an instance at the same time. (See issue 621#2.)
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>...
Fix job error message after unclean master shutdown
According to commit 599ee321eb, any job-related error messages shouldbe encoded within a Ganeti-specific error and not passed on as astring, to allow for easier parsing.
For jobs suffering from an undesirable status after an unclean master...
Add default file_driver if missing
If the file driver of an instance with file based storage is not specified, thedefault one is automatically added by the UpgradeConfig function.
Fixes Issue 571.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Xen handle domain shutdown
Update Xen backend to properly recognize when a domain has beenshutdown by the user and to properly cleanup a shutdown domain whenGaneti requests Xen to stop this domain.
Partial cherry-pick from 9d22cc90609e3ee8f0f2b34b793a3daced3c0e61...
Remove queue inspection from masterd
When masterd was responsible for queue handling, it would, onrestart, pick up all non-started jobs from the queue. Now thatluxid is responsible for handling the queue, make masterd notdo this any more.
Fix evacuation out of drained node
Introduce _UpgradeSerializedRuntime() method
This method is invoked during _AnalizeSerializedRuntime() and ismeant to modify runtime files in the way cfgupgrade does forconfig.data. This could remove deprecated fields, change theformat of the file, add/remove sections, etc....
Fix a bug in InstanceSetParams concerning names
In case no name is passed in disk modifications we shouldkeep the old one. If name=none then set disk name to None.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
SingleNotifyPipeCondition: don't share pollers
As widely known Ganeti uses a better1 lock condition notificationlibrary based on operating system pipes.
Inside this library we were using a shared poller for all threadswaiting for a condition. While poller is not thread safe, since (1)...
Parse spindles as integers
When parsing disk parameters, also convert spindles into integers.This is needed to obtain a syntactically correct request.
Add default_iallocator_params cluster parameter
Add a cluster parameter to hold the iallocator parameters usedby the default instance allocator. Implement the option tomodify config.data, query config.data and upgrade man pages,tests and cfgupgrade tool. The new default_iallocator_params is...
Remove parseUnit for spindles
In patch e33c9e6881e6, it was a bit overkill to useparseUnit for converting the 'spindles' value since it isonly an int without a unit. This patch fixes that.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Use query client for instance removal
This patch fixes another occurrence of triggering querieswith a non-query client.
Destroy Xen domain only if necessary
Check if a given Xen domain still exists after a 'xm shutdown' beforedestroying it. This check is necessary in order to cope with Xendomains created with other version of Ganeti, given that an instancecreated in or prior to Ganeti 2.10 will be cleaned up automatically...
Fix missing '@raise' in docstring of 'StopInstance'
Fix missing '@raise' in docstring of 'StopInstance'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Fix integer parsing of spindles in gnt-instance add
'gnt-instance' did not parse the spindles value to integerbefore submitting the opcode.
Fix error printing
Fixes issue 616.
Allow link local IPv6 gateways
Each host using IPv6 always has a link local address in fe80::/10. It iscommon to use fe80::1 as default gateway to ease client configuration.Ganeti prevented this usage, because it made sure that the IPv6 gatewayis in the IPv6 network the instance is connected to....
Fix NODE/NODE_RES locking in LUInstanceCreate
Both NODE and NODE_RES locks were acquired opportunistically if sorequested by the user. LUInstanceCreate requires, however, that theactually locked elements on NODE and NODE_RES level are the same.
This patch changes the locking of NODE_RES such that those locks are not...
Fix burnin error of instance name retrieval
Switching to Haskell queries revealed a bug in burninregarding the fetching of instance names from queries.
KVM: use custom KVM path if set for version checking
This commit fixes two TODOs from 2008 about using the hardcoded"default" path for KVM where a custom one could've been set through`gnt-cluster modify`.
As a result, `gnt-cluster verify` will no longer fail if a custom...
Instance queries: remove opcodes and LU
Removes the remains of the instance queries.
Export and network queries: remove opcodes and LUs
Removes the remains of the export (aka backup) and networkqueries.
Group queries: remove opcodes and LUs
Removes the remains of the group query code.
Node queries: remove opcodes and LUs
Removes the remains of the node query code.
Make burnin use luxi queries
Burnin was still using some op queries. With this patch,they are now replaced by luxi queries.
Remove op queries from masterd
Masterd still had some 'if' branches for the queries thatused to be op queries, but are now luxi queries. Thispatch removes them.
Remove instance query python code
This patch removes the python code for the instancequeries. So far, it replaces it by 'NotImplemented'exceptions. In a later patch of this series, theremaining part is remove completely.
Make watcher use queries properly
The watcher did not yet use the query client to makequeries. This patch fixes that.
Adjust console behavior wrt to operation state
Unlike its cli counterpart, the rapi console operation didnot properly take into account whether the instance isactually running. This patch fixes this problem.
User query client necessary for instance queries
There were some places left when code querying for instanceinformation did not use the query client yet.
Switch to Haskell for group queries
This patch removes the group query implementationin python in order to use the new Haskell implementation.
Use query client when neccessary for group queries
This patch makes code use the query client for groupqueries where necessary in order to remove the pythonqueries soon.
Switch to haskell for export (aka backup) queries
This patch removes the python implementation of export(aka backup) queries. So far, it is replaced by'NotImplemented' exceptions, but later in this seriesit will be replaced completely.
Switch to Haskell for network queries
This patch removes the python implementation of networkqueries and replaces it with 'NotImplemented' exceptions.It will be removed completely once all queries areswitched to Haskell.
Disable node query code
This patch removes the python query implementation fornodes. So far, the code is replaced by 'NotImplemented'exceptions, because the overall structure of query classescan be removed more easily at once when all queryimplementations are ready to be removed....
masterd: implement query via luxi
The master daemon so far still did queries via the pythonimplementation. This patch implements that it uses thehaskell implementation and removes the node queries fromthe list of OP-queriable entities.
Make watcher use query clients
So far, the watcher was still issuing node queriesvia the python implementation. This patche switches tothe haskell implementation.
Move GetClient() to runtime module
Soon both, the cli and the masterd, will use a luxiclient to run queries. The method to obtain and configurethe luxi client with the right address can be reused fromcli to masterd here. Therefore, this patch moves the...
Use a query client for node queries where necessary
To remove the node query's python implementation soon, thispatch makes all users of the queries call the haskellimplementation instead of the python one.
Remove --enable-split-query option
Switching from python to haskell queries, this patchremoves the option to dis/enable the haskell queriesat configure time.
Add Xen paused state as a Ganeti running state
The QA test 'TestClusterBurnin' pauses instances and the Xen statedomain 'paused' was not being handled properly by Ganeti because itwas not being classified as either a running or shutdown state. Thispatch adds the 'paused' state to the set of running states by...