Handle promoting/demoting nodes wrt to client certificates
This patch makes Ganeti correctly handle the clientcertificates when nodes get promoted to master candidatesor demoted to normal nodes.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Extend RPC call to create SSL certificates
So far the RPC call 'node_crypto_tokens' did only retrievethe certificate digest of an existing certificate. Thiscall is now enhanced to also create a new certificate andreturn the respective digest. This will be used in various...
Create client SSL certificates on cluster init
This patch makes Ganeti create a client SSL certificate forthe master node on cluster initialization. Note that some ofthe code in this patch is later moved into an LU to serverequirements for crypto renewal and updates, but for this...
Store candidate certificates in ssconf
This patch enables Ganeti to store the candidatecertificate map in ssconf. A utility function toread it is provided as well.
Handle client certificates on node add/remove
This patch adds the certificate of a newly added orreadded master candidate node to the map of master candidatecertificates. It removes a master candidate node's certificatedigest from the candidate certificate map if the node is...
Add certificate for master node
On cluster initialization, the master node'sSSL certificate digest is added to the list of mastercandidate certificates.
Add candiate certificate map to configuration
At the end of this patch series, incoming RPC calls arelegitimized against a map of master candidate nodes'SSL certificate digests. This patch adds the map itselfto the cluster's configuration.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Retrieve a node's certificate digest
In various cluster operations, the master node needs toretrieve the digest of a node's SSL certificate. For thispurpose, we add an RPC call to retrieve the digest. Thefunction is designed in a general way to make it possible...
Utility functions to manipulate the candidate map
This patch adds a couple of utility functions to manipulatethe map of master candidate SSL certificate digests.
Remove superfluous imports
This removes some superfluous imports from the X509 (SSL)unittests.
Fix types for queries in QA
Due to the actual implementation of the '?' operatorin our query language, it happily accepted essentiallyany value that was not 0 or False as being true. However,it was always only specified to work on boolean values.Therefore, our QA shouldn't test for this unspecified...
Merge branch 'stable-2.10' into master
Replace errors re-export in luxi.py with proper imports
Instead of re-exporting errors in luxi.py, import rpc/errors.py in themodules that use them.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
break line longer than 80 chars
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
luxi.py: Fix pylint warning about unused imports
Reexport exception classes more explicitly for pylint's convenience.
Signed-off-by: Santi Raffa <rsanti@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
rpc: Fix one more py-apidoc warnings
hsqueeze: Also test for tagging
hsqueeze is required to tag nodes before powering them down. Also testfor this behavior.
hsqueeze: tag nodes before offlining them
hsqueeze is supposed to tag nodes before powering them down, so thatit later can recognize which nodes can be activated later. When showingthe commands to execute, also add the tagging commands.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Add an hsqueeze test for drbd instances
In this example, there are two drbd instances, rendering a total offour nodes ineligible for being offlined. Additionally, the mastermay not be offlined either, leaving a single candidate.
hsqueeze: only consider nodes that are not secondaries
If an instance has a secondary node, it cannot be easilymoved to every node (in the same node group), as otherwiseno node would be distinguished as secondary. As hsqueezeshould only consider nodes were moving the instances away...
rpc: Fix py-apidoc warnings
The previous commits shuffled code around using import renames asglue. apidoc ignores import renames, however, and chokes on somenow invalid link targets.
This commit fixes the issue.
Signed-off-by: Santi Raffa <rsanti@google.com>...
Separate the LUXI protocol version from the generic client
This allows other daemons and their clients (such as WconfD) to use adifferent versioning sequence of their protocols.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Rename CallLuxiMethod to CallRPCMethod
Also update error messages and testing code to refer to RPC instead ofLUXI.
Split Luxi Client into a generic and a specific part
The generic part will be reused in WConfd.
Move Transport from luxi.py to a separate module
Also create a new module for RPC errors.This allows it to be reused for other clients as well.
Add a Python directory for RPC code to keep it at one place
Move rpc.py to rpc/node.py and modify imports in existing code.
Gluster: announce in NEWS
Add the relevant line to NEWS
Signed-off-by: Santi Raffa <rsanti@google.com>Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Gluster: add the Shared File storage type
The shared file and gluster disk templates should not report their diskspace information like file does, because they do not behave the same.
If a cluster pulls from the same, shared source of storage then it is...
Gluster: add userspace access support
Add support for the QEMU gluster: protocol. Also change the accessmode routines so they check the access parameter for all templates.
Signed-off-by: Santi Raffa <rsanti@google.com>Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Gluster: mount automatically
Add parameters to the Gluster disk template so Gluster can manage themount point point autonomously.
Gluster: use ssconf value for mountpoint directory
Gluster still does not mount anything autonomously, but this commitchanges where Gluster expects its mountpoint to be.
ssconf: Add Gluster mount directory
This commit adds the gluster storage directory to ssconf (withoutactually using its value just yet).
Gluster: add GlusterVolume class
This commit teaches Gluster what a volume is and how to use it.
Gluster: minimal implementation
Add Gluster to Ganeti by essentially cloning the shared file behavioureverywhere in the code base.
netutils: Add ValidatePortNumber method
This method accepts a port number and checks that it is in fact valid.
FileStorage: extract file logic to a FileDeviceHelper object
This will allow code reuse for Gluster through composition, ratherthan inheritance.
FileStorage: move to filesstorage.py
Move the FileStorage class in its own file, together with its helperfunctions.
PathJoin: improve error message when given one argument
PathJoin fails with an unclear message if only one argument is passedto it. Calling PathJoin("/foo") causes this exception:
Error: path joining resulted in different prefix (/foo != /foo)
However, /foo and /foo obviously share prefixes: what this function...
ComputeLDParams: do not spell out disk templates
A large part of the complexity in this function is due to the needto translate from "template-specific" parameter names to"template-agnostic" parameter names. This logic is complex and havingcomplex code for complex logic is okay....
bdev: Fix position of DEV_MAP
This rather important dictionary from constants to classes was hidingbetween function definitions. The dict cannot go to the top of the fileas the classes haven't been defined there yet, so it's been pushedto the bottom of the file....
gnt-cluster verify: demote orphan volume error to warning
Ganeti checks for orphan volume by making sure that it knows about allvolumes on disk; any additional orphan volume, even if created by theadministrator, causes a failure in gnt-cluster verify. Given that...
For the commandline, switch to query socket by default
As luxid now understands all the requests used by the command-line tools,switch the default luxi socket for those to be the socket of luxid.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Implement fields query for instance
Support the query for the fields available for instances.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Remove the hvsGlobals from instance query fields
...to be consistent with the python implementation.
Add nic.vlans to the query fields
In commit 3293332 this was only done for the Haskell side; doso for python as well, to have both views consistent.
When interpreting [] as "all fields", sort nicely
When asked for all fields, we promise to return the list of fieldssorted according to niceSort. Keep this promise.
Version bump for 2.10.0~rc1
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Update NEWS for 2.10.0 rc1 release
Mention inherited changes as well as the (very few) changes made tostable-2.10 since beta1.
Fix pylint 0.26.0/Python 2.7 warning
pylint 0.26.0 on Python 2.7 generates a warning on the string '\ ',recommending to use the r prefix. This patch adds the missing prefix.
Update INSTALL and devnotes for 2.10 release
The following changes are made: * Add M4 as required dependency * Change reference to qemu-img package to qemu-utils * Never use `sudo` for easy_install (not used for apt-get neither) * Add libpcre3-dev as required package for Wheezy (otherwise linking...
Take a fresh Luxi client for each failing test
Luxid is more strict with closing the connection after receivinga syntactically incorrect request, gnt-debug cannot use the sameclient for several successive tests verifying that a syntacticallyincorrect request is recognized as such....
Fix race in watchFile
As the calling of watchFile and the evaluation of the initialgetFStatSafe takes non-zero time, the file could have changedbefore inotify was set up properly. Solve this problem by anadditional check for the watched value to have changed immediately...
Merge branch 'stable-2.9' into stable-2.10
Bump revision for 2.9.2
Update NEWS for 2.9.2 release
Besides a few local fixes, the main improvement are the changesinherited from stable 2.8.
Use a data type when generating Python types of OpCodes
Currently they are generated only as Strings.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Refactor OpCodeDescriptor from a tuple to a data type
This greatly enhances code readability.
Also fix monadic types "Q ExpQ" [which is "Q (Q Exp)"] to "Q Exp".
Add showValueList to PyValue for proper String instances
It's the same trick ShowS uses. We add a type class function forshowing a list to PyValue and then just use it in the instance for`[a]`. This way we have the proper String instance without anyoverlapping/incoherent instances....
Rename PyValueInstances.hs to PyValue.hs
Now the file contains the type class declaration as well.
Move PyValue into PyValueInstances.hs, import it in THH.hs
This puts all PyValue code into one module, getting rid of orphaninstances.
Make the duration field optional null-serialized
The time in SetWatcherPause is optional (with Nothing meaningthat the pause should be canceled), but the serialization isnot that of a Maybe Double; instead Just values serialize asthey are and Nothing serializes to null. Fortunately, we already...
Handle QueryConfigValues
Make luxid handle the QueryConfigValues call providing certainsimple status information about the cluster.
Add a predicate for watcher pause
Add a predicate, in IO, to test whether the watcher ispaused.
Provide path to watcher pause file
Extend Path.hs to also provide the path to the file indicatingwhether watcher is paused.
Implement SetWatcherPause in luxid
Make luxid handle SetWatcherPause correctly.
Add the RPC-call set_watcher_pause
With luxid taking over responsibility for handling watcher-pause requests,it needs to know about this RPC. So have it available in Haskell as well.
The time field for SetWatcherPause is optional
A JSON null value is used to indicate that the pause should be canceled.
Generate a separate return type for the job queue update RPC
The instantiation of RPC requires a bidirectional functional dependencybetween call type and return type. Hence we cannot use Unit everywhere.
Document format of the file-storage-paths file
The format of the /etc/ganeti/file-storage-paths file was not documentedin the man page. This patch adds a short note about the format there.
Pass hvparams to GetInstanceInfo
...so that the xen command to be called can be determined. Thisfixes another semantical conflict of the last merge.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Jose Lopes <jabolopes@google.com>
Adapt parameters that moved to instance variables
Due to a change in the code organization in stable-2.9, somemethod variables became instance variables, causing a semanticmerge conflict. Fix this.
OS-redesign: change instance IP
Change the IP of the instance to make it correspond to the one used in theexample.
The .253 is chosen so that, if needed, in the future the network can be changedfrom a /24 to a /28.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Avoid lines longer than 80 chars
...as they're a lint error.
OS installation redesign
Add the document describing a new design for the OS installation process fornew instances.
Signed-off-by: Michele Tartara <mtartara@google.com>Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Merge branch 'stable-2.8' into stable-2.9
Version bump for 2.8.3
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Update NEWS for 2.8.3 release
List all the changes that happened between 2.8.2 and 2.8.3.
Move the generalized IO client from Luxi to UDSServer
No code is changed in this patch (except imports and qualifiers), onlymoved.
Generalize the IO client handling in Luxi
... to be usable for WConfd as well. A daemon handler is encapsulatedinto `Handler` data type, which is then passed to a generic `listener`.
The changes are done in Luxi.hs so that the differences are visible and...
Add MonadLog instance for `ReaderT r m`
This allows to use logging with the ReaderT monad transformer.
Add a MonadLog typeclass for monads that allow logging
This separates logging from IO, allowing to create unit tests in futurefor functions that use it.
Add fromJResultE and fromJVal that uses MonadError
Using MonadError is more correct than just "fail" on an arbitrarymonad, and more scalable when using monad type classes or monad stacks.
Add an Error instance for GanetiException
This allows it to be used with MonadError.
Add MonadPlus and MonadError instances for GenericResult
.. and ResultT.
While at it, generalize also the MonadPlus instance of GenericResult andadd some Functor/Applicative instances.
Support reseting arbitrary params of ext disks
If param=default and the param already exists then we removeit from params dict. This is stolen by GetUpdatedParams() whichis used for hvparams modification/inheritance.
This means that 'default' value is not accepted for an arbitrary...
Allow modification of arbitrary params for ext
Disks of ext template are allowed to have arbitrary parametersstored in the Disk object's params slot. Those parameters can bepassed during creation of a new disk, either in LUInstanceCreate()or in LUInsanceSetParams(). Still those parameters can not be...
Do not clear disk.params in UpgradeConfig()
Commits 5dbee5e and cce4616 fix disk upgrades concerning paramsslot. Since 2.7 params slot should be empty and gets filledany time needed.
Still ext template allows passing arbitrary params per disk.These params should be saved in config file for future use....
Generalize "validateCall" to be usable outside LUXI
Return the method (as any instance of JSON) and the arguments of a call.
Add the Unix domain socket path to the Server data type
This simplifies code for closing such a socket.
Encapsulate a server socket and its parameters
Instead of passing a bare server socket around, we pass it encapsulatedin a data type together with parameters such as read/write timeouts.
Rename getClient/Server to getLuxiClient/Server
Later they will be split into LUXI-specific and general parts.
Split Luxi.hs into LUXI-specific functions and general ones
This will allow WConfD to use the general functions without importingLuxi.hs.
Make luxid support WaitForJobChange
Make support the WaitForJobChange, waiting for a job tochange on certain monitored fields.
Add a generic function capable of watching a file
Add a method to return the new value of a function if it changes withinthe given timeout. If not, return the old value. Make use of the fact,that the function only changes, if the specified file changes on disk....
Add a safe version of getFStat
The function getFStat causes an IOError if the file to be stated doesnot exist. In some cases, however, the only thing we care about is whetherit has changed, with disappearing being a legitimate change. So add a wrapperthat catches the IOError and returns nullFStat....
SetDiskID() before accepting an instance
SetDiskID() fills physical_id slot of a Disk object.
LUInstanceSetParams() does not invoke SetDiskID() upon creation of anew disk. As a result the physical_id slot of the Disk object inconfig data is missing.
In case of ext disk template, in AcceptInstance() we invoke...
Lock group(s) when creating instances
This is required to prevent race conditions such as removing a networkfrom a group and adding an instance at the same time. (See issue 621#2.)
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>...
Gluster: Update design document
Anticipate and explain the choices made in the Gluster patch series.Remove parts about a possible way of supporting userspace access asit has been implemented otherwise.
Signed-off-by: Santi Raffa <rsanti@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Fix 'htest' related targets dependencies
This fixes issue 634.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Fix job error message after unclean master shutdown
According to commit 599ee321eb, any job-related error messages shouldbe encoded within a Ganeti-specific error and not passed on as astring, to allow for easier parsing.
For jobs suffering from an undesirable status after an unclean master...