Add uid_pool to ssconf
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add --uid-pool option to gnt-cluster init
Add printing of uid_pool to gnt-cluster info
Add --add-uids/--remove-uids to gnt-cluster modify
Add --uid-pool option to gnt-cluster modify
Add lib/uidpool.py module
Add uid_pool cluster parameter
Add user-id pool design doc
backend: Consolidate code opening real block device
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
utils: Add class to split string stream into lines
This will be used by the new import/export daemon to splitoutput by its child processes into lines.
Fix cluster behaviour with disabled file storage
There are a few issues with disabled file storage:- cluster initialization is broken by default, as it uses the 'no' setting which is not a valid path- some other parts of the code require the file storage dir to be a...
Update docstrings in tools/ and enable epydoc
This patch updates the docstrings in tools/ (mostly in lvmstrap, whichis very very old code-base) and then enabled the tools in this directoryfor 'make apidoc' too.
Signed-off-by: Iustin Pop <iustin@google.com>...
Forward-port the ganeti 2.0 cfgupgrade
2.1's cfgupgrade doesn't support upgrading from 1.2, only from 2.0.However, it's trivial to forward-port the 2.0 cfgupgrade to work with2.1, thus providing an upgrade path for 1.2 users directly to 2.1,without the intermediate step of installing 2.0....
Add a new tool: sanitize-config
This can be used for two purposes:
- safety copy of the config file, with just the secrets changed- cleanup of the config file (full randomization), so that (e.g.) users could send a broken config file to the devel-list...
Fix cfgupgrade with non-default DATA_DIR
Commit 43575108 added bootstrap.GenerateclusterCrypto and commit7506a7f1 changed cfgupgrade to use it. However, this lost thefunctionality of upgrading in non-default DATA_DIR.
To fix this, we enhance bootstrap.GenerateclusterCrypto to accept custom...
Improving the RAPI documentation
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Mark cluster-merge as executable
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
QA: Make the rapi credential handling less involving
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Update import documentation for the recent changes
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add a identify-defaults options for import
When importing an instance, all the saved valued will be used asexplicitly specified values, overriding the cluster defaults. This meansexport+import will change the status (from default to explicitlyspecified) of parameters....
Fix create/import verification of hvparams
Currently the instance creation checks the cluster hv defaults + the newparameters for validity, ignoring the os-specific hvparams (this was anoversight during the implementation of the os hvp). This patch uses the...
objects.Cluster: add method to get hv defaults
Currently the FillHV method is the one that does the cluster hvparams +os hvparams merger. However, in some cases we need to do just this,without adding the instance hvparams on top.
This patch adds a function to compute just this (hv + os hvp...
Reuse NIC information from export
If the user doesn't pass any nics in import, do not use a defaultone-nic, but instead read the nics from the export file as is.
Fortunately the export and the way nics are read from the command lineare compatible…
Reuse backend parameters from export
Similar to the previous patches, if we're missing some parameters andthe export has them (either in the new style or old-style), we reusethem.
Reuse disk information from export
If the user doesn't pass the disk information on import, automaticallyreuse the number and size of disks. This loses the iv_name attribute,but that is only cosmetic and cannot be changed by the user.
Reuse hypervisor parameters in import
If available, we reuse the parameters from the export info.
Read disk template from export info
This patch changes the instance import to read the disk templateautomatically from the export info, if the opcode doesn't alreadyspecify a disk template.
To do this, we have a couple of additional changes:
- change from required parameter to optional one for disk_template...
CreateInstance: separate the reading of the export
We move the reading of the export to a separate function, to simplifyCheckPrepreq and also read it earlier. This will allow building themissing opcode parameters from the export information, instead of...
Move code from ExpandNames to CheckPrereq
This is needed since only in CheckPrereq we have the nodes locked, andfuture import enhancements will need to have access to the export infoduring the parameter build.
CreateInstance: Move some code to CheckArguments
ExpandNames holds too much non-locking code (first LU to be converted toExpandNames, and we didn't have CheckArguments at that poin), and thispatch moves the checks that are lock-independent to CheckArguments....
Export more instance parameters in instance export
Currently the backend parameters are not exported automatically, butonly a few directly in the '[instance]' section. Hypervisor type andhypervisor parameters are not exported at all.
This patch creates two separate sections for the be and hv parameters,...
Export the nicparams too during instance export
The patch tries to export all params (based on the dict defined inconstants), using None for missing keys.
Handle errors better for wrong nic_count in export
This fixes an old 'FIXME' entry.
QA: Make sure RAPI credentials are setup on cluster init
This patch makes sure that the Ganeti RAPI credentials are setup,if any, on cluster init time.
Fix new pylint errors
burnin.ExecOrQueue: remove variable argument list
In order to later add an optional parameter we transform the variableops argument list in an explicit list.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
burnin.ExecOrQueue: add post-process function
If a post-process function is passed to ExecOrQueue it is executed ifand only if the job is successful. This happens immediately if we'reproceding iteratively, and at the end, when we collect all job results,...
burnin: only remove instances we actually added
Currently burnin, if proceding in parallel, will remove all instanceswhich were passed, even if they failed to add. This is bad because itwill also remove instances which existed before burnin started. By...
Rename the confd_client unittest (to confd.client)
This is to keep same naming across all tests (modules separate with dot,followed by _unittest.py).
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Make watcher request the max coverage
Since the actions are potentially destructive, we should try to get aconsistent view of the cluster, so it's better to get the most coveragepossible.
ConfdClient.SendRequest: allow max coverage
This patch changes the coverage parameter to allow specification of maxcoverage (via -1), versus auto-computation (default, 0) and manualspecification.
Unittests are updated for this case too.
Document the watcher node maintenance feature
The patch changes significantly the watcher man page, as it was verysimplistic.
Watcher: automatic shutdown of orphan resources
This patch changes the watcher so that it maintains (on all nodes) thelist of instances and DRBD devices by shutting down ones that confddaemons indicate should not be running on this node.
Export the maintain_node_health option in ssconf
Add a new cluster parameter maintain_node_health
This will be used to conditionally enable the watcher node maintenancefeature.
Add a new confd callback (StoreResultCallback)
This new callback simply stores (without calling any lower-levelcallback) the last result; coupled with the filtering callback, thisensures that it has the 'best' response after all have been received.
The result can then be retrieved via the GetResponse method....
ConfdClient: add synchronous wait for replies mode
Currently, there is no way for a user of the confd client library toknow how many replies there should be, whether all have been received,etc. This is bad since we can't reliably detect the consistency of the...
ConfdClient: unify some internal variables
Currently the requests are tracked in _request and in _expire_requests.This is conventient, but it restricts the ability to extend the requesttracking, e.g. via packet stats and/or extension of expiration time....
Fix consistency checks in ConfdFilterCallback
Commit 49b3fda added consistency checks, but these are wrongly triggeredfor old responses - we need to make sure to check that we have the sameserial.
Fix utils.WaitForFdCondition inner retry loop
Commit dfdc4060 added WaitForFdCondition which uses utils.Retry withouthandling timeout exceptions. This breaks any nested retry loops.
This patch fixes the above function, and also changes utils.Retry todetect and warn future similar cases. In addition, we add a few small...
Fix bug introduced in 76e5f8b54: mkdir mode
After commit 76e5f8b54, mkdir_mode in utils.RenameFile isno longer passed to Makedirs. This is fixed by this patch.
utils: Move wrapper code around os.makedirs into separate function
Fix unittest for the rapi client library
Wrong escape, so we make sure to use proper escapes (we want thebackslashes to be embedded, not interpreted). Also change " to ' to beeasier to read.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>...
Adding RAPI client library.
Signed-off-by: David Knowles <dknowles@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Signed-off-by: Iustin Pop <iustin@google.com>(modified slightly the unittest to account for missing httplib2 library)
Extend ConfdFilterCallback with consistency checks
Note that users of the callback will have to manually check theattribute.
Abstract the confd client creation
Most creation of confd clients will do the same steps: read MC file,parse it, read HMAC key, etc. We abstract this functionality so thatwe don't duplicate the code.
Remove unused import from test file
kvm_flag hypervisor parameter
Move the runas user at execution time
Everything still works the same way, but the user is calculated eachtime we start kvm, rather than stored in the config file. This makes iteasier to implement the "pool" security model.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Send "501 Not Implemented" back when method not found
Before this was "400 Bad Request" and thus it didn't reflectthe reality.
Adding QA RAPI tests for activate-disks and deactivate-disks calls
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
SerializableConfigParser: Make Loads class indep
Currently SerializableConfigParser.Loads is a static method that returnsa SerializableConfigParser. With this patch we change it to a classmethod that returns a member of the class. This way a subclass calling...
Unbreak command line job submission
A change introduced in 5299e61f modified the contents ofJobExecutor.jobs, missing a place where this tuple was deconstructed.This creates a traceback in gnt-* <any> --submit, fixed by this patch.
Write grow support for file storage
Allow file storage to be grown
Distribute list of enabled hypervisors in ssconf
This can be used by nodes to know which hypervisors they are supposed tosupport.
Extend the hypervisor API with name-only shutdown
Currently the ShutdownInstance method of the hypervisors takes a fullinstance object. However, when doing instance shutdowns from the nodeonly, we don't have a full object, just the name.
To handle this use case, we add a new ‘name’ argument to the method,...
Watcher: do not warn for missing hooks dir
If the hooks dir does not exist, do not warn needlessly. This is similarto commit a9b7e346 (for backend.py).
Watcher: fix some doc typos
ganeti-confd: Call pyinotify flags correctly
The "apparently pylint was right" commit.
Although the pyinotify constants work on old distributions, they fail onnew ones, with new python. Fixing this by calling them in a way thatworks everywhere.
Fix burnin error when trying to grow a file volume
Abstract the growable disk types in a ganeti constants, and only rundisk grow, from burnin, on them.
Fix backend.VerifyNode behaviour for VG problems
In case LVM is broken, backend.GetVolumeList will raise an RPC exception(as expected since it's a function exposed over RPC). Therefore we mustbe prepared to catch any such exceptions, so that we don't fail the...
Introduce a bool CLI option type
This option type enforces its value to either True or False, relievingthe scripts from manually parsing the values in each function.
We also update the bash completion code to use the option type ifpossible.
A rewrite of LUClusterVerify
Per issue 90, current cluster verify is very very brittle. It's one ofthe oldest pieces of code, with only additions without cleanups over thelast years.
Among its problems:
- data initialization interspersed with verification of RPC results,...
Some epydoc fixes
Adding missing documentation to make the docs better
Also fixed a typo I noticed.
Remove race condition in FileStorage.Create
Rather than checking that the file doesn't exist, and then creating it,we create it with O_CREAT | O_EXCL, making sure the checking/creation isatomic.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
KVM: Check instances for actual liveness
Currently if we find a live process with the pid we saved we assume kvmis alive. What could happen, though, is that the pidfile has beenreused.
In order to avoid that we change the check to make sure, everywhere,...
KVM: improve GetInstanceInfo docstring
Revert "Only override any and all if not defined"
This reverts commit bd5617020a50bcd08269330638d64078c1b30b71.
Turns out our and python's any/all are not compatible.
Adding RAPI call to deactivate-disks for an instance
Adding RAPI call for activate-disks on an instance
Add a hint to masterd for inconsistent clusters
SimpleConfigReader: add docstrings
All non-oneliner functions, after this patch, have their docstring.
Replace @keyword with @param in confd client
@keyword was used inappropriately.
ConfdClient: add synchronous features
By sending requests with async=False, and receiving replies withReceiveReply we can more easily use confd from a synchronous client.
ConfdCountingCallback
This new confd callback counts received replies for the registeredqueries.
move http.WaitForSocketCondition to utils
WaitForSocketCondition: rename, handle EINTR
- Rename WaitForSocketCondition to SingleWaitForFdCondition - Avoid potentially infinite loop, if we continue to get interrupted - Handle eintr correctly - Avoid the poller try/finally, as the poller object gets destroyed...
AsyncUDPSocket.process_next_packet
This function allows receiving socket data synchronously.
burnin: implement basic confd testing
Just a few queries are checked, but this should give us confidence thatat least the basic confd framework is working properly.
AsyncUDPSocket: abstract do_read function
This basically implements read handling, without catching allexceptions. When using the socket in synchronous mode, it's useful toavoid losing exception data (which, in an async daemon, can only belogged)
Burnin: don't add/remove routed nics
Only override any and all if not defined
If any or all are already defined (because we're using a new version ofpython) just link them inside "utils" rather than redefining them.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>...
backend: Two small style fixes
- Pass keyword parameter as such- Replace “not x == y” with “x != y”
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Allow cluster copy file over the replication net
This patch introduces the option “--use-replication-network” for thecluster copyfile functionality, which is useful if the primary andsecondary network are significantly different (see issue 32).
Enhance cli.GetOnlineNodes query/filtering
This patch allows GetOnlineNodes to return the secondary IPs instead ofthe node names, and to provide filtering of the master node (required tobe done in this function in case we return the secondary IPs).
Instance creation: implement --no-install mode
This is a simple patch that adds the no-install mode for instancecreation, allowing import from foreign source of the actual OS (insteadof requiring the preparation of data in a form expected by the import...
Allow OS changes without reinstallation
This patch modifies LUSetInstanceParms to allow OS name changes, withoutreinstallation, in case an OS gets renamed on-disk.
cmdlib: Abstract OS checks
This patch moves the node-has-os checks to a separate function.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>